Up to 20 percent of Facebook’s pay-per-click advertising revenue could be affected by a pending ruling from a California federal appeals court on whether a group of Facebook’s advertisers were charged for “invalid,” clicks on their ads from 2009 onward.
Facebook has denied the claims, and successfully won a ruling denying class action status at the district court level. The appeal hinges on the technicality of what counts as a “class” under federal law, and whether the plaintiffs adequately represent advertisers on Facebook as a whole. The company told us:
“We have a team of professionals and use sophisticated technical measures to monitor click activity and guard against click fraud and other improper actions that affect the cost of running ads. With respect to the lawsuit challenging our click filters, despite more than two years of discovery, the plaintiffs were unable to identify a single click that they believe was invalid. The allegations in the lawsuit are without merit and we were pleased with the district court’s decision.”
The papers within the case, however, could prove explosive for Facebook’s reputation. They include emails and internal memos written by Facebook’s engineers and advertising operations executives, and communications between employees of Facebook’s auditor, Ernst & Young.
Several Facebook engineers and advertising staff have also been deposed in the litigation, and a handful of the company's biggest advertising clients have been served with subpoenas, the court docket shows. In one filing, a lawyer for Facebook notes that the discovery phase of the litigation has covered 200,000 documents and 16,000 emails from its engineers.
Some of the evidence allegedly shows that, internally, some Facebook staff regarded independent verification of its clicks as “toxic” because they knew they wouldn’t pass an audit. Internal Facebook emails in the case describe a proposed adjustment to a key algorithm to detect invalid, accidental or fraudulent clicks, which would cost the company sales. One email stated:
“I think this is pretty much a no-brainer, but it is a fair amount of revenue. So we got to go to the board on this one.”
The adjustment was never done because engineers did not receive approval, the suit alleges.
Thousands of advertisers affected
If the appeal goes badly for Facebook, the class could cover as many as 100,000 of Facebook’s advertisers. Of those, 5,000 have made inquiries at Facebook about their clicks, the suit indicates. (It doesn't say whether they doubted the clicks' integrity.)
Sources tell Business Insider that between 40 and 80 percent of ads on Facebook are bought on a pay-per-click basis. Advertisers only pay for those ads if someone clicks on them. The plaintiffs have argued that up to 20 percent of those clicks are invalid, and that advertisers should get refunds for them. Assuming that Facebook recognizes roughly $1 billion per quarter in revenue, such a refund could amount to between $80 and $160 million per quarter.
Much of the paperwork filed in the case has been sealed or redacted to protect the confidentiality of Facebook’s business dealings. However, late last year MediaPost wrote a story about the case and published a pleading from it that was improperly redacted with black boxes to obscure the sealed text. Anyone with a PC could cut and paste the text underneath the black boxes into a word-processing document in order to read it.
Business Insider publishes those redactions here for the first time, on the following pages of this article. They include summaries of internal Facebook emails and what Facebook engineers told lawyers in depositions about the effect that adjusting their algorithms would have on Facebook’s revenue.
“We take click quality very seriously”
The suit was brought by three small Facebook advertisers, RootZoo, Nathan Fox of Fox Test Prep and Steven Price from DriveDownPrices.com, who each bought a few hundred dollars worth of pay-per-click advertising on Facebook. RootZoo was a sports fan web site; Fox is a tutoring service and Price ran the now-defunct DriveDownPrices.com, a car sales site.
All the companies wanted the ads to drive traffic to their sites. But they noticed that when they used their own click measurement programs, such as Google Analytics, Facebook was allegedly charging them for more clicks than were actually landing on their web sites.
At the time, a number of advertisers complained that they were being overbilled for clicks. Facebook currently offers to investigate advertisers' complaints, and resolve disputes, if a client believes clicks were invalid. A Facebook spokesperson said in 2009 that the company had fixed the problem:
“We take click quality very seriously and have a series of measures in place to detect it. We have large volumes of data to analyze click patterns and can identify suspicious activity quickly.”
“Over the past few days, we have seen an increase in suspicious clicks. We have identified a solution which we have already begun to implement and expect will be completely rolled out by the end of today. In addition, we are identifying impacted accounts and will ensure that advertisers are credited appropriately.”
Adam Gerston, a former manager at Facebook who worked on the company’s preferred marketing developers program, defended the company when he spoke to BI recently:
“I was working at Facebook when this issue arose. It’s an issue that Facebook has generally taken responsibility for, and it has taken lots and lots of steps to ensure clicks are real clicks.”
(Gerston is not involved in the suit; he now works at GraphEffect, a marketing company that does business with Facebook.)
Facebook offered the plaintiffs credit for more advertising on Facebook, but they decided to litigate instead. Facebook has since accused the companies – none of which is now actively doing business – of existing largely to pursue litigation against Facebook.
Fox, Facebook argues, failed to identify any damage he had suffered from missing clicks, and Facebook's analysis of his server showed that invalid clicks were actually "caused by Fox's own (mis)configuration of his servers and software."
No audits allowed
The companies were frustrated by Facebook’s “Statement of Rights and Responsibilities” for advertisers, which states that Facebook will not entertain inquiries about invalid clicks. Facebook specifically says it isn't responsible for invalid clicks:
We cannot control how clicks are generated on your ads. We have systems that attempt to detect and filter certain click activity, but we are not responsible for click fraud, technological issues, or other potentially invalid click activity that may affect the cost of running ads.
Facebook also declines to describe how it measures clicks. A Facebook glossary says:
Due to the proprietary nature of our technology, we’re not able to give you more specific information about these systems.
The suit also claims that Facebook declines to allow its clicks to be audited by a third party. That became an issue when the plaintiffs discovered that Facebook did not abide by the self-regulatory guidelines published by the Interactive Advertising Bureau.
The IAB represents 500 media companies in the online ad business. Its guidelines were crafted to give ad buyers confidence they weren’t being ripped off. They encourage the use of third-party ad verification services, like Adometry or RocketFuel, which identify invalid clicks caused by bots, fraud, or accidental clicks from users who clearly weren’t interested in an ad.
Facebook is on the board of the IAB, but doesn’t adhere to its guidelines, the suit alleges. Facebook’s main competitors – Google, Yahoo! And Microsoft– have all put their names on the guidelines. BI asked IAB CEO Randall Rothenberg why Facebook was a holdout. He told us he didn’t know, and added:
"All standards we develop are voluntary by definition; as a 501(c)6, we are not and cannot be an enforcement body. Industry standards are a 'wisdom of the crowd' activity - they are developed by groups of volunteers from member companies [and] … they consequently represent the best collective judgment about how to take friction and cost out of a supply chain. LOTS of companies adhere to standards but don't (and don't have to) sign their names to anything."
(Joe Laszlo, IAB’s senior director/mobile marketing center of excellence has also been subpoenaed in the case.)
“Toxic”
Facebook employees deposed in the case said they regarded the IAB as potentially “toxic,” according to a redacted motion from the plaintiffs. That pleading alleges:
… many Facebook witnesses in positions of authority with respect to click filtering either have either no idea what the IAB standards were or had little regard for them. Exh. 6 at 242:19-24 (characterizing the standards as “toxic”); Exh. 9 at 14:19-20 (“I personally didn’t read any IAB materials.”)
Facebook explicitly denies its advertisers the chance to audit their own clicks. On one of its FAQ pages, the company answers that question this way:
What can I do if I think I’m receiving invalid clicks?
Third-party aggregations or reports can’t be accepted for this purpose because they don’t contain the level of detail needed for investigation.
Facebook isn’t accredited by the Media Ratings Council, either. MRC sets ad industry standards for audience measurement. MRC CEO George Ivie told us:
We don't do any auditing of Facebook's measurement systems. Facebook has not received accreditation from MRC, and they have never applied for accreditation of any kind.
Advertisers and engineers subpoenaed
We called a random sampling of Facebook’s advertisers for comment. None spoke for the record. Talking privately, however, they all noted that Facebook stood alone among the major web ad business by not allowing third-party verification, but they did not believe Facebook was ripping them off.
Among the Facebook ad buying clients subpoenaed for the case are Buddy Media (now the Marketing Cloud unit of Salesforce.com), Efficient Frontier, Marin Software, Digital Envoy, and AdParlor.
Facebook staffers who have been deposed or written affidavits in the case include Robert Kang-Xing Jin director of engineering at Facebook; John McKeenan, manager in advertising operations; Thomas Carriero, an engineer; and Jordan Blackthorne, a product marketing manager.
The company that audits Facebook’s financial accounts, Ernst & Young, also audits clicks for online businesses. But an email described in the case sent between two E&Y employees allegedly states that Facebook has resisted a click audit.
According to the plaintiffs' filings, Jackson Bazley, E&Y’s executive director for advisory services/media and entertainment, allegedly wrote to Illian Ilev, the E&Y partner in charge of tech audits:
“I talked to our contacts [at Facebook] and the sense is that they will not pass such an assessment...”
Facebook has argued that the IAB is not a legal body, and that its terms and conditions for advertisers are clear. No one is forced to advertise on Facebook, and there are plenty of competing services who will take their ad dollars.
Facebook has long regarded itself as different from the rest of the online ad industry, and its executives may feel that the usual standards should not apply. Users must login to see Facebook, unlike most media websites, for instance. And Facebook has its own proprietary ad units that work differently than the standard banners used elsewhere on the web.
What is an "invalid" click?
Lastly, the exact amount of allegedly “invalid” clicks is also in dispute. The idea that up to 20 percent of clicks shouldn’t be billed comes from a study the plaintiffs allege was commissioned by Facebook. In that study, ad verification service Adometry allegedly found that up to 33 percent of clicks coming off Facebook were invalid, but in the same period Facebook only discounted 12 percent of clicks as invalid.
The plaintiffs thus allege that the difference – 21 percentage points – is the total amount of invalid clicks that Facebook wrongly charges for. In oral arguments before the judge, the plaintiffs' lawyer said:
"One time they found — Adometry found — 33 percent of their clicks were bad. Facebook only found in the same period 12 percent. Facebook didn't say to Adometry, hey, guys, we've got a problem. You are finding a lot more bad clicks than us. No. They took it back — I don't know what they did with it, they didn't change the filters."
In response, Facebook's lawyer argued:
"Is there a law that says I have to make a second click within a minute free? 10 minutes, is that one free? 20 minutes? How do you determine that? How do you determine that?
The data can be interpreted in different ways, however. The same study — which allegedly analyzed about 464,000,000 clicks on Facebook in July and August, 2010, also showed that the margin of difference may be as little as 3 percentage points, depending on how you define “invalid” clicks, according to the suit.
Adometry — which is not part of the suit — denies the plaintiffs' interpretation of the data. "The percentage of clicks that Facebook's systems designate as invalid is in line with our independent analysis of data received from Facebook and with rates we typically see in the industry," Peter Norwood, COO at Adometry, told us.
The Ninth Circuit Court of Appeals has yet to set a date to hear the case. Even if the appeals court finds in Facebook's favor, the advertisers will still be able to continue litigating as individuals. A final decision is thus months, if not years, away.
- The following pages contain the unredacted text from a plaintiffs' motion for class certification. They should be read in context with Facebook's motion in opposition, and the lower court ruling that found in favor of Facebook. (These motions discuss the facts of the case more fully than the appeals briefs, which focus more on the technicalities of what counts as a class.)
Disclosure: The author owns Facebook stock.
This material was redacted so that the public could not see the plaintiffs' argument that Facebook refuses to have its clicks audited by a third party group, under IAB guidelines.
Page 2
In spite of contracting with its advertising customers to report and charge advertisers only for “legitimate clicks,” Facebook has largely eschewed industry standards. Although an IAB member since at least 2008 and currently on its Board of Directors, Facebook refuses to be audited under IAB criteria, in contrast to other industry leaders. It also has flat out refused to allow any third party to test its wholly internal process for determining the legitimacy of the hundreds of millions of clicks it charges advertisers for each month. The consequence of this refusal is that Facebook cannot state, let alone “ensure,” that any of the clicks it charges customers for are in fact “legitimate,” which is the very promise at the heart of Facebook’s contractual obligation to its customers.
Here, the plaintiffs argue that Facebook has placed revenue generation above the integrity of its clicks.
Page 3
Common evidence reveals that Facebook’s secretive click measurement system elevates the goal of revenue generation above the self-imposed mandate to ensure the reporting and charging of only “legitimate” clicks. Revenue objectives were paramount. Because loss of revenue was such a concern, Facebook consistently ignored its engineers’ recommendations that would have brought its click legitimacy rules closer to industry practice. Even when its lead engineer proposed a rule change he believed was a “no brainer,” Facebook management refused to adopt it. In essence, Facebook manipulated its determination of click legitimacy to achieve revenue goals. As demonstrated below, plausible expert evidence shows that classwide liability and damages can be proven using Facebook’s own historical data, without the need for individualized proof. Accordingly, class certification is appropriate.
The plaintiffs describe how they believe Facebook's click filters work.
Page 5
Id. at ¶ 19. This operationally-based approach has been adopted by all major sellers of PPC advertising, including Facebook. Exh. 5 at 194:3-12.5
5 In 2007, when Facebook launched its PPC program, Facebook’s rules-based filter program contained six filters designed to flag illegitimate clicks. Facebook refers to these filters as follows “internal IP,” “delayed click,” “duplicate click,” “paused ads,” “click caps,” and “impression caps.” Ex. 16. The “internal IP” filter is intended to invalidate clicks that are received from website addresses within Facebook. Plaintiffs will show that these filters were inadequate both because there should have been additional filters, and because the ones they did use were not designed to catch all illegitimate clicks. The “delayed click” filter is intended to invalidate clicks that occur more than a set period of time after the advertisement is displayed. That set period of time, for example, was unreasonably narrow as established. The “duplicate click” filter is intended to invalidate clicks that occur within a set time frame from the same user on the same advertisement. It also used to narrow a timeframe for defining duplicate clicks. The “paused ad” filter is intended to invalidate clicks that are received after a set period of time that the particular advertisement was paused by the advertiser. The “click caps” and “impression caps” filters are intended to invalidate clicks from users who have clicked on more than a particular number of advertisements a day. Id. In late 2009, Facebook belatedly created a new filter, the “bad agent” filter, using an industry list of automated “robots” and “spiders”. Exh. 5 at 209:4-22; Exh. 6 at 250:20-24. This new filter was intended to discard clicks from known automated processes. At the same time Facebook also belatedly created another filter known as a “velocity karma” filter. Exh. 6 at 215:5-7. This filter is intended to discard any more than three clicks per user within a 5 minute period. Exh. 16 at FBCPC68. Plaintiffs have already compiled significant evidence as to paucity of the number of filters and the problems with the filters Facebook actually created. But the key fact at this stage in the litigation is that these filters applied classwide to every click on an ad that was recorded by the system.
This section discusses the plaintiffs' argument that Facebook's dispute resolution service is essentially a secret.
Page 7
Facebook has not kept its contractual promises to charge PPC advertisers only for “legitimate” clicks and to maintain measures to “ensure” illegitimate clicks are not billed. It has done so by ignoring the recommendations of its engineers and by spurning the primary source of legitimacy in the PPC advertising market—the IAB click measurement standards including third-party auditing to verify compliance with those standards. Facebook’s Click Filtering System Is Infected With Revenue-Driven Bias.
Facebook refuses to provide any details to advertisers when there is a dispute between Facebook and the advertisers over charges for clicks including even data about an advertiser’s own clicks. See, e.g., Exh. 11. Such a refusal has prompted advertisers to complain to Facebook customer support that they are displeased with Facebook’s blanket “trust us” mantra. See, e.g., Exh. 12 at FBCPC183076 (“So basically what you’re telling me is that the official Facebook advertising policy is ‘Just give us your money and trust us because we refuse to give you any information’? … when I try to find out what your policies are, your response boils down to ‘It’s a secret.’”).
The plaintiffs believe Facebook's engineers are encouraged to think about how their actions might impact revenue.
Page 8
While Plaintiffs’ expert, Dr. Jansen, notes that a “black box” approach may be necessary to protect the integrity of the process,9 he also explains that inherent in the “trust us” promise is the assumption that the click filters are “fairly implemented with the sole objective of protecting advertisers from being charged for illegitimate clicks.” Exh. 10 at p. 9.
Although Facebook professes that its rule-based filters are designed to protect advertisers (Exh. 13 at 76:1-5), Facebook’s internal emails and deposition testimony demonstrate that Facebook manipulated the secrecy of its processes to its financial advantage in two different but related ways: (1) by setting and modifying its filters in a manner designed to enhance revenue; and (2) by failing to employ reasonable measures used in the industry to determine and audit click legitimacy. Brazenly conceding this revenue objective, a Facebook internal web page makes clear that click filtering algorithms are “subject to change at any time, so that we can optimize both our revenue and the veracity of our billable statistics.” Exh. 14; Exh. 9 at 95:20-96:8. (emphasis added.) Facebook routinely makes decisions about filter parameters by considering revenue data instead of fulfilling Facebook’s promise to “ensure” that advertisers are being charged only for legitimate clicks. Exh. 9 at 235:1-6; Exh. 13 at 76:9-77:14. As one Facebook engineer explained: “I feel it’s important to understand the impacts of any particular change on all aspects of the system, including revenue.… I took it upon myself, as I think everyone on ads engineering did, to … understand the impact that that would have [on revenue].” Exh. 9 at 235:6-8, 235:19-22 (emphasis added). Among the engineers working on the advertising filters, it is clear that “the revenue specific is relevant.” Exh. 13 at 192:22-24. In fact, Facebook engineers readily concede they wear a “revenue hat” when they make decisions regarding click filters. Id. at 215:19-21 (“Q. So in this e-mail [about filters] you are wearing your revenue hat, right? A. Sure.”); see also Exh. 19.
Facebook's two-second rule for excluding unintended double-clicks is too strict, the plaintiffs allege, and should be extended to as much as 30 minutes.
Page 9
The process Facebook undertook in modifying the “duplicate click” filter is illustrative of the revenue-driven bias in its PPC platform. When Facebook launched its PPC advertising product in late 2007, it created an algorithmic filter that discarded any click from a user that occurred within two seconds of the prior click. Exhs. 15-17. This rule permitted Facebook to charge advertisers for second clicks from the same user that occurred outside of a two second interval (assuming that another click rule did not invalidate the click for another reason). Recognizing that this time interval was resulting in billing for clicks that clearly should have been considered illegitimate as “duplicate clicks,” Facebook decided to expand the time interval of the filter. After studying industry data and determining that the industry standard was at least a twenty-minute interval (Exh. 18), Facebook’s engineers concluded internally in late 2009 that it was “no brainer” that the filter should be changed to at least a twenty—or even a thirty-minute interval—to conform with industry practice and ensure click legitimacy. Exh. 19-22. Facebook recognized, however, that such a modification would result in a significant revenue decrease. Exh. 19 (“[moving the click time out to 30 minutes] is a fair amount of revenue, so [the engineers] may have to ask the board about this one, or whatever process we have for larger amounts of revenue.”). Ultimately, Facebook’s revenue objectives prevailed over any concern for click legitimacy, resulting in the decision to make the interval only thirty seconds between clicks. Exh. 16. The identical (and improper) process occurred when engineers at Facebook determined to change a filter that would prevent billing for duplicate clicks on the same “impression” (meaning that a user clicks twice on the same ad on the same web page) irrespective of the time interval between clicks.10 Facebook engineers concluded that this change was important for click legitimacy and were set to make the change but senior management permitted revenue considerations to prevail and the engineers’ recommendations were ignored. Exhs. 23-24.
A plaintiffs' expert criticizes Facebook for not having a 'Chinese Wall' between its engineers and its business staff.
Page 10
Plaintiffs’ expert, Dr. Jansen, calls into question the legitimacy of Facebook’s click filter system during the class period in part because of Facebook’s failure to create a “Chinese Wall” between the engineering decision to create or modify a filter and the consideration of how such a change will impact revenue. After reviewing multiple internal communications between Facebook engineers as well as deposition testimony in this case, Dr. Jansen concluded: “there appears to be no wall of separation between the engineering aspects of designing and implementing filters and the business side of impact on revenue at Facebook.” Exh. 10 at p. 18. Such a bias, in Dr. Jansen’s opinion, is violative of both “industry standards and best engineering practices.” Id. at p. 22.
Again, a portion on Facebook's lack of IAB compliance was redacted.
Page 11
Facebook’s Click Filtering System Does Not Comply With Industry Standards Endorsed by Facebook Itself.
The working group first disseminated a draft set of standards for comment by its members. Exh. 26 at 43:15-19. After the comment period expired, the IAB released its final standards in May 2009. The standards were unanimously approved by the IAB membership, including Facebook. Id. at 28:10-11; 62:19-22; 63:5-7.
More on Facebook and the IAB.
Page 12
2. Facebook Violates the IAB Standards.
While Facebook approved the IAB standards and professes to understand their value (Exh. 13 at 252:15), it has chosen to effectively disregard them. During the class period, Facebook’s click filter processes were not in compliance with several IAB standards, including the most important one of all–the external validation requirement. Indeed, many Facebook witnesses in positions of authority with respect to click filtering either have either no idea what the IAB standards were or had little regard for them. Exh. 6 at 242:19-24 (characterizing the standards as “toxic”); Exh. 9 at 14:19-20 (“I personally didn’t read any IAB materials.”)
Facebook’s “duplicate click” filter algorithm violated this standard during the class period. By initially determining that all clicks within two seconds were considered legitimate (which Facebook later changed during the class period to thirty seconds), Facebook’s rule, even as modified, did not prevent
clicks from being billed that were the result of navigational mistakes. These clicks are “repetitive” clicks which Facebook promised would not be billed to advertisers since they are not “legitimate.” As discussed above, Facebook’s own engineers recognized their illegitimacy, advocating a click interval
forty times longer than what currently exists at Facebook to filter repetitive clicks.
This section alleges that Facebook was late to exclude bots and spiders — malware programmed to commit fraud by falsely clicking on ads multiple times — from its system. It also talks about revenue collected from double clicks on ads that occurred within 2 seconds.
Page 13
b. Facebook does not utilize industry lists of known Bots & Spiders.
The IAB has published such a list for many years but it was not until the end of 2009 that Facebook first implemented an algorithm to filter out bots and spiders. Exh. 5 at 209:8-13. Moreover, as recently as late 2010, Facebook was still classifying clicks as valid even though it was aware that the Internet community had published a bots & spiders list that recognized that the clicks from addresses on the list were illegitimate. Exh. 27 (Facebook engineer expressing alarming concern that approximately 2% of the clicks Facebook categorized as “legitimate” were actually clicks originating from bots and spiders on industry lists). These clicks are “automated” clicks that Facebook promised would not be billed to advertisers.
c. Facebook knowingly billed advertisers for clicks that violated its own rules. Facebook also has charged advertisers for clicks that even its own rules considered invalid. For example, in a July 2009 email, a Facebook data analyst confided that Facebook was charging for illegitimate clicks that violated its rule against charging for duplicate clicks. Exh. 28 (“For some reason we still collect 30K of legal rev[enue] from clicks happening within 2 seconds…”). There is no record of Facebook refunding any overcharge to its advertisers.
3. Facebook Does not Permit Third-Party Audits to Certify Filter Validity.
Another redacted section about Facebook and the IAB.
Pages 14 and 15
7.1.3. The auditing, certification and testing of the click measurement systems are primarily conducted by accounting firms such as Ernst & Young.
Facebook’s principal competitors for PPC advertising — Google,14 Microsoft and Yahoo! — follow the IAB auditing standards and submit to independent audits to certify their compliance with the IAB/MRC standards for click measurement.15 In striking contrast, Facebook has flatly refused to allow an audit of its click filter systems. Exh. 26 at 57:1-4, 19-23; Exh. 27 at 84:8-13. It is telling that no third party has ever evaluated and tested Facebook’s processes for determining “legitimate” clicks or whether its systems are in compliance with the minimum industry standards. Facebook has repeatedly rebuffed the recommendation of its accounting auditors, E&Y, that it submit to an IAB audit and certification that E&Y offered to conduct. Exh. 30 at 76:11-24, 117:1-119:24, 162:11-15. As the declaration of the E&Y partner in charge of the Facebook account confirms, although Ernst & Young audited Facebook’s financial statements during the class period, it has not independently verified the veracity of Facebook’s click filter process nor has it confirmed compliance with the IAB standards. Exh. 31 at ¶ 3.
4. Facebook Does Not Publish a Description of Methodology.
The IAB standards also recognize that transparency is essential.
“[The IAB] believe[s] that third party verification of [compliance] is important to giving buyers confidence.” Exh. 26 at 56:10-12. The MRC is in agreement. Exh. 29 at 92:13-16. “[I]f a measurer is going to say they complied with the guidelines, there should be some validation of that and communication of that.”
pdf). Moreover, full disclosure leads to better-informed advertisers.
“An audit is the examination of evidential matter to determine whether certain criteria are met.” Exh. 29 at 95:17-19. During an audit, “clicks are measured as they compare[] to IAB guidelines and [MRC] standards” to determine compliance with those standards. Id. at 69:3-8. After an audit is completed, a company provides a letter to the IAB from the auditing company that says whether or not the company audited was in compliance with the standards. Exh. 26 at 60:1-6. Once a company is determined to be in compliance, the IAB adds them to a list of certified companies that is publicly available to advertisers.
The plaintiffs allege Facebook doesn't publish a "description of methodology" (DOM) for counting clicks.
Page 15
However, unlike its counterparts, Facebook has chosen not to publish a DOM. Exh. 9
This section features an extended discussion of whether Facebook could ever pass an audit held to the standard of the IAB guidelines.
Page 16
The reason Facebook has refused to submit itself to an IAB audit is clear: it knows it would fail. Facebook’s own engineers have admitted internally that its click system is not IAB-compliant. See, e.g., Exh. 34 (principal Facebook click filter engineer noting to his colleague that implementation of a certain filter “is yet another step toward getting closer to IAB compliance...”) (emphasis added); Exh. 35 (same engineer commenting internally on another filter: “nice bullet point to point on a document in a few years when we try to prove our IAB compliance”). In fact, when a junior member of the Facebook ad team sent an email to E&Y suggesting that Facebook was interested in having E&Y do an IAB audit, that initiative was quickly quashed by a senior member of the Facebook finance team. Exh. 36; Exh. 30 at 161:1-4 (confirming that Facebook management “pulled the plug” on the IAB audit initiative). An email exchange between E&Y personnel further confirms that Facebook did not believe its systems would pass an IAB audit. In a 2010 email from Jackson Bazley to Illian Ilev, the E&Y partner in charge of IT audits, Bazley wrote: “I talked to our contacts [at Facebook] and the sense is that they will not pass such an assessment...” Exh. 36; Exh. 30 at 156-157. Facebook’s admitted failure to adhere to the IAB standards, including submitting to external validation of its processes, is evidence of the breach of its contractual promises to maintain adequate click filtering systems and charge only for legitimate clicks. As the following sections demonstrate, the breach and resulting damages can be proven on a classwide basis using common evidence — Facebook’s own meticulous data.
D. Facebook Maintains Detailed Click Data That Can Be Analyzed Retroactively.
Facebook maintains historical records of click data on a click-by-click basis for the proposed class period. Exh. 13 at 91:7-10. In effect, Facebook has snapshots of billions of clicks that can be easily accessed and analyzed electronically. The data (or snapshot) contains more than 50 fields of information all of which provide exceptional detail about every click on a Facebook ad, including the precise time the click was initiated; the source of the click; and the whether the click was ultimately billed to the advertiser. Exhs. 37-38. Furthermore, even if a click is not billed, Facebook maintains data that indicates which rule (or filter) was responsible for the determination. Exh. 5
This section details an analysis of Facebook's traffic by Adometry. It alleges a 3 percentage point discrepancy between the 12 percent of clicks that Facebook counts as invalid and the 15 percent identified by Adometry. (The allegation that as many as 20 percent of are invalid came separately, in oral arguments over class action status.)
Page 17
Because Facebook keeps such “granular” data, a third party it retained has been conducting confidential, independent analysis of the Facebook clicks using some or all of the 50 data fields. Exh. 39 at 35:7-17, 40:12-14; Exhs. 40-41. Beginning in 2009 and continuing to the present, Facebook pays several hundred thousand dollars annually to a click measurement firm, Adometry (formerly known as Click Forensics), to perform an independent analysis using Facebook’s historical click data. Exh. 42. Adometry takes a subset of historical data from Facebook, typically click logs for a particular month, and runs that data through its own proprietary algorithms to determine click-by-click whether a particular click should have been considered legitimate. Exhs. 43-44. Adometry reports the results of its analysis to Facebook which then compares internally Adometry’s analysis with its own determination of valid and invalid clicks.16
As Dr. Jakobsson describes, he would use the historical click log data that Facebook maintains and input that data into rules-based algorithms created to reflect proper filtration rules.
Adometry analyzed approximately 464,000,000 clicks on Facebook ads during the period July 9, 2010 to August 12, 2010 and found that more than 15% of Facebook’s clicks should have been categorized as invalid and thus not charged to advertisers. During the approximate same time period, Facebook categorized only 12% as invalid resulting in approximately a 20% disparity between the two analyses.
A small redaction about industry standards.
Page 18
… the clicks for which each advertiser paid that would have been discarded had Facebook been employing industry-standard algorithmic rules.
The total number of Facebook's advertisers was redacted.
Page 19
18 Price’s click logs produced in discovery show clear instances of charges for illegitimate clicks.
At the end of March 2011, Facebook reported more than 100,000 advertisers.
A small redaction that mentions click filtering.
Page 26
Facebook breached the agreement by charging Plaintiffs for illegitimate clicks and failing to maintain proper click filtering systems; and (4) Plaintiffs suffered damages as a result of the brief.
More on click filtering ...
Page 27
As previously discussed, Plaintiffs’ expert Dr. Jakobsson can design rule-based algorithms and use Facebook’s own historical data to confirm the volume of invalid clicks charged by Facebook and then assign a dollar amount paid by each class member for invalid clicks, thus demonstrating substantial injury to the Named Plaintiffs and the Class. Common evidence demonstrates that Facebook’s failure to properly filter out invalid click was the result of behind-the-scenes decisions to give more importance to advertising revenue than the integrity of its pay-per- click advertising program.
Dr. Jakobsson’s rule-based algorithms may be applied to common historical data maintained by Facebook, enabling both classwide liability and the dollar amount improperly paid by each class member to be established without individualized proof.
This redaction included the allegation that Facebook audits its own clicks biannually via Adometry, but doesn't reveal the results.
Page 28
He proposes to create algorithms to reflect the rules that Plaintiffs claim should have been implemented but were not and, as in Citysearch, to plug Facebook’s click log data into these new algorithms to ascertain whether Facebook charged for clicks that were categorized as illegitimate pursuant to the new algorithms. Exh. 4 at ¶¶ 15-18. Moreover, unlike in Citysearch, Dr. Jakobsson’s methodology has already been employed in real life conditions by Adometry, which currently uses the same historical data supplied by Facebook to perform bi-annual independent click-by-click analyses of Facebook’s clicks to determine their legitimacy. Adometry’s work, commissioned by Facebook itself, demonstrates the plausibility of Dr. Jakobsson’s approach.
Page 29
(where an advertiser complains of $150 of overcharges by Facebook).
Please follow Advertising on Twitter and Facebook.
Join the conversation about this story »
