Despite growing awareness and industry initiatives to curb rampant online ad fraud, it is still a huge problem that most advertisers and their media buying agencies have not solved.
1. Fighting Rampant
Online Ad Fraud
Dr. Augustine Fou
http://linkd.in/augustinefou
October 2013
-1-
Augustine Fou
2. Ad Waste from Fraud
Display Ad Fraud
Video Ad Fraud
480 billion
20.1 billion
display ad impressions /mo
video ad impressions /mo
29%
40%
Source: Solve Media 2013
Source: Vindico, 2013
confirmed bot traffic
$1 - $3.50
~$10
cost per thousand
cost per thousand
$2-6 billion
$1 billion
wasted ad spend (annualized)
-2-
estimated fake views
wasted ad spend (annualized)
Augustine Fou
3. $9.5 Billion Ad Waste in 2013
Source: Solve Media August 2013
-3-
Augustine Fou
4. Industry Players
Nielsen/IAB
IntegralAds (AdSafe)
DoubleVerify
Industry working group to
define ad viewability.
Verify ad placement against
blacklist of known fraudulent
websites.
Ad placement, behavioral
compliance, fraud
detection
Viewable rates (of display ads)
ranged from 14% to 79%.
PageFair
Viewable rates (of display
ads) ranged from 14% to
79%.
Ad blocking detection
Spider.io
Solve Media
Algo detection of bot-like
activity and other malware.
Using CAPTCHAs to detect
humans versus bots.
Botnet Costing Display
Advertisers $6 Million per
month. Feb 2013
Global Bot Traffic on Pace to
Waste Up to $9.5 billion in 2013
Ad Budgets. Sep 2013
-4-
WhiteOps
Realtime bot detection
algorithms
Augustine Fou
5. Fake Mobile Ad Traffic
While “mobile” is more likely to be human, there is no guarantee!
-5-
Augustine Fou
6. Traffic Firehose On/Off
Source: Alexa
Legit human traffic does not change rapidly; but bot traffic
(firehose) can be rapidly turned off and directed to other sites.
-6-
Augustine Fou
7. Detecting Fraudulent Sites
• Rapid, large, unexplained changes in traffic
• Increase in traffic without corresponding increase in search volume (how
human users find content sites); under-indexing traffic % from search
• Large discrepancy between comScore traffic numbers and
Quantcast/Alexa counts (hard to fake actual installed toolbars)
-7-
Augustine Fou
8. Fraud Fighting Techniques
• Blacklisting fake websites by URL/domain
• Whitelisting legit publisher sites by domain
• Detecting bot traffic and known botnet IP addresses
• Scanning sites for malware (Google hosts list)
• Bot flag on bid records
• Real time rejection of fraudulent ad impressions
• Revised traffic numbers based on audience overlap
-8-
Augustine Fou
9. Defining Viewability
Source: Spider.io May 2, 2013
“Even if industry standards define viewability, bad guys
can still display dozens of „viewable‟ ads in hidden
iframes or with pixel opacity set to zero (invisible).”
-9-
Augustine Fou
10. Blacklisting Sites
“For every site that is added to a black list, bad guys
can put up many more to continue to commit ad fraud
(because they don‟t have to play by the rules).”
analyzecanceradvice.com
- 10 -
analyzecancerhelp.com
bestcanceropinion.com
Augustine Fou
11. Bot Detection
Source: Spider.io March 2013
“It was commonly assumed that mouse movement and
scrolling were human activities; now sophisticated bots
can also fake mouse movement and page scrolls.”
- 11 -
Augustine Fou
12. CAPTCHAs
• 43% deemed
“suspicious”
• 29% “confirmed
bot traffic”
“Startup called Vicarious
automatically solves
CAPTCHAs.” Oct 2013
http://bit.ly/1bFo9lZ
Source: Solve Media via Marketing Charts April 26, 2013
“It was commonly assumed that solving CAPTCHAs
could only be done by humans; not any more.”
- 12 -
Augustine Fou
13. Assumptions No Longer Valid
1.
bots can’t fake mouse movements and webpage scrolling - they can easily now
2.
captchas can only be solved by humans - bots can solve them too now
3.
it requires malware infected computers to commit ad fraud - bad guys can set up
hundreds of thousands of server instances to simulate users without having to infect
any computers with malware
4.
malware can be caught by virus software when installed - some malware does not
need to be installed, they are carried along with the code of a
toolbar, plugin, extension, etc. or can be asynchronously introduced later via updates
(especially when user has permitted auto-updates)
5.
if a correct bid record is passed it should be a human user - bots can easily send fake
information to simulate being a user (e.g. cookies, referrer, search
query, characteristics of the computer and browser, etc.)
6.
fraudulent traffic is a small portion of legitimate human traffic -- bot traffic is far
larger than actual human traffic. This comes from both “legitimate” sources like
Google crawlers or “site up status checkers” and “illicit” sources like server-side
scripts, browser side scripts, browser extensions and plugins, and other malware.
- 13 -
Augustine Fou
14. Related Articles
Bad Guys Happily Rob Display Advertisers
By: Augustine Fou, July 23, 2012
Everything Fake (Display Ad Fraud, Search Click Fraud)
By: Augustine Fou, April 2013
Blacklisting vs Whitelisting
By: Augustine Fou, October 2013
Fake Profiles on Facebook
By: Augustine Fou, July 2013
How Display Ad Fraud Works
By: Augustine Fou, May 2013
- 14 -
Augustine Fou
15. Dr. Augustine Fou – Digital Consigliere
“I advise clients on optimizing
advertising across all channels. Using
insights and fast-feedback loops from
digital, we can not only target brand ads
better, but we can shift towards more
detailed measurement and ROI.”
FORMER CHIEF DIGITAL OFFICER, HCG (OMNICOM)
MCKINSEY CONSULTANT
CLIENT SIDE / AGENCY SIDE EXPERIENCE
PROFESSOR AND COLUMNIST
ENTREPRENEUR / SMALL BUSINESS OWNER
PHD MATERIALS SCIENCE (MIT '95) AT AGE 23
ClickZ Articles: http://bit.ly/augustine-fou-clickz
Slideshares: http://bit.ly/augustine-fou-slideshares
LinkedIn: http://linkd.in/augustinefou
- 15 -
@acfou
Augustine Fou