SlideShare une entreprise Scribd logo
1  sur  43
Télécharger pour lire hors ligne
April 2019 / Page 0marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
State of Ad Fraud
Q2 2019
April 30, 2019
Augustine Fou, PhD.
acfou [at] mktsci.com
212. 203 .7239
April 2019 / Page 1marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
“I’ve been asked this a lot recently –
‘how big is ad fraud?’
I can’t say for the industry; I can
only say for the client campaigns
I’ve measured.”
April 2019 / Page 2marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Agenda
1. How big is ad fraud?
2. How much is NOT fraud (i.e. known)?
3. How marketers are solving fraud
themselves
April 2019 / Page 3marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How Big is Ad Fraud?
April 2019 / Page 4marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
“you tell me”
(look at the evidence)
April 2019 / Page 5marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
New “largest ever” botnet every year
Vast botnets targeting high-value video ads, disguising/hiding
April 2019 / Page 6marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Millions of apps on millions of phones
Big companies openly committing fraud; in-app is far less measurable
October 2018
https://www.buzzfeednews.com/
article/craigsilverman/how-a-
massive-ad-fraud-scheme-
exploited-android-phones-to
November 2018
https://www.buzzfeednews.com/
article/craigsilverman/android-
apps-cheetah-mobile-kika-
kochava-ad-fraud
March 2019
https://www.buzzfeednews.com/
article/craigsilverman/in-banner-
video-ad-fraud
April 2019
https://www.buzzfeednews.com/
article/craigsilverman/google-
play-store-ad-fraud-du-group-
baidu
April 2019 / Page 7marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Hundreds of billions of redirects
Page redirects/pop-unders evade fraud detection easily
a.k.a. “zero-click” “pop-under”
“forced-view” “auto-nav”
Source: https://www.buzzfeed.com/craigsilverman/remember-tom
April 2019 / Page 8marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
100% fraud, 99% location fraud
Big companies openly committing fraud; in-app is far less measurable
https://blog.branch.io/mobile-ad-fraud-
what-24-billion-clicks-on-700-ad-networks-
reveal/
“Top 8 networks were
100% fraud and the top
34 were all greater
than 50% fraud.”
“90 – 99% of
geolocation data is
bad or outright faked.”
April 2019 / Page 9marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
99% of ads were never served
Hidden cost: marketers pay for bids won; ads served is
less, sometimes by a lot, than what they paid for
Campaign 1
• For every bid won, there should be an ad
served – a 1:1 relationship
• Comparing DSP data vs ad server data
shows that for certain domains the
discrepancy is large
• Marketers pay on bids won; the discrepancy
is the hidden cost of ads not being served
Low CPM
Campaign 2
April 2019 / Page 10marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Billions of fake users purged
February 2018
http://www.adweek.com/digital/lotame-
purges-its-data-exchange-after-identifying-
400-million-user-profiles-as-bots/
May 2018
https://www.inc.com/bill-murphy-
jr/facebook-says-it-disabled-almost-13-
billion-fake-accounts-and-numbers-only-
get-more-insane-from-there.html
July 2018
https://www.engadget.com/2018/07/06/t
witter-reportedly-suspended-70-million-
fake-accounts-in-may-and/
April 2019 / Page 11marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Luminati[.]io Oxylabs[.]io Smartproxy[.]com
Millions of IPs to disguise bots
Use: to disguise data center bots to appear to come from
residential IP addresses, avoid detection
40 million
IP addresses
10 million
IP addresses
30 million
IP addresses
April 2019 / Page 12marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
700X more bad apps than good ones
$46 Search Display/Video $46
$32$39
$8
Google Search FB+Google Display$29
(outside Google/Facebook)
$100 Billion Digital SpendSource: IAB 2H 2018 Report
Source: Verisign, Q4 2016
329M
domains
est. 1 million est. 164 million
7M
apps
Source: Statista, March 2017
est. 10,000 est. 6.99 million
1% of
impressions
99% of
impressions
$10B $19B
Good Publishers “sites/apps with ads”
April 2019 / Page 13marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Traffic sellers openly selling
Many sources to buy “traffic,” tune “quality” level, host bots
Google
“buy real human traffic”
Select vendor and
“traffic quality level”
Host your own bots
(cost $3.99/mo)
April 2019 / Page 14marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Massive quantities sold by unknowns
Highlighted
domains and apps
are interspersed
with large sites that
you know many
humans go to.
These are DAILY
quantities of
impressions.
Notice the large
quantities; some
are larger than
mainstream sites.
April 2019 / Page 15marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Missing media dollars, fraud?
Comparing DSP logs vs invoices, unspent dollars go missing
Display Campaign 1
$3,300,000 (+10%)
(agency invoice to advertiser)
$3,000,000
DSP invoice to agency (paid by
advertiser)
$1,800,000 (-40%)
DSP log total (bids won)
On average, 24% of advertisers’ media spend was wasted/missing
Mobile Campaign 2
$950,000
(campaign spend)
$330,000 (35%)
(invalid devices/fraud)
Display Campaign 3
$7,900,000
(campaign spend)
$1,500,000 (19%)
(invalid devices/fraud)
Display Campaign 3
$3,900,000
(campaign spend)
$550,000 (14%)
(invalid devices/fraud)
Display Campaign 4
$7,000,000
(campaign spend)
$1,700,000 (24%)
(invalid devices/fraud)
Source: Method Media Intelligence
DSP logs show only $1.8 million of bids
won, compared to the $3.0 million paid;
$1.2 million went “missing” (not
returned/credited to advertiser)
April 2019 / Page 16marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Other forms of fraud, not IVT
If you’re only looking for bots/IVT, you will find just bots
1.3% + 57% = 58%
bots site/app fraud overall fraud
bot detection sees this
bot detection misses this
April 2019 / Page 17marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Fraud, waste eats most budgets
$1 spent by Marketer
blocked
see slide 1
site/app fraud
see slides 2-4
ad/user fraud
see
https://www.slideshare.net/augustinefou/presentations
not viewable
see slides 5-7
10% 30% 40% 20%
• fake sites, spoofing
• auto-reload/refresh
• ad stacking
• tag manipulation
• redirect/sourced traffic
• pop-ups/unders
• naked/invisible ads
• bots/fake users
• fake devices
• background load
• fake apps
• deviceID rotation
• random deviceID
• retargeting fraud
• fake profiles/data
• malware/adware
• fake analytics
• fake attribution
• click injection
• incentivized views
• residential proxy
• app is not in use
• tab is not active
• browser minimized
• 50% ad in viewport
• misrepresented
• (1s display, 2s
video)
10% 70% 20%
blocked fraud useless
productive digital ads for Marketer
1% ?
April 2019 / Page 18marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
10 – 90
dollars
1 - 99
impressions
unknown
unknown
April 2019 / Page 19marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Fake botnet for PR purposes
“used highly sophisticated techniques
to fraudulently load ads on the
affected sites without the site
owners' consent, leveraging a new
methodology that allows it to
monetize inventory on premium
domains.”
“none of this actually happened; it was completely
fabricated for the press release announcing their
new algo – ‘dramatic improvements to its
automated traffic detection .. primarily through
…machine learning methodologies’. The failure was
due to their analyzing only pre-bid data, which was
faked. There were no ads injected into any of the
sites they named in the press release. This was
confirmed by each of the good publishers, falsely
accused.”
April 2019 / Page 20marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Trade associations think…
Ads fraud is virtually
“non-existent”
– IAB Australia
“Ad fraud is $6.5
billion or 9% of
display ad spend”
-- Assoc. of Nat’l Advertisers
“@tag_today says it
has a ‘monumental’
breakthrough on a
method that can cut
online fraud by more
than 83%”
April 2019 / Page 21marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
“the problem with an industry-
wide extrapolation is …
you don’t know what you
don’t know.”
April 2019 / Page 22marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
“there are always new threats
and exploits – called ‘zero-
day’s in the cyber security.
Better to focus on what is
known or knowable.”
April 2019 / Page 23marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
What is NOT ad fraud?
(i.e. known)
April 2019 / Page 24marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
10 – 90
dollars
1 - 99
impressions
known
known
“$10 billion out of $100 billion of
digital spend in the U.S. is spent
with good publishers.” DCN
“1% of impressions can be
confirmed to be shown to
humans, and viewable”
April 2019 / Page 25marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Confirmed humans: in-ad, on-site
IN-AD
3% confirmed humans
(tag placed in display ads, in
ad iframe; used to determine
quality of sites and apps that
loaded them)
ON-SITE
82% confirmed humans
(embed code placed directly
on the page to determine
quality of visitors arriving on
the site)
April 2019 / Page 26marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Good publishers - magazines
Directly measured on-site, 77% confirmed humans
April 2019 / Page 27marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Good publishers – local TV
Directly measured on-site, 61% confirmed humans
April 2019 / Page 28marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Good publishers – local radio
Directly measured on-site, 48% confirmed humans
April 2019 / Page 29marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Measured viewability: 3-17%
Viewability must be measured in-ad, using intersectionObserver
Exchange A, All CPMs exclude app app only blended
hasFocus:1 4.2% 5% 0.5% 4.8% 4.7% 5%
-hasFocus:1 84.7% 95% 10.6% 95.2% 95.3% 95%
88.9% 11.1% 100.0%
visibilityState:visible 69.9% 79% 9.6% 86.1% 79.5% 80%
-visibilityState:visible 19.0% 21% 1.5% 13.9% 20.5% 20%
88.9% 11.1% 100.0%
intersection >50% 20.1% 23% 4.2% 37.4% 24.2% 24%
-intersection >50% 68.8% 77% 7.0% 62.6% 75.8% 76%
88.9% 11.1% 100.0%
viewable 0.8% 1.5% 0.9%
(excl hasFocus:1) viewable 17.8% 32.2% 19.3%
Exchange B, All CPMs exclude app app only blended
hasFocus:1 0.4% 0% 0.0% 0.0% 0.4% 0%
-hasFocus:1 95.1% 100% 4.5% 100.0% 99.7% 100%
95.5% 4.5% 100.0%
visibilityState:visible 34.3% 36% 4.2% 92.2% 38.4% 38%
-visibilityState:visible 61.2% 64% 0.4% 7.8% 61.6% 62%
95.5% 4.5% 100.0%
intersection >50% 8.0% 8% 0.5% 10.9% 8.5% 9%
-intersection >50% 87.5% 92% 4.0% 89.1% 91.5% 91%
95.5% 4.5% 100.0%
viewable 0.0% 0.0% 0.0%
(excl hasFocus:1) viewable 3.0% 10.0% 3.3%
April 2019 / Page 30marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Measured ad blocking: 10-18%
Ad blocking must be measured on-site, mobile separated, no bots
B2C (Consumer) Jan 2019 EXCLUDE BOTS
RAW (percent of data) percent
NOT Blocked Blocked
mobile 61.8% 0.4% 0.7% blocking rate
desktop 27.6% 3.1% 10.2% blocking rate
not measured 4.8%
97.8% 2.2% bots
B2B Jan 2019 EXCLUDE BOTS
RAW (percent of data) percent
NOT Blocked Blocked
mobile 34.5% 0.5% 1.3% blocking rate
desktop 39.9% 9.0% 18.5% blocking rate
not measured 7.9%
91.8% 8.2% bots
April 2019 / Page 31marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Known: -99% reach, no impact
“JPMorgan had already decided
last year to oversee its own
programmatic buying operation.
Advertisements for JPMorgan
Chase were appearing on about
400,000 websites a month. [But]
only 12,000, or 3 percent, led to
activity beyond an impression.
[Then, Chase] limited its display
ads to about 5,000 websites. We
haven’t seen any deterioration on
our performance metrics,” Ms.
Lemkau said.”
“99% reduction in ‘reach’ … Same Results.”
Source: NYTimes, March 29, 2017
(because it wasn’t real, human reach)
April 2019 / Page 32marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Known: -$200M, no impact
“Once we got transparency, it
illuminated what reality was,” said
Mr. Pritchard. P&G then took matters
into its owns hands and voted with
its dollars, he said.”
“As we all chased the Holy Grail of
digital, self-included, we were
relinquishing too much control—
blinded by shiny objects,
overwhelmed by big data, and ceding
power to algorithms,” Mr. Pritchard
said.
Source: WSJ, March 2018
April 2019 / Page 33marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Case Examples of
Marketers Solving Fraud
THEMSELVES
April 2019 / Page 34marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Small businesses found/killed fraud
“Both of these small businesses used their own analytics and gut
instinct; they resolved ad fraud without using any expensive tech.”
Small Business A
• Noticed a 118,600% increase in Android devices hitting her site
during campaign – AND no additional goal completions
• Compiled additional data that corroborated it was fraud;
presented to ad network and got refund for entire campaign
Small Business B
• Year over year, marketer noticed the discrepancy between counts
reported by ad network versus his own Google Analytics shot up
dramatically (even though cost-per-action remained similar).
• Conversions also dropped dramatically. With deeper digging, he
found the ratio of audience network inventory grew from 5% to
65% of total impressions. Solved by turning off audience network.
April 2019 / Page 35marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How a marketer beat fraud
2016
Still buying
through exchanges
Measure In-Ad
and arrivals On-Site
2017
Buy their own ads
through DSP
Took buying in-
house
2018
Started serving
their own ads
Took ad serving
in-house
2019 …
Buying direct from good publishers
April 2019 / Page 36marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Measure in-ad and on-site
In-Ad
measurement
• Blue means humans
• Red means bots
On-Site
measurement
“increase spend on sources driving more humans
(blue); reduce spend on sources with more bots (red)”
April 2019 / Page 37marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Detect and reduce fraud in-flight
Launch Week 3 onwardWeeks 1-2
Initial baseline
measurement
Measurement after
first optimization
After eliminating several
“problematic” networks
Stacked percent chart
Blue (human)
Red (bots)
April 2019 / Page 38marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Better media = better outcomes
Measure
Ads
Measure
Arrivals
Measure
Conversions
346
1743
5
156
A
B
30X more human
conversion events
• More arrivals
• Better quality
more humans (blue)
good publishers
low-cost media,
ad exchanges
April 2019 / Page 39marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Humans click and convert
IN-AD ON-SITE CLICKED
1.2% Human
CTR on ads
44% Human Clicks
to Claim Form
April 2019 / Page 40marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Display 4
2,036 humans
human conversion rate
More humans = more outcomes
Site Traffic Conversions
8,482 818
4,216 humans
5%
human conversion rate
14,539 193
225 humans
9%
human conversion rate
2,248 23
168 humans
5%
human conversion rate
1,527 9
Display 3
Display 2
Display 1
Humans
40%
April 2019 / Page 41marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
About the Author
Augustine Fou, PhD.
acfou [@] mktsci.com
212. 203 .7239
April 2019 / Page 42marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Dr. Augustine Fou – Researcher
2013
2014
Published slide decks and posts:
http://www.slideshare.net/augustinefou/presentations
https://www.linkedin.com/today/author/augustinefou
2016
2015
2017
20192018

Contenu connexe

Plus de Dr. Augustine Fou - Independent Ad Fraud Researcher

Plus de Dr. Augustine Fou - Independent Ad Fraud Researcher (20)

Entire ecosystem supporting ad fraud 2018
Entire ecosystem supporting ad fraud 2018Entire ecosystem supporting ad fraud 2018
Entire ecosystem supporting ad fraud 2018
 
Digital Media Trust Collaborative
Digital Media Trust CollaborativeDigital Media Trust Collaborative
Digital Media Trust Collaborative
 
Programmatic reach analysis 2021
Programmatic reach analysis 2021Programmatic reach analysis 2021
Programmatic reach analysis 2021
 
2021 update on ad fraud brand safety privacy
2021 update on ad fraud brand safety privacy2021 update on ad fraud brand safety privacy
2021 update on ad fraud brand safety privacy
 
Browser and OS Share Jan 2021
Browser and OS Share Jan 2021Browser and OS Share Jan 2021
Browser and OS Share Jan 2021
 
Checking abnormal referrer traffic in google analytics
Checking abnormal referrer traffic in google analyticsChecking abnormal referrer traffic in google analytics
Checking abnormal referrer traffic in google analytics
 
History and Impact of Digital Ad Fraud
History and Impact of Digital Ad FraudHistory and Impact of Digital Ad Fraud
History and Impact of Digital Ad Fraud
 
Digital Fraud Viewability Benchmarks Q4 2020
Digital Fraud Viewability Benchmarks Q4 2020Digital Fraud Viewability Benchmarks Q4 2020
Digital Fraud Viewability Benchmarks Q4 2020
 
What CFEs can do about digital ad fraud
What CFEs can do about digital ad fraudWhat CFEs can do about digital ad fraud
What CFEs can do about digital ad fraud
 
Four types of digital ad spend updated august 2020
Four types of digital ad spend updated august 2020Four types of digital ad spend updated august 2020
Four types of digital ad spend updated august 2020
 
How to Use FouAnalytics For Marketers
How to Use FouAnalytics   For MarketersHow to Use FouAnalytics   For Marketers
How to Use FouAnalytics For Marketers
 
FouAnalytics DIY site media analytics fraud detection baked in
FouAnalytics DIY site media analytics fraud detection baked inFouAnalytics DIY site media analytics fraud detection baked in
FouAnalytics DIY site media analytics fraud detection baked in
 
Investigating digital ad fraud spi virtual meeting
Investigating digital ad fraud   spi virtual meetingInvestigating digital ad fraud   spi virtual meeting
Investigating digital ad fraud spi virtual meeting
 
Fraud by Browser Study
Fraud by Browser StudyFraud by Browser Study
Fraud by Browser Study
 
Digital Ad Fraud FAQ Question 1
Digital Ad Fraud FAQ Question 1Digital Ad Fraud FAQ Question 1
Digital Ad Fraud FAQ Question 1
 
Digital ad dollars trickle down chart
Digital ad dollars trickle down chartDigital ad dollars trickle down chart
Digital ad dollars trickle down chart
 
Botnets used for ad fraud spam ddos attacks
Botnets used for ad fraud spam ddos attacksBotnets used for ad fraud spam ddos attacks
Botnets used for ad fraud spam ddos attacks
 
Marketer Outcomes Study
Marketer Outcomes StudyMarketer Outcomes Study
Marketer Outcomes Study
 
Ad fraud update for publishers Feb 2020
Ad fraud update for publishers Feb 2020Ad fraud update for publishers Feb 2020
Ad fraud update for publishers Feb 2020
 
Ad fraud is cash out for hacking
Ad fraud is cash out for hackingAd fraud is cash out for hacking
Ad fraud is cash out for hacking
 

Dernier

What’s the difference between Affiliate Marketing and Brand Partnerships?
What’s the difference between Affiliate Marketing and Brand Partnerships?What’s the difference between Affiliate Marketing and Brand Partnerships?
What’s the difference between Affiliate Marketing and Brand Partnerships?Partnercademy
 
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDigital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDemandbase
 
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024CIO Business World
 
Fiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFarrel Brest
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)DEVARAJV16
 
Influencer Marketing Power point presentation
Influencer Marketing  Power point presentationInfluencer Marketing  Power point presentation
Influencer Marketing Power point presentationdgtivemarketingagenc
 
Codes and Conventions of Film Magazine Websites.pptx
Codes and Conventions of Film Magazine Websites.pptxCodes and Conventions of Film Magazine Websites.pptx
Codes and Conventions of Film Magazine Websites.pptxGeorgeCulica
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...CIO Business World
 
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorTAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorSocial Samosa
 
Master the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfMaster the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfHigher Education Marketing
 
Infographics about SEO strategies and uses
Infographics about SEO strategies and usesInfographics about SEO strategies and uses
Infographics about SEO strategies and usesbhavanirupeshmoksha
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfVWO
 
From Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOFrom Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOSzymon Słowik
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project VideoSineadBidwell
 
Exploring The World Of Adult Ad Networks.pdf
Exploring The World Of Adult Ad Networks.pdfExploring The World Of Adult Ad Networks.pdf
Exploring The World Of Adult Ad Networks.pdfadult marketing
 
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...CIO Business World
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...Ahrefs
 
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdf
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdfDIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdf
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdfmayanksharma0441
 
Digital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigiKarishma
 
Prezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsPrezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsCristian Manafu
 

Dernier (20)

What’s the difference between Affiliate Marketing and Brand Partnerships?
What’s the difference between Affiliate Marketing and Brand Partnerships?What’s the difference between Affiliate Marketing and Brand Partnerships?
What’s the difference between Affiliate Marketing and Brand Partnerships?
 
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDigital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
 
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024
The 10 Most Inspirational Leaders LEADING THE WAY TO SUCCESS, 2024
 
Fiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview Assignment
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)
 
Influencer Marketing Power point presentation
Influencer Marketing  Power point presentationInfluencer Marketing  Power point presentation
Influencer Marketing Power point presentation
 
Codes and Conventions of Film Magazine Websites.pptx
Codes and Conventions of Film Magazine Websites.pptxCodes and Conventions of Film Magazine Websites.pptx
Codes and Conventions of Film Magazine Websites.pptx
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
 
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorTAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
 
Master the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfMaster the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdf
 
Infographics about SEO strategies and uses
Infographics about SEO strategies and usesInfographics about SEO strategies and uses
Infographics about SEO strategies and uses
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
 
From Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOFrom Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEO
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project Video
 
Exploring The World Of Adult Ad Networks.pdf
Exploring The World Of Adult Ad Networks.pdfExploring The World Of Adult Ad Networks.pdf
Exploring The World Of Adult Ad Networks.pdf
 
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...
The 10 Most Influential CMO's Leading the Way of Success, 2024 (Final file) (...
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
 
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdf
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdfDIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdf
DIGITAL MARKETING STRATEGY_INFOGRAPHIC IMAGE.pdf
 
Digital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G Age
 
Prezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsPrezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media Trends
 

State of NOT Ad Fraud Q2 2019

  • 1. April 2019 / Page 0marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou State of Ad Fraud Q2 2019 April 30, 2019 Augustine Fou, PhD. acfou [at] mktsci.com 212. 203 .7239
  • 2. April 2019 / Page 1marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou “I’ve been asked this a lot recently – ‘how big is ad fraud?’ I can’t say for the industry; I can only say for the client campaigns I’ve measured.”
  • 3. April 2019 / Page 2marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Agenda 1. How big is ad fraud? 2. How much is NOT fraud (i.e. known)? 3. How marketers are solving fraud themselves
  • 4. April 2019 / Page 3marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou How Big is Ad Fraud?
  • 5. April 2019 / Page 4marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou “you tell me” (look at the evidence)
  • 6. April 2019 / Page 5marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou New “largest ever” botnet every year Vast botnets targeting high-value video ads, disguising/hiding
  • 7. April 2019 / Page 6marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Millions of apps on millions of phones Big companies openly committing fraud; in-app is far less measurable October 2018 https://www.buzzfeednews.com/ article/craigsilverman/how-a- massive-ad-fraud-scheme- exploited-android-phones-to November 2018 https://www.buzzfeednews.com/ article/craigsilverman/android- apps-cheetah-mobile-kika- kochava-ad-fraud March 2019 https://www.buzzfeednews.com/ article/craigsilverman/in-banner- video-ad-fraud April 2019 https://www.buzzfeednews.com/ article/craigsilverman/google- play-store-ad-fraud-du-group- baidu
  • 8. April 2019 / Page 7marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Hundreds of billions of redirects Page redirects/pop-unders evade fraud detection easily a.k.a. “zero-click” “pop-under” “forced-view” “auto-nav” Source: https://www.buzzfeed.com/craigsilverman/remember-tom
  • 9. April 2019 / Page 8marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou 100% fraud, 99% location fraud Big companies openly committing fraud; in-app is far less measurable https://blog.branch.io/mobile-ad-fraud- what-24-billion-clicks-on-700-ad-networks- reveal/ “Top 8 networks were 100% fraud and the top 34 were all greater than 50% fraud.” “90 – 99% of geolocation data is bad or outright faked.”
  • 10. April 2019 / Page 9marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou 99% of ads were never served Hidden cost: marketers pay for bids won; ads served is less, sometimes by a lot, than what they paid for Campaign 1 • For every bid won, there should be an ad served – a 1:1 relationship • Comparing DSP data vs ad server data shows that for certain domains the discrepancy is large • Marketers pay on bids won; the discrepancy is the hidden cost of ads not being served Low CPM Campaign 2
  • 11. April 2019 / Page 10marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Billions of fake users purged February 2018 http://www.adweek.com/digital/lotame- purges-its-data-exchange-after-identifying- 400-million-user-profiles-as-bots/ May 2018 https://www.inc.com/bill-murphy- jr/facebook-says-it-disabled-almost-13- billion-fake-accounts-and-numbers-only- get-more-insane-from-there.html July 2018 https://www.engadget.com/2018/07/06/t witter-reportedly-suspended-70-million- fake-accounts-in-may-and/
  • 12. April 2019 / Page 11marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Luminati[.]io Oxylabs[.]io Smartproxy[.]com Millions of IPs to disguise bots Use: to disguise data center bots to appear to come from residential IP addresses, avoid detection 40 million IP addresses 10 million IP addresses 30 million IP addresses
  • 13. April 2019 / Page 12marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou 700X more bad apps than good ones $46 Search Display/Video $46 $32$39 $8 Google Search FB+Google Display$29 (outside Google/Facebook) $100 Billion Digital SpendSource: IAB 2H 2018 Report Source: Verisign, Q4 2016 329M domains est. 1 million est. 164 million 7M apps Source: Statista, March 2017 est. 10,000 est. 6.99 million 1% of impressions 99% of impressions $10B $19B Good Publishers “sites/apps with ads”
  • 14. April 2019 / Page 13marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Traffic sellers openly selling Many sources to buy “traffic,” tune “quality” level, host bots Google “buy real human traffic” Select vendor and “traffic quality level” Host your own bots (cost $3.99/mo)
  • 15. April 2019 / Page 14marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Massive quantities sold by unknowns Highlighted domains and apps are interspersed with large sites that you know many humans go to. These are DAILY quantities of impressions. Notice the large quantities; some are larger than mainstream sites.
  • 16. April 2019 / Page 15marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Missing media dollars, fraud? Comparing DSP logs vs invoices, unspent dollars go missing Display Campaign 1 $3,300,000 (+10%) (agency invoice to advertiser) $3,000,000 DSP invoice to agency (paid by advertiser) $1,800,000 (-40%) DSP log total (bids won) On average, 24% of advertisers’ media spend was wasted/missing Mobile Campaign 2 $950,000 (campaign spend) $330,000 (35%) (invalid devices/fraud) Display Campaign 3 $7,900,000 (campaign spend) $1,500,000 (19%) (invalid devices/fraud) Display Campaign 3 $3,900,000 (campaign spend) $550,000 (14%) (invalid devices/fraud) Display Campaign 4 $7,000,000 (campaign spend) $1,700,000 (24%) (invalid devices/fraud) Source: Method Media Intelligence DSP logs show only $1.8 million of bids won, compared to the $3.0 million paid; $1.2 million went “missing” (not returned/credited to advertiser)
  • 17. April 2019 / Page 16marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Other forms of fraud, not IVT If you’re only looking for bots/IVT, you will find just bots 1.3% + 57% = 58% bots site/app fraud overall fraud bot detection sees this bot detection misses this
  • 18. April 2019 / Page 17marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Fraud, waste eats most budgets $1 spent by Marketer blocked see slide 1 site/app fraud see slides 2-4 ad/user fraud see https://www.slideshare.net/augustinefou/presentations not viewable see slides 5-7 10% 30% 40% 20% • fake sites, spoofing • auto-reload/refresh • ad stacking • tag manipulation • redirect/sourced traffic • pop-ups/unders • naked/invisible ads • bots/fake users • fake devices • background load • fake apps • deviceID rotation • random deviceID • retargeting fraud • fake profiles/data • malware/adware • fake analytics • fake attribution • click injection • incentivized views • residential proxy • app is not in use • tab is not active • browser minimized • 50% ad in viewport • misrepresented • (1s display, 2s video) 10% 70% 20% blocked fraud useless productive digital ads for Marketer 1% ?
  • 19. April 2019 / Page 18marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou 10 – 90 dollars 1 - 99 impressions unknown unknown
  • 20. April 2019 / Page 19marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Fake botnet for PR purposes “used highly sophisticated techniques to fraudulently load ads on the affected sites without the site owners' consent, leveraging a new methodology that allows it to monetize inventory on premium domains.” “none of this actually happened; it was completely fabricated for the press release announcing their new algo – ‘dramatic improvements to its automated traffic detection .. primarily through …machine learning methodologies’. The failure was due to their analyzing only pre-bid data, which was faked. There were no ads injected into any of the sites they named in the press release. This was confirmed by each of the good publishers, falsely accused.”
  • 21. April 2019 / Page 20marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Trade associations think… Ads fraud is virtually “non-existent” – IAB Australia “Ad fraud is $6.5 billion or 9% of display ad spend” -- Assoc. of Nat’l Advertisers “@tag_today says it has a ‘monumental’ breakthrough on a method that can cut online fraud by more than 83%”
  • 22. April 2019 / Page 21marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou “the problem with an industry- wide extrapolation is … you don’t know what you don’t know.”
  • 23. April 2019 / Page 22marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou “there are always new threats and exploits – called ‘zero- day’s in the cyber security. Better to focus on what is known or knowable.”
  • 24. April 2019 / Page 23marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou What is NOT ad fraud? (i.e. known)
  • 25. April 2019 / Page 24marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou 10 – 90 dollars 1 - 99 impressions known known “$10 billion out of $100 billion of digital spend in the U.S. is spent with good publishers.” DCN “1% of impressions can be confirmed to be shown to humans, and viewable”
  • 26. April 2019 / Page 25marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Confirmed humans: in-ad, on-site IN-AD 3% confirmed humans (tag placed in display ads, in ad iframe; used to determine quality of sites and apps that loaded them) ON-SITE 82% confirmed humans (embed code placed directly on the page to determine quality of visitors arriving on the site)
  • 27. April 2019 / Page 26marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Good publishers - magazines Directly measured on-site, 77% confirmed humans
  • 28. April 2019 / Page 27marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Good publishers – local TV Directly measured on-site, 61% confirmed humans
  • 29. April 2019 / Page 28marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Good publishers – local radio Directly measured on-site, 48% confirmed humans
  • 30. April 2019 / Page 29marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Measured viewability: 3-17% Viewability must be measured in-ad, using intersectionObserver Exchange A, All CPMs exclude app app only blended hasFocus:1 4.2% 5% 0.5% 4.8% 4.7% 5% -hasFocus:1 84.7% 95% 10.6% 95.2% 95.3% 95% 88.9% 11.1% 100.0% visibilityState:visible 69.9% 79% 9.6% 86.1% 79.5% 80% -visibilityState:visible 19.0% 21% 1.5% 13.9% 20.5% 20% 88.9% 11.1% 100.0% intersection >50% 20.1% 23% 4.2% 37.4% 24.2% 24% -intersection >50% 68.8% 77% 7.0% 62.6% 75.8% 76% 88.9% 11.1% 100.0% viewable 0.8% 1.5% 0.9% (excl hasFocus:1) viewable 17.8% 32.2% 19.3% Exchange B, All CPMs exclude app app only blended hasFocus:1 0.4% 0% 0.0% 0.0% 0.4% 0% -hasFocus:1 95.1% 100% 4.5% 100.0% 99.7% 100% 95.5% 4.5% 100.0% visibilityState:visible 34.3% 36% 4.2% 92.2% 38.4% 38% -visibilityState:visible 61.2% 64% 0.4% 7.8% 61.6% 62% 95.5% 4.5% 100.0% intersection >50% 8.0% 8% 0.5% 10.9% 8.5% 9% -intersection >50% 87.5% 92% 4.0% 89.1% 91.5% 91% 95.5% 4.5% 100.0% viewable 0.0% 0.0% 0.0% (excl hasFocus:1) viewable 3.0% 10.0% 3.3%
  • 31. April 2019 / Page 30marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Measured ad blocking: 10-18% Ad blocking must be measured on-site, mobile separated, no bots B2C (Consumer) Jan 2019 EXCLUDE BOTS RAW (percent of data) percent NOT Blocked Blocked mobile 61.8% 0.4% 0.7% blocking rate desktop 27.6% 3.1% 10.2% blocking rate not measured 4.8% 97.8% 2.2% bots B2B Jan 2019 EXCLUDE BOTS RAW (percent of data) percent NOT Blocked Blocked mobile 34.5% 0.5% 1.3% blocking rate desktop 39.9% 9.0% 18.5% blocking rate not measured 7.9% 91.8% 8.2% bots
  • 32. April 2019 / Page 31marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Known: -99% reach, no impact “JPMorgan had already decided last year to oversee its own programmatic buying operation. Advertisements for JPMorgan Chase were appearing on about 400,000 websites a month. [But] only 12,000, or 3 percent, led to activity beyond an impression. [Then, Chase] limited its display ads to about 5,000 websites. We haven’t seen any deterioration on our performance metrics,” Ms. Lemkau said.” “99% reduction in ‘reach’ … Same Results.” Source: NYTimes, March 29, 2017 (because it wasn’t real, human reach)
  • 33. April 2019 / Page 32marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Known: -$200M, no impact “Once we got transparency, it illuminated what reality was,” said Mr. Pritchard. P&G then took matters into its owns hands and voted with its dollars, he said.” “As we all chased the Holy Grail of digital, self-included, we were relinquishing too much control— blinded by shiny objects, overwhelmed by big data, and ceding power to algorithms,” Mr. Pritchard said. Source: WSJ, March 2018
  • 34. April 2019 / Page 33marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Case Examples of Marketers Solving Fraud THEMSELVES
  • 35. April 2019 / Page 34marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Small businesses found/killed fraud “Both of these small businesses used their own analytics and gut instinct; they resolved ad fraud without using any expensive tech.” Small Business A • Noticed a 118,600% increase in Android devices hitting her site during campaign – AND no additional goal completions • Compiled additional data that corroborated it was fraud; presented to ad network and got refund for entire campaign Small Business B • Year over year, marketer noticed the discrepancy between counts reported by ad network versus his own Google Analytics shot up dramatically (even though cost-per-action remained similar). • Conversions also dropped dramatically. With deeper digging, he found the ratio of audience network inventory grew from 5% to 65% of total impressions. Solved by turning off audience network.
  • 36. April 2019 / Page 35marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou How a marketer beat fraud 2016 Still buying through exchanges Measure In-Ad and arrivals On-Site 2017 Buy their own ads through DSP Took buying in- house 2018 Started serving their own ads Took ad serving in-house 2019 … Buying direct from good publishers
  • 37. April 2019 / Page 36marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Measure in-ad and on-site In-Ad measurement • Blue means humans • Red means bots On-Site measurement “increase spend on sources driving more humans (blue); reduce spend on sources with more bots (red)”
  • 38. April 2019 / Page 37marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Detect and reduce fraud in-flight Launch Week 3 onwardWeeks 1-2 Initial baseline measurement Measurement after first optimization After eliminating several “problematic” networks Stacked percent chart Blue (human) Red (bots)
  • 39. April 2019 / Page 38marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Better media = better outcomes Measure Ads Measure Arrivals Measure Conversions 346 1743 5 156 A B 30X more human conversion events • More arrivals • Better quality more humans (blue) good publishers low-cost media, ad exchanges
  • 40. April 2019 / Page 39marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Humans click and convert IN-AD ON-SITE CLICKED 1.2% Human CTR on ads 44% Human Clicks to Claim Form
  • 41. April 2019 / Page 40marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Display 4 2,036 humans human conversion rate More humans = more outcomes Site Traffic Conversions 8,482 818 4,216 humans 5% human conversion rate 14,539 193 225 humans 9% human conversion rate 2,248 23 168 humans 5% human conversion rate 1,527 9 Display 3 Display 2 Display 1 Humans 40%
  • 42. April 2019 / Page 41marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou About the Author Augustine Fou, PhD. acfou [@] mktsci.com 212. 203 .7239
  • 43. April 2019 / Page 42marketing.scienceconsulting group, inc. linkedin.com/in/augustinefou Dr. Augustine Fou – Researcher 2013 2014 Published slide decks and posts: http://www.slideshare.net/augustinefou/presentations https://www.linkedin.com/today/author/augustinefou 2016 2015 2017 20192018