1. The Ethics Of A/B Testing

2. What is A/B Testing
 Also named randomized controlled experiments, online controlled
experiments, and split testing.
 In marketing, A/B testing is a simple randomized experiment with two
variants, A and B.
 It sees if different versions of website with little changes make us use
more, visit more, click more or buy more.
1

3. Example
 This example for a website that make two version with little changes
2

4. Facebook feeling A/B test
 PNAS magazine published results of an experiment on 689,003 users to
see if showing them more positive or negative sentiment posts in the
News Feed would affect their happiness levels as deduced by what they
posted.
3

5. The Experiment Had Almost No Effect
 Lets take a look on this experiment.
 Experiment duration : 1 Week
From January 11–18, 2012
 People shown fewer positive posts (a more
depressing feed) posted 0.1% fewer positive
words in their posts.
 People shown fewer negative posts (a happier
feed) post 0.07% fewer negative words.
4

6. Experiment risk and fails
 the National Institute Of Mental Health says 9.5% of Americans have mood
disorders, which can often lead to depression.
 Some people who are at risk of depression were almost surely part of
Facebook’s study group.
 Status update isn’t the real emotion, may this people just following the
trends they see on Facebook.
 Another research says that some people feel sad when they see lots of
happy status, because they feels that them life isn’t happy as they see on
Facebook.
6

7. Facebook Didn’t Get Ethics Board
Approval
 Facebook only did an internal review to decide if the study was ethical.
 the study would likely fail to meet many of the pre-requisites.
 But actually Facebook manipulates the News Feed all the time to test what
types of stories and designs makes you spend time, money like ,comment
or generally more active.
 Facebook technically has consent from all users, as its Data Use Policy
people automatically agree to when they sign up says
“we may use the information we receive about you…for…data
analysis, testing, research and service improvement.”
5

8. Everyone Is A/B Testing
 there are lots of A/B Tests but most are pushing for more business-oriented
results like increasing usage or clicks or purchases.
 A lot of times we don’t use the “real” Facebook. Or Twitter. Or Google, Yahoo,
or LinkedIn. We are almost all part of experiments they quietly.
7

9. Regulation, Or At Least Safeguards
 Offering an opt out Not for every test, but ones with some real risk like
showing users a more depressing feed.
8

10. Regulation, Or At Least Safeguards
(cont.)
 Independent protection against harmful emotional manipulation on a
grand scale.
 At the very least, the tech companies should educate their data scientists
and others designing A/B tests about the ethical research methods
associated with having experiments approved by the IRB
9

11. Till this happen
 Till this regulations happen, all of us are Laboratory rats
10

12. Thank you :D
11