Something Wicked: Defensible Social Architecture in the context of Big Data, Behavioral Econ, Bot Hives, and Bad Actors. BSides Las Vegas 2017 keynote presentation from Allison Miller (@selenakyle)

1. Something
Wicked
Defensible Social Architecture in the context of Big Data,
Behavioral Econ, Bot Hives, and Bad Actors
allison miller
@selenakyle

2. a world of firewalls & moats

5. soylent perimeter is people

6. scammers
phishing
malware
@selenakyle

7. troll thunderdrome

9. from behavior to tech to behavior

10. decision science
+
applied economics

11. behavior
data
analytics
economics
preferences
incentives

12. system theory
+
machine learning
=
smarter social systems

13. data rules everything around me

14. modeling + feedback
driving risk to a chokepoint
decisions have cost
quantified performance

15. models + feedback

16. UX
Platform
Back
Office
Event
Post-decision
UX
Post-Txn
Experience
Models*
Decision
Strategy
Score / Policy
Verdict
Response Post-event
actions
Post-event data
Data aggregates (txn
& acct records)
Model training,
testing, & builds

17. * speaking ground truth to power

18. * speaking ground truth to power

19. * speaking ground truth to power

20. decisions & cost

21. Authorize Block
Good
false
positive
Bad
false
negative
Good
Action Gets
Blocked
Bad Action
Gets Through
Downstream
Impacts

22. quantified performance

23. %BadisTrue
% Total

24. Gain
%BadisTrue
% Total

25. Gain
%BadisTrue
% Total

26. modeling + feedback
driving risk to a chokepoint
decisions have cost
quantified performance

27. preferential treatments

28. no, YOU’RE irrational

29. behavioral...
...finance
...economics
...game theory

30. choice architecture
opinionated design
data devaluation
...competition

31. framing + anchoring

32. opinionated design
are you sure?

33. opinionated design
let’s not.

34. data devaluation

35. choice architecture
opinionated design
data devaluation
...other system agents

36. dismal scienceing

37. Microeconomics
• Model for estimating consumption given individual preferences,
under a budget constraint
• Utility maximization
• Preferences: Consumption mix
• Good A vs Good B
• Labor vs leisure
• Budget constraint

38. Positive Normative
What it
is
What it
should be
Descriptions Recommendations

39. Themes of Security Economics
• Security ROI
• Cybercrime supply chains
• Market for Lemons
• Make it more expensive for the attacker
• Tragedy of the Commons
• Risk Tolerance
• Exploit/Vuln markets
• Behavioral Economics / Gamification

40. Wicked Games
Preferences
Utility
Money
Returns
Competition
Tolerances
Uncertainty
Data
Returns
Adversaries
Policy Analysis
Graph Theory
Dynamic Threat Models
Cyberinsurance
Security Econometrics
Classification
Inferior Goods
Security “CPI”
Incentive Design
Coalitional Game Theory
@selenakyle

41. Toolkit to Consider
Security ROI  Tradeoff distributions
Security Poverty Line  Inferior Goods
Information Asymmetries  Signaling
Repeated Games  Coalitional Game Theory
Risk management  Going for V(X) vs E(X)
@selenakyle

42. coalitional game theory
consumption & maturity
signal development
omg risk

43. (you’ve just been risk-rolled btw)
Concept where theory meets behavior
• Expected value vs expected variance
• Probability gives you both, we tend to focus on E(x)
• Risk aversion is a condition that relies on V(x)

44. Payoffs
UP
DOWN
CIRCLE
RED
BLUE
MARIO
LUIGI
KIRBY
GIZMO
10, 3
2, 10
2, 5
-3, 3
A
B
B
A
A
A

45. An example
You have $20k, but a 50/50 chance of losing $10k
• Expected value?
• $15k (i.e. .5($20k)+.5($10k))
Insurance costing $5k will cover full loss. Should you buy it or not?
• E(X) w/insurance?  $15k (for sure)
• E(X) w/o insurance  $15k (but as EITHER $10k or $20k)
A risk averse individual will opt for the same E(X) w/less uncertainty (less risk)
• People seek utility maximization, not payoffs
• Risk, i.e. uncertainty, reduces overall utility (wealth)

46. An example…continued
You have $20k, but a 50/50 chance of losing $10k
• E(X)= $15k
You are offered partial insurance costing $2.5k will cover half of the loss ($5k).
@ No Loss: $17.5k ($20k – 2.5k)
@ Loss: $12.5k ($20k – 2.5k – 10k – 5k)
• Expected value = $15k (but as EITHER $17.5k or $12.5k)
Risk, i.e. uncertainty, is reduced but there is still a $5k variance

47. #BOOMTIME #RISK @SELENAKYLE
What this Looks like
Utility
Wealth
E(V)
U(total)
U(partial)
U(no insurance)
12.5 17.515

48. The language of risk
• Some optimization functions assume *certainty*
• But making decisions under uncertainty is core to:
• Competition
• Investment
• Reality
• How are we talking about risk? Focus on E(X) or
V(X)?

49. How to Win at Risk
Win or lose?
• Game theory approach: maximize payoff
…Tends to gravitate towards expected value
• The “defender’s dilemma” assumes a risk intolerant
system manager
…Lower expected loss.
• Optimal investments manage to value and variance
…Build systems with better risk capacity
…Portfolio theory, not just point performance

50. <regroup>

51. humans: how do they work?

52. humans: how do they work?
are you sure?

53. by the pricking of my thumbs,
something wicked this way comes.
open, locks,
whoever knocks.

54. Thank You BSidesLV!
Allison Miller
@selenakyle

55. .
Some references (mostly about behavior)
• Improving SSL Warnings: Comprehension and Adherence. A. Felt, A. Ainslie, R. Reeder, S.
Consolvo, S. Thyagaraja, A. Bettes, H. Harris, and J. Grimes. CHI, page 2893-2902. ACM,
(2015) https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43265.pdf
• D. Akhawe and A. P. Felt. Alice in warningland: A large-scale field study of browser security warning
effectiveness. In Proc. of USENIX Security, pages 257–272, 2013.
• Ariely, Dan. Predictably Irrational: The Hidden Forces That Shape Our Decisions. New York, NY:
HarperCollins, 2008.
• Arthur, W Brian. All Systems will be Gamed: Exploitive Behavior in Economic and Social
Systems. Santa Fe Institute: SFI WORKING PAPER: 2014-06-
016. http://tuvalu.santafe.edu/~wbarthur/Papers/All%20Systems%20Gamed.pdf To appear in
Complexity and the Economy, W. B Arthur, Oxford University Press, 2014
• Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.
• Leyton-Brown, Kevin, and Yoav Shoham. Essentials of Game Theory: A Concise, Multidisciplinary
Introduction. [San Rafael, Calif.]: Morgan & Claypool, 2008.
• Meadows, Donella. Thinking in Systems: A Primer. London: Chelsea Green Publishing, 2008.

56. .
More references (mostly about decisions & game theory)
• Axelrod, Robert M. The Evolution of Cooperation. New York: Basic, 1984.
• Dixit, Avinash and Nalebuff, Barry. The Art of Strategy: A Game Theorist’s Guide to Success in
Business and in Life.
• Dormehl, Luke. Thinking Machines. Tarcher Perigee. New York, 2017.
• Fisher, Len. Rock, Paper, Scissors: Game Theory in Everyday Life. New York: Basic, 2008.
• Gibbons, Robert. Game Theory for Applied Economists. Princeton, NJ: Princeton UP, 1992.
• Gintis, Herbert. Game Theory Evolving: A Problem-centered Introduction to Modeling Strategic
Behavior. Princeton, NJ: Princeton UP, 2000.
• Ignacio Palacios-Heurta (2003) “Professionals Play Minimax”. Review of Economic Studies, Volume
70, pp 395-415.
• Jackson, Leyton-Brown & Shoham. Game Theory. (Stanford University and University of British
Columbia: Coursera), http://www.coursera.org, Accessed 2013.
• Polak, Ben. Game Theory (Yale University: Open Yale Courses), http://oyc.yale.edu, Accessed
2012. License: Creative Commons BY-NC-SA Thomas, L. C. Games, Theory, and Applications.
Chichester: E. Horwood, 1984.
• Wikipedia’s sections on Game Theory, Economics, & Probability.

57. .
Even more references (mostly about security
economics)• Anderson, Ross. "Risk and Privacy Implications of Consumer Payment Innovation." PDF available at
http://www.cl.cam.ac.uk/~rja14/Papers/anderson-frb-kansas-mar27.pdf Paper presented at "Consumer
Payment Innovation in the Connected Age" conference took place March 29-30, 2012, in Kansas City,
Mo.
• Anderson, Ross, Chris Barton, Rainer Bohme, Richard Clayton, Michel J.G. van Eeten, Michael Levi,
Tyler Moore, and Stefan Savage. “Measuring the cost of cybercrime”. (2012) PDF available
at: http://weis2012.econinfosec.org/papers/Anderson_WEIS2012.pdf.
• Camp, L. Jean and Wolfram, Catherine D., “Pricing Security: Vulnerabilities as Externalities.”
Economics of Information Security, Vol. 12, 2004. Available at SSRN: http://ssrn.com/abstract=894966.
• MacCarthy, Mark. “Information Security Policy in the U.S. Retail Payments Industry.” Workshop on the
Economics of Information Security, June 2010.
• Sullivan, Richard J. “The Changing Nature of U.S. Card Payment Fraud: Issues for Industry and Public
Policy.” For presentation at the 2010 Workshop of Economics of Information Security. May 21, 2010.
• Skeen, Dale. "How Real-Time Analytics Protects Banks from Large Scale Cyber Attacks." Information
Security Buzz, accessed Feb 23, 2014. http://www.informationsecuritybuzz.com/real-time-analytics-
protects-banks-large-scale-cyberattacks/
• Varian, Hal. “Managing Online Security Risks.” New York Times; New York, N.Y.; Jun 1,2000.