Data driven decision making can be retrospective, real-time, or predictive. We use Amazon Machine Learning to predict the probability that a vulnerability will become exploited, using only the data available when a vulnerability is released.

1. PREDICTING EXPLOITABILITY
@MROYTMAN

2. “Prediction is very difficult, especially
about the future.”
-Neils Bohr

3. 3 Types of “Data-Driven”

4. Too many vulnerabilities.
How do we derive risk from
vulnerability in a data-driven
manner?
PROBLEM

5. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

6. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

7. Analyst Input
Vulnerability Management
Programs Augmenting Data
Retrospective
Temporal Score Estimation
Vulnerability
Researchers

8. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

9. ATTACKERS
ARE FAST

10. 0 5 10 15 20 25 30 35
CVSS*10
EDB
MSP
EDB+MSP
Breach*Probability*(%)
Positive Predictive Value of remediating a
vulnerability with property X:

12. DATA OF FUTURE PAST
Q: “Of my current vulnerabilities, which ones
should I remediate?”
A: Old ones with stable, weaponized exploits

13. FUTURE OF DATA PAST
Q: “A new vulnerability was just released. Do we
scramble?”
A:

14. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

15. Machine Learning?

17. Enter: AWS ML

18. 70% Training, 30% Evaluation Split N = 81303
All Models:
L2 regularizer
1 gb
100 passes over the data
Receiver operating
characteristics for comparisons

19. Model 1: Baseline
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date

20. LMGTFY:

21. Moar Simple?

22. Model 2: Patches
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists

23. Model 3: Affected Software
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products

24. Model 4: Words!
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products
-Description, Ngrams 1-5

25. Model 5: Vulnerability Prevalence
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products
-Description, Ngrams 1-5
-Vulnerability Prevalence
-Number of References

26. Moar Simple?

27. Moar Simple?

28. Exploitability

29. -Track Predictions
vs. Real Exploits
-Integrate 20+
BlackHat Exploit
Kits - FP
reduction?
-Find better vulnerability
descriptions - mine
advisories for content?
FN reduction?
Future Work
-Predict Breaches,
not Exploits
-Attempt Models
by Vendor
-There are probably
two exploitation
processes here.

30. PREDICTIONS
1. CVE-2017-0003
2. CVE-2017-2963
3. CVE-2016-7256
These will have exploits in 2017:
Sharepoint Enterprise Server, Word 2016
Adobe Acrobat Reader
Windows Server 2008, 2012, 2016, Windows 7, 8, 10

32. Scan Data Is
Overwhelming
Finding Vulnerabilities – Needlessly Difficult
Impossible to Know
What to Prioritize
Not Integrated with
Threat Intelligence
Communication Is Painful—No Single
Pane of Glass Suits All Stakeholders

33. CISO Sec Ops IT Ops
How Kenna Works
Exploit Intel
10+ Threat Feeds
Enterprise
21+ Connectors

34. Thanks!
@MROYTMAN