This document discusses the design and maintenance of a vulnerability database. It outlines important considerations for the design such as what information to include in each entry and prioritizing which details are most important to collect. It also evaluates different sources of vulnerability data like databases, vendor advisories, and vulnerability contributors. Key factors discussed include coverage, speed of updates, visibility of listings, inclusion of technical details and risk ratings.
2. Agenda | Vulnerability Database Maintenance
1. Intro
Introduction 2 min
Who am I? 2 min
What is the Goal? 2 min
2. Vulnerability Database Maintenance
Design the Database 5 min
Handling of Sources 4 min
Interpretation of Data 4 min
Correlation of Data 4 min
Quality Management 5 min
Extrapolation of Data 5 min
Deliver your Results 5 min
Statistical Analysis 5 min
Provide Accessibility 5 min
Use Connectivity 5 min
3. Outro
Summary 2 min
Questions 5 min
area41 2014 2/34
3. Introduction | Who Am I?
Name Marc Ruef
Job Co-Owner / CTO, scip AG, Zürich
Private Website http://www.computec.ch
Last own Book „The Art of Penetration Testing“,
Computer & Literatur Böblingen,
ISBN 3-936546-49-5
Translation
area41 2014 3/34
2013 2007 20022004
4. Introduction | What Is a Vulnerability Database?
◦ What?
◦ A database collecting vulnerabilities
◦ Why?
◦ To do vulnerability management
◦ What is vulnerable?
◦ What is to patch?
◦ To do statistical analysis
◦ Costs of patch management
◦ Robustness of products
area41 2014 4
7. Design | What Should Your Vulnerability Database Do?
◦ How much?
◦ Full coverage
◦ Selective collection
◦ Inventory-only
◦ Vendor-selection
◦ Importance threshold
◦ Fixed only
◦ For whom?
◦ Everyone
◦ Public service
◦ Advertisement
◦ Customers
◦ Vulnerability management service
◦ Alerting service
◦ Tools
◦ Internal Use
◦ Knowledge-base
◦ For pentesters
◦ For administrators
area41 2014 7
8. Design | What Is an Entry?
◦ A VDB entry consists of different elements. Minimal elements
usually are:
◦ ID 12413
◦ Title Linux Low-Address Protection Denial of Service
◦ Disclosure Date 02/21/2014
◦ Description A vulnerability, classified as (…)
◦ Risk Rating problematic
◦ References CVE-2014-2039, BID 65700, …
area41 2014 8
10. Design | But Details Take Time!
◦ We have compiled more than 13’400 entries since 2003
◦ A scip VulDB entry consists of ~150 possible data points
◦ We rate data points to prioritize:
◦ Important = 33 (must be processed if available)
◦ Normal = 32 (shall be processed)
◦ Optional = 85 (can be processed, if you have «too much time»)
◦ Statistical analysis of defined data points over all entries:
◦ Average = 49.92
◦ Min = 26
◦ Max = 90
◦ We currently add ~15 new entries per day (work-days only)
area41 2014 10
12. Sources | Vulnerability Databases: Advantages and Disadvantages
VDB Pros Cons
IBM X-Force
http://xforce.iss.net
• Good coverage
• CVSSv2 base scores
• CVSSv2 temporal scores
• CVE support
• Sometimes a bit slow (2-3 updates per
week)
• «Arbitrary» listing (default view: 5
entries, no backlog)
• No RSS feed
OSVDB
http://www.osvdb.org
• Very quick (daily updates)
• Best coverage (everything!)
• CVSSv2 base scores (via MITRE)
• CVE support
• No listing (since Feb 2014)
• No own risk rating (CVSSv2 only)
• No RSS feed (since 2012)
Secunia
http://secunia.com/community
/advisories/historic/
• Good coverage
• Good listing (default view: 25 entries)
• CVE support
• Login required (since Apr 2014)
• Some details for paying customers only
• Combining multiple vulnerabilities in
one entry (by release/patch)
• They don’t like other projects (they
forbade to use their listing for
vulscan.nse in 2013)
• No RSS feed
• No CVSSv2 scores
SecurityFocus
http://www.securityfocus.com/
bid
• Good coverage
• CVE support
• Listing also shows updated entries
(default view: 31 entries)
• Site is slow
• Data for an entry is spread over 5 sub-
pages
• No CVSSv2 scores
SecurityTracker
http://securitytracker.com
• Sometimes quite quick
• Simple listing (default view: 5 entries)
• CVE support
• Selective coverage (popular products
only)
• No CVSSv2 scores
13. Sources | Evaluation Rating Introduction
◦ Criteria are those we think are
important
◦ We have addressed them as far
as possible in our project
(because of this prioritization)
◦ Rating is as fair as possible
◦ You might rate a bit differently
Description
Rating
Feature is supported: always/fully 3
Feature is supported: often/partially 2
Feature is supported: sometimes/somehow 1
Feature is never/not supported 0
15. Sources | Vulnerability Databases: Conclusion
◦ Being quick is not easy
◦ Technical details range from bad to good
◦ CVSS scores are pretty unpopular, especially «temporal scores»
◦ CVE has been established as the de facto standard (nice!)
◦ You can’t compare CERT VU, Exploit-DB, NIST NVD and MITRE
CVE with anything else
◦ Exploit-DB inherits abstraction from researchers and is not self-
consistent
◦ Secunia and SecurityFocus are very similar in many aspects
◦ X-Force and SecurityTracker remain pretty unpopular
◦ The «O» in OSVDB does not stand for «open» anymore
◦ Some features have been broken for ages (e.g. search on OSVDB
and X-Force)
◦ Not everyone is a big fan of feeds
area41 2014 15
16. Sources | Vendor Advisories: Advantages and Disadvantages
Vendor Pros Cons
Adobe
http://helpx.adobe.com/security.
html
• Product-related listing
• Some technical details
• Priority rating
• CVE support
• Advisory per release/upgrade
• No RSS feed
Apple • Simple technical details
• CVE support
• No risk rating
• No CVSSv2 scores
• No listing
• Advisory per release/upgrade
• No RSS feed
Cisco
https://tools.cisco.com/security/c
enter/publicationListing.x
• Advisory listing
• Advisory per vulnerability
• Sometimes additional technical details
• CVSSv2 base scores
• CVE support
• Technical details with login only
• Some details for customers only
• No RSS feed
Google • CVE support • No listing
• Advisory per release/upgrade
• Technical details with auth only
• No risk rating
• No CVSSv2 scores
• No RSS feed
Microsoft
http://technet.microsoft.com/sec
urity/advisory
• Some technical details
• Listing (default view: 5 entries)
• RSS feed
• Patch day collection (2nd Tuesday of
each month)
• Severity rating
• No CVSSv2 scores
Oracle
http://www.oracle.com/technetwo
rk/topics/security/alerts-
086861.html
• Simple listing
• CVSSv2 base scores
• CVE support
• Patch day collection (quarterly)
• No technical details
• No RSS feed
18. Sources | Vendor Advisories: Conclusion
◦ Some vendors have really ugly advisory URLs
◦ Technical details range from bad to good
◦ CVSS scores are pretty unpopular, especially «temporal scores»
◦ Own risk ratings are also unpopular, because they are hard
◦ Nearly everybody likes CVE
◦ Microsoft and Oracle handle things better than it felt
◦ Juniper has a field «Last Updated» but no «Disclosure Date»
◦ SAP is very restrictive with information for non-customers, which
introduces a severe disadvantage (VDB’s can’t categorize them,
which decreases visibility)
◦ Vendors aren’t big fans of RSS feeds either
area41 2014 18
19. Sources | Vuln Contributors: Advantages and Disadvantages
Project Pros Cons
iDEFENSE Vulnerability
Contributor Program
http://www.verisigninc.com/en_US/cyber-
security/index.xhtml
• Started in 2003 • Incomplete listing
• No announcement of upcoming
advisories
• No CVSSv2 support
• No search capabilities
• No RSS feed
• All old links are broken since
Zero Day Initiative
http://www.zerodayinitiative.com
• Provide announcement for
upcoming advisories
• Provide CVSSv2 Base Scores
• RSS feeds available
• No search capabilities
21. Sources | Vuln Contributors: Conclusion
◦ Only 2 major players
◦ They are quite similar in most aspects
◦ Zero Day Initiative has 2 advantages of CVSSv2 and RSS support
◦ More competition might increase quality
area41 2014 21
22. Interpretation | How to Analyze
◦ The basic approach of processing a source is simple:
1. Check source for new entries
2. Review source entry
3. Add necessary data to database
1. If entry is available → Update existing entry
2. If entry is not available → Create new entry
3. If source is false-positive → Ignore entry and flag for future reference
4. Goto 1
area41 2014 22
24. Interpretation | MITRE CVE as an Example: What Is missing?
◦ What’s missing on a MITRE CVE entry?
◦ Disclosure date
◦ Exact naming of vulnerability class
◦ Risk rating
◦ Person responsible for disclosure
◦ Detailed mitigation/countermeasure
◦ …
area41 2014 24
25. Interpretation | OSVDB as an Example
cve
sectracker
product version
description
date
exploit
news
27. Interpretation | Contradicting Conventions (Disclosure Date)
CVE-2014-2284
net-snmp 5.7.1 on Linux ICMP-MIB Denial of Service
02/19/2014
02/20/2014
02/21/2014
02/22/2014
02/23/2014
02/24/2014
02/25/2014
02/26/2014
02/27/2014
...
03/24/2014
SourceForge
ReleaseNote
SecFocus
SecTracker
VulDB
OSVDB
Secunia
RedHat
Our definition of
a (public) disclosure date:
The earliest known date to
disclose an issue to the public in
an unrestricted way.
(we’re going to adopt a more
differentiated approach in the
near future)
03/05/2014oss-security ...CVE
29. Sources | Vulnerability Databases: Conclusion
◦ OSVDB provides the best collection of data
◦ Secunia provides the worst collection of data
◦ SecurityFocus and Secunia usually don’t provide context
◦ X-Force, SecurityTracker and Secunia don’t provide exploit details
◦ SecurityTracker and Secunia have confusing disclosure dates
◦ SecurityFocus, SecurityTracker and Secunia don’t link to other
VDB
area41 2014 29
30. Correlation | That's Why You Have to Correlate
◦ Approach
◦ Merge different sources
◦ Compare similar data points
◦ Identify and verify contradictions
◦ Dangers
◦ Duplicates: Come up with annoying inconsistency
◦ Merges: Come up with dangerous mashups
area41 2014 30
31. Correlation | Now Things Are Getting Tricky
◦ Sometimes vulnerabilities can’t be identified individually
◦ CVE helps a lot! But not every vulnerability (immediately) has a CVE
number
◦ Some sources merge vulnerabilities into one entry
◦ Vendors do this within their patch release notes or patch days
◦ Secunia tends to compile different vulnerabilities of the same day or patch
generation into one entry (e.g. 58519). SecurityFocus does it sometimes
(e.g. 67553) and so does SecurityTracker in some cases (e.g. 1030269).
◦ Vulnerabilities with very few technical details often can’t be
distinguished from similar vulnerabilities (e.g. Apple HT6145: no info
available, but CVE assigned)
area41 2014 31
32. Correlation | Keep Track, Detect Collisions
◦ Keep track of your sources and the entries already reviewed
◦ Verify that every new entry is really new and not just a duplicate
or a minor fork of an existing entry. This is a very underestimated
task!
◦ We do that with collision detection
◦ Compare new values with existing values of other entries (e.g. URLs,
IDs, references). If there is a specified level of matches, we have to
check for a duplicate.
◦ Our reference maps help to distinguish. Projects like vFeed
support this very good. [https://github.com/toolswatch/vFeed/]
area41 2014 32
33. Correlation | To Split or Not to Split
Parameter
→ 5 entries
File
→ 4 entries
Component
→ 3 entries
Vuln Class
→ 2 entries
Advisory/Patch
→ 1 entry
Advisory
#VA42
Cross Site
Scripting
User Auth login.php
login_user
login_pass
News Portal
news.php news_id
archive.php news_year
SQL
Injection
Board forum.php post_id
area41 2014 33
36. Correlation | Split Pros and Cons
◦ Advisory / Patch
◦ Few entries
◦ Good for overview
◦ Good for patch management
◦ Vulnerability
◦ Some entries
◦ Possible splits for 3rd party components
◦ Element
◦ A lot of entries
◦ Good for statistical analysis
area41 2014 36
37. Quality | How to Provide the Best?
◦ Try to verify statements from researchers, vendors and
vulnerability database maintainers
◦ Check for plausibility
◦ Verify from other sources
◦ Re-test within a lab
◦ Eliminate wrong statements
◦ Delete false entries
◦ Preserve false entries (prefered by CVE, SecurityFocus)
◦ Add further explanations
◦ Flag (prefered by OSVDB, scip VulDB)
◦ advisory_disputed=1 (e.g. scipID 13305, 13000, 12643)
◦ advisory_reportconfidence=UR (CVSSv2 temp score metric)
◦ Try to find and compile additional details
area41 2014 37
38. Extrapolation | Versions of Affected Software
◦ Exact Version
◦ Internet Explorer 10 → X-Force, OSVDB, SecFocus, Secunia, VulDB
◦ Wildcards
◦ Internet Explorer 6.x → Secunia, SecFocus, SecTracker, VulDB
◦ Ranges
◦ Internet Explorer 8 – 10 → Secunia, CVE
◦ Internet Explorer prior 10 → SecurityTracker, Secunia
◦ Internet Explorer before 10 → CVE
◦ Internet Explorer up to 10 → VulDB
◦ Internet Explorer 8 and later → SecurityTracker
area41 2014 3810 119876
10
up to 10
8 to 10
Internet Explorer Versions
before 10
…
39. Extrapolation | What about The Unknown?
◦ Try to guess. Examples:
◦ «IE prior 9» → 6 – 9
◦ «IE prior 11» → 7 – 10
◦ Research and validate yourself
◦ A lot of work
◦ We combine with other projects (research or pentest)
◦ We enforce very important or interesting vulnerabilities
◦ Be quiet
area41 2014 39
40. Delivery | Chose your Channels
◦ Web Site
◦ Mail
◦ RSS
◦ Widgets
◦ Facebook
◦ Twitter
◦ LinkedIn
◦ App
◦ …
area41 2014 40
41. Statistics | Comparing Apples and Oranges
◦ Doing some statistics is easy. Doing it the right way is hard. Some
say it is even impossible.
[http://blog.osvdb.org/category/vulnerability-statistics/]
◦ Counting vulnerabilities doesn’t say anything:
◦ Weak code leads to a lot of vulnerabilities
◦ Complexity leads to a lot of vulnerabilities
◦ Popularity leads to a lot of vulnerabilities
◦ Bug bounty programs lead to a lot of vulnerabilities
◦ Open disclosure process leads to a lot of vulnerabilities
◦ We still provide statistical raw data and expect the viewers to
think about it
area41 2014 41
43. Statistics | Timelines Trivia (excerpt from 2014)
◦ [CVE-2014-0160] OpenSSL TLS/DTLS Heartbeat information
disclosure got introduced in 01/01/2012 and fixed in 04/07/2014
◦ existed 827 days
◦ [CVE-2014-0179] libvirt XML Entity Expansion Handler denial of
service got introduced in 12/23/2009 and fixed in 05/06/2014
◦ existed 1.595 days
◦ [CVE-2014-3122] Linux Kernel try_to_unmap_cluster() denial of
service got introduced in 10/19/2008 and fixed in 04/10/2014
◦ existed 1.996 days
◦ [CVE-2014-3460] Novell NetIQ Sentinel Agent Manager directory
traversal vendor got informed in 09/04/2013 but did not respond
until 05/19/2014
◦ Novell ignored grace period of 257 days
area41 2014 43
44. Accessibility | Choose Additional Representation
◦ To allow users to work with your data, it might be the best way to
provide additional forms of representation:
◦ SQL
◦ XML
◦ JSON
◦ CSV
◦ CVRF [http://www.icasi.org/cvrf]
area41 2014 44
45. Connectivity | Use Data for Vuln Scanning
◦ We are able to construct specific requests with our fields
software_argument and software_input_value to create test cases
and exploits (very simple for web-based vulns)
◦ Because of the fields software_* we are able to provide CPE lists
[http://cpe.mitre.org/], which can be matched with tools like
Nmap. Random examples:
◦ ID 12313 → cpe:/a:sap:netweaver:7.30
◦ ID 12802 → cpe:/o:cisco:ios:15.4(1.1)t
◦ ID 13306 → cpe:/a:microsoft:internet_explorer:8
area41 2014 45
46. Outro | Summary
◦ Vulnerability databases help to manage vulnerabilities
◦ Different sources allow to collect a broad amount of issues
◦ Every source has some advantages and disadvantages
◦ Compiling and maintaining vulnerabilities takes a lot of effort
◦ Making your data accessible helps others
area41 2014 46
47. Outro | Thank You
◦ I‘d like to thank a bunch of people which helped to discuss the
many interesting aspects of vulnerability database management:
◦ Stefan Friedli, scip AG
◦ Steven M. Christey, MITRE
area41 2014 47