This document discusses using big data analytics to improve upon traditional SIEM (Security Information and Event Management) solutions for network security monitoring. It notes that SIEMs have performance limits, cannot handle large varieties of data, and lack real-time correlation and analysis abilities. Big data solutions using the Hadoop ecosystem can overcome these issues through security analytics techniques like classification, correlation, clustering, affinity grouping, aggregation, and statistical analysis applied to large magnitudes of network and system data. This would allow for behavior-based anomaly detection rather than solely signature-based detection and anomaly detection of user network access. The document suggests network security managers can build their own "next-gen" network security monitoring systems using these big data analytics techniques.

1. NETWORK SECURITY
MONITORING WITH BIG
DATA ANALYTICS
Nguyen Minh Duc
Security BootCamp 2014
Oct 2014

3. Sad but True

4. Rules, signatures or not ?

5. Rules, signatures or not ?

6. Rules, signatures or not ?

7. • Security Information and Event Management
Ok, we’ve already got a SIEM

8. • Performance Limits Galore
• Data Variety Troubles
• Real-Time Correlation, Not Data Mining
• Analysis? Where Is That?
SIEM Challenges Today
Source: Gartner

9. So we need a more powerful tool
Big Data

10. Why Big Data ?

11. Hadoop ecosystem

12. Security analytics is the process of performing analysis on magnitudes of data, adding
context to the information collected and then deriving answers and actionable
knowledge
• Classification: Allows events to be grouped into like sets for context.
• Correlation: Real-time & historical associations can be recognized, providing
context and relational understanding.
• Clustering: Data point similarity detection across large collections provides a
straightforward, yet confident, way to derive true understanding of many events.
• Affinity Grouping: Similar to clustering, but this can take the context of each data
point as it pertains to users, systems, attacks and their interactions. Provides
excellent context between multiple, seemingly disparate, data points.
• Aggregation: Allows a high level view of large amounts of data, distilling often
complex sets into simple numerical quantities, e.g. Did this bad event happen
often enough in an hour to be of concern?
• Statistical Analysis: Provides methods for dealing with uncertainty within the data
sets yielding a confidence for comprehension.
Security Analytics with Big Data
Source: HP

13. • Behavior-based anomaly detection vs signature-based
detection
Security Analytics with Big Data

14. • Anomaly detection of user’s network resource access
Security Analytics with Big Data

15. Build your own “next-gen” NSM?

16. Build your own “next-gen” NSM?

17. Thanks
Nguyen Minh Duc
FPT Technology Innovation Dept
minhduc@fpt.com.vn