This document discusses log management and security information event management. It begins by introducing the speaker and their background. It then discusses how most organizations are unprepared to deal with security incidents due to a lack of log management. It emphasizes the need for visibility into systems and integrating multiple log sources. It outlines technical, economic, and legal challenges around log collection, normalization, storage, search, reporting, and correlation. Finally, it discusses various open source and free log management tools that can be used to build out log monitoring capabilities.

1. All Your Security Events
are Belong to ...You!
BSidesLondon 2011 - Xavier Mertens

2. $ whoami
• Xavier Mertens (@xme)
• Security Consultant
• CISSP, CISA, CeH
• Security Blogger
• Volunteer for security projects:

3. $ cat disclaimer.txt
“The opinions expressed in this presentation
are those of the speaker and do not reflect
those of past, present or future employers,
partners or customers”

4. Today’s Situation

5. How is Your Log-Fu?
• Logs? Which logs?
• It’s BORING!
• Most organizations are NOT prepared to
deal with security incidents
• If anything can go wrong, it will!
(Murphy’s law)
• Enough internal resources?

6. Need for Visi bility!
• Computer: “programmable electronic machine
that performs high-speed mathematical or
logical operations or that assembles, stores,
correlates, or otherwise processes information”
Too cool!
• Integration with multiple sources increases
the change to detect suspicious events.
• Detect activity below the radar.

7. Technical Issues
• Networks are complex
• Some components/knowledge are
outsourced
• Millions of daily events
• Lot of console/tools
• Lot of protocols/applications

8. Find the Differences
Aug 27 14:33:01 macosx ipfw: 12190 Deny TCP
192.168.13.1:2060 192.168.13.104:5000 in via en1
%PIX-3-313001: Denied ICMP type=11, code=0 from
192.168.30.2 on interface 2

9. Economic Issues
• “Time is money”
• Real-time operations
• Downtime has a huge financial impact
• Reduced staff & budget
• Happy shareholders
• Log management == Insurance
(Risk management)

10. Legal Issues
• Compliance requirements
• Big names
• Initiated by the group or business
• Local laws
• Due diligence & due care

11. Legal Requirements
• Internal
• You are not Big-Brother!
• Team-members must be aware of the
procedures
• External
• Notify your users & visitors which
information is logged, how and for which
purposes

12. Belgian Example: CBFA
From a document published in April 2009:
“Any institution that connects to the Internet
must have a security policy which takes into
account:
...
the creation, the archiving of event logs which
permit the analyze, follow-up and reporting.”

13. Challenges
• Creation & archiving of log files
• Analyze (Normalization)
• Follow-up
• Reporting
• (Correlation)

14. Layer Approach
Correlation
Reporting
Search
Storage
Normalization
Log Collection

15. Raw Material
• Your logs are belong to you!
• If not stored internally (cloud,
outsourcing), claim access to them
• All applications/devices generate events
• Developers, you MUST generate GOOD
events

16. 3rd Party Sources
• Vulnerabilities Databases
• Blacklists (IP addresses, ASNs)
• “Physical” Data
• Geolocalization
• Badge readers

17. Security Convergence
• Mix of logical control:
• Passwords, access-lists
• Blacklists (IP addresses, AS’s, domains)
• and physical control:
• Badge readers
• Geo-localization

18. The Recipe

19. Collection
• Push or pull methods
• Use a supported protocols
• Open vs. Proprietary
• Ensure integrity
• As close as the source

20. Normalization
• Parse events
• Fill in common fields
• Date, Src, Dst, User, Device, Type, Port, ...

21. Storage
• Index
• Store
• Archive
• Ensure integrity (again)

22. Search
• CLI tools remain used (grep|awk|sort|
tail|...)
• You know Google?
• Investigations / Forensic
• Looking for “smoke signals”

23. Reporting
• Automated / On-demand
• Reliable only if first steps are successful
• Reports must address the audience
(technical vs business)

24. Correlation
• Generation of new events based on the
way other events occurred (based on their
logic, their time or recurrence)
• Correlation will be successful only of the
other layers are properly working
• Is a step to incident management

25. Build Your Toolbox

26. <warning>
Please keep v€ndor$
away from the
next slide
</warning>

27. Let’s Kill Some Myths
• Big players do not always provide the best
solutions. A Formula-1 is touchy to drive!
• Why pay $$$ and use <10% of the
features? (the “Microsoft Office” effect)
• But even free softwares have costs!
• False sense of security

28. LM vs. SIEM
• A LM (“Log Management”) addresses the
lowest layers from the collection to
reporting.
• A SIEM (“Security Information & Event
Management”) adds the correlation layer
(and often incidents management tools)

29. Grocery Shopping
• Compliance
• Suspicious activity
• Web applications monitoring
• Correlation
• Supported devices
• Buying a SIEM is a very specific project

30. Free Tools to the
Rescue

31. Syslog Daemons
• Syslog is well implemented
• Lot of forked implementations
• syslogd, rsyslogd, syslog-ng
• Multiple sources
• Supports TLS, TCP
• Several tools exists to export to Syslog
(ex: SNARE)
• But a hell to parse

32. SEC
• “Simple Event Correlation”
• Performs correlation of logs based on Perl
regex
• Produces new events, triggers scripts,
writes to files
• Example: track IOS devices reload
type=single
continue=takeNext
ptype=regexp
pattern=d+:d+:d+.*?(S+)s+d+:.*?%SYS-5-RELOAD: (.*)
desc=(WARNING) reload requested for $1
action=pipe '%s details:$2' mail -s 'cisco event' xavier@rootshell.be

33. OSSEC
• HIDS
• Log collection & parsing
• Active-Response
• Rootkit detection
• File integrity checking
• Agents (UNIX, Windows)
• Log archiving

34. Protocols
• CEF - “Common Event Format” | ArcSight
• CEE - “Common Event Expression” | Mitre
• RELP - “Reliable Event Logging Protocol”
• SDEE - “Security Device Event Exchange” |
Cisco

35. Miscellaneous
• MySQL
• iptables / ulogd
• GoogleMaps API
• Some Perl code
• liblognorm
• Cloud Services (don’t be afraid)

36. Some Recipes Using
OSSEC

37. USB Stick Detection
• Purpose:
• Protection against data leak
• Security policies enforcment
• Ingredients:
• OSSEC Windows Agents
• Windows Registry

38. USB Stick Detection
• Each time an USB stick is inserted,
Windows creates a new registry entry:
HKLMSYSTEMCurrentControlSetEnumUSBSTOR
Disk&Ven_USB&Prod_Flash_Disk&Rev_0.00
• Create a new OSSEC rule:
[USB Storage Detected] [any] [] r:HKLMSYSTEMCurrentControlSet
ServicesUSBSTOR;

39. MySQL Integrity Audit
• Purpose:
• Track changes on some MySQL tables.
• Ingredients:
• MySQL Triggers
• MySQL UDF (“User Defined Functions”)
• OSSEC parser + rules

40. MySQL Integrity Audit

41. Temporary Tables
• Purpose:
• To detect suspicious users & IP’s
• Ingredients:
• MySQL
• Patch ossec-analysisd
• External public sources

42. Temporary Tables

43. Using Google Maps
• Purpose: What’s the difference between:
195.75.200.200 (Netherlands)
195.76.200.200 (Spain)
• Ingredients:
• Google Maps API
• Perl scripting
• Geo-IP API (Geocity Lite)

44. Using Google Maps

45. OSSEC Dashboard
• Because one picture is worth a thousand
words!
• Ingredients
• MySQL OSSEC support
• LAMP server

46. OSSEC Dashboard

47. More Visibility
• LaaS (Loggly)
• Splunk
• Secviz.org

48. Conclusions
• The raw material is already yours!
• The amount of data cannot be reviewed
manually.
• Suspicious activity occurs below the radar.
• Stick to your requirements!
• It costs $$$ and HH:MM
• Make your logs more valuable via external
sources

49. Thank You!
Q&A?
http://blog.rootshell.be
http://twitter.com/xme