1. Your Logs or ...
Back to the Gold Rush
ISSA-BE Event
January 2011

2. $ whoami
Xavier Mertens (@xme)
Senior Security Consultant @ C
C-CURE
CISSP, CISA, CEH
http://blog.rootshell.be
I’m also on Maltego & Google!
Some friends:

3. $ cat disclaimer.txt
The opinions expressed in this presentation are
those of the speaker and do not reflect those of
past, present or future employers, partners or
customers...

4. -1-
The situation today

5. acme.org

6. acme.org’s CSO
Did you already get this feeling?

7. Today's Issues
Technical
Networks are complex
Based on non-heterogeneous
heterogeneous
components (firewalls, IDS, proxies, etc)
Millions of daily events
Lot of consoles/tools
Protocols & applications

8. Today's Issues
Economical
”Time is Money”
Investigations must be performed in
real-time
Downtime may have a huge
business impact
Reduced staff & budgets
Happy Shareholders

9. Today's Issues
Legal
Compliance requirements
PCI-DSS, SOX, HIPAA, etc
DSS,
Initiated by the group or business
Local laws
Due diligence & due care
Security policies must
be enforced!

10. Need for More Visibility
More integration, more sources
More chances to detect a problem
Integration of external source of information
could help the detection of incidents
Automatic vulnerability scans
Import of vulnerabilities
database
FIM
Awareness

11. Need for More Visibility
[**] [1:2050:14] SQL version overflow attempt [**]
[Classification: Attempted Administrator Privilege Gain] [Priority: 1]
07/27-17:00:05.199275 203.85.114.127:1073 -> 10.0.0.2:1434
UDP TTL:105 TOS:0x0 ID:65518 IpLen:20 DgmLen:404
Len: 376
[Xref => http://www.microsoft.com/technet/security/bulletin/MS02-039.mspx][Xref => http://cgi.nessus.org/p
039.mspx][Xref
lugins/dump.php3?id=10674][Xref => http://cve.mitre.org/cgi-bin/cvename.cgi?name=2002
bin/cvename.cgi?name=2002-0649][Xref => http:/
/www.securityfocus.com/bid/5310]
[**] [119:14:1] (http_inspect) NON-RFC DEFINED CHAR [**]
[Priority: 3]
07/27-17:07:54.146866 10.0.0.2:9041 -> 199.7.71.72:80
TCP TTL:64 TOS:0x0 ID:36997 IpLen:20 DgmLen:167
***AP*** Seq: 0x5F1B1F41 Ack: 0x6CBD4FE5 Win: 0x4000 TcpLen: 32
TCP Options (3) => NOP NOP TS: 1475031583 2358505469
[**] [119:14:1] (http_inspect) NON-RFC DEFINED CHAR [**]
[Priority: 3]
07/27-17:20:05.913434 10.0.0.2:1758 -> 199.7.59.72:80
TCP TTL:64 TOS:0x0 ID:41064 IpLen:20 DgmLen:167
***AP*** Seq: 0xA9756DFB Ack: 0x8AF3A8FC Win: 0x4000 TcpLen: 32
TCP Options (3) => NOP NOP TS: 2086630937 3122214979
[**] [119:14:1] (http_inspect) NON-RFC DEFINED CHAR [**]
[Priority: 3]
07/27-17:22:27.226248 10.0.0.2:23157 -> 199.7.71.72:80
TCP TTL:64 TOS:0x0 ID:48855 IpLen:20 DgmLen:167
***AP*** Seq: 0x480A3145 Ack: 0x9227C6FF Win: 0x4000 TcpLen: 32
TCP Options (3) => NOP NOP TS: 2530339421 2353821688
[**] [119:14:1] (http_inspect) NON-RFC DEFINED CHAR [**]
[Priority: 3]
07/27-17:29:26.969904 10.0.0.2:41287 -> 199.7.52.72:80
TCP TTL:64 TOS:0x0 ID:7498 IpLen:20 DgmLen:167
***AP*** Seq: 0xBDCC9352 Ack: 0xB241F70B Win: 0x4000 TcpLen: 32
TCP Options (3) => NOP NOP TS: 3995062809 1050363790

12. -2-
Fraud?

13. What’s ”Fraud”?
”Deliberate deception, trickery, or cheating
Deliberate
intended to gain an advantage”
Fraud represents 39% of crimes in the
CERT.us database
Occurs “below the radar”

14. Fraud Types
Unauthorized addition or changes in
databases
Data theft or disclosure
Rogue devices
Identifity theft

15. Find the Intruder
Keep an eye on the « malicious insider »
Who is he?
Current or past employee (m/f)
Contractors / Business partners
Non-technical as well as technical position
technical
He/she has authorized access to
sensitive assets

16. Fraud == Suspicious
The term “fraud” is closely linked to money
Let’s use “suspicious which means
suspicious”
“inclined to suspect, to have doubts about;
distrust”
Detected outside the scope of regular
operations
Need for baselines,
thresholds and
watchdogs
And... Procedures!

17. Baselines
Interval of values
Trigger an alert of above a threshold
or outside an interval

18. Baselines
Recurrence in time

19. Baselines
Correlation between multiple sources

20. Impacts of Fraud?
Quantitative
$$$
Qualitative
Brand
Reputation
Customers / Stakeholders

21. Some Examples
CC used in country ”A” and used 4 hours
later in country ”B”.
A Belgian CC used to buy a 40” flat TV in
Brazil
A SIM card connected to a mobile network in
Belgium and 2 hours later in Thailand
Stolen or shared credentials / access badges.
SSL VPN access from a foreign country.

22. More Examples
”root” session opened on a Sunday 02AM.
Data copied on removable devices
Installation of keyloggers
Rogue FTP servers

23. Security Convergence!
Logical Security
Credentials
IP access lists
Physical Security
Access badges
GeoIP
Mobile devices
Time references
Let’s mix them!

24. Resources!
Adding plus-value to your logs is resources
value
consuming!
Temporary tables might be required
Beware of time lines!

25. How to fight?
Need for raw material Your logs
Know the process flows!
Talk to the ”business”
Increase the logs value
Add visibility
Correlate with other information sources
+ Processes and communication!

26. When?
Real-time
Immediate investigationSource: Real
Real-time alerts
Before
Proactivity (reporting - trending)
After
Forensic searches

27. -3-
The tools

28. It’s not a product...
”... It’s a process!” (c) Bruce
Incident Handling
Correlation
Reporting
Search
Log Collection

29. The Good, The Bad, The Ugly!
Big Play€r$ (no names!)
r$
All of them prone to be the best
But often when you look inside:

30. Straight to the Point
SIEM environments are exp
exp€n$ive!
Best choice?
Must address the business requirements
(not yours)
You must be able to handle them

31. The Ingredients...
Free software to the rescue!
Some tools...
OSSEC
MySQL
Iptables / Ulogd
Google Maps API
Perl
The ”Cloud” (don’t be scared!)

32. You said ”OSS.. What?”
OSSEC is ”an Open Source Host
an Host-based
Intrusion Detection System. It performs log
analysis, file integrity checking, policy
,
monitoring, rootkit detection, real
real-time
alerting and active response
response”.
More info @wimremes (ISSA 01/2010)
wimremes

33. The Recipes
Good news, you already have the main
ingredient: your logs!
Resources
Policies
External
Logs
Security
Incidents

34. -4-
MySQL Audit

35. Problem
Authorized users added or modified data in a
database.
Lack of control and separation of duties
Examples of fraud
Rogue acces created
Price changed
Stock modified
Data integrity not consistent anymore

36. Solution
Database changes can be audited
High performance impact
All transactions are logged
Not convenient to process
Monitor changes on critical data
Users credentials
Financial data
Audit INSERT, UPDATE & DELETE
queries

37. Howto
Use the MySQL UDF ”lib_mysqludf_log.so”
mysql> create function lib_mysqludf_log_info returns
string soname 'lib_mysqludf_log.so';
mysql> create function log_error returns string
soname 'lib_mysqludf_log.so';
Use MySQL triggers
mysql> create trigger users_insert after insert on
users for each row insert into dummy
values(log_error(”your message here”));
Triggers will write message in the
MySQL errors.log

38. Howto
Process the MySQL log via OSSEC
<!-- MySQL Integrity check -->
<rule id="100025" level="7">
<regex>^dddd-dd-
dd dd:dd:dd Table:
.</regex>
<description>MySQL users table
updated</description>
</rule>

39. Howto
Results:
Received From: (xxxxx) xx.xxx.xxx.xxx
xx.xxx.xxx.xxx-
>/var/lib/mysql/errors.log
Rule: 100025 fired (level 7) -> "MySQL users table
updated”
Portion of the log(s):
2011-01-08 00:31:24 Table: acme.users:
08
insert(8,brian,qavXvxlEVykwm) by admin@localhost
--END OF NOTIFICATION

40. -5-
USB Stick Detection

41. Problem
Risks of data leak
Risks of malware infections

42. Solution
The Windows registry is a goldmine to audit a
system!
The OSSEC Windows agent can monitor the
Windows registry.

43. Howto
Interesting registry keys:
HKLMSYSTEMCurrentControlSet
CurrentControlSetServicesUSBSTOREnumCount
Or
HKLMSYSTEMCurrentControlSet
CurrentControlSetEnumUSBSTOR

44. Howto
Create a new OSSEC rule:
[USB Storage Inserted] [any] []
r:HKLMSYSTEMCurrentControlSet
CurrentControlSetServicesUSBSTOREnum
-> Count -> !0;
If “Count” > 0 => USB Storage inserted
Problem: will be reported by the rootkit
detector and not in real time

45. Howto
The second registry key changes when a
USB stick is inserted:
HKLMSYSTEMCurrentControlSet
CurrentControlSetEnumUSBSTORDisk&Ven_U
SB&Prod_Flash_Disk&Rev_0.00
New rule:
[USB Storage Detected] [any] []
CurrentControlSetServicesUSBSTOR;
r:HKLMSYSTEMCurrentControlSet

46. Howto
Results
** Alert 1268681344.26683: - ossec,rootcheck,
2010 Mar 15 20:29:04 (WinXP 192.168.38.100-
WinXP)
>rootcheck
Rule: 512 (level 3) -> 'Windows Audit event.‘
>
Src IP: (none)
User: (none)
Windows Audit: USB Storage Inserted.

47. -6-
Detecting Rogue
Access

48. Problem
Stolen or shared credentials can be used
from ”unknown” locations
If your team members are local, is it normal
to have sessions opened on your SSL VPN
from Thailand or Brazil?
An admin session started from the
administration VLAN?

49. Solution
Public IP addresses? They can be mapped to
coordonatess using open GeoIP databases
Private IP addresses? Hey, they’re yours,
you should know them
For public services, Google Maps offers a
nice API

50. Howto
Configure OSSEC for your application log file
(write a parser if required)
Create an “Active-Response” action triggered
Response”
when a specific action is detected
The “Active-Response” script will perform a
Response”
geoIP lookup using the source IP address

51. Howto
If the IP address belongs to suspicious
country or network zone, inject a new event
into OSSEC
OSSEC generates an alert based on
this event.

52. Howto
Results:
** Alert 1270065106.2956457: mail - local,syslog,
2010 Mar 31 21:51:46 satanas
satanas->/var/log/fraud.log
Rule: 50001 (level 10) -> 'Fraud Detection‘
>
Src IP: (none)
User: (none)
[31-03-2010 21:51:45] Suspicious activity detected
2010
for user johndoe via IP x.x.x.x in DE, Germany

53. -7-
Mapping on Google
Maps

54. Problem
What the difference between:
195.75.200.200 (Netherlands)
195.76.200.200 (Spain)
IP’s are extracted from firewall logs, botnet
analyzis, web sites logs, ...

55. Howto
Geo-localization is performed using the
MaxMind DB (free version) + Perl API
use Geo::IP;
my $gi = Geo::IP->open("GeoLiteCity.dat",
>open("GeoLiteCity.dat",
GEOIP_STANDARD);
my $record = $gi->record_by_name
record_by_name(“1.2.3.4");
print $record->latitude . "," . $record
>latitude $record->longitude;
Store results to a XML file.

56. Howto
Submit the file to the Google map API from
HTML code.

57. -8-
Searching the
Cloud

58. ”LaaS” ?
”Logging as a Service” seems to be an
emerging thread in 2011.
Loggly offers beta accounts
200MB/day - 90 days of retention
No SSL support
Supported ”inputs”
Syslog (UDP or TCP)
HTTP(S)

59. ”OSSEC phone Loggly”
OSSEC can export to Syslog
Events can be sent to Loggly using HTTP
POST requests:
https://logs.loggly.com/inputs/420fecf5-c332-4578-
https://logs.loggly.com/inputs/420fecf5
a0cb-21b421d4cc46

60. ”OSSEC phone Loggly”
Perl to the rescue:
# ./syslog2loggly.pl –h
syslog2loggly.pl [-f keyfile] [
f [-D] [-h] [-v] [-p
port]
-D
D : Run as a daemon
-h : This help
-f keyfile : Configuration file
f
(default: /etc/syslog2loggly.conf)
-p port
p : Bind to port (default 5140)
-v
v : Increase verbosity

61. Results

62. Conclusions
The raw material is already yours.
The amount of data to process makes it
impossible to process it without appropriate
tools.
Suspicious activity occurs below the radar.
Make your logs more valuable by cross
cross-
linking them with other sources.
Be ”imaginative”!

63. References
The scripts and references are available on
my blog: http://blog.rootshell.be/
Keyword: ”OSSEC”

64. Thank You!
Questions?