2. Disclaimer
All opinions and thoughts in this presentation are my own
and do not represent my employer
All use of Ice Cube’s image, lyrics, movies, and music are
for storytelling, not for profit
The data used in this presentation comes from my
employer, but is anonymized to protect the guilty and
innocent
4. Speed
If you're foul, you better run a make on that license plate
You coulda had a V8
Instead of a tre-eight slug to the cranium
I got six and I'm aimin em
5. Speed
How fast did you find the breach?
How fast did you stop the breach after it happened?
How fast did you clean it up?
How fast did you go from What? to So What? to Now
What?
6. Speed
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
You better check yourself before you wreck yourself
Cause I'm bad for your health, I come real stealth
Dropping bombs on your moms, f*** car alarms
Doing foul crime, I'm that fool with your Alpine
- Check Yourself – Ice Cube
7. Intellectual Honesty
Time’s are all in the same time zone – goes without saying
The time of compromise is when something changed in
the system – not when you or your system found it
Missing that key fact means you miss
Quality of intelligence
Coverage of intelligence
Time dropper hit the
file table
Time A/V reported
finding the
backdoor
Difference = 7 months, 8 days, 13 hours, 34 minutes, 7
seconds
9. Comp-to-Find
Speed of intelligence
deployment to your tools
How fast did you get it?
How fast did you know it?
How fast did you use it?
Frequency of scans
Alertness of users
Collection
Processing
Exploitation
Dissemination
Tasking
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
10. How to find?
Host
AV logs
Event logs
Nagios
Tripwire
Network
IDS/IPS alerts
Firewall logs
Proxy logs
Email gateway logs
11. Find-to-Alert
Speed of the sensor
Are your alerts backing up
on a DB somewhere?
How often are sensors
reporting back to their
console?
Knowledge of user
(protein-based sensor)
Do they know how to
report shadiness?
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
12. Alert-to-Give a s&*t
How long do alerts linger?
How long do emails about
incidents bounced around
inboxes?
SIEM logs
When analyst
acknowledges the alert
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
13. Give a s&*t-to-taking
action
Speed of triage & initial
analysis
Knowledge of internal
organization
Do your responders know
who to call?
Comprehensiveness of
response plans and SOPs
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
I found the APT !!!
14. Taking Action-to-Stopping
the s&*t
Host
Event log (shutdown)
DHCP log
AV log (deleted malz)
Phish deleted
Network
ACL in switch
IPS rule change log
IP block added to router
Firewall block added
Proxy log
Not when the rule was added, but
when it was confirmed to be
working
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
15. Stopping-to-cleaning up
the s&*t
How long the business
was impacted by the
breach?
Did the containment
strategy conflict with or
support recovery?
How fast did you find other
breaches?
How effective was your
recovery?
The fed’s
preferred
recovery method
Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
16. Quality
I hate motherf**kers claimin that they foldin bank
But steady talkin s&*t in the holding tank
First you wanna step to me
Now you’re a** screamin for the deputy
17. Quality
It’s great that you’re fast, but are you any good at it?
Easy to confuse quality with forensic soundness
Easy to confuse quality with expensive blinking boxes
Quality really measures
Are you focusing on what’s really important (customer)?
Are you focusing on what really works (performance)?
Do you track failures as much as you do successes (defects)?
Do you learn from mistakes and do you repeat them
(improvement)?
18. Comp Find Alert
Give a
s&*t?
Taking
action
Stop the
s&*t
Clean
up the
s&*t
First time right
In this process, how often were mistakes made
Do you track and categorize mistakes and misfires?
How many
times did you
miss the
breach?
Did the alerts
go to the right
place the first
time?
Did the person
viewing the
alert make the
right call?
Did the person
who gives a
s&*t do the
right thing?
Did the actions
actually stop
the breach?
Was your
cleanup
effective?
19. Measuring Quality
Get granular
Avoid “other” or
“unknown”
If given an option,
analysts will choose
“other” two out of
every three times.
Set goals
What’s acceptable
performance?
20. Forensics & Kill Chain
Increasing ferocity of Ice Cube movie characters
Increasing cost of response and recovery
23. Coverage
Tricks wanna step to Cube and then they get played
Cause they b&*ch made pullin out a switchblade
That's kinda trifle, cause that's a knife-o
[here’s an] AK-47, assault rifle
24. Coverage
Are you looking for the
right things in the right
places?
Filenames in IDS?
IP addresses in AV logs?
What percentage of your
install base are you
monitoring?
First, check yo’self
Use the Kill Chain
Find your gaps
25. Check Yo’self
How do you get got?
Phishing?
Watering holes?
Thumbdrives?
Websites getting popped?
For one thing, you don’t know how the f**k my
company be muthaf**king owned.
29. Finding Gaps
Lack of process
Misapplying Intel
Bad deployment of web
applications
Lack of Training
Developers building
insecure apps
Lack of technology
Buy only when you have
a clear blind spot
Not every gap in yo’ security needs
to be filled with cash money
30. Check yo ‘net
Do you have every
network ingress/egress
point monitored?
3rd Parties/Suppliers
VPN
Mobile/BYOD
Do you have monitoring on
every network service?
FTP, SFTP, Web, SMTP,
Telnet (yes, telnet)
Cloud services (*aaS)
Gary’s manager found an un-
instrumented PoP on the network
31. Check yo ‘boxes
What is your host logging
policy?
Do your logs go to a
central location?
Do you have a method to
search the endpoints and
servers for IOCs?
How agro are your
patching policies?
Will a Java patch f’ your
network?
http://bit.ly/1pTiodM - for other
derp-ables referring to “the APT”
32. Takeaways
Here to let you know boy, oh boy
I make dough but don't call me DoughBoy
This ain't no f**kin motion picture
A guy or b^*ch-a, my fool get wit'cha
And hit ya, takin that yack to the neck
So you better run a check
33. Telling your story to
management
Know the real cost of your
breach
Your time
Your team’s time
Cost of recovery
Client’s lost productivity
Data loss
Cost of R&D
Profit Margin
Know the real cost of
countermeasures
Training costs should
include time away and
travel
Process improvements
requires good data,
discipline, and expertise
If you’re buying a new
tool, double the cost of
deployment and add 50%
to annual O&M
34. Telling your story to
management
$7 K
$113 K
$64 K
$119 K
$122 K
$142 K
$114 K
$42 K
$56 K
$45 K
$6 K$6 K $7 K
$110 K
$17 K
$47 K
$152 K
$144 K
$97 K $100 K
$119 K
$39 K
$99 K
$135 K
$ K
$20 K
$40 K
$60 K
$80 K
$100 K
$120 K
$140 K
$160 K
Jan-12Jan-12Jan-12Jan-12
Feb-12Feb-12Feb-12
Mar-12Mar-12Mar-12Mar-12
Apr-12Apr-12Apr-12
May-12May-12May-12
Jun-12Jun-12Jun-12Jun-12
Jul-12Jul-12Jul-12
Aug-12Aug-12Aug-12Aug-12
Sep-12Sep-12Sep-12
Oct-12Oct-12Oct-12
Nov-12Nov-12Nov-12Nov-12
Dec-12Dec-12Dec-12
Jan-13Jan-13Jan-13Jan-13
Feb-13Feb-13Feb-13
Mar-13Mar-13Mar-13
Apr-13Apr-13Apr-13
May-13May-13May-13May-13
Jun-13Jun-13Jun-13
Jul-13Jul-13Jul-13Jul-13
Aug-13Aug-13Aug-13
Sep-13Sep-13Sep-13
Oct-13Oct-13Oct-13Oct-13
Nov-13Nov-13Nov-13
Dec-13Dec-13Dec-13Dec-13
Jan-14Jan-14Jan-14
Feb-14Feb-14Feb-14
Mar-14Mar-14Mar-14Mar-14
Apr-14Apr-14Apr-14
May-14May-14May-14
Jun-14
Per-event cost of our large-scale intrusions (Jan ‘12 – Jul
’14)
(# of days of full-scale response) x (daily rate of employee) x (# of employees involved in the response)
35. What point in the Kill Chain are attacks being stopped?
Does it cost more to respond to events higher in the KC?
Telling your story to
management
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
0
100
200
300
400
500
600
700
800
900
1000
Recon Deliver Exploit Install C2 AoO
Days
NumberofIncidents
36. What systems are catching attacks from “the APT”
Telling your story to
management
IDS
29%
Host-Based Scanner
12%
AV
12%
Proxy
Logs
7%
User
Report
6%
Email
Scanner
6%
Frequency Analysis
5%
Monthly Host Checker
4%
IP/Domain Hotlist
4%
SIEM
Correlations
4%
Event Logs
3%
Other
2%
Netflow
2%
3rd Party Notification
2%
Cloud-based Proxy
1%
IPS
1%
Commercial
Malware
Analysis
appliance
1%
Registry Scanner
1% Email Logs
1%
37. Don’t buy me another chirping box
Telling your story to
management
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
0
1
2
3
4
5
6
7
8
9
IDS Crmcl
Malware
Analysis
Device
McAfee User Report Email
Scanner
3rd Party
(Other)
Event Logs Proxy Logs
DaysofInvestigation
#ofFalsePositives
Detection Tool
38. y = -0.0958x + 12.279
R² = 0.01819
0
5
10
15
20
25
0 5 10 15 20 25 30 35 40
#ofDaysofFull-scaleResponse
# of Analysts on IR Team
More people, more
problems
Practically no correlation between having more people and
being able to responding faster
39. Training vs. Tools
Cost of Training an Analyst for
a small network – 10K hosts
SANS Course & Certification =
~$5,500
Travel & Meals = ~$1,500
Time Away from office =
~$1,750
Cost of OS IDS appliance(s) &
management servers =
$20,000
Cost of a commercial IDS
solution = ~$50,000 - $150,000
Cost of a commercial SIEM
product = ~$150,000-$200,000
Annual cost of MSSP services =
~$60,000-$120,000