2. Presentation will be available at:
www.misti.com/download
Download password is available in your Show Guide
3. Slide 3
Core Volatility developer
Co-Author “Art of Memory Forensics”
Lead-investigator on large-scale investigations
Performed many RE efforts, pentests, and source code
audits
Previously presented at Black Hat, RSA, Source,
DFRWS, BSides, and others
Who Am I?
4. Slide 4
(Brief) overview of traditional incident response settings
Challenges faced when traditional approaches are
applied to cloud environments
Overcoming the challenges
Leveraging unique features of the cloud for scalable and
effective incident response
Agenda
5. Slide 5
Focused on mostly static networks
IT has full control over system start, stop, reset,
refresh, etc.
Collection usually performed directly on affected
systems or at least on the same internal network
Traditional IR
6. Slide 6
Analysts have the ability to gather files, volatile data,
physical memory, and (full) disk images as needed
Analysts have full control over both the host and guest
virtual machines of servers
Logs are locally and easily accessible
Traditional IR Cont.
7. Slide 7
The cloud is not static
Systems may start and stop automatically in response
to processing load
Including systems that were or still are compromised
Volatile data is gone forever…
IT staff generally has little control over the architecture
and resource allocation
Traditional IR vs Cloud IR – Environment
8. Slide 8
Collection is done outside of the local environment,
over the Internet
Collect to storage within the cloud?
Secure credentials?
Cost?
Collect to the local environment?
Speed?
Capture in real-time vs in-cloud copies?
Cost?
Traditional IR vs Cloud IR - Collection
9. Slide 9
Acquiring traditional data sets is often difficult
Full disk images are usually impossible
Full memory captures possible, but chances of a
smeared image greatly increase with high system
activity
Number of systems that may be comprised can be
enormous
Live analysis tools trivially lied to by malware
Particularly on Linux
Traditional IR vs Cloud IR - Collection
10. Slide 10
Gathering logs faces many of the same issues as disk
and volatile data collection
In-cloud SIEM may prevent reasonable local download
of logs
Periodic transfer of logs from cloud to the local network
may leave gaps in real-time view
Traditional IR vs Cloud IR - Collection
11. Slide 11
You (or your client) generally have little to no control
over the VM host when using the cloud
This prevents acquisition of data from guests through
the host
This necessitates the use of software within the guest
to acquire data
Traditional IR vs Cloud IR – VM Control
12. Slide 12
During an incident is a bad time to move acquisition
tools to system(s)
Many fully automated deployments don’t enable SSH
Administrators may have no remote access to the
system
What then?
Agents?
“Backdoor” to enable remote administration?
Traditional IR vs Cloud IR – Acquisition Tools
13. Slide 13
Incident response needs must be considered at all
stages of development, deployment, and ongoing
operations
The goal of these efforts is to enable effective and
immediate response as well as ongoing detection of
threats
Richard Mogull has done great work in this space
related to application and network security
https://securosis.com/blog
Making Cloud IR Seamless
14. Slide 14
Applications should be verified that all relevant logging
features are enabled
In-house applications should be built with detailed
logging built-in and enabled
This includes every action that you as an
investigator might want to later know about
Malicious insiders and remote attackers should never be
able to use your own app against you and you not be
able to later track down exactly what they did
Making Cloud IR Seamless – App Dev
15. Slide 15
As systems are built, automated forensics tools should
be used to base line the system’s “normal” state
Both on-disk and in-memory artifacts
Prevent guessing during incidents
Immediately pinpoint suspicious artifacts
Systems should be checked to ensure that all relevant
logging is enabled
Making Cloud IR Seamless – Pre-Deployment
16. Slide 16
Tools required for collection of forensics artifacts need
to be installed with the base system
How to collect if entirety of disk is not acquirable?
“Select” files
What to do with memory?
Acquisition of artifacts through APIs is vulnerable to
malware interference
“Live” memory forensics isn’t
A good compromise when you can’t get a full sample
of RAM
Making Cloud IR Seamless – Enabling Collection
17. Slide 17
If the system can be automatically spun down, ensure
the logging is remote
Scalable, remote logging is preferred in most cases
even if the system is stable
Have automated methods to gather data of interest
Be proactive about finding threats – don’t wait for
signatures (AV, HIPs, IDS) to fire!
Making Cloud IR Seamless – Post-Deployment
18. Slide 18
Required in both traditional and cloud environments
Over 60% of breaches were “discovered” after 3rd party
notification
Existing technology will only catch skilled adversaries if
they make a mistake
Proactive Incident Response - Motivation
19. Slide 19
Constantly gather and evaluate system state
Processes
Network connections
AutoRun locations
… many more data points
Compare current state to baselines
Use IOCs, threat intel data, etc. to find known badness
Proactive Incident Response – Howto
20. Slide 20
Leverage IR-only credentials
Leverage IR-only instances
Stop any auto termination of (potentially) affected
hosts
Use automated scripts to gather as much data as
possible
Leverage features of the cloud to enhance response
and minimize disruption
Making Cloud IR Seamless – Active Incident
21. Slide 21
While IR in the cloud has many challenges, it also has
unique features that can be very beneficial
When used correctly, large-scale, automated detection
and collection becomes possible
Leveraging the Cloud for Better IR
22. Slide 22
Pre-built instances that have the tools (software) and
storage needed to support IR
No need to configure and install tools during an
incident
Removes bottlenecks related to people power as well
as processing power
Can use credentials separate from the rest of the
environment
IR-Only Instances
23. Slide 23
Production instances are often under medium to heavy
load
This pollutes forensics data and makes live analysis
challenging
Fix:
Isolate (potentially) affected instances
Spin up new production instances to replace
compute power
Benefits:
More time to gather data in a stable manner
No adverse effects on customers or performance
Virtual Guest Isolation
24. Slide 24
Can inspect the state of VM guests without direct
interaction
Avoids the issue of malware interference or notifying
attackers of forensics activity
Much simpler to automate and scale
Collected data can be safely stored on the VM host until
needed
A huge security boost to private clouds and managed
security from public providers
Virtual Machine Introspection
25. Slide 25
Snapshots include both volatile memory (RAM) and the
file system (disk)
The guest cannot detect itself being snapshotted*
Again - no chance for malware interference or
attacker notification
Can periodically snapshot and keep for days or weeks
after
Determine exact time of infection and state changes
since then
Virtual Machine Guest Snapshots
26. Slide 26
Coming from traditional IR settings, the cloud can be
quite challenging
Pre-planning is required to effective
Agreed upon processes to capture and analyze data
Pre-allocation of resources
Full-scale exercises to test all points of response
Automated as much as possible
Continuous threat hunting
The cloud also provides unique features that, if
leveraged properly, can make IR much more effective
Summary