This presentation "Threat hunting on the wire" is part of a a series of courses on the subject of Threat Hunting. It covers command-line packet analysis, and network forensics.
2. About me
• Joe McCray
• Deep Technical IT Security Consultant & Trainer
• Spoken/Trained at:
• Black Hat, Def Con, Hacker Halted, and over 200 security conferences
• Founder of InfoSecAddicts.com
• joemccray@infosecaddicts.com
3. About the Threat Hunting courses
• Course 1: Threat Hunting Fundamentals
• Course 2: Threat hunting on the wire (hands-on)
• Course 3: Threat hunting on the endpoint (hands-on)
• Course 4: Threat hunting with static analysis (hands-on)
• Course 5: Threat hunting with dynamic analysis (hands-on)
• Course 6: Threat hunting with memory analysis (hands-on)
• Course 7: Threat hunting with SIEM/NSM solutions (hands-on)
• Course 8: Advanced threat hunting with machine learning and artificial intelligence (hands-on)
4. Threat Hunting on the wire
• Get Linux
• Setting up your virtual machine
• What is PCAP?
• PCAP Analysis with PRADS
• PCAP Analysis with ChaosReader
• PCAP Analysis with TShark
• PCAP Analysis with Suricata
• PCAP Analysis with Yara
5. This is a HANDS-ON class
• This course is designed for you to follow along
• The slides can be found at: https://www.slideshare.net/infosecaddicts
• The commands can be found at: https://pastebin.com/DfqiGN7u
6. Get Linux
• Get a virtualization platform
• VMWare/Vbox
• OSBoxes.org
• Great site to download FREE Linux virtual machines (VMware and Virtualbox)
• Download my virtual machine
• https://s3.amazonaws.com/infosecaddictsvirtualmachines/InfoSecAddictsVM.zip
• user: infosecaddicts
• pass: infosecaddicts
• Great website for Linux basics
• Linuxsurvival.com
7. Setting up your virtual machine
• Default install of Ubuntu 16.04
• Lot of dependencies to install (run as root)
sudo apt-get install -y libpcre3-dbg libpcre3-dev autoconf automake libtool libpcap-dev libnet1-dev libyaml-dev libjansson4
libcap-ng-dev libmagic-dev libjansson-dev zlib1g-dev libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev cmake make
gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev unzip sendmail sendmail-bin prads tcpflow python-scapy
whois python-yara tshark
8. Setting up your virtual machine
• Install Suricata (run as root)
wget https://www.openinfosecfoundation.org/download/suricata-4.0.5.tar.gz
tar -zxvf suricata-4.0.5.tar.gz
cd suricata-4.0.5
./configure --enable-nfqueue --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
make install
make install-conf
mkdir suri
wget https://s3.amazonaws.com/infosecaddictsfiles/suspicious-time.pcap
cd rules
cp *.rules /etc/suricata/rules/
cd /etc/suricata/
wget https://rules.emergingthreats.net/open/suricata-4.0/emerging.rules.tar.gz
tar -zxvf emerging.rules.tar.gz
10. What Is PCAP?
• PCAP == Packet Capture
• Complete record of network activity
• Layers 2 – 7
• Most common format is libpcap
• Open-source
• Available on *nix and Windows
• C library, bindings in many languages
• Others proprietary formats not covered
13. PCAP Analysis with PRADS
• PRADS is a Passive Real-time Asset Detection System
PRADS employs digital fingerprints to recognize services on the wire, and can be used to map your network and monitor for changes in real time.
Real-time passive traffic analysis will also let you detect assets that are just connected to the network for a short period of time, since PRADS can glean useful
information from every packet.
PRADS aims to be the one-stop-shop for passive asset detection, and currently does MAC lookups, TCP and UDP OS fingerprinting as well as client and service
application matching and a connection state table. Various output plugins include logfile and FIFO and make PRADS a useful replacement for p0f, pads and sancp.
PRADS was built from the ground up for a small footprint and modern networks with IPv6 and gigabits of throughput.
Source: http://manpages.ubuntu.com/manpages/trusty/man1/prads.1.html
14. PCAP Analysis with PRADS
• Run PRADS as a regular user
cd ~/pcap_analysis/
mkdir prads
cd ~/pcap_analysis/prads
wget https://s3.amazonaws.com/infosecaddictsfiles/suspicious-time.pcap
prads -r suspicious-time.pcap
cat prads-asset.log | less
16. PCAP Analysis with ChaosReader
• What if you have to parse multiple large PCAP files
• Try chaosreader.pl (oldie but goodie)
• A free tool to trace TCP/UDP/... sessions and fetch application data from snoop or tcpdump logs. This is a type of "any-snarf" program, as it will fetch telnet sessions, FTP files, HTTP transfers
(HTML, GIF, JPEG, ...), SMTP emails, ... from the captured data inside network traffic logs. A html index file is created that links to all the session details, including realtime replay programs for
telnet, rlogin, IRC, X11 and VNC sessions; and reports such as image reports and HTTP GET/POST content reports
• Source: http://chaosreader.sourceforge.net/
• What can chaosreader do?
• I like being able to quickly go through really large, multiple, or even worse multiple large PCAP files.
• It also creates a down and dirty web page (really handy)
17. PCAP Analysis with ChaosReader
cd ~
mkdir -p pcap_analysis/chaos_reader/
cd ~/pcap_analysis/chaos_reader/
wget https://s3.amazonaws.com/infosecaddictsfiles/suspicious-time.pcap
wget https://s3.amazonaws.com/infosecaddictsfiles/chaosreader.pl
perl chaosreader.pl suspicious-time.pcap
cat index.text | grep -v '"' | grep -oE "([0-9]+.){3}[0-9]+.*)"
cat index.text | grep -v '"' | grep -oE "([0-9]+.){3}[0-9]+.*)" | awk '{print $4, $5, $6}' | sort | uniq -c | sort -nr
for i in session_00[0-9]*.http.html; do srcip=`cat "$i" | grep 'http: ' | awk '{print $2}' | cut -d ':' -f1`; dstip=`cat "$i" | grep 'http: ' | awk '{print $4}' | cut
-d ':' -f1`; host=`cat "$i" | grep 'Host: ' | sort -u | sed -e 's/Host: //g'`; echo "$srcip --> $dstip = $host"; done | sort -u
python -m SimpleHTTPServer
****** Open a web browser and browse the the IP address of your Linux machine port 8000 for the web page *****
22. PCAP Analysis with Suricata
• Suricata is a free and open source, mature, fast and robust network threat detection engine.
• The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing.
• Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats.
• With standard input and output formats like YAML and JSON integrations with tools like existing SIEMs, Splunk, Logstash/Elasticsearch, Kibana, and other database become
effortless.
• Suricata’s fast paced community driven development focuses on security, usability and efficiency.
• The Suricata project and code is owned and supported by the Open Information Security Foundation (OISF), a non-profit foundation committed to ensuring Suricata’s
development and sustained success as an open source project.
Source: https://suricata-ids.org/
23. PCAP Analysis with Suricata
• Run Suricata against the suspicious PCAP
cd ~/pcap_analysis/
mkdir suri
suricata -c /etc/suricata/suricata.yaml -r suspicious-time.pcap -l suri/
cat suri/fast.log | less
25. PCAP Analysis with Yara
• YARA is a tool aimed at (but not limited to) helping malware researchers to identify and classify malware samples.
• With YARA you can create descriptions of malware families (or whatever you want to describe) based on textual or binary patterns.
Source: https://virustotal.github.io/yara/
26. Isn’t Yara is for file analysis
• Yes, that’s right Yara is for file analysis
• Let me introduce you to YaraPCAP
• Reads a PCAP File and Extracts Http Streams.
• gzip deflates any compressed streams
• Scans every file with yara
• writes a report.txt
• optionally saves matching files to a Dir
Source: https://github.com/kevthehermit/YaraPcap
27. PCAP Analysis with Yara
• Run Yara against the suspicious PCAP
git clone https://github.com/kevthehermit/YaraPcap.git
cd YaraPcap/
wget https://github.com/Yara-Rules/rules/archive/master.zip
unzip master.zip
cd rules-master/
cat index.yar
clear
./index_gen.sh
cd ..
python yaraPcap.py rules-master/index.yar ../suspicious-time.pcap -s matching_files/
cd matching_files/
cat report.txt