CS266 Software Reverse Engineering (SRE) Identifying, Monitoring, and Reporting Malware Teodoro (Ted) Cipresso, teodoro.cipresso@sjsu.edu Department of Computer Science San José State University Spring 2015

1. CS266 Software Reverse Engineering (SRE)
Identifying, Monitoring, and Reporting Malware
Teodoro (Ted) Cipresso, teodoro.cipresso@sjsu.edu
Department of Computer Science
San José State University
Spring 2015
The information in this presentation is taken from the thesis “Software reverse engineering education”
available at http://scholarworks.sjsu.edu/etd_theses/3734/ where all citations can be found.

2. 2
Identifying, Monitoring, and Reporting Malware
What Qualifies as Malware?
 Malware describes a category of software that doesn’t always operate in a way
that benefits the user.
 Of course, those of us who have ever used software might contend that this
definition of malware will cause programs that we use every day to be
categorized as malware.
 So let's qualify it a bit: the malicious or annoying behaviors of malware are
intentional, not the result of one or more bugs.

3. 3
Identifying, Monitoring, and Reporting Malware
Types of Malware
 There are several types of malware that affect computer systems [6] [7]:
 Viruses: require some deliberate action to help them spread.
 Worms: similar to a virus but can spread by itself over computer networks.
 Trojan Horses: performs hidden malicious or annoying operations.
 Backdoor: a vulnerability purposely embedded in software.
 Rabbit: a program that exhausts system resources.
 Ransomware: lock computer files, victim has to pay to unlock.
 Criminalware: Steal sensitive information.

4. 4
Identifying, Monitoring, and Reporting Malware
Prevalence of Malware Types
 Malware usually isn't of just one type; for example, 4 of the top 10 malicious
codes families reported in 2011 were Viruses with a Worm component.
 Using the machine code and bytecode reversing experiences gained from the
previous modules, one could attempt to reverse malware.
 Using virtualization tools such as VMWare or Virtual Box to create secondary
operating system images (Guests) on which to analyze malware can still result
in infection of the primary operating system (Host).
 Great care should be taken to isolate guest OSes from their host OS.
 Networking, removeable storage, clipboard usage, etc…

5. 5

6. 6
Identifying, Monitoring, and Reporting Malware
Safe & Practical Malware Reversing
 We want to become familiar with using tools to identify, monitor, and report
software that might be malicious.
 Reversing malware directly is especially challenging because several anti-
reversing techniques will have been applied to the code.
 Given that unexpected catastrophes can arise when installing a virus, worm,
backdoor, etc… for academic purposes; we could still learn something from
working with contrived or benign “malware”.
 In 1996, Mark Russinovich founded a company called “Winternals Software”
where he was the chief software architect on a comprehensive suite of tools for
diagnosing, debugging, and repairing Windows® systems and applications [9].

7. 7
Identifying, Monitoring, and Reporting Malware
Windows Sysinternals
 Mark's company was purchased by Microsoft and the suite of tools have been
rebranded as Windows Sysinternals which are are offered for free.
 Mark's story is an interesting one because he is recognized as an expert on the
internals of Windows even though he did not participate in its development—a
true testament to what can be learned about software through reversing.
 The Sysinternals suite contains 69 utilities, but we’ll focus on just one.

8. 8
Identifying, Monitoring, and Reporting Malware
Sysinternals Process Monitor
 The Process Monitor can capture detailed information about a process in a
Windows system including file system, registry, and network activity.
Process Monitor session for the Password Vault CPP application.
File system
activity

9. 9
Identifying, Monitoring, and Reporting Malware
Sysinternals Process Monitor (cont’d)
 The Process Monitor can capture detailed information about a process in a
Windows system including file system, registry, and network activity.
Process Monitor session for the Password Vault CPP application.
Network
activity

10. 10
Identifying, Monitoring, and Reporting Malware
Sysinternals Process Monitor (cont’d)
 The Process Monitor can capture detailed information about a process in a
Windows system including file system, registry, and network activity.
Process Monitor session for the Password Vault CPP application.
Registry
Activity

11. 11
Identifying, Monitoring, and Reporting Malware
Sysinternals Process Monitor (cont’d)
 Process Monitor itself does not detect or identify malware. It simply monitors
and records what processes are doing.
 With a little bit of ingenuity, one can identify a software Trojan by looking for
activities that don't seem to fit with the advertised functionality of a program.
 It's common practice to download free software from the Internet:
 Some believe that open-source software, should have the fewest number of
vulnerabilities. The more eyes the better, right?
 Becoming familiar with the Sysinternals suite can help you evaluate whether
the software on your Windows machine is acting in your best interest.

12. 12
Identifying, Monitoring, and Reporting Malware
Benign Malware Exercise
 The Alarm Clock application is a benign software Trojan that, in addition to
being a rudimentary alarm clock, performs unadvertised functions on
background threads:
 Logs information from the Windows® registry
 Logs locations of “office” documents in the file system.
 Scans for computers that respond to an ICMP ping.
 Paced background threads are used.

13. 13
Identifying, Monitoring, and Reporting Malware
Benign Malware Exercise (cont’d)
Background threads log information about the user’s system.

14. 14
Identifying, Monitoring, and Reporting Malware
Is Open Source More Trustworthy?
 The data on the number of vulnerabilities found in the 5 most popular Internet
browsers does not support the proposition that open source is more secure.
 Big 5: Google Chrome, Mozilla Firefox, Internet Explorer, Opera and Safari.
 Mozilla Firefox was affected by 270 new vulnerabilities in 2013, more than any
other browser; 245 new vulnerabilities were found in Google Chrome, 126 in
Internet Explorer, 75 in Apple Safari, 11 in Opera [Secunia].
 The two browsers containing the most open source (Chrome based Chromium,
Firefox based on Mozilla), have the most vulnerabilities…
 Of course we need temper this judgement with the observation that popular
software is targeted more often.

15. 15
Identifying, Monitoring, and Reporting Malware
Reporting Suspected Malware
 If you suspect a particular program to be malware, it can be submitted to
online threat analysis services such as ThreatExpert or Virus Total.
 ThreatExpert and Virus Total are Web-based tools that support submission of
suspicious executables or URLs to detect possible malware.
 Both services match against databases of existing malware, however
ThreatExpert (itself) attempts to execute binaries in an isolated environment to
perform heuristic detection of malware.

16. 16
http://www.threatexpert.com/submit.aspx
http://www.threatexpert.com/report.aspx?md5=acdd4c2a377933d89139b5ee6eefc464
Heuristic anlaysis
components

17. 17
44 out of 56
antiviruses detect
this as malware

18. 18
Identifying, Monitoring, and Reporting Malware
Setting up a Lab for Analyzing Malware
 Each of you have been assigned your own VMWare image (info on Canvas).
 The images are only accessible through VMWare’s built-in VNC server.
 The images are on a virtual network and have no connectivity to the Internet
or the Host’s network. This is to prevent:
 Infection of the Host (primary OS), worms from spreading*
 downloading of additional threats,
 transmission of sensitive data to hacker sites.
 Virtualized Network Isolation for a Malware Analysis Lab
 https://zeltser.com/vmware-network-isolation-for-malware-analysis/

19. 19

20. 20

21. 21