This thesis describes the analysis of 18 personal firewalls. It discovers the differences in their behaviour while they are under various techniques of port scanning and Denial of Service (DoS) attacks. With port scanning, the detection ability, time consumption, leaked port states and obfuscation techniques are analysed. With using different DoS attacks, performance measurements of CPU and network adapter are taken. The potential of firewall fingerprinting based on the different behaviour across multiple products is also addressed.

1. MASARYK UNIVERSITY
FACULTY OF INFORMATICS
}w¡¢£¤¥¦§¨!#$%123456789@ACDEFGHIPQRS`ye|
Comparative Analysis of Personal
Firewalls
MASTER THESIS
Bc. Andrej ˇSimko
Brno, January 2015

2. Declaration
Hereby I declare, that this paper is my original authorial work, which I have worked
out by my own. All sources, references and literature used or excerpted during elabo-
ration of this work are properly cited and listed in complete reference to the due source.
Supervisor: Mgr. V´ıt Bukaˇc
ii

3. Acknowledgement
I want to express my sincere gratitude to V´ıt Bukaˇc. Without his guidance, this work
would have never been brought to the level you see now. I could not have imagined a
better advisor.
I also want to thank my family for their constant support in everything - my deci-
sion so far and studies alike.
iii

4. Abstract
This thesis describes the analysis of 18 personal firewalls. It discovers the differences in
their behaviour while they are under various techniques of port scanning and Denial
of Service (DoS) attacks. With port scanning, the detection ability, time consumption,
leaked port states and obfuscation techniques are analysed. With using different DoS
attacks, performance measurements of CPU and network adapter are taken. The poten-
tial of firewall fingerprinting based on the different behaviour across multiple products
is also addressed.
iv

5. Keywords
firewall, comparative analysis, port scan, denial of service, flooding attacks, finger-
printing, port TCP/0
v

6. Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Personal firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 Packet filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Stateful inspection firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Unified Threat Management (UTM) and Next Generation firewalls . . . 3
2.4 Testing - Proactive Security Challenge 64 . . . . . . . . . . . . . . . . . . 3
3 Attacks by types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1 Port scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1.1 Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1.2 Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1.3 Port scanning attack . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.4 Port scan attack techniques in Nmap . . . . . . . . . . . . . . . . . 7
TCP SYN scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
TCP connect() scan . . . . . . . . . . . . . . . . . . . . . . . . . . 8
TCP FIN scan, TCP Xmas scan, Null scan . . . . . . . . . . . . . 10
TCP ACK scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
TCP Window scan . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
TCP Maimon scan . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
UDP scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
SCTP INIT scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
SCTP COOKIE ECHO scan . . . . . . . . . . . . . . . . . . . . . 11
IP protocol scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Service/Version detection scan . . . . . . . . . . . . . . . . . . . 12
3.1.5 Other Nmap options . . . . . . . . . . . . . . . . . . . . . . . . . . 12
-6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
-O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
-p port range . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
--top-ports count . . . . . . . . . . . . . . . . . . . . . . . . . . 13
--mtu mtu number . . . . . . . . . . . . . . . . . . . . . . . . . 13
--scan-delay time . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.6 Ping scan (ICMP echo request) . . . . . . . . . . . . . . . . . . . . 14
3.2 Denial of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2.1 Hping3 tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2.2 Low Orbit ION Cannon (LOIC) . . . . . . . . . . . . . . . . . . . . 16
3.2.3 IPv6 Router Advertisement (ICMPv6 type 134, code 0) . . . . . . 16
3.2.4 IPv6 neighbor Advertisement (ICMPv6 type 136, code 0) . . . . . 17
4 Experiment description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Virtual environment . . . . . . . . . . . . . . . . . . . . . . . . . 18
Physical environment . . . . . . . . . . . . . . . . . . . . . . . . . 18
vi

7. 4.1 Choosing particular firewalls . . . . . . . . . . . . . . . . . . . . . . . . . 19
Firewall settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 Port scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Logging port scanning attacks . . . . . . . . . . . . . . . . . . . . 21
Port TCP/0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Detection thresholds . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.3 DoS attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.3.1 DoS results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5 Fingerprinting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1 Using time differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2 Using port states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6 Ideal behaviour of firewall under certain attacks . . . . . . . . . . . . . . . . 48
6.1 Ideal port scanning behaviour of a firewall . . . . . . . . . . . . . . . . . 48
6.2 Ideal behavior under the DoS attacks . . . . . . . . . . . . . . . . . . . . . 51
7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
7.1 Future improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
A List of attachments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
B Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
C Additional figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
vii

8. Chapter 1
Introduction
Personal firewall is a need-to-have requirement on every machine to defend against
various network attacks. In the recent years, specialized firewall applications were re-
placed by all-in-one security suites. All major antivirus companies have incorporated
firewall into their products to facilitate a better protection for their products. There are
not that many comparative analyses of firewalls for users to see which products have
higher score than others. One notable example of a test suite is the Proactive Security
Challenge 64 [21], but it doesn’t perform tests I incorporated into this thesis. See Chap-
ter 2.4 for more details.
Zero-day exploits are still on the rise. For the attacker, the precise knowledge which
endpoint-protection system is installed on the victim’s device is invaluable informa-
tion. Especially if he wants to avoid detection, or use certain exploits which can be ex-
ploited on particular security suites. Therefore the ability to fingerprint firewalls comes
handy. I didn’t find any research to perform firewall fingerprinting using port scanning
attacks, so I created one.
Although the Denial of Service (DoS) attacks usually target servers, they can also be
launched against the entire networks. If that happens, it is interesting to observe how
personal firewalls behave when they are under different kinds of DoS attacks. In the
second chapter, the general history and the principles of how firewalls work will be de-
scribed. The third chapter will focus on describing the theory behind the attacks used
in this thesis. Different port scanning techniques and DoS attacks will be described
closely. The fourth chapter will elaborate on the preparation of environment for testing
purposes, mention important differences observed between various firewall brands,
give example of detection thresholds of triggering alarms of the attacks and show the
results from DoS attacks observed on the performance of the victim’s computer. The
fingerprinting will be illustrated in the fifth chapter, along with a few examples of de-
termining which firewall is installed on the victim’s computer without any previous
knowledge. The results from all port scanning attacks are shown in tables with all the
port states, as well as the timings of each portscanning attacks. Interesting statistical
observations are also pointed out. In the chapter six, the description of how the ideal
firewall should behave is outlined. This is important to note as it would successful
counter possible fingerprintings by using port scan attacks. The final chapter describes
possibilities for the future improvements as well as summarizes the most important
findings and observations of this work.
1

9. Chapter 2
Personal firewalls
Firewalls filter the traffic into or out of a network, based on certain rules. Personal
firewalls are host based - they provide protection for a single operating system. Based
on certain rules set by the administrator, they can either allow or deny certain con-
nections. There are two possible approaches - implicit allow and implicit deny default
policies. For the security reasons, the whitelisting approach is highly recommended -
what is not allowed is denied by default. When new packet/session/connection is re-
ceived by the firewall, it usually goes through filtering rules based on their order in the
list. Suppose we have a rule “accept all packets from 192.168.20.0/24” at the top of the
list, followed be a “drop all” rule. When packet arrives, its header is examined by the
firewall and checked against this ruleset. For example, if the packet comes from the IP
address 192.168.20.2, it is allowed. If it comes from 192.168.16.32, it is denied.
2.1 Packet filters
A packet filter examines the header of each packet individually and can filter traffic
based on simple rules. IP addresses, subnet addresses, ports, and protocols are used
for creating filtering rules in the ACL (Access Control List). By combining these rules,
only the TCP (Transmission Control Protocol) traffic that originates from the IP address
within my Local Area Network (LAN) which comes to a port 25 can be allowed. The
packet filter is handy to have on routers, but is almost useless to have on personal com-
puter. For once it needs configuring by more advanced user, and for second it is not
sufficient for the end stations today. It usually requires configuring long ACLs, which
takes high amount of time and needs certain skills. However it can still become handy
even for end stations. For example during DoS attack to a certain port or from a certain
IP address. It can effectively leverage this thread, if configured properly. Second use-
case could be blocking regular port scanning attacks from a certain address. Blocking
all unused ports is highly recommended approach of using the packet filter. This type
of firewall is also called “stateless” as it can’t determine whether the packet is a part of
an ongoing session. It understands only link layer, internet layer and transport layer of
the TCP/IP protocol stack.
2.2 Stateful inspection firewall
Instead of inspecting each packet individually, the stateful firewall monitors the overall
session. With the TCP, the session must begin with a 3-way handshake - see Figure 2.1.
2

10. 2. PERSONAL FIREWALLS
Figure 2.1: 3-way TCP handshake [2]
After the session is established, the firewall can monitor its status and traffic. Stateful
Packet Inspection (SPI) is aware of who created the connection and it can determine if
packet is either the start of a new connection, part of an existing connection, or is an in-
valid packet. It can allow only inbound TCP packets that are generated in response to
a connection initiated from inside of the internal network. Dynamic ports are opened
based on the needs of the connections and when the session is complete, firewall can
close these no longer needed ports. As with the packet filters, the stateful inspection
firewall doesn’t understand the application layer of TCP/IP protocol suite.
2.3 Unified Threat Management (UTM) and Next Generation
firewalls
The firewall is no longer a specialized piece of software. It became a part of UTM - an
integration of application control, user awareness, Intrusion Detection System (IDS),
Intrusion Prevention System (IPS), antivirus system, antispam system, content filters,
deep packet analysis, anomaly detection system...It understands data on the appli-
cation layer of TCP/IP and can block inappropriate content (e.g. vulnerabilities, web-
sites, viruses...). Evolutionary, the UTM was followed by proposition of Gartner who
defined the next step in firewall evolution in [1]. Today, these two stages are merging
into one another. They both focus on easy and effective management of the entire secu-
rity system, because it is perceived to be the most common security problem. Gartner
in the research paper [29] stated: “Through 2018, more than 95% of firewall breaches
will be caused by firewall misconfigurations, not firewall flaws”.
As the firewall understands the application layer, the rules can be set for particular
applications: “Allow Skype for Alice” or “Deny Facebook for Bob”.
2.4 Testing - Proactive Security Challenge 64
Probably the most known test suite of application-based security on Windows can be
found on www.matousec.com. It examines products called Internet security suites,
personal firewalls, Host Intrusion Prevention Systems (HIPS)...It entails 11 testing lev-
els, each consisting of 10 tests. 38 firewalls are ordered based on their score in all 110
test rated from 0% to 100%. All the source codes are available on their website. The
individual tests are divided into 4 main categories:
3

11. 2. PERSONAL FIREWALLS
Product Score
Agnitum 90%
Avast! 8%
AVG 7%
Avira 9%
Bitdefender 19%
COMODO 97%
Emsisoft -
ESET 67%
F-Secure 6%
Gdata -
Kaspersky 89%
McAfee 3%
Microsoft -
Norton 9%
Panda 1%
Quick Heal -
TrustPort 8%
ZoneAlarm 34%
Table 2.1: Firewall score in Proactive Security Challenge 64
• Leak tests attempt to send data to the Internet server.
• Spying tests are using keyloggers or packet sniffers to spy on user’s inputs or
data.
• Autorun tests are trying to install themselves persistently so that they would
remain active after the reboot.
• Self-defense tests are attempting to terminate security product processes or threads,
and remove, destroy or corrupt critical objects for that security product.
As the reader can observe, these categories are not directly testing firewalls themselves
but rather the overall endpoint security protection systems. There are no network at-
tacks included in these tests. In my thesis, I’ve only done tests which focus on fire-
walls. For the completeness of this research, see Table 2.1 with product score of fire-
walls which I used in my tests and their score in Proactive Security Challenge 64[21].
Some firewalls I tested were not tested by the Proactive Security Challenge 64, hence
there is “-” sign in the table.
4

12. Chapter 3
Attacks by types
3.1 Port scanning
3.1.1 Ports
After we are provided with host-to-host delivery by the network layer of TCP/IP stack,
we need to address the process-to-process connectivity in the transport layer, so that
application would also be addressable. This is done by assigning different port num-
bers to the host. The particular process/application is uniquely determined by the IP
address with port number (this touple is together called the socket) and the transport
layer protocol (TCP, UDP, SCTP and DCCP). A port is uniquely identified by a 16-bit
long port number (0-65535), which is stored in segment’s header (see Figure 3.1)
Ports can be divided in 3 categories (set by RFC 6335 [8]):
• System (well-known) ports (0-1023)
• User (registered) ports (1024-49151)
• Dynamic (private) ports (49151-65535)
Since November 1977 [5], the Internet Assigned Numbers Authority (IANA) - an orga-
nization for assigning IP addresses, AS (Autonomous System) numbers, port numbers
and others; have been periodically updating tables of port numbers in the form of RFC
documents. In January 2002, the last RFC 1700 [5] was rendered obsolete by RFC 3232
[6] which officially stated that all future changes will be made available in the online
database on www.iana.org. Today, the official list can be found here: [4]. IANA can
only assign system and user ports, never dynamic ports. Some of the examples of ports
that are mentioned in this thesis are: 135 (TCP - msrpc), 445 (TCP - microsoft-ds) or 3389
(TCP - ms-wbt-server).
3.1.2 Flags
In the TCP header (see Figure 3.1), apart from other fields, there are six 1-bit fields
indicating flags:
• URG - Urgent - packet must be processed urgently
• ACK - Acknowledgment - for segment that has been successfully received
• PSH - Push - the receiver should immediately push data to the upper level
• RST - Reset - hard termination of the connection
5

13. 3. ATTACKS BY TYPES
Figure 3.1: TCP Header [15]
Figure 3.2: 4-way TCP handshake [3]
• SYN - Synchronize - for setting up the connection and synchronizing sequence
numbers
• FIN - Finish - no more data from sender, connection tear-down
TCP has to establish the connection first and send data later - for this purpose, the 3-
way handshake needs to occur prior to sending data (see Figure 2.1). For the connection
tear-down, 4-way communication is needed (see Figure 3.2).
3.1.3 Port scanning attack
Port scanning attack is a process of probing host for open ports. Using various tech-
niques, the attacker/administrator/penetration tester is able to differentiate between
various states of ports: open, closed, open|filtered (where “|” means logical “or” and
is used when the particular technique can’t differentiate between these two states),
6

14. 3. ATTACKS BY TYPES
closed, and filtered.
Port scans can be divided into horizontal and vertical scans. Horizontal scan is scan-
ning of single port on many victims, whereas vertical means scanning many ports on
a single victim. In my research I was scanning only 1 victim at a time, thus I will only
describe vertical scans.
Since the early days, this particular attack have been used to discover vulnerable
systems that can be potentially attacked or exploited. One could object that over the
time, many countermeasures and security devices/features have been implemented
that would stop this thread - firewalls, Intrusion Detection Systems (IDS), Intrusion
Prevention Systems (IPS), Network Address Translation (NAT), or proxy servers. The
port scanning attack still poses great threat nowadays - for example the HACIENDA
program of NSA/GCHQ [12]. In this document they describe common ground for
scanning entire countries and sharing the results between agencies of United States,
Canada, United Kingdom, Australia and New Zealand.
Port scans are not only used by government agencies, they are often used by crimi-
nals or hackers. Especially when they possess a knowledge of some 0-day vulnerability
and the knowledge from port scans, they can attack their victims.
One of the studies which can demonstrate the scale of today’s port scanning attacks
is the paper “An Internet-Wide View of Internet-Wide Scanning” [22]. The authors
have shown, that scans of the entire IPv4 address space are quite common nowadays
- by legitimate researchers, security companies, and attackers alike. According to their
words: “Internet-scale horizontal scans have become common”. High speed scanning
of the entire IPv4 address space, and thus public Internet addresses, was made possible
with introduction of two open-source utilities in 2013: Zmap [23] and Masscan [24]. The
time required for launching such an Internet-wide scan attack is around 44 minutes for
Zmap [22] and under 3 minutes for Masscan [24]. Observations on real networks were
made, that within hours or days after a new vulnerability is discovered (e.g. Linksys
backdoors, Heartbleed or NTP DDoS attacks), there is an obvious spike in scanning of
vulnerable ports.
3.1.4 Port scan attack techniques in Nmap
Nmap (network mapper) is an open source cross-platform tool for network discovery
and security auditing. It is fully capable of both horizontal and vertical scans. I chose
to use Nmap for testing purposes, because it has many options which can be used for
scanning particular host and evading detections by firewalls on the victim. See Table
3.1 for the list of nmap’s scanning techniques with used commands and possible port
states. See Figure 3.3 for graphical interpretation of Nmap scanning techniques de-
scribed later.
I stored every command I ran through Nmap into a separate txt file (see files on
CD for more information), so that I could later analyze the results. All the important
information, like how many ports are in open state, which port states differ from all
7

15. 3. ATTACKS BY TYPES
Scanning technique Nmap command Possible states of ports
TCP SYN -sS open, closed, filtered
TCP connect() -sT open, closed, filtered
TCP FIN, TCP Xmas, TCP Null -sF, -sX, -sN open|filtered, closed, filtered
TCP ACK -sA unfiltered, filtered
TCP Window -sW open, closed, filtered
TCP Maimon -sM open|filtered, closed
UDP scan -sU open, open|filtered, closed, filtered
SCTP Init -sY open, closed, filtered
SCTP COOKIE ECHO -sZ open|filtered, filtered
IP protocol scan -sO open, open|filtered, closed, filtered
Service and Version detection -sV open, closed, filtered
Table 3.1: List of Nmap techniques
others, as well as the time consumption of particular scan with particular parameters
can be used for further analysis from the log files. In every directory (with scanning
the particular firewall) there are at least 33 log files from different scans.
The importance of Nmap can be seen simply by looking at its occurrence on many
security conferences, such as “Let’s Screw With Nmap” on Defcon 21 [11], “Network
Anti-Reconnaissance: Messing with Nmap Through Smoke and Mirrors” on Defcon 20
[10], or many others.
TCP SYN scan
This is probably the most common and the most used scanning technique (it is also the
default setting) for Nmap tool. It never completes the TCP handshake (which makes
it stealthy) because Nmap resets the connection before it can be completed. The at-
tacker sends only TCP SYN segments and never responds to SYN+ACK segments.
After sending the TCP SYN segment, the response (from the victim) can either be
SYN+ACK if the port is opened; RST if the port is closed; or getting the ICMP un-
reachable error or no response at all if the port is filtered. The only problem with TCP
SYN scan can be the need of root privileges. As this is now widely used technique,
many network protection systems detects these types of scans.
TCP connect() scan
Tries to establish the entire TCP connection and finish the 3-way handshake (after
which RST segment is immediately sent back) by using the connect() function of op-
erating system. It thus doesn’t need the privileged access to be run. As with the TCP
SYN, closed port is determined by receiving only RST segment; opened port is deter-
mined by SYN+ACK segment (to which Nmap sends ACK and RST segments); and
filtered port is determined if nothing is received. This technique is the most obvious
and the most easy to detect.
8

16. 3. ATTACKS BY TYPES
Figure3.3:GraphicalrepresentationofNmapscanningtechniques[2]
9

17. 3. ATTACKS BY TYPES
TCP FIN scan, TCP Xmas scan, Null scan
These are more stealthier methods than TCP SYN scan, because they do not even at-
tempt to create a handshake. They just send 1 segment that would probably never
occur in the real world (except from FIN scan). According to RFC 793 [9] “If the [des-
tination port] state is CLOSED then all data in the incoming segment is discarded. An
incoming segment containing a RST is discarded. An incoming segment not containing
a RST causes a RST to be sent in response”. On the next page it is stated that if segments
are sent to open ports, but do not contain RST, SYN or ACK, then “you are unlikely
to get here, but if you do, drop the segment, and return”. In other words, when host
receives information on the closed port (and it doesn’t contain RST), it should respond
with RST segment; and if the port is opened (and neither one or RST, SYN or ACK
is present) it should not send anything. If ICMP unreachable error is generated, port
is marked as filtered. Nmap thus uses either “closed”, “open|filtered”, and “filtered”
information. Although the exact response should be operating system specific, I dis-
covered that there are slight differences when using different firewalls.
Differences in these techniques are as follows:
• FIN: only FIN flag is present (6 bits: 000001)
• Xmas: FIN, PSH and URG flags are present (6 bits: 101001)
• Null: no flag is present (6 bits: 000000)
Unfortunately, not all operating systems follow RFC 793. Some of them (Microsoft Win-
dows, many CISCO devices, and few others) “send RST responses to the probes regard-
less of whether the port is open or not” [7]. This results in all ports being marked as
closed.
Such a behaviour was observed on IPv4 mainly in Emsisoft and Kaspersky fire-
walls - most of their ports were marked as “closed”. With other firewalls on IPv4,
the majority of ports were marked as “open|filtered”. On IPv6, this changed rapidly
- Panda, TrustPort and ZoneAlarm marked all 1000 scanned ports as “closed”. Based
on these observations, operating system can’t be deduced just by observing different
port states, as firewall changes the default behaviour of Windows (and most likely also
other) operating systems.
TCP ACK scan
Although the TCP ACK scan doesn’t determine whether the port is opened (or even
open|filtered), it can be useful to determine if firewall is stateful or not and which ports
are filtered. If the scanned system is unfiltered, both open and closed ports should
return RST segment (in which case Nmap marks them as unfiltered). If there is no
response, or ICMP error message occurs, they are labeled as filtered.
TCP Window scan
Is exactly the same as the TCP ACK scan, but uses exploitation of implementation
details to determine between open and closed ports. When RST segment is returned, it
10

18. 3. ATTACKS BY TYPES
might happen that Window field size can be either zero or positive number. If it is zero,
ports are usually closed; and if it is a positive number, they might be open. The results
from my research however indicate that this isn’t pure operating system specific, and
a few firewalls behave differently than they should.
TCP Maimon scan
Named after its discoverer, Uriel Maimon, this scanning method should be BSD-like
system specific. It is the same as TCP FIN, TCP Xmas and TCP Null with one difference
- FIN+ACK is used in probes. According to the RFC 793 [9], the RST segment should be
generated to FIN+ACK probe; which is not always true and some operating systems
simply drop the segment if port is open.
UDP scan
This is the only option in Nmap which is able to scan UDP ports. Since UDP is a
connection-less service, there are no flags in the UDP header. In fact, the UDP header is
designed to be as small as possible and has only source and destination IP addresses,
length and checksum. For some common ports (e.g. 53 - DNS and 161 - SNMP) Nmap
sends protocol-specific payload, and for all other ports the data part is empty. If the
response is ICMP port unreachable error (type 3, code 3), then the port is closed. If
different ICMP unreachable error (type 3, codes 1, 2, 9, 10 or 13) [13] is generated, port
is marked as filtered. If UDP packet is generated as a response, port is marked as open;
and if no response at all is generated, port is classified as open|filtered.
SCTP INIT scan
The Stream Control Transmission Protocol (SCTP) is rather new protocol defined in
RFC 4960 [7]. This particular INIT scan is an equivalent to TCP SYN scan, because it
never creates full connection. Nmap is able to scan 52 SCTP ports, which are stored
in the “nmap-services” file in the nmap installation directory. Note that according to
IANA, there are 65 ports allocated to SCTP protocol [4].
Although it is unlikely that normal end user station would use any of these ports,
I tried to test it anyway. This was the only portscan technique in which all firewalls
shared exactly the same port state results - all 52 were reported as filtered on both
IPv4 and IPv6 scans. The only difference with IPv4 was negligible - with Panda and
Kaspersky it took 2.13s, whether on all other firewalls it took exactly 2.34s. The same
time consumption result was observed with SCTP COOKIE ECHO scan.
SCTP COOKIE ECHO scan
More advanced variant of SCTP scanning exploits the fact, that implementation should
drop packets containing COOKIE ECHO chunks on open ports, but send ABORT seg-
ment if the port is closed. It can’t differentiate between open and filtered ports, but can
identify closed ports. Nmap is again able to scan 52 ports and unlike the SCTP INIT
scan, there are visible differences in port states across firewalls on IPv4 - Kaspersky and
11

19. 3. ATTACKS BY TYPES
Panda have 2 and 3 ports marked as filtered, respectively. All other firewalls have all
52 ports in open|filtered state. Apart from the same time difference with the same fire-
walls, McAfee’s scan took 2.61s to finish. With IPv6 scanning, results were even more
rigid - only 9 firewalls reported all 52 ports as open|filtered.
IP protocol scan
Up until now every scan was probing particular ports on TCP, UDP or SCTP protocols.
On the other hand, IP protocol scan is able to determine which IP protocols (ICMP,
IGMP, TCP, UDP, SCTP, ...) are supported by the target machine. The scan iterates
through 8-bit IP protocol field in the IP header (thus scanning for 256 different protocol
numbers). If Nmap receives any response from protocol from scanned host, it marks
protocol as open. If an ICMP protocol unreachable message (type 3, code 2) is gener-
ated, the protocol is marked as closed. If other ICMP error message is generated (type
3, codes 1, 3, 9, 10, or 13) [13] then the protocol is marked as filtered. If no response is
generated then the protocol is marked as open|filtered.
Service/Version detection scan
Normally, it is almost certain that if port 80/TCP is open, the http runs on it, or if
25/TCP is opened, smtp service runs on it. Such a default behaviour is however not
always the case. This is when the Service and version detection option in Nmap comes
handy. Thanks to the vast database, it can also differentiate the version number of
particular services. Nmap first tries to determine the service protocol (e.g. HTTP, SSH,
FTP), the application name (e.g. Solaris telnetd, Apache httpd), the version number,
hostname, device type (e.g. printer, router), the OS family (e.g. Windows, Linux) [14].
3.1.5 Other Nmap options
-6
This option turns on the IPv6 scanning (IPv6 address has to be used instead of IPv4).
Enabling IPv6 port scanning can be very useful nowadays, provided we have the IPv6
address of the victim. Because of the huge address space, it is no longer possible to go
through all IP addresses on particular subnet as it was possible with IPv4. On the other
hand, hostnames can be also used instead of the entire IPv6 address, which makes
things easier. The output looks the same as when scanning IPv4 ports. There are no-
table differences in both default port state behaviour and time of the scans.
-O
When used, it enables the OS detection by exploiting the TCP/IP stack fingerprinting.
Nmap sends carefully crafted TCP/UDP packets to the victim and analyses every piece
of the response. After many of such packets, it compares result to the “Nmap-os-db”
file (in version 6.47, the file was last edited 2014-08-13 15:39:44). Users are encouraged
to send new finger-prints onto the Nmap website.
12

20. 3. ATTACKS BY TYPES
-p port range
This option scans specific ports. For my purposes I always used it while testing detec-
tion threshold of number of scanned ports; scanning special port TCP/0; and scanning
all ports (with “-p 1-65535”) on the victim host. Unless specified otherwise by the user,
this port range is scanned in randomized (non-consecutive) way.
--top-ports count
Scans selected number of ports that are most used on the Internet. The installation di-
rectory of Nmap contains a file “Nmap-services” which has a database with 19 908
most used ports (in Nmap 6.47 it was edited on 2014-08-13 20:52:08). Every row is de-
fined by Service name, portnum/protocol, open-frequency, and optional comments.
Based on this file, Nmap is able to scan certain number of most used ports on the In-
ternet.
Those firewalls, which were successful in detecting the port scan attack on default
settings, have some threshold value of scanned ports which triggers detection when
crossed. This threshold value is different when testing a range of ports (e.g. from port
1 to port 100) and number top ports. I assumed that the attacker would be able to scan
less top ports than a range (because the top ports are the most used ones, they should
be most protected). As I will describe later, I observed the exact opposite - I can scan
more ports while using --top-ports command, than if I were to scan a range of ports.
See Table 4.6 for the exact numbers.
--mtu mtu number
Setting the Maximum Transmission Unit (MTU) is a very good trick for evading detec-
tion on some firewalls by forcing the segments to fragment. The multiple of 8 has to be
used when specifying the fragmentation number. For my tests, I tried 4 alternatives: 8,
16, 32, and 64. See Table 4.8 for the results.
--scan-delay time
Setting the scan delay will set the time between two probes. Nmap can use various time
formats in the time parameter: milliseconds (ms), seconds (either (s), or nothing - since
it is the default option), minutes (m), or hours (h). Most of the firewalls which detect
port scanning have the ability to detect them only from above a certain threshold value.
With using this parameter, I tried to find out the lowest value which firewall didn’t
detect. As I tried to have a precision of 10 milliseconds. If you want to be absolutely
certain your portscan won’t be detected, try raising up the value a little. I used this
parameter only when port scanning with default configuration was detected, so in the
attached tables, some firewalls have no value assigned to them. See Table 4.7 for more
information.
13

21. 3. ATTACKS BY TYPES
Product Success Detection
ping ping -6 ping ping -6
Agnitum no no partially partially
Avast! yes yes no no
AVG no no partially partially
Avira no no no no
Bitdefender no no no no
COMODO no no no no
Emsisoft no yes no no
ESET yes yes no no
F-Secure no yes no no
Gdata yes yes no no
Kaspersky no no no no
McAfee no no yes yes
Microsoft no no no no
Norton no yes no no
Panda yes yes no no
Quick Heal no yes no no
TrustPort no yes partially no
ZoneAlarm no no partially no
Table 3.2: Ping scan behaviour
3.1.6 Ping scan (ICMP echo request)
ICMP scan is the easiest one to test, because of native support from operating sys-
tems with ping command. Note that the response to ping can usually be set in settings
of the particular firewall, or underlying operating system. Interestingly enough, re-
searchers found out two daily ICMP scans of the entire IPv4 address space coming
from Guangzhou, China [22]. The purpose of these scans, as well as their usefulness or
maliciousness remains a mystery. For the completeness of this work, I observed fire-
walls’ response to ping probes, and their detection capabilities, which can be seen in
the Table 3.2. In this table, the “Partial” with AVG means that in logs there was “System
block in ICMP on local port 8” for IPv4 ping and “System block in ICMPv6 on local
port 128” on IPv6 ping. ZoneAlarm stored only Type: “Route”, Action: “Blocked”, date,
time, source and destination IP. There was no mention of ping, but there was something
else stored in logs. Only McAfee detects ping scan.
3.2 Denial of Service
Denial of Service attacks (DoS) are types of attacks, which can make host, network
or infrastructure unavailable to their legitimate users. These attacks are mainly used
against web servers, DNS servers, email servers or network infrastructure. DoS attacks
can be used to interrupt computers or the entire network. When two or more attackers
are taking part in the same attack, it is called a Distributed Denial of Service (DDoS).
DDoS usually makes use of extensive botnets (compromised hosts) to launch this type
14

22. 3. ATTACKS BY TYPES
of attack [25].
So called “asymmetric” DoS attacks can be mounted from a single slow device on
a slow network, but they can still inflict major damage (e.g. the amplification attack
on NTP - the Network Time Protocol - with the amplification factor 4670 [18]). I will
focus on DoS on end stations (computers of normal users), in particular on CPU and
network adapter of the victim.
There are many tools available online which can be used to attack some targets (e.g.
XOIC [16] or LOIC [17]). These tools can easily be used against particular hosts - on IP
addresses in LAN. So called “flooding” is used, when the attacker tries to overwhelm
its victim by a huge volume of packets. This mainly results in a consumption of vic-
tim’s network adapter resources. The CPU may also be heavily affected, depending on
the type of packet and length of the data part. Firewalls have to look on each packet
header to inspect it and if there is a need to perform a deep packet inspection (e.g. to
be able to detect some attack types), then firewall looks into the application data part
of the packet and performs an analysis. This behaviour is depleting CPU resources. If
these flooding attacks are not detected, the user can be completely unaware that his
inability to work with his device is due to some attack.
Similar to TCP port scanning attack, the attacker can set any TCP flags in TCP flood-
ing attack. For the simplicity, I did not set any flags in my attacks. I did however test
3 (different) possibilities of every flooding attack: with 0 data size (only header was
sent), 1000 bytes data size and 65495 bytes data size (the maximum value possible
with Hping3 tool).
I discovered that different firewalls cope differently with DoS attacks. As we will
see, some of them can take as low as 19% of average CPU usage in contrast with 100%
with other firewalls while being under the same type of active attack. Bandwidth con-
sumed on the particular network interface can also be different: ranging from 1% to
98%.
If these DoS attacks were targeted on a single open port discovered by Hping3,
Nmap, or any other tool, I think they would inflict even more damage by consuming
more resources, because firewall could try to do deep-packet analysis. On the other
hand while non-open ports are being scanned, the firewall should not care of these
packets and simply drop them. However, since every firewall I tested have different
ports open and some of them have no open ports at all, I decided not to test this ap-
proach and focus on scenario which can be used all the time - an attack using default
settings.
3.2.1 Hping3 tool
Hping3 [19] is a packet generator tool available by default in Linux distributions such
as BackTrack or Kali. It can create TCP, UDP or ICMP packets with various options,
fragmentations or sizes and send them from randomly generated IP address (for ex-
ample for covering the tracks of sender’s IP address). It can be used for port scanning,
15

23. 3. ATTACKS BY TYPES
testing firewall rules, or mounting a DoS attacks. It can give potentially unlimited con-
trol over flooding DoS attacks.
3.2.2 Low Orbit ION Cannon (LOIC)
LOIC is a network stress testing and DoS tool available on Windows which is sim-
ple to use by anyone. User can select either target’s URL or IP address, attacked port,
TCP/UDP/HTTP method, payload (with TCP/UDP) or the speed of generating pack-
ets. The speed is controlled by a slider and there are no numbers to show how many
packets per second are generated. I chose LOIC because the potential script kiddies
running on Windows could choose this tool as well because of friendly user interface.
3.2.3 IPv6 Router Advertisement (ICMPv6 type 134, code 0)
DoS attack can be created by flooding a Local Area Network (LAN) with Router Ad-
vertisement messages, which will consume 100% of CPU. Many operating systems are
affected: Windows (8, 2008, 7, 2003, 2000, XP), all FreeBSD versions, all NetBSD ver-
sions and CISCO devices with firmware released before November 2010. All vulnera-
ble operating systems will spend great amount of system resources to many SLAAC
(StateLess Auto Configuration) processes. Multiple CVE (Common Vulnerabilities and
Exposures) has been created with a severity score of 7.8 [20]. The official description
of vulnerability follows: “When flooding the local network with random router adver-
tisements, hosts and routers update the network information, consuming all available
CPU resources, making the systems unusable and unresponsive” [20]. According to the
same source, “a personal firewall or similar security product does not protect against
this attack, as the default filter rules allow these packets through”. Although the report
is from April 2011, no tested firewall detected this attack. There were only few firewalls
which were able to consume less than 90% of average host’s CPU performance.
For assigning IPv4 addresses to hosts, the Dynamic Host Configuration Protocol
(DHCP) is used. For IPv6 addresses either stateful (DHCPv6) or stateless (neighbor
discovery protocol [26]) can be used. The disadvantage of stateful autoconfiguration
is the need of the DHCPv6 server, which can be unavailable in normal household for
TV, refrigerator or other devices that can have IPv6 address. The stateless neighbor
discovery protocol uses ICMPv6 messages and is responsible for autoconfiguration of
IPv6 addresses, determining network prefixes, determining layer 2 addresses of nodes
on the same link and more. This protocol consists of 5 ICMP messages:
• Router Solicitation (RS)
• Router Advertisement (RA)
• Neighbor Solicitation (NS)
• Neighbor Advertisement (NA)
• ICMP Redirect
16

24. 3. ATTACKS BY TYPES
When new node connects to the network, it sends Router Solicitation message to
the network. Router (as the “clever wiseman”) responds with a Router Advertisement
message, which is also sent out periodically by the router. The process of stateless au-
toconfiguration of a node is complex and requires following steps:
• Link-Local Address Generation (prefix “FE80” followed by 54 zeroes, followed
by 64-bit MAC address or a randomly generated ID),
• Link-Local Address Uniqueness Test (to determine if generated address is not
already used in the local network),
• Link-Local Address Assignment (if uniqueness test passed, the device assigns
the link-local address to its IP interface),
• Router Contact (to get more information),
• Router Direction (router directs host how to proceed), and
• Global Address Configuration (host configures itself with globally unique inter-
net address).
When BackTrack’s native built-in command flood router6 eth0 is used, it floods the
entire Local Area Network (LAN) with Router Advertisement messages, thus making
all IPv6 enabled LAN devices on LAN unresponsive, including game consoles like
Playstation 3 or Xbox.
3.2.4 IPv6 neighbor Advertisement (ICMPv6 type 136, code 0)
For layer 2 (L2) address resolution in IPv4, the ARP (Address Resolution Protocol) is
used. In the IPv6 world, neighbor Solicitation and neighbor Advertisement messages
are used. In the usual scenario with no attacker present, a node looking for layer 2
address takes the last 24 bits of the IP address whose L2 address it is looking for and
concatenates it with the common multicast prefix (FF02:0:0:0:0:1:FF00::/104). A neigh-
bor Solicitation message is sent to such multicast address. When a node which belongs
to that particular multicast group receives a neighbor Solicitation message, it answers
with a neighbor Advertisement message. This message contains all IPv6 and L2 ad-
dresses the node has and 3 flags:
• Router flag: indicates if sender is a router.
• Solicited flag: indicates that the advertisement was sent in response to a neighbor
Solicitation from the Destination address.
• Override flag: indicates that the advertisement should override an existing cache
entry and update the cached link-layer address.
Although flooding all devices on LAN with BackTrack’s command flood advertise6
eht0 is not so “deadly” as when using Router Advertisement messages, it still con-
sumes over 95% of CPU. More detailed analysis will be used in chapters 4/5/6.
17

25. Chapter 4
Experiment description
I prepared two different environments: virtual for testing port scanning, and physical
for testing DoS attacks. Every environment had victim and attacker, on which special
settings needed to be used.
Virtual environment
I’ve been assigned access to Masaryk University’s Windows Server 2008 RC2 with
Hyper-V. On this virtualization server, two Windows 8.1 64-bit virtual machines (the
attacker and the victim) were installed and activated. All updates were installed, along
with required utilities for testing (Nmap, TODO ADD). Both computers were con-
nected via the private LAN network so that there would be no other computers gener-
ating traffic.
The victim was only connected to the internet while downloading, installing and
updating new endpoint security software that was to be tested. It was assigned IPv4
address 192.168.20.1 and IPv6 address fe80::f5dd:cd1d:175a:2d6d.
The attacker was never connected to the internet and it was assigned IPv4 address
192.168.20.2 and IPv6 address fe80::b0e1:ffb9:719e:686.
First, the checkpoint of victim virtual machine was created. Afterwards, different
endpoint security solutions were installed one at a time. After the installation was suc-
cessful and all tests were run, the state of the victim was again checkpointed, so that I
could return to particular firewall if I needed to run more tests of check logs.
Physical environment
Because I wanted to observe the consumption of resources of each DoS attack, to test
this behaviour in the “ideal” virtual environment would be of no practical value. I
was assigned two identical physical computers (Intel Core 2 Duo E8500 3.16 GHz with
4096 MB DDR3 and Windows 8.1 Professional 32bit) and connected them with 1Gb
UTP (Unshielded Twisted Pair) cable without any interconnecting network device like
router or switch. Again, Windows 8.1 was fully updated on both computers. On the
attacker’s computer, I used bootable BackTrack R3 to perform attacks with using com-
mands hping3, flood advertise6 and flood router6.
18

26. 4. EXPERIMENT DESCRIPTION
Company Product Tested version
Agnitum Outpost Pro Security Suite 9.1 (4652.701.1951)
Avast! Internet Security 2014.9.0.2021
AVG Internet Security 2015 2015.0.5315
Avira Antivirus Pro 14.0.7.306
Bitdefender Internet Security 2015 18.17.0.1227
COMODO Internet Security Premium 7.0.317799.4142
Emsisoft Internet Security 9.0.0.4570
ESET Smart Security 8.0.301.0
F-Secure SAFE Internet Security 14.115 build 100
Gdata Internet Security 24.4727
Kaspersky Internet Security 2015 15.0.0.463 (a)
McAfee Total Protection 12.8.988
Microsoft Windows 8.1 Firewall -
Norton Security 22.0.1.14
Panda AntiVirus Pro 2015 15.0.4
Quick Heal AntiVirus Pro 15.00 (8.0.8.0)
TrustPort Internet Security 14.0.5.5273
ZoneAlarm Free Antivirus + Firewall 2015 13.3.209.000
Table 4.1: Antivirus security suites
4.1 Choosing particular firewalls
I aimed to test firewalls that are well known in Czech Republic, along with widely
used firewalls in the world. Since firewalls are incorporated into endpoint “security
suites”, I chose only one particular suite from each company, which had the firewall
features, but without any unnecessary functionalities for this research (e.g. driver up-
dates, file encryption, system speedup, parental control, online backup. .. ). Therefore,
some products are named only “Antivirus” but they contain full-featured firewall. I
downloaded and tested trial versions of selected 18 security suites with full updates
(see Table 4.1).
Firewall settings
Every firewall has different filtering modes, protections of network, or levels of detec-
tion. Therefore I decided to leave these settings mostly on default, but when asked I
selected “work” profile of network (out of options public/work/home), and automatic
(mostly out of automatic/interactive/learning). See Table 4.2 for the list of all changed
settings. Where character “-” occurs, no interaction with user was needed and it was
left on default settings.
It is very important to note, that I was observing and testing only the default be-
haviour of selected firewalls. As this research was a quantitative and not qualitative
19

27. 4. EXPERIMENT DESCRIPTION
Company Settings
Avast! private, unfriendly
AVG automatic
Bitdefender no autopilot
COMODO work, safe mode
ESET home/work, automatic
Panda work
Table 4.2: Firewall settings
analysis, I did not dive in depth when testing particular firewalls. There might be spe-
cialized settings for detection levels or option to deny certain packets (e.g. for blacklist-
ing the ICMPv6 Router Advertisement of neighbor Advertisement messages) which I
did not test.
4.2 Port scanning
Port scanning was performed only in virtual environment using Nmap tool, therefore
the time of actual scanning can differ from the real world scenario. All outputs were
saved into TXT files and later checked for open/open|filtered/closed/unfiltered/filtered
ports, and the time of the actual attacks.
As the first step, all techniques in Nmap were used with their default settings (1000
ports): -sS, -sT, -sF, -sX, -sN, -sA, -sW, -sM, -sU, -sY, -sZ, -sO, -sV. Then two other
commands were used: -sS -O, -sS -p1-65535 (the scanning of all ports). Finally, one
particular port was scanned with -sS -p0. The same thing was done for both IPv4 and
IPv6 addresses. On attached CD, there are at least 32 log files - at least 16 for IPv4 and
at least 16 for IPv6.
If the firewall did not detect any of these techniques, no other tests were run (hence
no values will be presented in tables). If the firewall detected some technique of the
attack (but only from -sS, -sT and -sU), then more tests followed for determining:
• The highest number of scanned ports without detection (with command -p 101-
X).
• The highest number of top ports scanned without detection (with command --
top-ports X).
• The shortest scan delay without detection (with command --scan-delay X).
• If they are detected with using fragmentation (with command --mtu X where X
is one of 8, 16, 32, 64).
Some firewalls have blocking timers incorporated in them. When they detect and
counter attack, they will try to block it. It is not very efficient to wait 5 or more minutes
for these timers to reset. Such a waiting approach also contains the possible drawback
20

28. 4. EXPERIMENT DESCRIPTION
of firewall’s internal mechanism learning that it is “normal” when firewall is being
attacked regularly and the behaviour could be changed to represent this fact, or the
firewall just won’t detect more attacks. Also the time needed for timers to reset is an
issue itself. To cope with these three drawbacks during large-scale testing, the virtual
machine was restored from checkpoint every time when the firewall detected the port
scanning attack. Checkpoints were created for each firewall when it was freshly in-
stalled on the system and updated - hence no attacks were present at that point which
could somehow change firewall’s behaviour. This approach was far more efficient than
sometimes waiting for dozens of minutes before the firewall would detect the attack
again. The checkpoint approach is in particular very useful while finding threshold
values of detections with millisecond precision times.
The exact command used is as follows: “nmap [technique/s] -n -v [IP of the victim]
file.txt”. Options -v for higher verbosity level and -n for never doing DNS resolution
were used in every command. The entire log of the actual scan was always saved into
the TXT file and stored on the attached CD.
Logging port scanning attacks
During every port scan, an observation on the victim’s PC was being done, whether se-
lected technique is detectable (and thus stored in logs) and if the user was made aware
of this ongoing attack by popup window with details. There were significant differ-
ences - for detailed view of whether port scanning attack was detected and stored in
logs, see Table 4.3 for IPv4 and Table 4.4 for IPv6.
Many firewalls did not log just one attack of type “port scan”, but stored hundreds
of events in the packet log. When this behaviour was observed, I wrote “no” in the table
of detecting attack and storing it in logs, because there was no higher-level information
about particular attack. When I stated “partially” in the same table, it means something
was stored in logs but not “port scan”.
Detailed explanation of “partial” detection follows:
• Agnitum: “Attack type: “KOX””
• McAfee: “ping”
• Norton: In “Firewall - Activities” (packet log), there are thousands packets logged
which look like: “Rule “Default Block All Inbound Windows Services (Public
Networks)” rejected TCP (6) traffic with (192.168.20.2 Port (54560))”
• Panda: “TCP flag check”
• ZoneAlarm: Every packet shown along with flags used (e.g. “AF” with TCP Mai-
mon scan)
Based on these results, it is noticeable that some firewalls either lack higher-level
logs, or they are present, but Portscanning attack is not detected and thus not put there.
21

29. 4. EXPERIMENT DESCRIPTION
Company-sS-sS-O-sS-A-sT-sA-sW-sM-sU-sN-sF-sX-sY-sZ-sO-sV
Agnitumyesyespartiallyyesyesnonoyespartiallyyesyesnononoyes
Avast!yesyesyesyesyesyesyesyesyesyesyesnononoyes
AVGnonononononononononononononono
Aviranonononononononononononononono
Bitdefenderyesyesnoyesnononoyesnononononoyesno
COMODOnonononononononononononononono
Emsisoftnonononononononononononononopartially
ESETyesyesyesyesnononoyesnonononononoyes
F-Securenonononononononononononononono
Gdatanononoyesnonononononononononono
Kasperskyyesyesyesyesnonononononononononoyes
McAfeeyesyespartiallyyesyesyesyesyesyesyesyesnonopartiallyyes
Microsoftnonononononononononononononono
Nortonpartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallynonopartiallypartially
Pandayesyesyesyesnononoyespartiallypartiallypartiallynononoyes
QuickHealnonononononononononononononono
TrustPortnonononononononononononononono
ZoneAlarmpartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallynopartiallypartially
Table4.3:IPv4scansshowninlogs
22

30. 4. EXPERIMENT DESCRIPTION
Company-sS-sS-O-sT-sA-sW-sM-sU-sN-sF-sX-sY-sZ-sO-sV
Agnitumnopartiallynonoyesnonopartiallynoyesnonopartiallyyes
Avast!yesyesyesyesyesyesyesyesyesyesnononoyes
AVGnononononononononononononono
Aviranononononononononononononono
Bitdefender--------------
COMODOnononononononononononononono
Emsisoftnononononononononononononono
ESETnononononononononononononono
F-Securenononononononononononononono
Gdatanononononononononononononono
Kasperskyyesyesyesnonononononononononoyes
McAfeeyesyesyesyesyesyesyesyesyesyesnonopartiallyyes
Microsoftnononononononononononononono
Nortonpartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallypartiallynonopartiallypartially
Pandanononononononononononononono
QuickHealnononononononononononononono
TrustPortnononononononononononononono
ZoneAlarmnononononononononononononono
Table4.4:IPv6scansshowninlogs
23

31. 4. EXPERIMENT DESCRIPTION
Company IPv4 IPv6
Agnitum, AVG, Avira, COMODO, ESET, F-Secure,
Gdata, McAfee, Microsoft, Norton, Quick Heal filtered filtered
Avast!, Emsisoft, Kaspersky, Panda closed closed
Bitdefender, ZoneAlarm filtered -
TrustPort filtered closed
Table 4.5: TCP/0 port states across firewalls
Port TCP/0
The port TCP/0 is reserved by IANA and should not be used by any applications.
However it can be used for malicious purposes. Scanning of this port “is frequently
used for fingerprinting network stacks and because it is not possible to block the port
on some firewalls” [22]. CISCO’s technical lead Craig Williams wrote in his blog, that a
massive spike they detected on 2013/11/02 is extremely likely to be a reconnaissance
before the attack [27] or may be connected with a new kind of malware. In my analysis
I identify the opportunity for firewall fingerprinting, because the response to scanning
on this port differs throughout some firewalls - see Table 4.5.
Scanning only a single port with TCP SYN scan was not detected by any firewall,
therefore this is rather stealthy technique of fingerprinting. Most firewalls didn’t reply
to this scan probe, and therefore the state was reported as filtered by Nmap. However
with Avast!, Emsisoft, Kaspersky and Panda, both IPv4 and IPv6 scans of TCP/0 were
reported as closed. This is very interesting result mainly for Avast! because during
normal TCP SYN scan, 999 ports were reported as filtered and 1 as open, which makes
marking TCP/0 port inconsistent with its default behaviour for other ports. Same ap-
plies for Panda, although there were 988 ports reported as filtered, 11 as closed, and
1 as open. The most significant result was observing TrustPort, because it responded
differently for IPv4 and IPv6 scans of TCP/0 port - the state was filtered on IPv4, but
closed on IPv6. Bitdefender and ZoneAlarm firewalls couldn’t be scanned with IPv6
TCP SYN scan. Their IPv4 port states were both filtered. As using this two single-port
scans techniques has interesting results, it was used in fingerprinting and connected
with other scans for higher precision of the process.
Detection thresholds
Every firewall which detected certain port scanning attack have some detection thresh-
old value. If this threshold value isn’t exceeded, the attack becomes invisible for the
firewall’s detection mechanism. Nmap can use multiple of evasion techniques, mainly
adjusting the delay between the probes, using fragmentation, or applying a limit to the
number of scanned ports. The Table 4.6 shows 2 different approaches - first 3 columns
state the highest number of ports scanned in “consecutive” way without being de-
tected by the firewall. The second 3 columns show the upper limit when top ports are
scanned. It is interesting to see that these 2 values differ, sometimes rather significantly.
In the Table 4.7 the individual threshold values of TCP SYN scan without detection
24

32. 4. EXPERIMENT DESCRIPTION
Company -p101-XXX ports --top-ports
-sS -sT -sU -sS -sT -sU
Agnitum 10 10 12 10 10 20
Avast! 6 6 6 7 7 17
Bitdefender 5 5 1 3 3 4
ESET 8 8 7 8 10 7
Kaspersky 81 77 - 100 85 -
McAfee 171 43 124 143 76 122
Panda 11 10 10 11 10 11
Table 4.6: Highest number of scanned ports without detection
Company Time in seconds
Agnitum 30.00
Avast! 13.28
Bitdefender 0.69
ESET 1.00
Kaspersky 0.16
McAfee 2.45
Panda 28.00
Table 4.7: Lowest scan delay without detection
are shown. Note that only those firewalls which detected TCP SYN scan are shown in
this table. All values are in seconds rounded to 2 decimal places. If you want to scan
the computer as fast as possible without triggering alarm or detection, you should use
these numbers. To be on the safe side, you might want to increase these values to little
higher number.
Fragmentation can also be used to avoid detection. I tried 4 different fragmenta-
tions by setting the MTU to 8, 16, 32, and 64 bytes. The results can be seen in Table
4.8. Note that “partial” with Agnitum means it detected “OPENTEAR” attack, but no
port scanning. As Agnitum doesn’t have any severity levels of detected events, both
attacks had the same weight. With ESET, “Incorrect TCP packet length” was shown in
the packet logs hundreds of times, but no port scanning attack attack was reported.
The severity was brought down from “warning” to “informative” and no pop-up win-
dow was shown.
Using particular fragmentation option, I was able to not to trigger detection in fire-
walls. Only Agnitum, Avast! and ESET can be fooled this way. Other firewalls will
detect all port scans even when fragmentation is being used.
4.3 DoS attacks
On victim’s computer, there was a need to make multiple measurements of consumed
resources to which I used “typeperf -cf counters.txt -sc 20 -o output.csv” command in
Windows CMD, which was run as Administrator to have a higher priority than other
25

33. 4. EXPERIMENT DESCRIPTION
-sSdetectable:-sVdetectable:-sUdetectable:-sTdetectable:
8163264816326481632648163264
Agnitumpartiallypartiallyyesnopartiallypartiallyyesnoyesyesyesnoyesyesyesno
Avast!nonoyesyesnonoyesyesyesyesyesyesyesyesyesyes
Bitdefenderyesyesyesyesyesyesyesyesyesyesyesyesyesyesyesyes
ESETpartiallypartiallynonopartiallypartiallyyesyesyesyesyesyesyesyesyesyes
Kasperskyyesyesyesyesyesyesyesyes----yesyesyesyes
McAfeeyesyesyesyesyesyesyesyesyesyesyesyesyesyesyesyes
Pandayesyesyesyesyesyesyesyesyesyesyesyesyesyesyesyes
Table4.8:Portscandetectabilityusingfragmentation
26

34. 4. EXPERIMENT DESCRIPTION
processes. I decided to make 20 measurements (1 measurement per second) while un-
der an active attack. Following counters were used in the input counter file (coun-
ters.txt):
• processor(0)% Processor Time
• processor(1)% Processor Time
• processor( Total)% Processor Time
• Network Interface(Realtek PCIe GBE Family Controller) Bytes Total/sec
• Network Interface(Realtek PCIe GBE Family Controller) Current Bandwidth
• Network Interface(Realtek PCIe GBE Family Controller) Packets/sec
Along with these 6 values, the date and time were put to every recorded measure-
ment row. All files were then transformed into the tables which can be found on the
attached CD, along with all original CSV files generated by typerf command. To ob-
serve each firewall’s behaviour under the DoS attack, I ran LOIC UDP flood attack
with “wait” and “no wait” options on Windows. On BackTrack, following commands
were launched:
• flood advertise6 eth0
• flood router6 eth0
• hping3 --icmp --flood 192.168.1.1
• hping3 --icmp --flood -d 1000 192.168.1.1
• hping3 --icmp --flood -d 65495 192.168.1.1
• hping3 --udp --flood 192.168.1.1
• hping3 --udp --flood -d 1000 192.168.1.1
• hping3 --udp --flood -d 65495 192.168.1.1
• hping3 --rawip --flood 192.168.1.1
• hping3 --rawip --flood -d 1000 192.168.1.1
• hping3 --rawip --flood -d 65495 192.168.1.1
There are 2 tables for LOIC and 11 for BackTrack. As an example, see the Table 4.9 in
which the TCP flood attack is shown with 1000 bytes of data in each packet. I created
additional table which summarizes the maximum, minimum, average, median and
deviation values that can be observed on selected DoS attacks - see Table 4.10 for more
information.
27

35. 4. EXPERIMENT DESCRIPTION
Producthping3–rawip–flood-d1000192.168.1.1
TotaltimeAveragehighestcoreusageAverageCPUusageAverageBytesTotal/secAveragePackets/secBandwidth
Agnitum0:00:1996.0052.08106523566.81103020.960.8522
Avast!0:00:1983.2343.69106542241.10103039.020.8523
AVG0:00:2043.6822.24106528820.47103025.290.8522
Avira0:00:2195.3894.82107298581.82103770.480.8584
Bitdefender0:00:1930.2615.49106299177.33102803.840.8504
COMODO0:00:19100.0046.94106548638.89103046.900.8524
Emsisoft0:00:1960.2955.66428.550.410.0000
ESET0:00:1962.7536.59106587680.37103082.960.8527
F-Secure0:00:2089.8276.65107693649.99104152.720.8615
Gdata0:00:1937.9719.39106423818.59102924.570.8514
Kaspersky0:00:2097.4252.2579452351.2876840.140.6356
McAfee0:00:2199.8657.9763316551.9361234.660.5065
Microsoft0:00:2090.3078.57106342570.91102845.770.8507
Norton0:00:33100.0099.8977666553.0675112.850.6213
Panda0:00:2095.2475.4279643292.1077024.460.6371
QuickHeal0:00:2198.4073.75106488685.10102987.300.8519
TrustPort0:00:1961.0542.09106443440.51102943.370.8515
ZoneAlarm0:00:1939.2320.11106863142.15103349.530.8549
Table4.9:ResourcesconsumptionduringTCPfloodattackwith1000bytes
28

36. 4. EXPERIMENT DESCRIPTION
measurementmaxminaveragemediandeviation
floodadvertise6eth0
Totaltime00:36(Panda)0:190:210:200:00
AveragehighestCPUusage100.00(Panda)59.61(Emsisoft)94.2697.72.10.90
AverageCPUusage100.00(Panda)36.48(Bitdefender)64.1054.5418.96
AveragetotalMbytes/sec14.69(Agnitum)3.03(Norton)7.887.263.79
Averagepackets/sec179099(Agnitum)36981(Norton)961618856946134
Bandwidth12.32%(Agnitum)2.54%(Norton)6.61%6.10%3.18%
floodrouter6eth0
Totaltime00:42(Panda)00:19(Bitdefender)0:250:230:06
AveragehighestCPUusage100(Avast!,Avira,COMODO,ESET,Gdata,Kaspersky,
McAfee,Norton,Panda,TrustPort,ZoneAlarm)54.58(Bitdefender)95.4310013.19
AverageCPUusage100(Avira,Panda)28.02(Bitdefender)71.7865.2419.89
AveragetotalMbytes/sec15.67(Agnitum)1.01(Avira)2.791.294.35
Averagepackets/sec139244(Agnitum)10997(F-Secure)351182517135531
Bandwidth13.14%(Agnitum)0.85%(Avira)2.89%1.12%4.05%
hping3--icmp--flood-d65495192.168.1.1
Totaltime0:200:190:190:190:00
AveragehighestCPUusage100(COMODO)26.53(ZoneAlarm)51.8850.4122.21
AverageCPUusage68.65(Agnitum)13.71(ZoneAlarm)32.5624.718.32
AveragetotalMbytes/sec215.39(Gdata)92.2(Emsisoft)119.41117.3125.19
Averagepackets/sec151617(Gdata)64916(Emsisoft)840598257817729
Bandwidth180.68%(Gdata)77.34%(Emsisoft)100.17%98.41%21.13%
hping3--udp--flood192.168.1.1
Totaltime0:03:59(Avast!)0:00:17(Emsisoft)00:4200:2000:55
AveragehighestCPUusage100(Gdata,Zonealarm)60.35(Bitdefender)89.5096.9913.79
AverageCPUusage99.93(Gdata)30.51(Bitdefender)64.5063.6621.24
AveragetotalMbytes/sec20.31(TrustPort)0.00(Emsisoft)8.2811.136.5
Averagepackets/sec355004(TrustPort)0(Emsisoft)145939194979104155
Bandwidth17.04%(Trustport)0.00%(Emsisoft)6.94%9.34%5.07%
hping3--rawip--flood-d1000192.168.1.1
Totaltime00:33(Norton)00:1900:2000:2000:03
AveragehighestCPUusage100(Comodo,Norton)30.26(Bitdefender)76.7290.0625.26
AverageCPUusage99.89(Norton)15.49(Bitdefender)53.5352.1725.67
AveragetotalMbytes/sec102.70(F-Secure)0.00(Emsisoft)89.36101.5325.91
Averagepackets/sec104153(F-Secure)0(Emsisoft)8987410294333599
Bandwidth86.15%(F-Secure)0.00%(Emsisoft)74.34%85.15%27.79%
Table4.10:ConsumptiondifferencesinDoS
29

37. 4. EXPERIMENT DESCRIPTION
4.3.1 DoS results
As well as with the port scan attacks, each firewall behaves significantly different from
one another. However, the differences in port scanning attacks could be observed by
the attacker. With DoS attacks, the attacker receives no information from the victim. On
the other hand, the overall impact on the system resources can be measured with DoS
attacks. If the attacker wants to deplete specific resources on the victim’s computer but
is unaware which endpoint security solution is in place, he can use Table 4.11 where
all average numbers are shown. If for example the overall system resources are to be
attacked, one can search for the highest average total time and discover that UDP flood
with no data is the best DoS attack. When CPU resources of multi-core system are to be
depleted, flood router6 eth0 should be used. For the bandwidth consumption, ICMP
flood with 65495 bytes of data is the best course of action.
The only firewall which had major and unrecoverable difficulties with certain DoS
attacks was Gdata. Using LOIC UDP flood, after waiting between 2:15-2:50 from the
beginning of the attack, a black screen appeared and the computer became unusable -
hard reset had to be done. I was able to recreate this behaviour multiple times. When
the attack stopped after the black screen appeared, the victim’s PC was still unusable
until the next restart. I took a picture of this behaviour which is shown in Figure [?]. As
it can be seen, the resolution was changed from 1600x1200 pixels, there are plenty of
graphical elements that shouldn’t be there and the script haven’t even started to write
into the file. I couldn’t get any data written into the CSV file because it only started to
write into the file after I stopped the attack, or it didn’t even create file in the first place.
I was able to obtain the CSV file using hping3 UDP flood without data payload, but
the overall result was the same - black screen and inability to use computer without
the hard reset. The same problems were not observed with ICMP nor TCP floods.
Other interesting observation was with Panda under the flood router6 eth0 attack -
every CPU core was used to 100% in every measurement. There were no measurements
of network utilizations stored into the CSV file. This attack was the most devastating
one on single CPU core - 11 firewalls had their CPU on 100%, and 5 other were on
over 99.5%. Only Bitdefender (54.58%) and Emsisoft (64.33%) were significantly dif-
ferent from the rest. Emsisoft shown orders of magnitude better results against TCP,
UDP and ICMP floods - with bandwidth utilization 0.00%. The worst bandwidth re-
sult was observed with Gdata - 181% of the bandwidth on ICMP flood with 65495 bytes
data part. Avast! is leading in the time of performing 20 measurements - it should have
taken 20 seconds, but it took 3 minutes and 59 seconds instead when it was under UDP
flood with no data.
To achieve the highest CPU consumption, full bandwidth was rarely needed. For
example, receiving only 428.55 Bytes per second in average was responsible for 60%
CPU utilization for Emsisoft while under TCP flood attack with 1000 bytes data part.
No more than 1.5 MB/s were needed for the flood router6 eth0 to successfully deplete
16 firewalls to more than 99.5%. This means that even the attacker with slow band-
width or computational capabilities is able to perform quiet serious DoS attacks.
30

38. 4. EXPERIMENT DESCRIPTION
After the flood router6 eth0, there were thousands of records showed in ipconfig
/all. To delete these bogus information, following commands had to be used:
• ipconfig /release6
• netsh int ipv6 reset
The first command releases leased IPv6 addresses and the second command resets the
IPv6 configuration state.
31

39. 4. EXPERIMENT DESCRIPTION
AttackTotaltimeAvg.highestcoreusageAvg.CPUusageAvg.BytesTotal/secAvg.Packets/secBandwidth
floodadvertise6eth000:2194.2664.18262835.1496160.630.07
floodrouter6eth000:2595.4371.782922010.0835117.840.03
hping3--icmp--flood00:2574.3851.4812825864.87213765.40.1
hping3--icmp--flood-d100000:2054.1636.5397475151.2493546.470.78
hping3--icmp--flood-d6549500:1951.8832.56125214329.284058.911.00
hping3--udp--flood00:4289.564.58680233.57145938.910.07
hping3--udp--flood-d100000:3675.2354.8471536400.2570482.310.57
hping3--udp--flood-d6549500:2066.240.83104183824.670179.970.83
hping3--rawip--flood00:2685.8362.3811709629.44195160.050.09
hping3--rawip--flood-d100000:2076.7253.5393703510.6190622.510.75
hping3--rawip--flood-d6549500:2057.234.85117867220.979135.30.94
LOICUDPwait00:2577.3355.7412166674.91164408.320.1
LOICUDPnowait00:2075.754.1311112084.46150157.820.1
Table4.11:AverageresourceconsumptiononparticularDoSattacks
32

40. Chapter 5
Fingerprinting
Exploiting zero-day vulnerabilities have become common. Although there is a signif-
icant research and bounties for bug tracking of commonly used systems, the exploita-
tion still continues. In 2014, Kevin Mitnick launched a webstore called “Mitnick’s Ab-
solute Zero Day Exploit Exchange” for zero-day exploits with CVSS (Common Vulner-
ability Scoring System) at least 8 [28]. This suggests that there is a big demand for these
vulnerabilities that could be exploited by almost anyone. If the attacker breaks into the
network, he first needs to know which security systems are used. The endpoint protec-
tion systems are one of such security countermeasures - most likely the very last line
of defence before the computer is compromised. As I discovered, it is not that hard to
fingerprint these endpoint security systems to differentiate and find out which system
is used from the attacker’s point of view. After the attacker has this knowledge, he can
either find zero-day vulnerability to bypass this endpoint protection system, or use
attacks/stealth techniques which are not detected by it.
5.1 Using time differences
There are significant discrepancies in time consumption of certain port scanning tech-
niques: see Table 5.2 for the full IPv4 times and Table 5.3 for the full IPv6 times. The
uniqueness of time differences can be observed from several points = several tech-
niques in both IPv4 and IPv6 as these two give different results. Table 5.1 was created
to represent the most important differences between the various techniques - firewalls
which took the least and the most time to perform a port scan against. Other columns
were computed to show other interesting numbers - average, median and deviation.
Scanning default 1000 ports with TCP SYN scan gives significant anomalies rang-
ing from 1.45s (Panda on IPv6), through 22.91s (most common on IPv4), to 1243.67s
(Avast! on IPv4). Scanning 1000 UDP ports also gives a significant differences: 7.83s
(Gdata on IPv4) in contrast to 3769.63s (Panda on IPv4). The smallest difference can
be observed on scanning SCTP ports, because there are only 52 of them in total which
Nmap scans. In almost every case, the IPv6 portscanning performed much faster than
on IPv4. Anomalies with scanning the IP protocol were also significant, ranging from
335.77 seconds (AVG on IPv6) to 2.25 seconds (Quick Heal on IPv6). To demonstrate the
magnitude of difference in the time consumption of among firewalls, I tested all 65535
ports. The best result from the victim’s point of view was observed with Panda on IPv4
which took 132307.52 seconds (37.75 hours). The best because it takes the longest time
for attacker to perform a full TCP SYN scan. The fastest result was also observed with
Panda on IPv6 which took 34.3 seconds. This is really interesting result as Panda is
33

41. 5. FINGERPRINTING
both the fastest (on IPv6) and the slowest (on IPv4) when under the TCP SYN scan of
all 65535 ports. The usual time of most firewalls was around 1434 seconds on both IPv4
and IPv6.
There are plenty of approaches to fingerprinting based on Nmap scans and observ-
ing the total amount of time needed to complete the scan. I created two examples - one
focuses on the reliability (see Figure 5.1) and the other one focuses on avoiding detec-
tion (see Figure 5.2). These two diagrams can serve as a guide to fingerprint firewall
by observing total scan time consumption under the ideal conditions. Note that delays
have to be taken into the account, as these times were taken in the ideal environment.
Also, since I measured every port scan only once, there can be slight variation in mil-
liseconds. In the Figure 5.1, first “nmap -6 -sS” is used and based on the time needed
to perform this default scan, we can end in one of the 7 possible states. Sometimes,
it is necessary to perform a scan which will be detected and logged by the firewall,
hence final states can be either undetected or detected. First variant is detectable by 4
different firewalls (Agnitum, Avast!, Kaspersky and McAfee), while the more stealthy
variant is detectable only on Avast!. Note that much more approaches of fingerprinting
are possible and the best course of action would be to perform every Nmap scan and
compare results with overall tables.
5.2 Using port states
The default behaviour on how to respond to port scanning probe differs across fire-
walls. It is not only operating system specific. Some firewalls are suppressing the de-
fault behaviour of the operating system. If certain firewall sends a response to port
scanning probe indicating it is closed, then such scanning takes much shorter time as
opposed to sending no response at all. This difference could range from 5 seconds for
Gdata, to 1243.67 seconds for Avast!. Both numbers are scanning times of TCP SYN
scan of 1000 ports. All firewalls that responded to TCP SYN scan probes as “closed”
are shown in the Table 5.4 for IPv4 and able 5.5 for IPv6.
We can differentiate between 2 fingerprinting methods - within IPv4 or IPv6 or be-
tween the two of them. For example, the TrustPort behaviour under TCP SYN portscan-
ning attack varies when under IPv4 and IPv6 significantly - there are 5 open and 0
closed ports on IPv4; but 10 open and 990 closed ports on IPv6. These differences are
quite common even with more exotic techniques: for example Panda on TCP FIN scan
has 1000 closed ports on IPv4, but only 1 closed port on IPv6. The most significant dif-
ferences between the IPv4 and IPv6 scans are with IP protocol scans: only Kaspersky
(239 closed ports) and Panda (240 closed ports) have the majority of closed ports on
IPv4; whereas there are many others on IPv6 - AVG (244), Emsisoft (145), Kaspersky
(244), Panda (237), TrustPort (228) and ZoneAlarm (242).
I created Figure 5.2 to show the example of possible fingerprinting based on the
differences in port states under various scanning techniques. Some firewalls can be
fingerprinted by scanning only 2 ports - Bitdefender and ZoneAlarm. I chose to exploit
the TCP/0 port on both IPv4 and IPv6 as well as the IP protocol scan. The IP protocol
34

42. 5. FINGERPRINTING
Figure5.1:Fingerprintingusingtimedifference-reliability
35

43. 5. FINGERPRINTING
Figure 5.2: Fingerprinting using time difference - avoiding detection
36

44. 5. FINGERPRINTING
technique maximum minimum average median deviation
-sS 1243.67 (Avast!) 3.77 (Emsisoft) 153.81 22.91 377.84
-6 -sS 312.86 (Kaspersky) 1.45 (Panda) 32.19 17.17 71.60
-sT 2669.77 (Panda) 45.56 (Bitdefender) 323.94 45.85 788.87
-6 -sT 1257.2 (Avast!) 45.34 (TrustPort) 123.31 45.81 286.87
-sA 23.36 (Norton) 1.58 (Panda) 19.53 22.91 7.95
-6 -sA 23.39 (ESET) 1.38 (Panda) 15.68 22.91 10.56
-sW 23.03 (F-Secure) 1.47 (Panda) 18.50 22.91 8.52
-6 -sW 23.41 (Avira) 1.39 (Panda) 16.63 22.91 9.91
-sM 23.45 (Norton) 1.44 (Kaspersky) 18.49 22.91 8.66
-6 -sM 23.98 (ESET) 1.47 (Panda) 15.78 22.91 10.54
-sU 3769.63 (Panda) 7.83 (Gdata) 290.29 22.91 905.41
-6 -sU 1160.00 (Emsisoft) 22.91 (most of them) 278.31 22.91 462.51
-sN 23.22 (F-Secure) 2.81 (Kaspersky) 20.74 22.91 6.42
-6 -sN 23.22 (F-Secure) 1.38 (Panda) 16.77 22.91 10.38
-sF 1236.8 (Agnitum) 1.56 (Kaspersky) 88.91 22.91 286.57
-6 -sF 23.19 (F-Secure) 1.39 (Panda) 15.79 22.91 10.42
-sX 1243.58 (Agnitum) 1.44 (Kaspersky) 88.76 22.91 288.29
-6 -sX 23.02 (F-Secure) 1.39 (Panda) 15.63 22.91 10.57
-sY 2.34 (most of them) 2.13 (Kaspersky, Panda) 2.32 2.34 0.07
-6 -sY 2.36 (Gdata) 0.00 (ZoneAlarm) 2.14 2.34 0.74
-sZ 2.61 (McAfee) 2.13 (Kaspersky, Panda) 2.33 2.34 0.10
-6 -sZ 2.36 (Avira, Microsoft) 0.00 (ZoneAlarm) 2.14 2.34 0.74
-sO 312.08 (Panda) 2.58 (Emsisoft) 51.37 6.72 106.26
-6 -sO 335.77 (AVG) 2.25 (Quick Heal) 84.50 6.72 128.22
Table 5.1: Time differences in port scanning
37

45. 5. FINGERPRINTING
timeinsecondsof“nmap-s*-v-n192.168.20.1”
Company-sS-sS-p1-65535-sT-sA-sW-sM-sU-sN-sF-sX-sY-sZ-sO-sV
Agnitum22.916807.5393.6922.914.344.7622.9122.911236.81243.582.342.343.9522.91
Avast!1243.671434.752299.2923.2222.9122.9122.9122.9122.9122.912.342.343.223.2
AVG25.191360.6445.6822.9122.9122.9122.9122.9122.9123.642.342.346.726.50+48.58
Avira22.911436.5845.5722.9122.9122.9122.9122.9122.9127.142.342.346.7222.91
Bitdefender22.911476.3245.5622.9122.9122.9122.9222.9122.9122.912.342.346.7222.91
COMODO22.911434.7546.5622.9722.9122.9122.9122.9122.9122.912.342.34226.8422.91
Emsisoft3.7774.0346.792.383.593.5222.913.415.23.362.342.342.586.03
ESET22.911435.5345.7122.9122.9122.9122.9122.9122.9122.912.342.345.9722.91
F-Secure22.911434.7346.3623.1923.0323.2523.1923.2223.1323.252.342.346.7224.19
Gdata5111.1345.5722.9122.9122.927.8322.9122.9122.912.342.342.915.19+53.60
Kaspersky108.397239.23124.912.82.781.441111.392.811.561.442.132.13300.11109.94+56.58
McAfee5.2166.5845.7622.9122.9122.9122.9122.9222.9122.912.342.616.724.75+6.01
Microsoft22.921435.5845.8222.9122.9122.9222.9122.9122.9122.912.342.346.7223.13
Norton22.921439.6346.5123.3622.9123.4522.9123.1723.223.162.342.346.7222.91
Panda1135.03132307.522669.771.581.471.483769.6322.9235.9822.912.132.13312.081179.48+5.01
QuickHeal23.191455.9445.8822.9122.9122.9122.9122.9222.9222.922.342.346.7222.89
TrustPort13111.7545.6522.9122.9122.9115.2822.9122.9122.912.342.346.724.78+53.57
ZoneAlarm22.911435.3445.822.9122.9122.9122.9122.9122.9122.922.342.346.7222.91
Table5.2:IPv4portscanningtimes
38

46. 5. FINGERPRINTING
timeinsecondsof“nmap-6-s*-v-nfe80::f5dd:cd1d:175a:2d6d”
Company-sS-sS-p1-65535-sT-sA-sW-sM-sU-sN-sF-sX-sY-sZ-sO
Agnitum4.33172.5645.54.555.25.222.9122.914.675.22.342.346.72
Avast!23.191435.281257.222.9122.9122.9422.9122.9122.9122.912.342.343.88
AVG8.92226.3945.8122.9122.9122.9122.9122.9122.9122.912.132.13335.77
Avira22.911435.4245.822.9123.4122.9122.9122.9122.9122.912.342.366.72
Bitdefender-------------
COMODO22.911443.0846.8422.9122.9122.9122.9122.9122.9122.912.342.346.72
Emsisoft3.86450.93.453.673.6111603.835.422.392.132.13171.55
ESET11.42111.5645.723.3923.1723.9822.9123.222.9122.912.342.343.45
F-Secure22.911434.7345.6823.0923.3623.1623.2823.2223.1923.22.342.345.44
Gdata4.77121.8845.6622.9122.9122.9122.9122.9122.9122.912.362.343.89
Kaspersky312.8621495.2269.891.422.521.471117.581.451.471.582.132.13300.8
McAfee4.98116.3445.7522.9122.9122.9122.9122.9122.9122.912.342.346.73
Microsoft23.341456.0945.8622.9122.9122.9122.9122.9122.9122.912.342.366.72
Norton22.911436.1645.8222.9122.9222.9122.9122.9122.9122.912.342.343.45
Panda1.4534.345.371.381.391.471071.521.381.391.392.132.13279.95
QuickHeal22.911435.4145.922.9122.9222.9122.9122.9222.9122.912.342.342.25
TrustPort1.471024.0545.341.671.561.561062.591.471.691.452.132.13287.66
ZoneAlarm-39.41-1.4715.191.5944.221.481.481.51004.72
Table5.3:IPv6portscanningtimes
39

47. 5. FINGERPRINTING
Company Number of closed ports after
the default TCP SYN scan on IPv4
Agnitum, Avast!, AVG, Avira,
Bitdefender, COMODO, ESET, 0
F-Secure, Gdata, McAfee, Microsoft,
Norton, Quick Heal, TrustPort, ZoneAlarm
Panda 11
Emsisoft 976
Kaspersky 986
Table 5.4: Number of closed ports after the TCP SYN scan on IPv4
Company Number of closed ports after
the default TCP SYN scan on IPv6
Bitdefender, ZoneAlarm -
Avast!, AVG, Avira,
COMODO, ESET, F-Secure, 0
Gdata, McAfee, Microsoft,
Norton, Quick Heal
Agnitum 1
Panda 11
Emsisoft 976
Kaspersky 987
TrustPort 990
Table 5.5: Number of closed ports after the TCP SYN scan on IPv6
scan on IPv6 was not detected by any firewall, it has only 256 ports and it shows many
different states of ports.
When observing only differences within IPv4 scans, only two firewalls designated
most ports as “closed”: Emsisoft (976 ports) and Kaspersky (986 ports) on TCP SYN
scan. Interesting to add here is that only Panda designated all 1000 ports as closed dur-
ing TCP Window and TCP Maimon scans - both took only 1.47s to scan. Other irregu-
larities can be found in results from UDP scan - 16 firewalls had all 1000 ports marked
as “open|filtered”, but for Kaspersky it was only 10, for Panda 987 and for Trustport
999. TrustPort was also the only firewall which had “open” port in the UDP scan. Only
SCTP INIT scan gave the same results of port states across all 18 firewalls, however dif-
ferences can be found in time of scan - 16 firewalls took exactly 2.34 seconds to finish
the scan, except of Kaspersky and Panda which both took 2.13 seconds, which is 9%
decrease of time. In SCTP COOKIE ECHO scan, 16 firewalls had all 52 ports marked
as “open|filtered” with the exception of Kaspersky and Panda, which marked 2 and 3
ports as “filtered” respectively.
For more information, see the whole tables: 5.6, 5.7, 5.8, 5.9, 5.10, 5.11.
40

48. 5. FINGERPRINTING
Figure5.3:Fingerprintingusingportstates
41

49. 5. FINGERPRINTING
statesofportswithdefault“nmap-s*-n-v192.168.20.1”
-sO-sV
openopen|filteredclosedfilteredopenclosedfiltered
125410001000
125410001000
02560010999
025600001000
025600001000
125500001000
125500197623
025600001000
025600001000
125500100990
314239079867
02560020998
025600001000
025600001000
3132400117982
025600001000
02560050995
025600001000
Table5.6:IPv4portstates(iii)
42

50. 5. FINGERPRINTING
statesofportswithdefault“nmap-s*-n-v192.168.20.1”
-sN-sF-sX-sY-sZ
open|filteredclosedfilteredopen|filteredclosedfilteredopen|filteredclosedfilteredopenclosedfilteredopen|filteredfiltered
10000099820999100052520
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
2397702397702397700052520
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
7993079930799300052502
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
100000999101000000052493
1000001000001000000052520
1000001000001000000052520
1000001000001000000052520
Table5.7:IPv4portstates(ii)
43

51. 5. FINGERPRINTING
statesofportswithdefault“nmap-s*-n-v192.168.20.1”
-sS-sT-sA-sW-sM-sU
openclosedfilteredopenclosedfilteredunfilteredfilteredopenclosedfilteredopen|filteredclosedopenopen|filteredclosedfiltered
Agnitum001000909910100001099099820100000
Avast!109992099801000001000100000100000
AVG109991099901000001000100000100000
Avira00100000100001000001000100000100000
Bitdefender00100000100001000001000100000100000
COMODO00100000100001000001000100000100000
Emsisoft1976231099997723097723239770100000
ESET00100000100001000001000100000100000
F-Secure00100000100001000001000100000100000
Gdata10099010099001000001000100000100000
Kaspersky798677099399370993779930109900
McAfee209982099801000001000100000100000
Microsoft00100000100001000001000100000100000
Norton00100000100001000001000100000100000
Panda1119883799001000010000010000987130
QuickHeal00100000100001000001000100000100000
TrustPort509955010000100000100010000199900
ZoneAlarm00100000100001000001000100000100000
Table5.8:IPv4portstates(i)
44

52. 5. FINGERPRINTING
statesofportswithdefault“nmap-6-s*-n-vfe80::f5dd:cd1d:175a:2d6d”
-sY-sZ-sO-sV
openclosedfilteredopen|filteredfilteredopenopen|filteredclosedfilteredopenclosedfiltered
005252002560091990
0052520025411001000
005250257244010999
0052520025600001000
------------
00525207025600001000
00524930106145597624
005252012550030997
0052520125500001000
005252012550090991
00524938424469877
005205202560010999
0052520025600001000
0052520125500001000
00524938112370109900
0052520125500001000
00522508202280109900
005205259242009919
Table5.9:IPv6portstates(iii)
45

53. 5. FINGERPRINTING
statesofportswithdefault“nmap-6-s*-n-vfe80::f5dd:cd1d:175a:2d6d”
-sM-sU-sN-sF-sX
open|filteredclosedopenopen|filteredclosedfilteredopen|filteredclosedfilteredopen|filteredclosedfilteredopen|filteredclosedfiltered
998201000001000009982099820
100000100000100000100000100000
100000100000100000100000100000
100000100000100000100000100000
---------------
100000100000100000100000100000
239770159850239770239770239770
100000100000100000100000100000
100000100000100000100000100000
100000100000100000100000100000
79930109900799307993079930
100000100000100000100000100000
100000100000100000100000100000
100000100000100000100000100000
01000509950010000010000010000
100000100000100000100000100000
01000049960010000010000010000
0100005995001000001000001000
Table5.10:IPv6portstates(ii)
46

54. 5. FINGERPRINTING
statesofportswithdefault“nmap-6-s*-n-vfe80::f5dd:cd1d:175a:2d6d”
-sS-sS-all-sS-O-sT-sA-sW
openclosedfilteredopenclosedfilteredopenclosedfilteredopenclosedfilteredunfilteredfilteredopenclosedfiltered
Agnitum1019891036552210198910099011989011989
Avast!00100000655350010002099801000001000
AVG109991065534109991099901000001000
Avira00100000655350010000010000100001000
Bitdefender-----------------
COMODO001000006553500100000100001000001000
Emsisoft097624065509260976241099997723097723
ESET309973065532309973099701000001000
F-Secure001000006553500100000100001000001000
Gdata909919065527909919099101000001000
Kaspersky6987766552096987750995993709937
McAfee109991065535109991099901000001000
Microsoft001000006553500100000100001000001000
Norton001000006553500100000100001000001000
Panda1119881065526010990010099010000010000
QuickHeal001000006553600100000100001000001000
TrustPort1099001065526010990010099010000010000
ZoneAlarm---06552610------10000010000
Table5.11:IPv6portstates(i)
47

55. Chapter 6
Ideal behaviour of firewall under certain attacks
Based on my results from testing 18 different firewalls, I can extrapolate how the
ideal firewall should behave under the port scanning attacks. Interpreting results with
portscan attack is much easier than interpreting DoS attacks. I can observe the run-
ning time of every scan and differentiate between various states of ports when certain
technique or obfuscation is used. During the DoS attacks I was only able to measure
the consumption of system resources on the victim, which I can interpret based on
numbers in results.
6.1 Ideal port scanning behaviour of a firewall
Based on my observations, I propose 5 points that are important to establish the com-
mon ground of testing firewall’s behaviour while under port scanning attack:
• Leakage of port states. Every firewall should achieve the ideal port states on all
ports, no matter the IPv4 or IPv6 - as stated in Table 6.1.
• Resilience against obfuscation techniques (with respect to the previous point).
Obfuscation techniques (e.g. fragmentation or higher delay between the probes)
should make no difference in port scanning results.
• Unified time of the scanning. As shown in Table 6.2, there is an “ideal” time each
port scan should consume.
• Logging higher-level information on the detection. Each attack should be logged
with a higher level information. Along with the source IP address, this should
state clear information about the type - “Port scanning attack”. See Table 6.4 for
current capabilities.
• Taking steps against the attack. After the attack is detected, there should be
an action taken against than particular attack - e.g. throwing away all incoming
packets from the IP address of the attacker.
The most important function of a firewall under active port scanning attack is not
whether the firewall is able to detect the actual attack and warn the user. It is of far
greater importance to not to leak any information about the state of ports, no matter
the scanning or obfuscation techniques used. If the firewall warns user of every port
scan but fails to hide open ports, user doesn’t know what to do and the attacker gets
valuable information. To get the unified time of scanning, as well as no leakage of port
states, no firewall should respond to packet scanning probes. For firewalls which sent
48

56. 6. IDEAL BEHAVIOUR OF FIREWALL UNDER CERTAIN ATTACKS
Scanning technique Ideal port state
TCP SYN filtered
TCP connect() filtered
TCP FIN, TCP Xmas, TCP Null open|filtered
TCP ACK filtered
TCP Window filtered
TCP Maimon open|filtered
UDP scan open|filtered
SCTP Init filtered
SCTP COOKIE ECHO open|filtered
IP protocol scan open|filtered
Service and Version detection filtered
Table 6.1: Ideal Port states
Port scanning technique Ideal time (s)
TCP SYN, TCP FIN,
TCP Xmas, TCP Null,
TCP ACK, TCP Window, 22.91
TCP Maimon, UDP, Service
and Version detection
TCP connect() 45.56
SCTP Init, 2.34
SCTP COOKIE ECHO
IP protocol scan 6.72
Table 6.2: Ideal port scanning times
response to port scan probes indicating their state to be “closed”, the time of scanning
was radically different from those which marked them as “filtered”, when no response
was sent. Only Avira, Bitdefender, ESET, F-secure, Microsoft, Norton, Quickheal, and
Zonealarm achieved all “ideal” port states.
After observing the behaviour of various scanning techniques on 18 firewalls, I pro-
pose Table 6.1 to be an ideal port states to every technique used in Nmap. I also propose
Table 6.2 to be the ideal time for scanning the default amount of ports in Nmap. Note
that these values are those which most firewalls achieved and if all firewalls could
achieve these values, the fingerprinting based on the time differences would be effec-
tively countered. On the other hand, firewalls could also try to raise the time needed
for port scannings by orders of magnitudes, which would probably alleviate the risk
of being a random target.
Logging is also very important for more skilled users and system administrators.
If the firewall contain only the packet log, in which there are hundreds of messages,
searching for the actual attack becomes harder. If there are no filters that can be used
in logs, it is even more difficult. Screenshots of particular (relevant) logs after the port
49

57. 6. IDEAL BEHAVIOUR OF FIREWALL UNDER CERTAIN ATTACKS
Product Ideal port states on all ports
on all 13 scanning techniques
on default IPv4 on default IPv6
Agnitum, Avast!, AVG,
Emsisoft, Gdata, Kaspersky, no no
McAfee, Panda, TrustPort
Avira, Microsoft yes yes
Bitdefender yes -
COMODO no yes
ESET, F-Secure, Norton, yes no
Quick Heal, ZoneAlarm
Table 6.3: Firewalls under different attacks
Product Shown in Shown in Popup Filters
higher-level logs packet logs window
Agnitum yes yes yes no
Avast!, Bitdefender yes no no yes
AVG, Avira
Emsisoft, F-Secure no no no no
Gdata, Quick Heal
COMODO, Microsoft no no no yes
ESET yes no yes yes
Kaspersky yes no yes only search
McAfee yes yes no no
Norton no yes no only search
Panda yes no no only search
TrustPort, ZoneAlarm no yes no no
Table 6.4: Logs with port scanning
scanning attacks can be found on the attached CD. You can see the Table 6.4 for the
information about particular logs of firewalls. On the second column, there is informa-
tion whether the firewall logged information about “port scanning attack”. In the third
column, there is information whether port scanning packets were stored in packet logs.
This usually resulted in few hundreds of logged events within few seconds. Next col-
umn shows if user was made aware of port scanning attack by pop-up window. The
last column shows whether there are any filters in the particular firewalls’ logs.
Table 6.5 shows my final result from the portscanning attacks on firewalls. All 13
different Nmap techniques were used (-sS, -sT,-sA, -sW, -sM, -sU, -sN, -sF, -sX, -sY, -sZ,
-sO, -sV). The first joined column shows the number of techniques on which all ports
were in their ideal states. For example, the Kaspersky has both numbers (IPv4 and
IPv6) equal to 1 because only on -sY technique all ports were in their ideal state. Avira
and Microsoft scored the highest numbers (13 on IPv4 and 13 on IPv6), because they
were the only ones to not to leak any port information. Comodo, F-Secure, Norton and
Quick Heal were closely behind them with only 1 technique leaking port information
50

58. 6. IDEAL BEHAVIOUR OF FIREWALL UNDER CERTAIN ATTACKS
# of techniques with # of logged Fragmentation # of TCP/0 states
all ports in ideal states portscan techniques successful that are “filtered”
IPv4 IPv6 IPv4 IPv6 IPv4 IPv4 + IPv6
Agnitum 7 5 7 3 yes 2
Avast! 10 11 10 10 yes 0
AVG 10 8 0 0 - 2
Avira 13 13 0 0 - 2
Bitdefender 13 0 4 0 no 1
COMODO 12 13 0 0 - 2
Emsisoft 3 1 0 0 - 0
ESET 13 9 4 0 yes 2
F-Secure 13 12 0 0 - 2
Gdata 9 9 1 0 - 2
Kaspersky 1 1 3 3 no 0
McAfee 10 9 10 10 no 2
Microsoft 13 13 0 0 - 2
Norton 13 12 0 0 - 2
Panda 4 1 4 0 no 0
Quick Heal 13 12 0 0 - 2
TrustPort 10 1 0 0 - 1
ZoneAlarm 13 1 0 0 - 1
Table 6.5: Overall results of port scanning attacks
in each. The second joined column shows the number of detected and logged portscan-
ning techniques. We can see that only 4 firewalls are able to detect any port scans on
IPv6, in contrast to 8 on IPv4. The best performers were Avast! and McAfee which de-
tected 10 out of 13 scanning techniques on both IPv4 and IPv6. Agnitum, Bitdefender,
ESET, Gdata and Panda show degradation in detection from IPv4 to IPv6. Next col-
umn shows whether any fragmentation technique as an obfuscation of the attack was
successful. In other words, if the particular firewall could be fooled with using frag-
mentation into not detecting the portscan at all. Only Agnitum, Avast! and ESET could
be mislead by this. As stated earlier, I didn’t test firewalls which didn’t detect the port
scanning attack in the first place and thus there are “-” characters. The last column de-
scribes how many (0, 1 or 2) TCP/0 ports were in their ideal state - “filtered”. Only 11
firewalls had both TCP/0 ports (on IPv4 and IPv6) filtered.
6.2 Ideal behavior under the DoS attacks
With DoS attack, it is hard to estimate “why” firewalls are behaving the way they are
behaving. It is obvious that some firewalls could cope with floodings better than they
are coping with it now. Based on my results, there are huge differences in the total run-
ning time of the script to measure 20 values, CPU usage and bandwidth consumption.
I have to note, that there is a clear pattern of the trade-off between consuming network
and CPU resources with some DoS attack.
The ideal firewall should be able to detect the DoS attacks and try to counteract
51

59. 6. IDEAL BEHAVIOUR OF FIREWALL UNDER CERTAIN ATTACKS
them. In the worst case scenario, even disabling the particular network adapter for
a few seconds/minutes would be a viable solution. At least user’s device would not
be negatively suffering from the attack. Other, not-so-drastic measures could also be
taken. For example ignoring all the packets with some characteristics (used protocol,
IP address of sender, source/destination ports, ...).
The overall impact should be as low as possible - some firewalls are more successful
in this than others. I’ve summarized the overall comparison of the average highest core
usage in the Table 6.6. There are 11 DoS flooding attacks. In every attack column, the
threshold value in % was created using following guidelines:
• the maximum value is 90%
• the number of “yes” values should be the same or lower (but as close as possible)
as the number of “no” values
• go from the initial value of 30% up by 10% steps until the first 2 requirements are
met
• if there are the same results for more thresholds, use the lowest threshold value
With meeting these guidelines, it is possible to reach values, in which exact or lower
half of firewalls behaved better than the others. You can observe the total number of
firewalls which met the requirements in the last two rows. The last column is the final
score of particular firewall - the higher, the better. We can observe, that the best fire-
wall coping with DoS attack is the Bitdefender with the overall score 10 out of 11 tests.
The worst firewalls are Agnitum and Quick Heal, which scored 0 point. Be aware that
this is only 1 particular result out of 7 performance parameters I stored in every mea-
surement. Only by cross-referencing results from all 7 measurements could we come
close to determining the rank-list of these 18 firewalls with respect to their performance
while under DoS flooding attacks.
52