IAC 2024 - IA Fast Track to Search Focused AI Solutions
networking
1. Rob Thomas
60 Days of Basic Naughtiness
Probes and Attacks Endured by an
Active Web Site
16 March 2001
2. Rob Thomas
60 Days of Basic Naughtiness
• Statistical analysis of log and IDS files.
• Statistical analysis of a two-day DDoS
attack.
• Methods of mitigation.
• Questions.
3. Rob Thomas
About the Site
• Production site for several (> 4) years.
• Largely static content.
• No e-commerce.
• Layers of defense – more on that later!
4. Rob Thomas
About the Data
• Data from router logs.
• Data from IDS logs.
• Snapshot taken from 60 days of combined
data.
• Data processed by several home-brew tools
(mostly Perl and awk).
5. Rob Thomas
Definition of “Naughty”
• Any traffic that is logged by a specific
“deny” ACL.
• Any traffic that presents a pattern detected
by the IDS software.
• The two log sources are not necessarily
synchronized.
6. Rob Thomas
Daily Probes and Attacks
• TCP and UDP Probes and Attacks – ICMP
not counted.
• Average – 529.00
• Standard deviation – 644.10!
• 60 Day Low – 83.00
• 60 Day High – 4355.00
7. Rob Thomas
Daily Probes and Attacks
Daily Probes and Attacks
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
11/17/00
11/22/00
11/27/00
12/2/00
12/7/00
12/12/00
12/17/00
12/22/00
12/27/00
1/1/01
1/6/01
1/11/01
Day
Hits
TCP
UDP
8. Rob Thomas
Weekly Probes and Attacks
• There is no steady-state.
• Attacks come in waves, generally on the
heels of a new exploit and scan.
• Certain types of scans (e.g. Netbios) tend to
run 24x7x365.
• Proactive monitoring, based on
underground and public alerts, will result in
significant data capture.
10. Rob Thomas
Hourly Probes and Attacks
• Myth: “Most attacks occur at night.”
• An attacker’s evening may be a victim’s
day – the nature of a global network.
• Truth: Don’t plan based on the clock.
12. Rob Thomas
UDP Probes and Attacks
Top Five Destination Ports
• First – 137 NETBIOS
• Second – 53 DNS
• Third – 27960
• Fourth – 500 ISAKMP
• Fifth – 33480 (likely UNIX traceroute)
13. Rob Thomas
UDP Probes and Attacks
Trend Analysis
UDP Probes and Attacks
0
50
100
150
200
250
300
350
11/17/00
11/24/00
12/1/00
12/8/00
12/15/00
12/22/00
12/29/00
1/5/01
1/12/01
Day
NumberofHits
Port 137 Hits
Port 53 Hits
14. Rob Thomas
TCP Probes and Attacks
Top Five Destination Ports
• First – 3663 (DDoS Attack)
• Second – 0 Reserved (DDoS Attack)
• Third – 6667 IRC (DDoS Attack)
• Fourth – 81 (DDoS Attack)
• Fifth – 21 FTP-control
15. Rob Thomas
TCP Probes and Attacks
Trend Analysis
TCP Probes and Attacks
0
20
40
60
80
100
120
11/17/00
11/24/00
12/1/00
12/8/00
12/15/00
12/22/00
12/29/00
1/5/01
1/12/01
Date
Hits
Port 0 Hits
Port 21 Hits
16. Rob Thomas
Source Address of Probes and
Attacks
Classful Sources of Probes and Attacks
0
500
1000
1500
2000
2500
3000
3500
A B C D E
IP Netblock Class
NumberofUniqueIPAddressesSeen
Source Address Class Percentage
20%
7%
20%
26%
27%
A
B
C
D
E
17. Rob Thomas
Source Address of Probes and
Attacks
Bogon Source Percentages
2346
803
2275
1128
167
270
0
500
1000
1500
2000
2500
3000
3500
4000
A B C
IP Netblock Class
UniqueIPAddresses
Bogon Addresses
Total Addresses
18. Rob Thomas
Source Address of Probes and
Attacks
• Bogon source attacks still common.
• Of all source addresses, 53.39% were in the
Class D and Class E space.
• Percentage of bogons, all classes –
66.85%!
• This is good news – prefix-list, ACL
defense, and uRPF will block 66.85% of
these nasties!
19. Rob Thomas
Source Region of the Naughty
A dangerously misleading slide
RIR for Source Addresses
58%
37%
5%
ARIN
RIPE
APNIC
20. Rob Thomas
Intrusion (attempt) Detection
• IDS is not foolproof!
• Incorrect fingerprinting does occur.
• You can not identify that which you can not
see.
21. Rob Thomas
Top Five IDS Detected Probes
IDS Detected Probes
0
200
400
600
800
1000
1200
1400
NetBus Backorifice TFTP IDENT Deep Throat
Type
Hits
22. Rob Thomas
Top Five Detected IDS Probes
IDS Detected Probes - Trend Analysis
0
20
40
60
80
100
120
140
160
180
1
4
7
10
13
16
19
22
25
28
31
34
37
40
43
46
49
52
Date
Hits
NetBus
Backorifice
TFTP
IDENT
Deep Throat
23. Rob Thomas
Top Five IDS Detected Attacks
IDS Detected Attacks
0
50
100
150
200
250
300
350
400
450
500
TCP Port 0 FIN flood Fragments ICMP flood RST flood
Type
Hits
Number
24. Rob Thomas
Top Five IDS Detected Sources
IDS Detected Source Netblocks
0
20
40
60
80
100
120
140
160
180
200
Azerbaijan USA 01 South Korea USA 02 Canada
Netblock Location
Hits
Count
25. Rob Thomas
Top Five IDS Detected Sources
IDS Detected Attacks - Trend Analysis
0
20
40
60
80
100
120
140
160
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
Day
Hits
A
B
C
D
E
26. Rob Thomas
Match a Source with a Scan
Source to Hit Matching
0
20
40
60
80
100
120
140
160
1 2 3 4 5 6 7
Day
Hits
B
NetBus
Backorifice
TFTP
IDENT
Deep Throat
27. Rob Thomas
Two Days of DDoS
• Attack that resulted in 10295 hits on day
one and 77466 hits on day two.
• Attack lasted 25 hours, 25 minutes, and 44
seconds.
• Quasi-random UDP high ports (source and
destination), small packets.
28. Rob Thomas
Two Days of DDoS
• Perhaps as many as 2000 hosts used by the
attackers.
• 23 unique organizations.
• 9 different nations located in the Americas,
Europe, and Asia.
• Source netblocks all legitimate.
29. Rob Thomas
Two Days of DDoS
Packets per minute
0
10
20
30
40
50
60
70
24:21:13
24:22:03
24:22:53
24:23:46
25:00:36
25:01:26
25:02:16
25:03:06
25:03:56
25:04:46
25:05:36
25:06:26
25:07:16
25:08:06
25:08:56
25:09:46
25:10:36
25:11:26
25:12:16
25:13:06
25:13:56
25:14:46
25:15:36
25:16:26
25:17:16
25:18:06
25:18:57
25:19:48
25:20:39
25:21:37
25:22:29
DATE:HOUR:MINUTE
Packets
30. Rob Thomas
Two Days of DDoS
DDoS Sources
0
500
1000
1500
2000
2500
3000
3500
4000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Hour
Packets
31. Rob Thomas
Site Defense and Attack
Mitigation
• While you can not prevent an attack, you
can choose how to react to an attack.
• Layers of defense that use multiple tools.
• Layers of monitoring and alert mechanisms.
• Know how to respond before the attack
begins.
32. Rob Thomas
Site Defense and Attack
Mitigation
• Border router
– Protocol shaping and filtering.
– Anti-bogon and anti-spoofing defense (uRPF),
ingress and egress filtering.
– NetFlow.
• IDS device(s)
– Attack and probe signatures.
– Alerts.
33. Rob Thomas
Site Defense and Attack
Mitigation
• Border firewall
– Port filtering.
– Logging.
– Some IDS capability.
• End systems
– Tuned kernel.
– TCP wrappers, disable services, etc.
– Crunchy through and through!
34. Rob Thomas
Site Defense and Attack
Mitigation
• Don’t panic!
• Collect data!
• The good news - you can survive!
35. Rob Thomas
References and shameless self
advertisements
• RFC 2267 - http://rfc.net/rfc2267.html
• Secure IOS Template –
http://www.cymru.com/~robt/Docs/Articles/secure-ios-
template.html
• Secure BGP Template –
http://www.cymru.com/~robt/Docs/Articles/secure-bgp-
template.html
• UNIX IP Stack Tuning Guide –
http://www.cymru.com/~robt/Docs/Articles/ip-stack-
tuning.html
37. Rob Thomas
Thank you for your time!
• Thanks to Jan, Luuk, and Jacques for
inviting me to speak with you today.
• Thanks to Surfnet/CERT-NL for picking up
the travel.
• Thanks for all of the coffee!
Notes de l'éditeur
This is part of a larger project – to analyze and track attack and probe methods and sources.
A holistic view of site probes and attacks.
To create an early warning and verification/validation system/site for others to use.
To track particularly popular source netblocks and assist the netblock owners with proper defense and mitigation.
The analysis was performed using copious amounts of Perl, sed, awk, and coffee.
The methods are still being honed – they are far from perfect.
Started with a series of simple questions raised by a manager :
“When did this attack start?”
“Did it slowly gain speed? Should we have seen it coming?”
“When will it end? Can we predict the end of the attack?”
Agreed to keep the site name and related data confidential.
I am always looking for more log files! Please feel free to share anything you have. Send it to my e-mail address.
The router and IDS logs are not in sync, unfortunately.
This is the first significant effort with copious amounts (942MB) of log data.
Previous efforts utilized a significantly smaller sample size.
ACLs – deny and log RPC, bogon source addresses, etc.
IDS – detect SYN floods, Backorifice probes, etc.
The data files do not overlap perfectly. This would be ideal for a completely holistic view of site probes and attacks.
Remember – NTP is your friend!
The standard deviation is higher than the mean! In other words, the ebb and flow of probes and attacks is of a greatly variable nature, and therefore difficult to model.
Note the 60 day low figure of 83.00. This means that there is never a completely “safe” day!
Note that never a day goes by without at least 83 probes and/or attacks!
The miscreants hit the site 24 x 7 x 365.
Why so many? Because the miscreants do NOT share data! Elucidate this point (IRC wars).
There is no “steady-state,” and there is no “normal” week or day.
The vulnerability du jour (e.g. BIND hole, sendmail hole) tends to wane over a two to three month period.
When an alert comes out – particularly one that lists a port or ports – fire up an ACL and watch the log entries grow.
We are now part of a global network – a network that NEVER sleeps.
The average DDoS attack lasts between 24 and 72 hours – the zombies don’t need a break.
Take a guess – what hour of the day will be the most utilized by the miscreants to hit this site?
With the exception of TCP 21, all other listed ports were part of a DDoS attack attempt.
TCP port zero remains very popular with those who target this site.
TCP port 21 (FTP) probes have been supplanted by other probes, such as DNS.
Note that Multicast and experimental source address packets account for 53% of the naughty packets.
Thus 53% of the nasty packets are blocked at our border!
What about the other classes? Are the sources therein all legitimate?
Class A bogon source percentage – 48.08%!
Class B bogon source percentage – 20.80%.
Class C bogon source percentage – 11.87%.
Class D and E bogon source percentage – 100.00%.
Block bogons, both inbound and outbound, at your border!
Add anti-spoofing at your egress points so that only source addresses within your allocated netblocks may pass.
Three attack types:
Spoof the source using bogon addresses.
Spoof the source using legitimate addresses.
No spoofing.
If all networks performed these basic steps, how successful would spoof attacks be?
The location of the miscreant is difficult to determine because of:
Spoof attacks.
Zombies.
Legacy netblock assignments.
Key point – the RIR or netblock does not necessarily indicate the location of the miscreant(s)!
Statistics can be misleading. Proper analysis requires a certain degree of “clue.”
Latest groups are from Brasil and Romania.
IDS is really a misnomer – it should be Intrusion Attempt Detection System, or IADS.
Intrusion Detection is easy – you will know your hull has been breached when your web page reads “Hackers love Ramen!”
IDS can be overwhelmed. Stick is nothing new – we have likely all seen it before.
Keep your IDS database current!
Place your IDS tool where it will provide the best detection and analysis.
Notice how the probes converge around day 13 and particularly days 26 and 27. This is not likely to be a coincidence!
Azerbaijan – ISP
USA 01 – Consulting Company
South Korea – ISP
USA 02 – ISP dialup pool
Canada – Cable modem
Notice how netblock B (USA 01) trends sharply upward around days 24 through 25.
In the raw data spreadsheet, the sudden appearance of netblock B coincides with the sharp increase in probe activity as seen in the Top Five Probes slide.
This was a relatively mild attack, and this makes for simple analysis. However, the analysis steps are the same regardless of attack intensity.
The use of legitimate source addresses makes defense that much more difficult.
This was an extremely mild DDoS – perhaps a test of things to come.
As with all DDoS attacks, the ramp up time was quite fast – on the order of several seconds. The end came just as suddenly.
Note the rise and fall of myriad sources. This is not uncommon during even the most intense DDoS attacks, and can be due to several factors, such as:
Congestion at the attacker site(s)
Reactionary filtering (local and intermediate) during the attacks
Zombie control issues
Insert the IRC “out of control zombie story” here.
Each layer should protect the layer beyond.
Each layer should integrate with the layers on either side.
Each layer should be untrusted by the next site-facing layer.
Funnel the traffic, filtering at an ever more granular level of detail.
Block bogons and prevent spoofing!
The presence of an IDS device does not obviate the need to have the other filtering devices logging (ACL logs, rule base logs, NetFlow, etc.).
Compare log entries often – remember, NTP is your friend.
