A presentation given at APRICOT 2015 during the Routing Session.

1. BGP Flowspec(RFC5575) Case study and Discussion
Shishio Tsuchiya
shtsuchi@cisco.com

2. • BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda

3. DDOS Traffic are always changing…
http://www.digitalattackmap.com/

4. Affect of DDOS attack
Customer
aggregation
node/line
Bandwidth of Backbone
Customer
line/node/servic
e
Target
Service
203.0.113.1
The affect would be all of network wide…

5. RTBH(Remote Triggered Black Hole Filtering)
Target
Service
203.0.113.1
203.0.113.1 via
192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
192.0.2.1 null0
203.113.1 192.0.2.1
• RTBH(RFC5635) is well known technic in ISP
• static route to null(Black hole) preliminarily
• If incidence happen then BGP advertises route
• DDOS traffic will be stopped

6. Netflow+BGP Attribute
Why BGP Flow Specification will be needed
 Non DDOS user also would be stopped.
 It is difficult to discover/ attempt rule against DDOS
attack which rapidly change and increasing

7. BGP Flowspec(RFC5575)+draft-ietf-idr-flow-spec-v6
Dst IP
Src IP
protocol
port
Dst port
Src Port
ICMP Type
ICMP Code
TCP Flags
Packet Length
DSCP
Fragment
traffic-rate
traffic-action
redirect
traffic-marking
Flow Type Action Rule
+---------------------------------------------------------+
| AFI(2 octets) 1 and 2 |
+---------------------------------------------------------+
| SAFI (1 octet) 133 and 134 |
+---------------------------------------------------------+
| Length of Next Hop Network Address (1 octet) |
+---------------------------------------------------------+
| Network Address of Next Hop (variable) |
+---------------------------------------------------------+
| Reserved (1 octet) |
+---------------------------------------------------------+
| Network Layer Reachability Information (variable) |
+---------------------------------------------------------+
SAFI
133 Dissemination of flow specification rules
134 L3VPN dissemination of flow specification rules
BGP Flowspec defined in RFC5575. draft-ietf-idr-flow-spec-v6 for IPv6 BGP Flowspec
Flow type to identify traffic , Action Rule to execute policy against the traffic
“Flow Type” and “Action Rule” will be advertised by BGP update

8. BGP Flowspec(RFC5575)
Target
Service
203.0.113.1
A,B,C to
203.0.113.1 drop
D and E to 203.0.113.1 100kbps
F markdown to dscp 0
100kbps
Netflow
collector
Flowspec uses netflow to collect traffic information
Flow rule and action will be distributed by BGP

9. • BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda

10. • DDOS Problem
• Affect Large/Often to end user
• Not only end user but also Infrastructure Risk
• OPEX increase
• DDoS Analysis
• Large DDOS attack by botnet armies/Script Kiddies
• TCP Syn Flood greater than 1Mpps
• UDP fragment
• Most of Attack source APNIC(Chinese) IP source , difficult to track due to national NAT
• Deployed Flowspec for Peer & Transit router from RR
• Mitigation from egress point to cleaning vrf
• What was missing ?
• Multi vendor support (deployed Juniper and Arbor)
• Inter-Carrier
• Matching DSCP
Flowspec Use case 1 world wide
Time Warner Telecom (TWTC) NANOG38 2006
Deployment Experience With BGP Flow Specification
https://www.nanog.org/meetings/nanog38/presentations/labovitz-bgp-flowsp

11. • Compare RTBH/PBR and Flowspec
• RTBH(Remote Triggered Black Hole)
Website can protect from DDOS attack, but no more traffic on website
• PBR(Policy Based Routing)
Can control traffic precisely by hardware
But need contact to service provide operator to run/remove policy when ddos detect
• Flowspec
Makes static PBR to dyanmic/Propagate PBR rules/do no need additional communication channel
• Deployed Flowspec on transit router
Would like to use on eBGP as architecture but can not trust customer/don’t like to use flow for ebgp session for
stability reason
• What’s Next
• IPv6 and VPNv6 support
• Traffic Monitoring
• More vendors(only Juniper and Alcatel support at that time)
Flowspec Use case 2 world wide
Neo Telecoms FRNOG18 2011
Flowspec
http://media.frnog.org/FRnOG_18/FRnOG_18-6.pdf

12. • Background
• Attacker use zombies, if number of army of zombies then DDOS traffic will be
massive (ex. DNS amp)
• Need Better tools
- Granularity : per flow
- Action : drop/rate-limit/redirect,
- Speedy/ Efficiency / Automation / Manageability
• Deployed FireCircle
• Wizard based UI to define policy from customer
• Apply XML configuration to BGP flowspec router via NETCONF
• eBGP flowspec propagate policy to GRNET router
• Expanding the service to GEANT community
https://fod.grnet.gr/
Flowspec Use case 3 world wide
GRNET(Greek Research and Technology Network) TNC2012
FireCircle: GRNET’s approach to advanced network security services’
management via bgp flow-spec and NETCONF
https://tnc2012.terena.org/core/presentation/41
NETCONF
FireCircle
GRNET
GEANT
Participant
NREN

13. • DDOS Volume(average)
• JAPAN Q2:491.63Mbps Q3:365.8Mbps
• Asia Q2:530.5Mbps Q3:588.74Mbps
• World Wide Q2:759.83Mbps Q3:858.98Mbps
• NTP Amp trend(average volume)
• JAPAN Q2:3.22Gbps Q3:281.76Mbps
• Asia Q2:2.57Gbps Q3:2.70Gbps
• Attack Duration
• 92% DDOS stops within 1hour
• JAPAN: >1hour 92% average 3h21m
• Asia: >1hour 94.1% average 31m
• Professional DDOS service is exist
ex)5min free 4$/hour
Atlas DDOS Trend report
Services UDP
Source Port
Q3
Maximum
DDOS Volume
Q3
Average
DDOS Volume
SNMP 161 3.75Gbps 769.1Mbps
Chargen 19 21.26Gbps 1.12Gbps
DNS 53 43.45Gbps 1.31Gbps
SSDP 1900 51Gbps 5.11Gbps
• What’s Next
• NTP Amp attack can create big volume.
• So Attacker using other protocol.
• SSDP(1900) is increasing
http://www.janog.gr.jp/meeting/janog35/files/2014/2077/3840/janog35-bgpfs-agatsuma-1.pdf

14. • ISP who is interesting in BGP Flowspec
• Amp attack are increasing under 5%-> over 70%
• and valuable
• Src 53 Dst 0/Src 123/Src 1900/Dst 80
Flowspec Use case 1
Protect Method For Point If Flowspec deployed
RTBH rapid action protect short duration DDOS more specific flow
can use policer for DDOS amp
ACL permanent action flexible/need time to deploy to be rapidly/manage acl rule
Mitigation premier service expensive would be effective

15. • ISP who already deployed by Juniper
• and would like to deploy to be more wide by Cisco
• Flowspec is very useful feature against today’s DDOS, but one consideration
point is scalability spec of forwarding router
• Rule was too long, so forwarding router could not apply filter as the result not
only DDOS but also normal traffic down
Flowspec Use case2
DDOS detect/BGP update send Rule was too long for forwarding router, cold not apply filter

16. • BGP Flowspec Overview
• BGP Flowspec case study
• JANOG35 Q&A
Agenda

17. • JANOG had a session of BGP Flowspec in JANOG35
Shishio Tsuchiya Cisco Systems G.K.
Shojiro Hirasawa BIGLOBE Inc.
Satoshi Agatsuma TOYO Corporation
http://www.janog.gr.jp/en/index.php?JANOG35_Meeting%2FJANOG35_Program_Contents%2FBGPFS
http://www.janog.gr.jp/meeting/janog35/program/bgpfs/
• Share question/discussion on JANOG35 meeting
Discussion summary

18. • Let’s confirm in detail for RFC and IETF WG draft.
Q1. Does Flowspec really useful?
Typ
e
IPv4
(RFC5575)
IPv6
(flow-spec-v6)
1 Destination Prefix Destination IPv6 Prefix
2 Source Prefix Source IPv6 Prefix
3 IP Protocol Next Header
4 Port Port
5 Destination port Destination port
6 Source port Source Port
7 ICMP type ICMP type
8 ICMP code ICMP type
9 TCP flags TCP flags
10 Packet length Packet length
11 DSCP DSCP
12 Fragment Fragment
13 N/A Flow Label
Flow Type has operator code which can
specify lt(less than) gt(grater than)
eq(equal) .

19. • Most of action rule is defined both IPv4 and IPv6.
• But redirect IP seems confusing , should watch idr wg activity
Q1. Does Flowspec really useful? cont’d
type extended community Actual Action RFC/draft
0x8006 traffic-rate Policing rate
0：drop
RFC5575
0x8007 traffic-action specific acction
Terminal bit:(0 is terminal)
Sample bit:(1 is logging/sampling)
RFC5575
0x8008
0x8208
0x800b
redirect AS-2byte
redirect AS-4byte
redirect IPv6 specific AS
redirect to specific vrf flowspec-redirect-rt-bis
flowspec-redirect-rt-bis
flow-spec-v6
0x8108 redirect IPv4 address
redirect IPv6 address
redirect to next hop address
redirect to next hop address
flowspec-redirect-rt-bis
flowspec-redirect-ip
flowspec-redirect-ip
0x8009 traffic-marking marking DSCP values flowspec-redirect-rt-bis
flow-spec-v6

20. • Cisco
IOS-XR:5.2.0-
IOS-XE3.14 –(RR)
Forwarding router in
3.15
• Juniper
JUNOS 7.3-
• Alcatel-Lucent
SR-OS 9.0R1-
Implementation status
• Arbor Networks
PeakFlow 6.0-
• Genie Networks
5.5.1-
• ExaBGP

21. Q2. How about interoperability in multi vendor?
Cisco IOS Cisco IOS-XR JNPR
JUNO
S
ALU
SR-OS
Arbor Genie
Cisco
IOS
     
Cisco
IOS-
XR
     
JNPR
JUNOS
     
ALU
SR-OS
     
Arbor      
Genie      
• There is some intorop report but may need more interop test to deploy ISP network

22. Q3.Flow is really enough to monitor ISP traffic?
DDOS Traffic
Normal Traffic
Inline type model offramp model
need many equipment to monitor all
of subscribers
can use shared resource
have to monitor huge traffic only suspect traffic will transit to
mitigation
when mitigation fail, the failed
equipment should just transit traffic
when mitigation fail, then advertise
BGP to change rule
offramp solution
would be reasonable

23. • Today’s most of mobile carrier deployed CGN as solution of IPv4 exhaustion problem.
• Malware/DDOS tool of android already exist.
• Flow based filtering will be more importance to reduce side affect of DDOS
Q4.How is DDOS on mobile network?
Global Address Global
Address
RFC6598 ISP Shared Address
or
RFC1918 Private Address

24. • It’s depends on router architecture.
APNIC38 Geoff Huston (APNIC) - What's so special about 512?
APRICOT2012 Greg Hankins, Brocade Pushing the Limits, A Perspective on Router Architecture Challenges
• Usually QoS/PBR is used on TCAM, so performance impact would
be minimize .
Q5.Performance issue?
https://supportforums.cisco.com/document/105496/asr9000xr-understanding-route-scale

25. • Flowspec should work in eBGP peer. But eBGP validation rule for received route should be relaxed.
• On transit AS/Router server on IXP, it would be desirable service. Because if one AS sends DDOS then
affects to another AS.
• Validation rule should be relax so maybe we should consider co-exist solution with RPKI to be more
powerful security solution.
• Should check “Revised Validation Procedure for BGP Flow Specifications” draft-ietf-idr-bgp-flowspec-oid
Q6.eBGP Use case?
ROA
Transit AS Route Server on IXP
co-Exist with RPKI

26. • There is Openflow DDOS protection solution.
• Hybrid OF use TCAM also.
• Difference point are network architecture(full distributed vs controller)
and API(OF vs BGP)
Q7.How is OpenFlow DDOS solution?

27. • Current DDOS are high volume/short duration/amp attack variable
and increasing
• BGP Flowspec is useful solution against today’s DDOS attack
• BGP Flowspec is almost ready to deploy in ISP network.
• Need detail implementation information of each of
vendors(scalability/nexthop address/IPv6) and interoperability test
result.
• eBGP should work and customer may desire on-demand
Firewall/PBR services like a FireCircle.
Summary