It's a rough world out there, filled with mega bot nets that threaten the availability of your web service. How do you keep your service running in the event of a 10,000x increase in traffic? Maximizing service availability under DDoS conditions requires thoughtful service architecture, and at times, fast acting operations teams. This presentation covers best practices for DDoS-resilient services.

1. DDoS Resilience with Amazon Web Services
nated@amazon.com
November 14, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. Agenda
•
•
•
•
Anatomy of DDoS
Things We Do So You Don’t Have To
Designing for Availability
Attack Response

3. DDoS Facts
• Yes, DDoS attacks are on the rise and the big
ones are getting bigger
• …although those attacks average out to
~14Gbps* and target services owners ~1 per
year
*source: Arbor Networks

4. DDoS Facts
Percentile
Max Gbps
10
20
30
40
50
60
70
80
90
95
99
2.39
4.28
6.55
8.27
10.49
11.85
13.97
17.38
25.45
35.74
84.90
Max
299.43
Average
Duration
(minutes)
5.87
7.68
9.00
10.53
13.23
16.80
23.12
35.87
66.13
141.74
906.80
13.81
*source: Arbor Networks

5. DDoS Anatomy
Application Exhaustion
/search.php?expensive-params
service
attacker

6. DDoS Anatomy
Host Exhaustion
attacker
service
attacker

7. DDoS Anatomy
attacker
Traditional Datacenter Exhaustion
attacker
traditional
datacenter
transit
attacker

8. DDoS Anatomy
attacke
r
attacke
r
Intermediary Exhaustion
attacke
r
transit
traditional
datacenter
attacke
r
transit
transit
attacke
r
transit
attacke
r
attacke
r

9. DDoS Anatomy
• Large enough attacks consume the capacity of
application layer, host, datacenter connectivity,
Internet connectivity, or intermediary networks

10. How can we help you?
• Scale and Diversity of AWS
• Resilient Service Designs
• Business or Enterprise Support

11. Things We Do So You Don’t Have To

12. model credit:
Scale

13. Scale
traditional
datacenter
transit

14. Scale
More Bandwidth
transit
AWS
region
transit
transit

15. Scale
More Compute
transit
AWS
region
transit
transit

16. Scale
More Points of Presence
transit
AWS
region
AWS
edge
AWS
edge
transit
transit
AWS
edge

17. Scale
Attack Absorbed
attacker
transit
AWS
region
AWS
edge
transit
transit
attacker
AWS
edge
AWS
edge
attacker

18. Diversity
transit
peer
Internet Transit and Peering
peer
peer
peer
transit
AWS
region
peer
transit

19. Diversity

20. Diversity

21. Diversity
Amazon Route 53 Example - Anycast Striping
• Leverages Resolver Behavior
• Edge Location Diversity
• Network Path Diversity

22. Delegation Set
[nated@xyz ~]$ dig NS internetkitties.com
;; QUESTION SECTION:
;internetkitties.com.
;; ANSWER SECTION:
internetkitties.com.
internetkitties.com.
internetkitties.com.
internetkitties.com.
IN
NS
172800
172800
172800
172800
IN
IN
IN
IN
NS
NS
NS
NS
ns-1131.awsdns-13.org.
ns-1751.awsdns-26.co.uk.
ns-340.awsdns-42.com.
ns-952.awsdns-55.net.

23. Delegation Set
[nated@xyz ~]$ dig NS internetkitties.com
;; QUESTION SECTION:
;internetkitties.com.
;; ANSWER SECTION:
internetkitties.com.
internetkitties.com.
internetkitties.com.
internetkitties.com.
IN
NS
172800
172800
172800
172800
IN
IN
IN
IN
NS
NS
NS
NS
ns-1131.awsdns-13.org.
ns-1751.awsdns-26.co.uk.
ns-340.awsdns-42.com.
ns-952.awsdns-55.net.

24. Edge Location Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

25. Edge Location Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

26. Edge Location Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

27. Edge Location Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

28. Edge Location Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

29. Network Path Diversity
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.
[nated@xyz ~]$ traceroute ns-1131.awsdns-13.org.
[nated@xyz ~]$ traceroute ns-1751.awsdns-26.co.uk.
traceroute to ns-1131.awsdns-13.org (205.251.196.107), 64 hops max, 52 byte packets
traceroute to ~]$ traceroute ns-340.awsdns-42.com.
1 (192.168.1.1) 1.748 ms 0.830 ms 0.750 ms
[nated@xyz ns-1751.awsdns-26.co.uk (205.251.198.215), 64 hops max, 52 byte packets
1 *traceroute to ns-340.awsdns-42.com (205.251.193.84), 64 hops max, 52 byte packets
(192.168.1.1) 1.298 ms 0.755 ms 0.694 ms
2 **
[nated@xyz ~]$ traceroute ns-952.awsdns-55.net.
2 ***
1 (192.168.1.1) 2.444 ms 1.676 (68.85.255.255) 14.634 ms 12.822 ms 10.774 ms
3 cat.seattle.wa.seattle.comcast.net ms 1.028 ms (205.251.195.184), 64 hops max, 52 byte packets
traceroute to ns-952.awsdns-55.net
3 cat.seattle.wa.seattle.comcast.net (68.85.255.255) 9.254 ms 24.156 ms 19.167 ms
2 ***
4 ae-20-0-ar03.burien.wa.seattle.comcast.net (69.139.164.125) 31.766 ms 13.898 ms
1 (192.168.1.1) 1.352 ms 0.642 ms 0.630 ms
4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 17.281 ms 18.580 ms 17.906
3 cat.seattle.wa.seattle.comcast.net (68.85.255.255) 19.842 ms 23.018 ms 26.469 ms
5 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 20.108 ms
2 ***
5 he-1-5-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.86.94.65) 20.753 ms 29.955 ms
4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 24.366 ms 20.842 ms
3 cat.seattle.wa.seattle.comcast.net (68.86.93.5) 18.781 ms
16.253
6 he-1-7-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.85.255.255)34.612 ms 30.382 ms 17.851 ms
6 ae12.edge2.seattle3.level3.net (4.68.63.65)(68.86.93.173) 30.211 ms ms 17.221 ms
5 he-1-12-0-0-10-cr01.seattle.wa.ibone.comcast.net 38.159 ms
4 be-1-ur08.seattle.wa.seattle.comcast.net (69.139.164.134) ms
13.561 ms
7 ae12.edge2.seattle3.level3.net (4.68.63.65) 34.371 ms 36.504 msms 49.457 ms 49.945 ms
7 ae-31-51.ebr1.seattle1.level3.net (4.69.147.150) 31.948 ms 29.775 ms
6 ae12.edge2.seattle3.level3.net (4.68.63.65) 33.596 ms 48.510 27.301
5 ae-1-0-ar03.seattle.wa.seattle.comcast.net (68.85.240.94) 21.009 ms ms
8 ae-31-51.ebr1.seattle1.level3.net (4.69.147.150) 48.557 ms 60.610 43.456 ms ms
8 ae-7-7.ebr2.sanjose1.level3.net (4.69.132.49) 45.286167.112 ms 161.82143.219
7 ae-32-52.ebr2.seattle1.level3.net (4.69.147.182) 162.580 ms ms ms 56.751 ms
6 he-1-12-0-0-11-cr01.seattle.wa.ibone.comcast.net
9 ae-7-7.ebr2.sanjose1.level3.net (4.69.132.49) 58.662 ms 46.830 ms 62.458 ms
9 ae-62-62.csw1.sanjose1.level3.net (4.69.153.18) 44.181(68.86.93.177) 17.366 ms 19.162 ms
8 ae-2-2.ebr2.denver1.level3.net (4.69.132.54) 163.723 ms 159.037 ms 174.670 ms
ms
7 be-12-pe03.seattle.wa.ibone.comcast.net (68.86.84.106)
ae-3-80.edge1.sanjose3.level3.net (4.69.152.144) 46.817 ms 19.949 ms 22.968 ms 24.976 ms
10 10 9 ae-3-3.ebr1.chicago2.level3.net (4.69.132.62) 60.700 ms 47.997 ms 54.477 ms
ae-2-2.ebr2.sanjose5.level3.net (4.69.148.141) 169.379 ms 167.307 ms 168.454 ms
8 ***
4.53.208.22 (4.53.208.22) 54.634 ms 60.111 ms 44.187 ms
11 11 10 ae-6-6.ebr1.chicago1.level3.net (4.69.148.201)166.002 ms 168.125 ms 164.232 ms
ae-6-6.ebr2.losangeles1.level3.net (4.69.140.189) 55.190 ms 58.829 ms 55.751 ms
9 ***
205.251.229.155 (205.251.229.155) 47.758 ms
12 12 11 ae-2-2.ebr2.newyork2.level3.net (4.69.132.66) 167.861 ms ms
ae-92-92.csw4.losangeles1.level3.net (4.69.137.30) 49.261 167.893 ms 160.681 ms
10 * 65-122-235-178.dia.static.qwest.net (65.122.235.178) 40.707 ms 30.916 ms
205.251.230.91 (205.251.230.91) 52.714 ms 43.560 53.091 ms
13 13 12 ae-1-100.ebr1.newyork2.level3.net (4.69.144.139)163.919 ms ms
ae-3-80.edge5.losangeles1.level3.net (4.69.135.253) 58.707 ms166.782 ms 161.686 ms
11
14 13 4.69.201.45 (4.69.201.45) 164.023 ms
SFO5 205.251.225.22 (205.251.225.22) 85.275 ms
14 amazon.com.edge5.losangeles1.level3.net (205.129.4.26) 46.477 ms 36.525 ms 42.110 ms
12 205.251.225.122 (205.251.225.122) 35.017 160.461 ms
14 ae-42-42.ebr2.london1.level3.net (4.69.137.69) 165.560 msms 38.568 ms
15 LAX3 13 205.251.226.136 (205.251.226.136) 36.560 ms
15
16
17
18
19
ae-46-46.ebr2.amsterdam1.level3.net (4.69.143.73) 165.627 ms
14 SEA50
ae-59-224.csw2.amsterdam1.level3.ne (t4.69.153.214) 172.909 ms 166.052 ms
4.69.162.154 (4.69.162.154) 166.353 ms
212.72.41.162 (212.72.41.162) 171.714 ms 174.033 ms 179.219 ms
AMS50

30. Striping in Action

31. Striping in Action

32. Striping in Action
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

33. Striping in Action
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

34. Striping in Action
awsdns-13.org.
awsdns-26.co.uk.
awsdns-42.com.
awsdns-55.net.

35. Diversity
client
AWS
edge
AWS
edge
AWS
region
AWS
edge
transit
client
AWS
edge
attacker

36. Diversity
• Amazon Route 53 - Anycast Striping
• Amazon CloudFront Edge Locations
• AWS Regions

37. How can we help you?
• Scale and Diversity of AWS
• Resilient Service Designs
• Business or Enterprise Support

38. How can we help you?
• Amazon Route 53 and Amazon CloudFront
• Resilient Service Designs
• Business or Enterprise Support

39. Designing for Resilience

40. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

41. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

42. N+1 Failover
• Scale Out, Plus Redundancy

43. N+1 Failover
• Scale Out, Plus Redundancy
• Failure of 1/100 < Failure of 1/10

44. N+1 Failover
• Scale Out, Plus Redundancy
• Failure of 1/100 < Failure of 1/10
• Automatic Failover with Health Checked DNS

45. N+1 Failover
attacker
client

46. N+1 Failover
attacker
client

47. N+1 Failover
Check out Amazon Route 53
Health Checks

48. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

49. Resilient Clients
• Use multi-record RRSets
• Randomize the record on connect retry
• Popular HTTP clients already do this!

50. Resilient Clients
[nated@xyz ~]$ dig www.internetkitties.com
;; QUESTION SECTION:
;www.internetkitties.com.
IN
;; ANSWER SECTION:
www.internetkitties.com. 32 IN
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
d3g5kqnbrlf3fg.cloudfront.net.
A
CNAME
30 IN A
30 IN A
30 IN A
30 IN A
30 IN A
30 IN A
30 IN A
30 IN A
d3g5kqnbrlf3fg.cloudfront.net.
54.230.69.190
54.230.71.141
54.230.71.172
54.230.71.233
54.240.188.66
54.230.68.41
54.230.68.212
54.230.69.141

51. Resilient Clients
Browser Packet Capture
Num
4
5
6
7
8
9
11
12
13
14
15
16
Time
2.535515
2.736659
2.93782
3.138996
3.339767
3.540963
3.541123
3.742296
3.824502
3.824515
4.024809
4.225094
Source
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
10.61.60.17
Destination
54.230.69.141
54.230.69.190
54.230.71.141
54.230.71.172
54.230.71.233
54.240.188.66
54.230.68.41
54.230.68.212
54.230.69.190
54.230.69.141
54.230.71.141
54.230.71.172
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]
[SYN]

52. Client Retry Behavior, SYN Timeout
Browser
OS
Rotates
IPs
Time to
Rotation
Chrome 30.0.1599
Windows 7
Yes
12
Internet Explorer 8
Windows 7
Yes
12
Firefox 25
Windows 7
Yes
20
Safari 5.0.5
Windows 7
Yes
20
Safari 6.0.5
OSX 10.7.5
Yes
<1
Firefox 25
OSX 10.7.5
Yes (2)
<1
Chrome 32.0.1678
OSX 10.7.5
Yes (2)
DNS TTL, or
Refresh

53. Resilient Clients
attacker
service
client

54. Resilient Clients

55. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

56. Capped Workloads
• Protect Application Layer Capacity
• Strive for Sameness
• Throttle or Sample Request Workloads

57. Strive for Sameness
Application Exhaustion
/search.php?expensive-params
service
attacker

58. Strive for Sameness
/search.php?expensive-params
attacker
service
Search_Result_Page_1

59. Capped Workloads
AppLayer
~1K to ~10K rps
Host/OS
~500K to 5M pps

60. Capped Workloads
AppLayer
~1K to ~10K rps
Host/OS
~500K to 5M pps
Core
DAL
Auth
Logging

61. Capped Workloads
AppLayer
~1K to ~10K rps
Throttle
~10 to ~100K rps
Core
DAL
Auth
Logging
Host/OS
~500K to 5M pps

62. Capped Workloads
AppLayer
~1K to ~10K rps
Throttle
~10 to ~100K rps
Host/OS
~500K to 5M pps
Core
DAL
Auth
Logging
1,000 samples /
sec

63. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

64. Process Isolation
• Isolate application components across
processes
• Let the OS protect critical resources

65. Process Isolation
Core
DAL
Auth
Logging

66. Process Isolation
Core
DAL
Auth
Logging

67. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

68. Evolution of Resilience
client
client

69. Evolution of Resilience
client
client

70. Evolution of Resilience
client
client

71. Evolution of Resilience
client
client

72. Evolution of Resilience
client
client

73. Evolution of Resilience
client
client

74. Evolution of Resilience
client
client

75. Evolution of Resilience
client
client

76. Evolution of Resilience
client
client

77. N Choose M Isolation
• 2 endpoints 2 AZs = 4 permutations

78. N Choose M Isolation
• 2 endpoints 2 AZs = 4 permutations
• 8 endpoints 2 AZs = 64

79. N Choose M Isolation
• 2 endpoints 2 AZs = 4 permutations
• 8 endpoints 2 AZs = 64
• 8 endpoints 3 AZs = 512

80. Shuffle Sharding – Amazon Route 53
• Define Availability Lattice
• Stripes – Edge Location
• Braids – Host Isolation
• Assign Endpoints to the Lattice
• Virtual Name Servers
• Allocate Endpoints to Resources
• Hosted Zone Delegate Set

81. Non-Overlapping Delegation Sets
;; QUESTION SECTION:
;gray.internetkitties.com.
IN NS
;; QUESTION SECTION:
;orange.internetkitties.org.
IN NS
;; ANSWER SECTION:
ns-1131.awsdns-13.org.
ns-1751.awsdns-26.co.uk.
ns-340.awsdns-42.com.
ns-952.awsdns-55.net.
;; ANSWER SECTION:
ns-1140.awsdns-14.org.
ns-1773.awsdns-29.co.uk.
ns-290.awsdns-36.com.
ns-989.awsdns-59.net.

82. Shuffle Sharding
.com
.net
.co.uk
.org

83. Shuffle Sharding
.com
.net
.co.uk
.org
ns-1773.awsdns-29.co.uk.
ns-1140.awsdns-14.org.

84. Shuffle Sharding
A
B
C
D
.com
.net
.co.uk
.org
ns-1773.awsdns-29.co.uk.
ns-1140.awsdns-14.org.

85. Shuffle Sharding
A
.com
.net
.co.uk
.org
B
gray.internetkitties.com
orange.internetkitties.org
C
D

86. Shuffle Sharding
A
.com
.net
.co.uk
.org
B
gray.internetkitties.com
orange.internetkitties.org
C
D

87. Non-Overlapping Delegation Sets
;; QUESTION SECTION:
;gray.internetkitties.com.
IN NS
;; QUESTION SECTION:
;orange.internetkitties.org.
IN NS
;; ANSWER SECTION:
ns-1131.awsdns-13.org.
ns-1751.awsdns-26.co.uk.
ns-340.awsdns-42.com.
ns-952.awsdns-55.net.
;; ANSWER SECTION:
ns-1140.awsdns-14.org.
ns-1773.awsdns-29.co.uk.
ns-290.awsdns-36.com.
ns-989.awsdns-59.net.

88. Shuffle Sharding Resilience
attacke
r
.co.uk
A
B
C
D
client
.org
A
B
C
D
gray.internetkitties.com
orange.internetkitties.org

89. Shuffle Sharding Resilience
attacke
r
.co.uk
A
B
C
D
client
.org
A
B
C
D
gray.internetkitties.com
orange.internetkitties.org

90. Shuffle Sharding Toolkit
•
•
•
•
Define a Lattice of Availability
Allocate Service Resources to the Lattice
Assign Customers Isolated Resources
https://github.com/awslabs/route53-infima

91. Lattice Configuration
// Create a 1-D lattice with "AvailabilityZone” as the dimension
OneDimensionalLattice<HealthCheckedRecordSet> myServiceLayout =
new OneDimensionalLattice<HealthCheckedRecordSet>("AvailabilityZone”);

92. Lattice Configuration
// Add endpoints in the us-west-1a Availability zone
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.1"));
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.2"));
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.3"));
…
// Add endpoints in the us-west-1b Availability zone
myServiceLayout.addEndpoint("us-west-1b”
new HealthCheckedRecordSet("192.0.2.11"));
…

93. Lattice Configuration
// Add endpoints in the us-west-1a Availability zone
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.1"));
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.2"));
myServiceLayout.addEndpoint("us-west-1a”,
new HealthCheckedRecordSet("192.0.2.3"));
…
// Add endpoints in the us-west-1b Availability zone
myServiceLayout.addEndpoint("us-west-1b”
new HealthCheckedRecordSet("192.0.2.11"));
…

94. Shuffle Shard
// Create a shuffle sharder
SimpleSignatureShuffleSharder shuffleSharder = new
SimpleSignatureShuffleSharder(5353L);
Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);

95. Shuffle Shard
// Create a shuffle sharder
SimpleSignatureShuffleSharder shuffleSharder = new
SimpleSignatureShuffleSharder(5353L);
Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);

96. Shuffle Shard
// Create a shuffle sharder
SimpleSignatureShuffleSharder shuffleSharder = new
SimpleSignatureShuffleSharder(5353L);
Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);

97. Vulcanized Lattice
// Create a shuffle sharder
SimpleSignatureShuffleSharder shuffleSharder = new
SimpleSignatureShuffleSharder(5353L);
Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);
// Create a RubberTree of DNS records
Route53RubberTree rubberTree =
new Route53RubberTree(”v123543234.video.internetkitties.com", shard);
List rrsets = rubberTree.vulcanize();

98. Lattice Shard RRSet
[nated@xyz ~]$ dig v123543234.video.internetkitties.com
;; QUESTION SECTION:
; v123543234.video.internetkitties.com. IN
A
;; ANSWER SECTION:
v123543234.video.internetkitties.com. 60 IN A
v123543234.video.internetkitties.com. 60 IN A
v123543234.video.internetkitties.com. 60 IN A
192.0.2.12
192.0.1.45
192.0.3.24
us-west-1b
us-west-1a
us-west-1c

99. Designing for Resilience
•
•
•
•
•
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation
Shuffle Sharding

100. Attack Response

101. Attack Response
• Detection
• Src-IP Blocking
• Engaging Customer Support

102. Attack Response
• Detection
• Src-IP Blocking
• Engaging Customer Support

103. Detect
• Traffic Spikes, Drops
• CPU Utilization
• Network Stats

104. Detect
• Use Resilience Patterns to Access Logs
• X-Forwarded-For
• Sort and Sum

105. X-Forwarded-For
• Use a trusted load balancer or proxy

106. X-Forwarded-For
• Use a trusted load balancer or proxy
• Enable logging

107. X-Forwarded-For
• Use a trusted load balancer or proxy
• Enable logging
– IIS7
• Install ‘IIS Advanced Logging’
• Configure X-Forwarded-For field

108. X-Forwarded-For
Enable Logging
if($http_x_forwarded_for !='-’) {
nginx:
log_format main '$http_x_forwarded_for - $remote_user
[$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$remote_addr"';
}
else {
log_format main '$remote_addr - $remote_user [$time_local]
$status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
}

109. X-Forwarded-For
• Use a trusted load balancer or proxy
• Enable X-Forwarded-For logging

110. Sort & Sum
• Used to identify “top talkers”
[nated@xyz.com ~]$ grep 'expensive-param' ./access.log | awk '{print $1}' |
sort | uniq -c | tail
2 10.54.4.1
3 10.63.34.1
5 10.23.97.212
1182 10.54.0.183

111. Sort & Sum
• Used to identify “top talkers”
[nated@xyz.com ~]$ grep 'expensive-param' ./access.log | awk '{print $1}' |
sort | uniq -c | tail
2 10.54.4.1
3 10.63.34.1
5 10.23.97.212
1182 10.54.0.183

112. Src-IP Blacklisting
•
•
•
•
Host-Level Firewalling
Web-Server Configuration
VPC Network ACLs
Web Application Firewall

113. Src-IP Blacklisting
•
•
•
•
Host-Level Firewalling (IPTables)
Web-Server Configuration (Nginx / Apache, IIS)
VPC Network ACLs
Web Application Firewall

114. Src-IP Blacklisting
•
•
•
•
Host-Level Firewalling
Web-Server Configuration
VPC Network ACLs
Web Application Firewall

115. VPC Network ACLs
• Apply to a VPC subnet
• Supports DENY rules

116. VPC Network ACLs
• Enter each source IP
• Set DENY

117. Src-IP Blacklisting
• Host-Level Firewalling
• VPC Network ACLs
• Web Application Firewall

118. Web Application Firewall
•
•
•
•
Src-IP Blacklist
HTTP Headers (X-Forwarded-For)
URI-Based Filtering
Advanced Throttling

119. Attack Response
• Detection
• Src-IP Blocking
• Engaging Customer Support

120. Engaging Customer Support
http://aws.amazon.com/premiumsupport/

121. Summary
How can we help?
Resilient Design
• Scale and Diversity
• Route 53 and CloudFront
• Business and Enterprise
Support
•
•
•
•
•
•
Attack Response
•
•
•
•
Enable X-Forwarded-For Logging
Detect, Sum and Sort
Src-IP Blacklist
Engage Customer Support
Availability Lattice
Shuffle Sharding
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation

122. Summary
How can we help?
Resilient Design
• Scale and Diversity
• Route 53 and CloudFront
• Business and Enterprise
Support
•
•
•
•
•
•
Attack Response
•
•
•
•
Enable X-Forwarded-For Logging
Detect, Sum and Sort
Src-IP Blacklist
Engage Customer Support
Availability Lattice
Shuffle Sharding
N+1 Failover
Resilient Clients
Capped Workloads
Process Isolation

123. Please give us your feedback on this
presentation
SEC305
As a thank you, we will select prize
winners daily for completed surveys!