Improving Site Response Time, Part 2

1. Improving Site Response Time
Part 2: Global Varnish Cluster with GeoDNS
Kim Stefan Lindholm 1 18.2.2012

2. SETUP
Same components as before with following additions/modifications:
• Varnish Server: Amazon EC2 Micro Instance (613 MB)
• Varnish Monitoring: New Relic, Munin, RevealCloud
• Edge Locations: Tokyo, California, Ireland
• GeoDNS: EdgeDirector
DNS and newly added Google Analytics + New Relic code may slightly affect load times
2

3. Front-end optimization can speed up a
website but doesn’t tackle network
latency. Content must be brought closer
to visitors.
3

4. LOAD TIME, BEFORE Target Speed
(1.5 Mbps connection)
Average: <3.5s
18 s Worst Case: <5s
14 s
Worst Case
9s Average
5s
Los Angeles São Paulo Stockholm Shanghai Sydney
✘ ✘ ✘ ✘ ✘
4

5. Solution: Set up remote Varnish caches
and direct visitors to the nearest one
with GeoDNS
5

6. Varnish serves files that it has previously cached (in its assigned geographic area)
Varnish fetches uncached or expired content through Incapsula, never directly from website
Incapsula fetches uncached content from origin server(s)
If edge server is unresponsive, DNS failover redirects to Incapsula or another Varnish
Website
Varnish Varnish
Varnish
Incapsula
6

7. LOCAL VS. REMOTE VARNISH
• Why would we place a Varnish box in the same region as the website?
• Severalcloud application platforms (CloudControl, Heroku, etc.) have
Varnish running locally but only default configuration is supported
• Incapsula
sets cookies but Varnish doesn't serve cached results to sessions
with cookies ➔ limited benefits with default settings
• We’dlike to see cache hit rates of up to 99.9 % so by manually
configuring Varnish we ensure that it actually does its job
7

8. LOAD TIME, AFTER Target Speed
(1.5 Mbps connection)
Dotted line denotes load time before Average: <3.5s
18 s Worst Case: <5s
14 s
Worst Case
9s Average
5s
Los Angeles São Paulo Stockholm Shanghai Sydney
✔ ✔ ✔ ✔ ✔
8

9. http://royal.pingdom.com/2010/11/12/real-connection-speeds-for-internet-users-across-the-world/
“The average connection speed for
Internet users worldwide, not just among
these 50 countries, but all countries, is
an average of 1.8 Mbit/s”
We’ll use 1.5 Mbit/s and 5.0 Mbit/s
connections in tests that follow
9

10. TARGET SPEED, AVERAGE
1.5 Mbps <3.5s
5.0 Mbps <2.5s
10

11. 1.5 Mbps
AVERAGE, BEFORE
CloudFront
4.0 s
3.5 s
2.5 s
3.8 s
4.4 s
2.9 s
2.7 s
3.5 s
6.0 s
4.9 s
3.7 s
4.9 s
5.8 s
<3.5s <3.5s <3.5s
5 measurements, >20 min intervals 11 DNS Provider: DNS Made Easy

12. 1.5 Mbps
AVERAGE, BEFORE
Akamai
3.6 s
3.3 s
2.1 s
4.5 s
5.0 s
3.1 s
2.7 s
3.7 s
6.2 s
4.8 s
4.9 s
4.0 s
4.8 s
<3.5s <3.5s <3.5s
5 measurements, >20 min intervals 12 DNS Provider: DNS Made Easy

13. 1.5 Mbps
CDN COMPARISON
CloudFront
Akamai
- 0.4 s
- 0.2 s
- 0.4 s
- 0.7 s
- 0.6 s
- 0.2 s
0.0 s
- 0.2 s
- 0.2 s
- 0.1 s
- 1.2 s
- 0.9 s
- 1.0 s
We’ll be using CloudFront for subsequent tests
5 measurements, >20 min intervals 13 DNS Provider: DNS Made Easy

14. 1.5 Mbps
AVERAGE, AFTER
CloudFront
1.8 s
3.4 s
1.9 s
1.5 s
3.0 s
2.3 s
1.8 s
2.1 s
2.8 s
3.1 s
2.6 s
2.8 s
2.5 s
<3.5s <3.5s <3.5s
5 measurements, >20 min intervals 14 DNS Provider: EdgeDirector

15. 5.0 Mbps
AVERAGE, AFTER
CloudFront
1.2 s
2.9 s
1.5 s
2.0 s
2.4 s
1.8 s
1.1 s
1.7 s
2.2 s
2.6 s
1.9 s
2.1 s
2.0 s
<2.5s <2.5s <2.5s
5 measurements, >20 min intervals 15 DNS Provider: EdgeDirector

16. TARGET SPEED, WORST CASE
1.5 Mbps <5.0s
5.0 Mbps <4.0s
16

17. 1.5 Mbps
WORST CASE, BEFORE
CloudFront
4.4 s
4.2 s
3.2 s
6.0 s
6.5 s
3.1 s
3.2 s
4.3 s
8.0 s
5.3 s
4.5 s
10.4 s
9.8 s
<5.0s <5.0s <5.0s
5 measurements, >20 min intervals 17 DNS Provider: DNS Made Easy

18. 1.5 Mbps
WORST CASE, AFTER
CloudFront
1.9 s
4.0 s
2.2 s
1.6 s
4.6 s
2.3 s
2.1 s
2.7 s
3.6 s
3.6 s
3.2 s
3.9 s
2.8 s
<5.0s <5.0s <5.0s
5 measurements, >20 min intervals 18 DNS Provider: EdgeDirector

19. 5.0 Mbps
WORST CASE, AFTER
CloudFront
1.4 s
3.0 s
1.7 s
3.9 s
3.2 s
1.9 s
1.2 s
2.2 s
2.8 s
2.9 s
2.2 s
2.2 s
2.2 s
<4.0s <4.0s <4.0s
5 measurements, >20 min intervals 19 DNS Provider: EdgeDirector

20. CDN doesn’t guarantee optimal
delivery locations. Where were assets
actually served from?
20

21. AKAMAI LOCATIONS
Files were loaded
from 2 continents
CloudFront seems to load all files from Seattle which is just a simple geo-IP lookup issue
21

22. How much does a minimal Varnish
cluster cost?
22

23. SERVERS
VPS / Cloud Server Locations RAM Storage Price/month
Linode 512 US, UK, Japan 512 MB 20 GB $19.95
Hetzner VQ 7 Germany 512 MB 20 GB €7.90
Zerigo Linux Denver, CO 512 MB 24 GB $19
Amazon EC2 micro US, EU, Asia, Brazil 613 MB 20 GB (EBS) $16.7 / $20.7
NB 1: Varnish should run on a 64-bit server
NB 2: Amazon prices are for California and Tokyo, reserved, heavy utilization, 30 GB traffic
23

24. GEODNS
Zerigo DNS DNS Made Easy DNS.com EdgeDirector
Pro 4 Business + GTD Basic / Pro + Monitoring
Monthly Queries 4 million 10 million 5 / 15 million Unlimited
$1.5 / monitor +
Monthly Price $20 $60 $34.95
$15 / M queries
+ DNS Failover X Included (3) X Included (1)
+ DNSSEC X X Included X
+ Vanity Name
Included Included $44.95 X
Servers
NB: For a small website, EdgeDirector can cost less than $3/month
24

25. Without DNS Failover
2 x Linode server + Zerigo DNS (1 M queries)
= 720 USD (~550 EUR) per year
With DNS Failover
2 x Linode server + EdgeDirector (1 M queries)
= 690 USD (~530 EUR) per year
25 Rates in February 2012

26. LEARNINGS
• Varnishmade a real improvement in response time (up to 7 sec) and as it
now handles more than 99 % of requests, a website with few DB write
operations performs fine even on shared cloud hosting
• Varnish boxes don’t scale up automatically but as DNS failover bypasses
failed edge servers, this doesn’t seem like a problem
• GeoDNS isn’t always necessary for accelerating a website: even one Varnish
serving all requests globally will significantly improve performance
26

27. LEARNINGS
• Amazon micro instance isn’t suitable for production use due to high CPU
steal
• Setting
up Varnish is pretty straightforward, see Part 3 for an example
configuration on Amazon EC2
27

28. RESOURCES
• Cloud & Virtual Private Servers (VPS)
• Linode, http://www.linode.com/
• Hetzner, http://www.hetzner.de/hosting/produktmatrix_vserver/vserver-produktmatrix
• Zerigo, http://www.zerigo.com/vps-servers
• Amazon EC2, http://aws.amazon.com/ec2/
• GeoDNS / Global Server Load Balancing (GSLB)
• http://www.zerigo.com/docs/managed-dns/setting_up_geodns
• http://help.dnsmadeeasy.com/gtd/configure-gtd-with-dns-failover/
• http://dns.com/Help/#configure
• http://edgedirector.com/how/both.htm
28