Life on the Edge with ESI

1. Life on the Edge with ESI
Kit Chan(kichan@yahoo-inc.com)
Yahoo!

5. Life on the Edge
Source: http://img.shockblast.net/2012/01/ShockBlast-4-803109.jpg

6. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

7. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

8. Edge Computing/CDN Source: http://www.excitingip.com/wp-content/uploads/2011/01/contentdeliverynetworkblockdiagram.jpeg

9. CDN in Yahoo!
- Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio
- Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio
User Location Latency - 100K file in US Through Y! Edge
California, USA 0.2s 0.02s
Singapore 2s 0.02s
Korea 1.3s 0.02s
Ireland 1.2s 0.2s
Switzerland 1.0s 0.2s

10. Edge Computing in Yahoo!
- Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio
- Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio
User Location Latency - 100K file in US Through Y! Edge
California, USA 0.2s 0.02s
Singapore 2s 0.02s
Korea 1.3s 0.02s
Ireland 1.2s 0.2s
Switzerland 1.0s 0.2s

11. ESI – Edge Side
Includes

12. Syntax Example
Markup in Response
<esi:include src=http://xyz.com/content.html/>

13. Syntax Example
Can handle response error and timeout
<esi:try><esi:attempt>
<esi:include src=http://xyz.com/content.html/ >
</esi:attempt><esi:except>
Sorry there is an error!
</esi:except></esi:try>

14. ESI Support in Proxy Server – Proven & Popular

15. ESI Support in CDN Vendor

16. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

17. Use Case 1 – Content Assembly

18. Use Case 1 – Content Assembly

19. Use Case 1 – Content Assembly

20. Use Case 1 - Content Assembly

21. Why? Performance!
No server cache +
No ESI
Server cache + No
ESI
ATS Cache + ESI
modules
Scenario
Origin Server
Handling Full Page
requests each time
and executing every
module every time.
Origin Server
handling full page
requests, only
executing non
cacheable modules
Origin Server only
serving requests for
non cacheable
modules
FE throughput per
box
45 queries per
second
140 queries per
second
190 queries per
second
Server latency 900 ms per request 80 ms per page 55 ms per page

22. Why? Availability
Why it goes down?
- Releases
- Outage
How ESI Help?
- Template cached
- Fallback for each ESI module

23. Why? Legacy Module Support

24. Use Case 2 – Combo Handler
Allow us to fetch multiple JS/CSS in one HTTP request
E.g. http://l.yimg.com/zz/combo?d/1.js&d/2.js
- It combines http://l.yimg.com/d/1.js and http://l.yimg.com/d/2.js
into one response in the server and returns one response
Minimize HTTP requests from browser to increase performance

25. Combo Handler Sequence Diagram

26. Sample ESI Doc from Combo URL Decode Service
<esi:try><esi:attempt>
<esi:include src=http://l.yimg.com/d/1.js/>
</esi:attempt><esi:except>
/* error fetching 1.js */
</esi:except></esi:try>
<esi:try><esi:attempt>
<esi:include src=http://l.yimg.com/d/2.js/ >
</esi:attempt><esi:except>
/* error fetching 2.js */
</esi:except></esi:try>

27. Why?
 ATS with ESI Plugin
- Provide sophisticated features for HTTP resource caching
- Provide other features such as timeout, collapse forwarding for request
- Provide capability of fetching contents simultaneously and assembling
them together

28. Use Case 3 - Timely Launch

29. Before Launch

30. After Launch

31. Timely Launch – Before Launch

32. Timely Launch – Before Launch

33. Timely Launch – After Launch

34. Timely Launch – After Launch

35. Usage Pattern
Client
Proxy
Server/CDN
ESI Service
Origin Server
for ESI Include

36. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

37. Concurrent Requests

38. First Byte Flush

39. ESI Support in Varnish
No concurrent requests for includes
First Byte Flush Support

40. ESI Support in Apache Traffic Server
Concurrent requests for includes
First Byte Flush Support*

41. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

42. History
The spec (ESI 1.0) is published over 10 years ago.
- W3C spec
- Editor - Mark Nottingham from Akamai
Mark Nottingham
http://www.mnot.net/

43. Future – Deep HTTP Integration
Client
Proxy Server
Supporting
ESI
Origin Server
returing ESI
Document
Origin Server
for ESI Include

44. Future – Deep HTTP Integration
Client
Proxy Server
Supporting
ESI
Origin Server
returing ESI
Document
Origin Server
for ESI Include
Final response
header to user
cannot be changed

45. Future – Deep HTTP Integration
Client
Proxy Server
Supporting
ESI
Origin Server
returing ESI
Document
Origin Server
for ESI Include
Final response
header to user can
be changed
Request Header,
method, timeout
cannot be changed for
includes

46. Future – Deep HTTP Integration
Client
Proxy Server
Supporting
ESI
Origin Server
returing ESI
Document
Origin Server
for ESI Include
Request/Response
Header cannot be
retrieved as variables
Final response
header to user can
be changed
Request Header,
method, timeout can
be changed for
includes

47. Future – Smart Assembly
One request for multiple
includes (e.g. one Ad call
fetching multiple ad position)

48. Future – More Use cases
Device Detection
Bucket Testing
http://illusionmedia.com/wp-content/uploads/2012/03/mobileDevices1-300x200.jpg
https://doazrm3qeh7n2.cloudfront.net/images/features/quick-a-vs-b-testing.jpg

49. Agenda
 Introduction
 Use Cases
 Performance Characteristics
 Future
 Q&A

50. Q & A
Thank you
kichan@yahoo-inc.com

51. Reference
 ESI Language Specification – http://www.w3.org/TR/esi-lang
 Edge Architecture Specification - http://www.w3.org/TR/edge-arch
 ATS ESI plugin -
https://github.com/apache/trafficserver/blob/master/plugins/experimental/esi/README
 Varnish ESI - https://www.varnish-cache.org/docs/3.0/tutorial/esi.html
 Akamai ESI - http://www.akamai.com/dl/technical_publications/akamai_esi_extensions.pdf
 F5 - http://support.f5.com/kb/en-
us/products/wa/manuals/product/wa9_4policy/WA_Policy_9_4-19-1.html

52. Extra: Moving to the Real Edge? – the Clients
 Problem with SEO
 Performance - More HTTP requests between clients and edge servers