Rit 2011 ats
•
3 j'aime•3,125 vues
My Apache Traffic Server presentation for RIT++ in Russia, 2011.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
Similaire à Rit 2011 ats
Similaire à Rit 2011 ats (20)
Dernier
Dernier (20)
Rit 2011 ats
- 1. Apache Traffic Server Leif Hedstrom Principal Architect, Akamai Technologies lhedstro@akamai.com zwoop@apache.org @zwoop
- 2. History and introduction Choosing a proxy Setup
- 3. Origins of the software
- 5. Why HTTP proxy / cache? Better performance Save money (or other scarce resources) Make operations less of a pain in the … Easier web development Better user experience Lets look at the three common HTTP proxy configurations
- 9. Plenty of FOSS Proxy Servers
- 11. SMP Scalability and performance
- 12. Four Horseman of Bad Performance Data copies Context Switches Memory allocation Lock contention Avoid these! http://pl.atyp.us/content/tech/servers.html
- 13. Traffic Server threads model
- 14. Event Processing
- 15. Disk cache: “Cyclone” http://www.flickr.com/photos/gsfc/4461843139/
- 16. Some interesting features RAM: Optional compression Various compression levels supported Very fast (virtually no overhead using the faster algo) RAM: Several cache eviction algorithms CLFUS algorithm avoids the classic problem on an LRU during “scans” And it’s not encumbered by any patents Disk: Supports various block sizes (e.g. SSD) Disk: Can be tuned for various object sizes Disk: No file system overhead (RAW disks)
- 18. Traffic Server performance 400,000 requests / sec in a production CDN 50 Gbps / second (95-percentile) Many more request handled for HTML 100,000 requests / sec with small content out of RAM cache, on a single (lab) quad-core 3.6Gbps out of a single box, with larger content (4x GigE NIC bonded) 19Gbps on “localhost” using 2 CPUs (for large objects, 1MB in the test)
- 21. Ease of use, extensible
- 23. Traffic Server making Ops easy Traffic Server will restart automatically if it crashes Traffic Server will kill itself if it isn’t responding Traffic Server can reload most configurations without server restarts Adding plugins is a matter of dropping in the shared object, and restarting Speaking of plugins, we have a very flexible plugin APIs for C/C++ developers (hopefully some scripted languages soon) Stats and configurations are accessible via command line utilities
- 24. Configurations
- 25. plugin.config records.config volume.config hosting.config cache.config update.config ip_allow.config storage.config logs_xml.config parent.config remap.config icp.config
- 27. Critical in records.config CONFIG proxy.config.exec_thread.autoconfig INT 1 CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 2.0 CONFIG proxy.config.exec_thread.limit INT 2 CONFIG proxy.config.http.server_port INT 8080 CONFIG proxy.config.http.cache.http INT 1 CONFIG proxy.config.cache.ram_cache.size INT 512M CONFIG proxy.config.reverse_proxy.enabled INT 1 CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.url_remap.pristine_host_hdr INT 0 CONFIG proxy.config.ssl.enabled INT 0 CONFIG proxy.config.ssl.server.cert.filename STRING server.pem
- 28. CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.reverse_proxy.enabled INT 1
- 29. CONFIG proxy.config.url_remap.remap_required INT 0 CONFIG proxy.config.reverse_proxy.enabled INT 0
- 30. Content on the Edge
- 31. Traffic Server configuration CONFIG proxy.config.http.server_port INT 80CONFIG proxy.config.cache.ram_cache.size INT 4096M CONFIG proxy.config.url_remap.remap_required INT 1 map http://cdn.example.com/js http://js.example.com reverse_map http://js.example.com http://cdn.example.com/js map http://cdn.example.com/css http://css.example.com reverse_map http://css.example.com http://cdn.exampe.com/css map http://cdn.example.com/img http://img.example.com reverse_map http://img.example.com http://cdn.example.com/img /dev/sd2
- 32. Enemies of Web performance
- 36. ATS Connection mgmt configuration CONFIG proxy.config.http.server_port INT 80CONFIG proxy.config.cache.ram_cache.size INT 0 CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.http.send_http11_requests INT 1 CONFIG proxy.config.http.share_server_sessions INT 1 CONFIG proxy.config.http.server_max_connections INT 2000 CONFIG proxy.config.http.keep_alive_no_activity_timeout_out INT 60 CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 60 map http://www.example.com/ http://real-www.example.com
- 37. Executive Summary Understand your requirements Evaluate available software, services and solutions Apache Traffic Server is Fast and SMP scalable Battle hardened HTTP/1.1 compliant Easy to configure and manage Extensible Actively developed And of course, it’s free!
- 38. Content Delivery: Correctness first
Notes de l'éditeur
- I have divvied up this talk into three sections:A little bit of background and introduction to what an intermediary, proxy server, isATS configurationsUse cases
- Traffic Server started as a commercial product, developed and sold by Inktomi way back in the days Yahoo! Acquired Inktomi in 2003, and while developing our own CDN, we “found” Traffic Server laying on the shelves. Dusting it off and porting to modern Linux, it immediately beat existing intermediaries hands down in our benchmarks, typically by 5x or more In 2009, Y! donated the Traffic Server source code to the Apache Software Foundation. In April of 2010, Apache Traffic Server became a TLP.
- Before entering into the Apache community, the Y! version of TS was only running on 32-bit Linux. A direct benefit of being OpenSource gave us not only 64-bit support, but also ports to most common Linux distributions, FreeBSD, OpenSolaris and MacOSX. Performance has more than doubled, and in some cases, trippled, since we released he code into the Apache Open Source community, and most of these improvements have come from external contributors.
- * Before we go into details of what drives Traffic Server, and how we use it, let me briefly discuss the three most common proxy server configurations.* In a forward proxy, the web browser has to be manually (or via auto-PAC files etc.) configured to use a proxy server for all (or some) requests. The browser typically sends the “full” URL as part of the GET request.The forward proxy typically is not required to be configured for “allowed” destination addresses, but can be configured with Access Control List, or blacklists controlling what requests are allowed, and by whom. A forward proxy is typically allowed to cache content, and a common use case scenario is inside corporate firewalls.
- A reverse proxy, aka a web accelerator, does not require the browser to cooperate in any special way. As far as the user (browser) is concerned, it looks like it’s talking to any other HTTP web server on the internet. The reverse proxy server on the other hand must be explicitly configured for what traffic it should handle, and how such requests are properly routed to the backend servers (aka. Origin Servers). Just as with a forward proxy, many reverse proxies are configured to cache content locally. It can also help load balancing and redundancy on the Origin Servers, and help solve difficult problems like Ajax routing.
- An intercepting proxy, also commonly called a transparent proxy, is very similar to a forward proxy, except the client (browser) does not require any special configuration. As far as the user is concerned, the proxying happens completely transparently. A transparent proxy will intercerpt the HTTP requests, modify them accordingly, and typically “forge” the source IP before forwarding the request to the final destination. Transparent proxies usually also implements traffic filters and monitoring, allowing for strict control of what HTTP traffic passes through the mandatory proxy layer. Typical use cases include ISPs and very strictly controlled corporate firewalls. I’m very excited to announce that as of a few days ago, code for transparent proxy is available in the subversion tree.
- Traffic Server is obviously not the only HTTP intermediary in the Open Source community. Existing servers include Apache mod_proxy, Squid, NGINX, Varnish and Haproxy. This makes the task of choosing a Proxy server an interesting, but challenging task. You really need to understand your problem space, your requirements, and any restrictions (like, budget). Easy for me to pick, but lets discuss some of the considerations you should take.
- For me, there are three important areas to consider when choosing the proxy server (or probably, any other server for that matters): Performance and scalability Features Is it a good product for operations to manage, and for engineers to develop applications for? We’ll discuss these in details, but the goal for Apache Traffic Server is obviously to be smack in the middle of this Venn diagram. We’re not quite there yet.
- SMP scalability, how well does the server scale with multiple CPUs and cores? Can it take advantage of all available CPU (and other) resources on modern HW? What sort of throughput can the server handle? Request / second, or Mbps / sec etc. How many concurrent users can the server handle? Can it cope with thousands or tends of thousands of concurrent users?
- First time I met some of the original developers of Traffic Server, John Plevyak and George Paul both mentioned the “four horsemen”Foundational belief held by some of the original developers of Traffic Server
- There are n worker threads per core, typically 2. This gives around 16 – 24 threads of execution threads on typical modern hardware, each running an event loop There are m I/O threads per disk spindle. This is used to deal with disk I/O outside of the worker threads, and the default is 4. A critical configuration decision here is to scale this appropriately, particularly if a “disk” is raided, and might have more than 1 spindle under the hood. There are also a small number of “helper” threads, to do tasks like accepting new connections, produce log output and stats aggregation and presentation* All threads share resources, such as RAM and Disk cache, configurations, stats and logs.
- Events are scheduled by the event loop, and event handlers execute specific code for specific events This makes it easier to code for, there’s no risk of deadlock or race condition Can handle a good number of connections (but not unlimited) Squid is a good example of an event driven server.
- * RAW disks, striped and monitored internally* Very low overhead on CPU and disk I/O, since there is no LRU to manageVery low memory overhead (10 bytes per object stored on cache, compare to 60-70 bytes for Squid)Can support very large disks (0.5PB per “partition”)
- A very interesting feature of Apache Traffic Server is the built-in clustering support.This allows for a set of servers to act as a distributed cache. Each object is (generally) only hosted on one disk / cache.Cluster members are health-checked internally, and members are added (or removed) automaticallyConfigurations are also distributed throughout all cluster members.It also supports IP take-over, but this is honestly better handled at router / switch levels (IP take-over always has a risk of split brain syndrome)
- I wasn’t going to go into performance numbers, because out of context they are fairly useless, but here are some numbers from the Y! CDN and our lab. The Y! CDN is on some ~200 servers, most of which are idle most of the time. The reason for such a large deployment is because we cover most of the world, and also need to handle major outages as well as traffic spikes.
- HTTP/1.1 is the standard HTTP protocol in use today, most browsers and servers use and support it. So should your intermediary server. There are many extensions and additional features an intermediary might want to do, for example, ICP for cache peering. Getting every corner case of HTTP/1.1 is difficult, particularly for an intermediary. There is a lot of semantics overloaded into the standard HTTP headers. In many cases a regular HTTP server might not need to worry about all of this, but an intermediary probably has to.
- This table is much to large to go into details, but it shows that there are a number of features to take into consideration when choosing an intermediary. This is not a complete list in any way, it is merely an example of what features you might want to consider for your proxy choices.
- Easy to use, easy to configure, and generally easy to manage from an operational perspective. Resilience to crashes, corruptions and other operational nightmares. Extensible, making it easy to modify default behaviors, add functionality, extend with new code and features.
- Easy to use, easy to configure, and generally easy to manage from an operational perspective. Configuring your software should not be this hard! Defaults should be “reasonable”, and let you get started fast.
- Traffic Server will monitor itself, and restart the main server process if something isn’t functional. Even through process restarts, the HTTP port is still being listened on, and new requests are queued up in the listen backlog. Most configurations can be modified and reloaded, without server restarts Adding plugins is easy, just drop them in place and restart the server.
- The next section is going to be about configurations
- Traffic Server comes with a fairly large number of configuration files. Most of the configurations have reasonable “defaults”. Most applications wouldn’t use even a fraction of the available configuration options and features.
- Getting started, only two or maybe three configuration files are necessary to tweak. records.config is a key-value configuration format, holding most global application configuration settings storage.config is used to specify disk storage configurations remap.config is used to specify mapping rules for rewriting requests, typically in a reverse proxy setup
- Critical for performance: number of worker threads. The defaults are “ok”, but in a very high performance setup, tuning this can make a noticeable difference. Default port is 8080 Default is to run as a reverse proxy Easy to change to open forward proxy
- * The defaults are for a reverse proxy, now lets examine all the configuration changes necessary to turn this into a forward proxy server.
- Yes, that is it, there’s one bit of configuration changes between a reverse proxy and an open forward proxy. There’s not much that can be done to simplify this! Of course, you can have both reverse and forward proxy enabled at the same time!
- A CDN should preferably be on the edge, to provide static content close to the user. But at a minimum it needs to be distributed enough to deal with network and colo outages. The CDN should make network problems, server problems, or simple maintenance tasks (mostly) transparent to users. The CDN also should make it easy for operations to distribute content world wide. This is one reason why caching proxies are such a powerful tools for the Ops team. Finally, the CDN should hopefully save money for your company, using the cheapest possible distribution mechanism where possible.
- Configuring Apache Traffic Server for a basic CDN is surprisingly straight forward. The defaults from the installation are mostly setup, and only minor tweaking are necessary. First, update the key-value config file, records.config, with a few updated setting. In this example, I modify the HTTP port we’re listening on, and how much memory to use for the RAM cache. Secondly, we’ll need to provide the mapping rules for the reverse proxy. In this example, we provide both a forward and reverse mapping rule for each origin server, the reverse mapping is only necessary if you expect to receive HTTP redirect responses, and in that case, Traffic Server will rewrite the Location: header accordingly. Finally, you need to specify disk storage for the cache. It can be one or more raw partitions or directories on a file system.
- Before we go into the next use case scenario, lets discuss a couple of common problems that can severely affect your web sites performance.There are several reasons why a Web page might be slow to render, three common problems that I personally have to deal with are TCP 3-way handshake, TCP congestion control, and DNS lookups. We’ll discuss the first two here, and explain what we’ve done at Yahoo! using Traffic Server to alleviate these two problems.
- In TCP, every new connection has to go through a setup phase, typically referred to as the 3-way handshake. As you can see from the picture above, this means that there’s a full round-trip worth of latency before the client can even send the first HTTP request to the server. Since latency introduced by 3-way handshake is associated with network round-trip time, it goes without saying that the longer the distance between the client and the server, the longer the latency until the HTTP request can be sent. The solution to this problem is generally to use HTTP keep-alive between the client and server, which is a major reason why it is so critical for Traffic Server, and other intermediaries, to be able to handle tens of thousands of concurrent connections.
- TCP congestion avoidance is a mechanism for TCP to avoid congestion on our networks. There are many different implementations of TCP congestion control mechanisms, but the general idea is to start out ‘slow’, and increase the amount of packets we have outstanding on the wires before we wait for an acknowledgement from the receiver. This is why sometimes congestion control is referred to as “slow start”. Similar to 3-way handshake, this will introduce latencies which are directly related to the round-trip time between client and server. Keep-alive generally doesn’t solve this, since by default (sometimes configurable) an idle connection will force the restart of another slow start.
- To solve not only the 3-way handshake latency problems (which is easy), but to also solve the congestion avoidance latency problems, we’ve deployed Traffic Server farms all over the world. Users will always connect to a TS farm that is close, preferably in the 10’s of ms latency. For connections over long distances, the Traffic Servers will keep persistent connections to other servers (which could be another Traffic Server, or any other HTTP server that supports Keep-Alive). These connections are reused and share between many users, and the congestion avoidance resets are avoided because these server to server connections are generally kept active all the time.
- Configuring Apache Traffic Server for a basic CDN is surprisingly straight forward. The defaults from the installation are mostly setup, and only minor tweaking are necessary. First, update the key-value config file, records.config, with a few updated setting. In this example, I modify the HTTP port we’re listening on, and how much memory to use for the RAM cache. Secondly, we’ll need to provide the mapping rules for the reverse proxy. In this example, we provide both a forward and reverse mapping rule for each origin server, the reverse mapping is only necessary if you expect to receive HTTP redirect responses, and in that case, Traffic Server will rewrite the Location: header accordingly. Finally, you need to specify disk storage for the cache. It can be one or more raw partitions or directories on a file system.
- Besides all other features an intermediary must provides, there’s one that should always have priority: Correctness. If your network doesn’t deliver the data, correctly, timely, and as the web developers, and users, expect, you have an Epic Fail at your hands.
- A plugin APIs allows for developers to extend the capabilities of the Apache Traffic Server. A set of example plugins are available, which augments the developer SDK documentations. A set of production quality plugins are being prepared to be released by Yahoo to the Apache community. We encourage developers to contribute and release plugins, and will gladly accept contributions.
- Alright, so lets talk about what problems a good HTTP (and/or proxy server) can solve. There are two primary concurrency problems for the server software developers to consider:How can the software handle tens of thousands of concurrent TCP connections?How can the software take advantage of modern multi-core CPUs? Commodity hardware today has 2, or 4 and even 8 or more cores in each server.* Additionally, while solving these two problems, we have to make sure we don’t introduce other resource starvations, for example, memory pressure.
- Multithreading allows a process to split itself, and run multiple tasks in “parallel”. There is significantly less overhead running threads compared to individual processes, but threads are still not free. They need memory resources, and incur context switches. It’s a known methodology for solving the concurrency problem, and many, many server implementations relies heavily on threads. Modern OS’es have good support for threads, and standard libraries are widely available.
- Deadlocks, where two threads (or processes) need to acquire the same two resources (e.g. locks), which can cause the application to completely stall (unrecoverable) Race conditions can occur, where the outcome is not deterministic, but depends on timing or scheduling of threads execution. Difficult to code and ‘get right’.