HAProxy, the world’s fastest and most widely-used software load balancer, was first released in December 2001. The load balancer landscape has changed significantly since then. Yet HAProxy, with 17 years of active development under its belt, has continued to evolve and innovate. Now, we're announcing the release of HAProxy 1.9.

1. WWW.HAPROXY.COM
HAProxy 1.9

2. WWW.HAPROXY.COM
● Release cycle
● Internal infrastructure
● Performance
● Management
● Observability
● Evolutions
Agenda
HAProxy 1.9 - What’s new

3. WWW.HAPROXY.COMWWW.HAPROXY.COM
Release cycle

4. WWW.HAPROXY.COM
● Moved from once a year release to twice: May and November
● November-ish release:
○
○
○
● May-ish release:
○
○
○
HAProxy new release cycle
Release cycle

5. WWW.HAPROXY.COM
HAProxy new release cycle
Release cycle

6. WWW.HAPROXY.COM
Adoption of a regression testing suite
Release cycle
● The Varnish Cache project developed a very nice “varnishtest” utility to test proxies
● Its author encouraged us to adopt and extend it to support haproxy
● Varnishtest (now renamed Vtest) can now test both Varnish and HAProxy
● Supports testing complex configurations and client/server/log scenarios
=> helps detecting bugs before they are released
=> will improve the code’s quality over time
=> will avoid bug fixes creating new bugs
● Still few tests now but collection quickly improving
● Oh and we now build with -Wextra -Werror!
NOTE: features backported into HAProxy Enterprise 1.8r2

7. WWW.HAPROXY.COMWWW.HAPROXY.COM
Internal
infrastructure

8. WWW.HAPROXY.COM
● Internal native representation of the HTTP protocol(s)
● Agnostic to the HTTP version
● Still considered experimental when 1.9 is released
BENEFITS:
● Easy manipulation of any representation of the HTTP protocol
● Maintain HTTP transport AND semantics from end to end
● HTTP/2 on both sides :-)
● Higher performance than translating 2 to 1 or 1 to 2
● No more memmove() of the whole buffer to insert/remove/replace headers
WARNING: modifying HTTP buffer in Lua without using HAProxy functions is now prohibited
HTX: any HTTP version
Internal Infrastructure

9. WWW.HAPROXY.COM
HTTP processing in 1.8 (and 1.9 legacy mode)
Internal Infrastructure

10. WWW.HAPROXY.COM
HTTP processing in 1.8 (and 1.9 legacy mode)
Internal Infrastructure

11. WWW.HAPROXY.COM
HTTP processing in 1.9 (HTX mode)
Internal Infrastructure

12. WWW.HAPROXY.COM
HTTP processing in 1.9 (HTX mode)
Internal Infrastructure

13. WWW.HAPROXY.COM
● Move from a callback-oriented model to async events with completion callbacks
● One of the riskiest short-term change, but will save many bugs over the long term
BENEFITS:
● Lower send() latency (almost never polls, saves one poll loop)
● Less round-trips between layers, better I-cache efficiency
● More straightforward usage in upper layers
● No more code duplication in the lower layers
● Better / finer error reporting in lower layers
Connection management rework
Internal Infrastructure

14. WWW.HAPROXY.COM
Connection: example of H2 Rx processing in 1.8
Internal Infrastructure

15. WWW.HAPROXY.COM
Connection: example of H2 Rx processing in 1.9
Internal Infrastructure

16. WWW.HAPROXY.COM
Connection: 1.8 vs 1.9, side by side
Internal Infrastructure

17. WWW.HAPROXY.COM
● Multiple requests can share a same multiplexed connection (HTTP/2)
● The same connection can be reused by other requests
● Per-server connection pool maintained even after the last client closes
● Pool size adjusted based on the workload
BENEFITS:
● Server side keep-alive in almost every situation
● higher performance when clients closes connections
● Shorter response time when the server is far away from HAProxy
Server side connection pooling
Internal Infrastructure

18. WWW.HAPROXY.COM
● New internal buffers API can use any area for storage, no more “in” vs “out”
● New buffers will allow multi-size buffers later
● Merge of buffers and chunks API, dividing code implementations by 3!
● Logs may be emitted from lower layers now (e.g. HTTP/2)
● Removal of old synchronization point for a less intrusive rendez-vous point
Misc
Internal Infrastructure

19. WWW.HAPROXY.COMWWW.HAPROXY.COM
Performance

20. WWW.HAPROXY.COM
● 1.8 was the introduction of thread support
● 1.9 made them much more scalable :
○
○
○
○
○
BENEFITS:
● Almost as scalable as with multi-process now!
Threading improvements
Performance

21. WWW.HAPROXY.COM
● CPUs with variable frequencies turn to low-power mode while sleeping in poll()
● I/O events take hundreds of microseconds to bring the CPU back to full speed
● Busy polling prevents the CPU from sleeping at all
BENEFITS:
● Much lower latencies at the expense of much higher power consumption!
NOTES:
● be certain to properly map the CPUs!
● Real CPU usage may only be measured by haproxy, via the “Idle” value
Busy polling
Performance

22. WWW.HAPROXY.COM
● New set-priority-class and set-priority-offset actions for http-request and
tcp-content rules
● Applies to queues, hence when a server’s maxconn is reached
● Class defines priorities, offset defines relative delays
● Also available in Lua
BENEFITS:
● Give priority to some traffic
● Fine SLA management on service time
NOTE: features backported into HAProxy Enterprise 1.8r2
Traffic classification and prioritization
Performance

23. WWW.HAPROXY.COM
● Can cache any object size (1/256 of total cache size or max-object-size)
● Set Age header field
BENEFITS:
● Smoother caching of “big” objects
NOTE: features backported into HAProxy Enterprise 1.8r2
HTTP cache improvements
Performance

24. WWW.HAPROXY.COM
● The connection load balancing on SPOE was improved to support smarter load
balancing, very similar to the leastconn algorithm
BENEFITS:
● Much lower average and maximum processing latencies when using pipelining
SPOE improvements
Performance

25. WWW.HAPROXY.COMWWW.HAPROXY.COM
Management
Click to add text

26. WWW.HAPROXY.COM
● Server class
○
● TXN class
○
● New StickTable class
○
Lua
Management

27. WWW.HAPROXY.COM
● master CLI (configurable directly with -S in the program arguments)
● Access to old processes (proxied commands)
BENEFITS:
● Ability to access an old process which is in a leaving state
● Only one CLI listener for all processes
● ‘show proc’ displays the state of the processes
Master / Worker
Management

28. WWW.HAPROXY.COM
● Send logs to a fd in a non-blocking way
● Counter on lost log lines
BENEFITS:
● Useful when debugging
● Smoother integration with container environments
NOTE: features backported into HAProxy Enterprise 1.8r2
Logs to fd / std(out|err)
Management

29. WWW.HAPROXY.COM
● short: message containing level between angle brackets and the
text of the log line
● raw: message containing only the text of the log line
BENEFITS:
● Native integration with Containers (raw format)
● Native integration with journalctl (short format)
New log formats
Management

30. WWW.HAPROXY.COM
● Ability to provide a payload to a CLI command
● Allow “uploading” content or run multiple commands at once
● Applies to add map and set ssl ocsp-response for now
NOTE: features backported into HAProxy Enterprise 1.8r2
CLI: payload
Management

31. WWW.HAPROXY.COM
● “random” load balancing algorithm (NOTE: features backported into
HAProxy Enterprise 1.8r2)
● Significant reduction of memory usage for error captures
● Support for populating a resolvers section from resolv.conf
Misc
Management

32. WWW.HAPROXY.COMWWW.HAPROXY.COM
Observability

33. WWW.HAPROXY.COM
● Stolen CPU time (hypervisor, other processes)
● Time spent between 2 polls (minimum latency)
● Processing time and latency (to improve configuration)
BENEFITS:
● Detect when latency is induced by other processes sharing the CPU
● Figure how much of a request service time is due to other requests
● Figure how much service time a request inflicts to other requests
New timers
Observability

34. WWW.HAPROXY.COM
New timers: stolen time measurements
Observability

35. WWW.HAPROXY.COM
New timers: latency measurements
Observability

36. WWW.HAPROXY.COMWWW.HAPROXY.COM
Evolution

37. WWW.HAPROXY.COM
● New keyword to configure SSL cipher suite for TLS 1.3
(because openssl requires it, also backported in 1.8)
● Support of ALPN on the server side
TLS
Evolution

38. WWW.HAPROXY.COM
● Comparable to “HTTP/2 push” for HTTP/1 & 2 (RFC8297)
● Sends “Link: rel=preload” URLs very early (request parsing)
BENEFITS:
● Supports multiple servers
● No waste of bandwidth transfering already cached objects
● Cascadable, supports HTTP/1 and 2
WARNING:
● Not implemented in all browsers now (silently discarded)
NOTE: feature backported into HAProxy Enterprise 1.8r2
HTTP 103 / “Early Hints” status code
Evolution

39. WWW.HAPROXY.COM
QUESTION &
ANSWER