Microcaching, Scalabe stack

1. Delivery high performance
applications with Efficient
Cache Layer
Bruno Paiuca => ‘KaBuM! SA’
Twitter =>
‘https://twitter.com/BPaiuca’

2. whoami
> whoami
Bruno Paiuca, 28 anos, Palmeirense, Sysadmin coder.
Redes de computadores - FIAP
MBA Sistemas corporativos de alto desempenho - FIAP
Certificações:
- LPI
- CCNA
- RHCE
- AWS Solution Architect Associate
Tecnologias:
- Linux
- NGINX
- Perl
- Python
- SaltStack
- Docker
- AWS
- redis
- terraform
Twitter => https://twitter.com/BPaiuca
LinkedIn => https://www.linkedin.com/in/brunopaiuca/

3. sumário
Por que? Como?
- expectativa do usuário
- baixíssima latência
- altamente escaláveis
- altamente disponiveis
- redução de custos

4. por que?
study done by Amazon almost 10 years
ago proved that, even then, a
100-millisecond decrease in page-loading
time translated to a 1% increase in its
revenue.
According to surveys done by Akamai and
Gomez.com, nearly half of web users
expect a site to load in 2 seconds or less,
and they tend to abandon a site that isn’t
loaded within 3 seconds. 79% of web
shoppers who have trouble with web site
performance say they won’t return to the
site to buy again and around 44% of
them would tell a friend if they had a poor
experience shopping online.

5. Como?

6. NGINX
Web Server Load Balancer

7. NGINX
High Availability Content Cache

8. NGINX+
Kubernetes
Ingress
Controller
Programmability
MonitoringSecurity Control

9. NGINX CACHE

10. NGINX CACHE
How long?
What is your app? Pattern?
- Monolithic - REST
- Microservice - SOAP

11. NGINX CACHE
Advantages
- Reduce Server load - Better user experience
- Delivery more with less resources

12. proxy_cache_path
proxy_cache_path path [levels=levels] [use_temp_path=on|off]
keys_zone=name:size [inactive=time] [max_size=size]
Determina o diretório de armazenamento, level de subdiretórios,
size e nome da key_zone.
E.g. proxy_cache_path /dev/shm/nginx levels=1:2 use_temp_path=on
keys_zone=APPCACHE:1m inactive=300s max_size=1m;

13. proxy_cache_key
Syntax: Proxy_cache_key string;
Default: proxy_cache_key $scheme$proxy_host$request_uri;
Determina a key para armazenamento e busca de Match nos
objetos do Cache.
proxy_cache_key $scheme$proxy_host$uri$cookie_userid;

14. proxy_cache
Syntax: proxy_cache [cache zone];
Define em qual Cached Zone o objeto será buscado ou
armazenado.
proxy_cache APPCACHE;

15. proxy_cache_valid
Syntax: Proxy_cache_valid [http code] time;
Define a condição para um cache ser armazenado na cached
zone e o tempo em que o objeto será considerado Fresh.
proxy_cache_valid 200 301 302 304 1h;
proxy_cache_valid 404 1m;
proxy_cache_valid any 30s;

16. proxy_cache_lock
Syntax: proxy_cache_lock [off|on];
Default: proxy_cache_lock off;
Configura para apenas 1 request simultanea seja autorizado a
popular um novo cache object , enviando para o Upstream server.
proxy_cache_lock on;

17. Proxy_cache_purge
Syntax: Proxy_cache_purge [$variable];
Define uma condição para que a request seja considerado uma Purge
Request. Essa requisição limpa o objeto da cached zone, baseado na
cache_key. O resultado é retornado como HTTP 204 (No Content).
Essa é uma feature comercial.
map $request_method $purge_method {
PURGE 1;
default 0;
}
server {
proxy_cache_lock $purge_method;
}

18. Proxy_cache_bypass
Syntax: Proxy_cache_bypass [$variable];
Define uma condição para que a request ignore os objetos do
Cache Zone e a request seja enviada para o Upstream. O retorno
atualiza o objeto já armazenado na Cache Zone.
proxy_cache_lock $arg_skipcache;

19. Proxy_cache_use_stale
Syntax: Proxy_cache_use_stale [$variable];
Define casos em que uma STALE Cached response pode ser
utilizada.
E.g. proxy_cache_use_stale error timeout invalid_header http_500
http_502 http_503 http_504 http_404 updating;

20. Proxy_cache_background_update
Syntax: proxy_cache_background_update [off|on];
Default: proxy_cache_background_update off;
Permite que uma subrequest em background seja acionada
para atualização do objecto expirado, enquanto o objeto STALE
é retornado ao cliente.
proxy_cache_background_update on;

21. Proxy_pass
Syntax: proxy_pass url;
Configura qual o servidor Upstream vai responder a request.
proxy_pass http://backend;

22. upstream
Configura os servidores de backend, healthy checks intervals,
load balance algorithm.
resolver 10.0.0.1;
upstream dynamic {
zone upstream_dynamic 64k;
least_conn;
server backend1.example.com weight=5 max_fails=3 slow_start=30s timeout=5s resolve;
server backup1.example.com backup;
}
server {
location / {
proxy_pass http://dynamic;
health_check;
}
}

23. Cache status
● MISS – The response was not found in the cache and so was fetched from an
origin server. The response might then have been cached.
● BYPASS – The response was fetched from the origin server instead of served from
the cache because the request matched a proxy_cache_bypass directive (see Can
I Punch a Hole Through My Cache? below.) The response might then have been
cached.
● EXPIRED – The entry in the cache has expired. The response contains fresh
content from the origin server.
● STALE – The content is stale because the origin server is not responding correctly,
and proxy_cache_use_stale was configured.
● UPDATING – The content is stale because the entry is currently being updated in
response to a previous request, and proxy_cache_use_stale updating is
configured.
● REVALIDATED – The proxy_cache_revalidate directive was enabled and NGINX
verified that the current cached content was still valid (If-Modified-Since or If-
None-Match).
● HIT – The response contains valid, fresh content direct from the cache.

24. NGINX SHARDING

25. nginx
- High Capacity - Replication

26. nginx
Quanto maior for o Cache Tier - Maior a capacidade de
armazenamento e menos custoso o incidente de failover.

27. nginx

28. nginx

29. Sharding with hash consistent
upstream backend {
zone lb 32k;
server ngx_cache.service.consul:8081 resolve backup;
server 10.0.0.6:8081;
hash $scheme$proxy_host$uri consistent;
keepalive 32;
}
$uri -> ignora os query param.

30. MICROCACHING

31. Benchmarked - AB
1. Apache+WordPress
2. Reverse Proxy Only
3. Short-term Caching
4. Microcaching optimizations

32. Apache + WordPress
root@nginx-client:~# ab -c 10 -t 30 -k http://nginx-server/
Requests per second: 5.53 [#/sec] (mean)
Time per request: 1809.260 [ms] (mean)
Time per request: 180.926 [ms] (mean, across all
concurrent requests)
Transfer rate: 319.74 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 3
Processing: 1430 1735 259.4 1580 2228
Waiting: 537 683 119.7 624 980
Total: 1430 1735 259.4 1580 2228
root@nginx-server:/var/www/html# vmstat 3
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
10 0 0 136076 44944 585920 0 0 0 0 476 1665 96 4 0 0 0
10 0 0 140112 44952 585924 0 0 0 4 506 1773 98 2 0 0 0
10 0 0 136208 44952 585924 0 0 0 0 576 2057 97 3 0 0 0

33. reverse proxy only
root@nginx-client:~# ab -c 10 -t 30 -k http://nginx-server/
Requests per second: 5.63 [#/sec] (mean)
Time per request: 1774.708 [ms] (mean)
Time per request: 177.471 [ms] (mean, across all concurrent requests)
Transfer rate: 324.44 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 1
Processing:1423 1709 341.3 1532 2794
Waiting: 554 703 165.0 608 1165
Total: 1423 1709 341.4 1532 2794

34. short-term caching
root@nginx-client:~# ab -c 10 -t 30 -k http://nginx-server/
Complete requests: 18022
Requests per second: 600.73 [#/sec] (mean)
Time per request: 16.646 [ms] (mean)
Time per request: 1.665 [ms] (mean, across all concurrent requests)
Transfer rate: 33374.96 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.5 1 10
Processing: 0 16 141.5 3 2119
Waiting: 0 6 54.6 1 818
Total: 1 17 141.5 4 2121

35. microcaching optimizations
root@nginx-client:~# ab -c 10 -t 30 -n 100000 -k http://nginx-server/
Concurrency Level: 10
Time taken for tests: 30.001 seconds
Complete requests: 65553
Failed requests: 0
Keep-Alive requests: 0
Total transferred: 3728905623 bytes
HTML transferred: 3712974057 bytes
Requests per second: 2185.03 [#/sec] (mean)
Time per request: 4.577 [ms] (mean)
Time per request: 0.458 [ms] (mean, across all concurrent requests)
Transfer rate: 121379.72 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.3 1 5
Processing: 1 4 8.1 3 661
Waiting: 0 1 2.6 1 250
Total: 1 5 8.1 4 661 root@nginx-server:/var/www/html# vmstat 3
procs -----------memory---------- ---swap-- -----io---- -system--- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 106512 53192 641116 0 0 0 37 11016 3727 19 45 36 0 0
1 0 0 105832 53192 641808 0 0 0 68 17116 3521 13 56 31 0 0
1 0 0 104624 53192 643132 0 0 0 64 14120 4487 15 51 33 0 0

36. microcaching - why?

37. Microcaching
server {
proxy_cache one;
proxy_cache_lock on;
proxy_cache_valid 200 1s;
proxy_cache_background_update on;
proxy_cache_use_stale updating;
proxy_http_version 1.1; # Always upgrade to HTTP/1.1
proxy_set_header Connection ""; # Enable keepalives
# ...
}
upstream one {
zone wordpress 128k;
keepalive 32;
server backend:80 resolve;
}

38. Dúvidas?????

39. referências
https://www.nginx.com/blog/nginx-caching-guide/
https://www.nginx.com/blog/benefits-of-microcaching-nginx/
https://github.com/hashicorp/consul-template
https://nginx.org/en/docs/http/ngx_http_proxy_module.html
https://www.nginx.com/blog/10-tips-for-10x-application-performance/
https://blog.kissmetrics.com/speed-is-a-killer/
https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/

40. Obrigado!!!!!