Varnish presentation for the Symfony Zaragoza user group

r.v. var·nished, var·nish·ing, var·nish·es
1. To cover with varnish.
2. To give a smooth and glossy finish to.
3. To give a deceptively attractive appearance to; gloss over.
by jnerin 1

Server (cubietruck)
● AllWinnerTech SOC
A20
○ ARM® Cortex™-A7
Dual-Core
● 2GB ram
● SAMSUNG SpinPoint
F2 EG 1.5TB sata
hard disk
● 1Gb Ethernet
2

Server (cubietruck)
● Apache 2.2.27-r4
○ KeepAlive On
○ MaxKeepAliveRequests 100
○ KeepAliveTimeout 15
○ mpm worker:
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxClients 150
MaxRequestsPerChild 10000
● PHP 5.5.16 (opcache enabled by default
64M)
● mariadb 10.0.14
(have_query_cache=YES but
query_cache_size=0)
3

Test.sh
#!/bin/bash
URL="$1"
LOG="$URL.$(date +%Y%m%d-%H%M%S).log"
HOST="test"
IP="192.168.2.3:8080"
REQUESTS=10000
CONCURRENCY=20
exec 6<&1 # Save STDOUT FD
exec >$LOG
echo "- Simple test, no concurrency"
/usr/sbin/ab2 -n $REQUESTS -H "Host: $HOST" http://$IP/$URL
echo "- Simple test, concurrency"
/usr/sbin/ab2 -n $REQUESTS -c $CONCURRENCY -H "Host: $HOST" http://$IP/$URL
echo "- Concurrency & KeepAlive"
/usr/sbin/ab2 -k -n $REQUESTS -c $CONCURRENCY -H "Host: $HOST" http://$IP/$URL
echo "- Simple test, no concurrency (Accept-Encoding: gzip)"
/usr/sbin/ab2 -n $REQUESTS -H "Accept-encoding: gzip" -H "Host: $HOST" http://$IP/$URL
echo "- Simple test, concurrency (Accept-Encoding: gzip)"
/usr/sbin/ab2 -n $REQUESTS -c $CONCURRENCY -H "Accept-encoding: gzip" -H "Host: $HOST" http:
//$IP/$URL
echo "- Concurrency & KeepAlive (Accept-Encoding: gzip)"
/usr/sbin/ab2 -k -n $REQUESTS -c $CONCURRENCY -H "Accept-encoding: gzip" -H "Host: $HOST" http:
//$IP/$URL
exec 1<&6 6<&- # Restore STDOUT FD
egrep "^(-|Requests per second)" $LOG
4

Test1.html
<html>
<head>
<title>Test 1</title>
</head>
<body>
Test 1
</body>
</html>
Simple test, no concurrency
222.56 req/s
Simple test, concurrency
197.06 req/s
Concurrency & KeepAlive
679.81 req/s
(Accept-Encoding: gzip)
232.75 req/s
415.48 req/s
825.26 req/s
5

Test2.php
<html>
<head>
</head>
<body>

<?php
require_once('defaults.php');
$link = mysqli_connect($mysql_server,
$mysql_user,
$mysql_password,$mysql_database) or die
("Error " . mysqli_error($link));
if ($result = mysqli_query($link, "SELECT
temperature FROM oregon_data ORDER BY
receiver_timestamp DESC LIMIT 1")) {
echo mysqli_fetch_row($result)[0];
mysqli_free_result($result);
}
mysqli_close($link);
?>

</body>
</html>
120.61 req/s
236.65 req/s
271.19 req/s
94.80 req/s
108.93 req/s
163.97 req/s
6

MySQL Query Cache (Qcache)
mysql -e 'show global variables like "%query_cache%";show status like "Qcache%"'
+------------------------------+----------+
| Variable_name | Value |
+------------------------------+----------+
| have_query_cache | YES |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 33554432 |
| query_cache_strip_comments | OFF |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
+------------------------------+----------+
+-------------------------+----------+
| Variable_name | Value |
+-------------------------+----------+
| Qcache_free_blocks | 1 |
| Qcache_free_memory | 33544000 |
| Qcache_hits | 60030 |
| Qcache_inserts | 10 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 97 |
| Qcache_queries_in_cache | 1 |
| Qcache_total_blocks | 4 |
+-------------------------+----------+
7

Test2.php + QCache
<html>
<head>
</head>
<body>

<?php
$link = mysqli_connect($mysql_server,
$mysql_user,
}
?>

</body>
</html>
152.21 req/s
116.98 req/s
165.25 req/s
122.06 req/s
252.50 req/s
288.17 req/s
8

Test3.php (QCache + pconnect)
<html>
<head>
</head>
<body>

<?php
$link = mysqli_connect("p:$mysql_server",
$mysql_user,
}
?>

</body>
</html>
177.35 req/s
243.24 req/s
484.15 req/s
176.79 req/s
325.63 req/s
388.58 req/s
9

Let's try varnish
varnishd -a :8080
-s malloc,200M
-f /etc/varnish.vcl
-T 127.0.0.1:8081
-u apache -g apache
-t 120
10

Let's try varnish
varnishd -a :8080
-s malloc,200M
-f /etc/varnish.vcl
-T 127.0.0.1:8081
-u apache -g apache
-t 120
465.20 req/s
1053.24 req/s
958.43 req/s
454.48 req/s
948.18 req/s
2361.02 req/s
11

Obligatory Graph "Lies, damned lies, and statistics" (Mark Twain)
12

Varnish Control Language (VCL)
https://www.varnish-software.com/static/book/VCL_Basics.html
13

1 vcl 4.0;
2 # Based on: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/
blob/master/default.vcl
3 # Corrected & improved for 4.0.2 by jnerin@gmail.com
4 import std;
5 import directors;
6 backend server1 { # Define one backend
7 .host = "127.0.0.1"; # IP or Hostname of backend
8 .port = "80"; # Port Apache or whatever is listening
9 .max_connections = 300; # That's it
10 .probe = {
11 #.url = "/"; # short easy way (GET /)
12 # We prefer to only do a HEAD /
13 .request =
14 "HEAD / HTTP/1.1"
15 "Host: localhost"
16 "Connection: close";
17 .interval = 5s; # check the health of each backend every 5 seconds
18 .timeout = 1s; # timing out after 1 second.
19 # If 3 out of the last 5 polls succeeded the backend is considered healthy,
otherwise it will be marked as sick
20 .window = 5;
21 .threshold = 3;
22 }
23 .first_byte_timeout = 300s; # How long to wait before we receive a first byte from
our backend?
24 .connect_timeout = 5s; # How long to wait for a backend connection?
25 .between_bytes_timeout = 2s; # How long to wait between bytes received from our
backend?
26 }
27 acl purge {
28 # ACL we'll use later to allow purges
29 "localhost";
30 "127.0.0.1";
31 "::1";
32 }
16

34 /*
35 acl editors {
36 # ACL to honor the "Cache-Control: no-cache" header to force a refresh but only from selected IPs
37 "localhost";
38 "127.0.0.1";
39 "::1";
40 }
41 */
42
43 sub vcl_init {
44 # Called when VCL is loaded, before any requests pass through it. Typically used to initialize
VMODs.
45
46 new vdir = directors.round_robin();
47 vdir.add_backend(server1);
48 # vdir.add_backend(server...);
49 # vdir.add_backend(servern);
50 }
51
52 sub vcl_recv {
53 # Called at the beginning of a request, after the complete request has been received and parsed.
Its purpose is to decide whether or not to serve the request, how to do it, and, if applicable, which
backend to use.
54 # also used to modify the request
55
56 set req.backend_hint = vdir.backend(); # send all traffic to the vdir director
57
58 if (req.restarts == 0) {
59 if (req.http.X-Forwarded-For) { # set or append the client.ip to X-Forwarded-For
header
60 set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
61 } else {
62 set req.http.X-Forwarded-For = client.ip;
63 }
64 } 17

66 # Normalize the header, remove the port (in case you're testing this on various TCP ports)
67 set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
68
69 # Normalize the query arguments
70 set req.url = std.querysort(req.url);
71
72 # Allow purging
73 if (req.method == "PURGE") {
74 if (!client.ip ~ purge) { # purge is the ACL defined at the begining
75 # Not from an allowed IP? Then die with an error.
76 return (synth(405, "This IP is not allowed to send PURGE requests."));
77 }
78 # If you got this stage (and didn't error out above), purge the cached result
79 return (purge);
80 }
81
82 # Only deal with "normal" types
83 if (req.method != "GET" &&
84 req.method != "HEAD" &&
85 req.method != "PUT" &&
86 req.method != "POST" &&
87 req.method != "TRACE" &&
88 req.method != "OPTIONS" &&
89 req.method != "PATCH" &&
90 req.method != "DELETE") {
91 /* Non-RFC2616 or CONNECT which is weird. */
92 return (pipe);
93 }
94
95 # Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-
websockets.html)
96 if (req.http.Upgrade ~ "(?i)websocket") {
97 return (pipe);
98 }
18

Effect of set req.url = std.querysort(req.url);
$ GET -de "http://test:8080/varnish.vcl.html?q1=1&q2=2&q3=3" |egrep
"(Cache|Age|ETag|Expires|Date)"
Cache-Control: max-age=14400
Date: Wed, 08 Oct 2014 14:25:28 GMT
Age: 984
ETag: W/"3d838b6-b4f1-504e9efa40148"
Expires: Wed, 08 Oct 2014 18:25:28 GMT
Client-Date: Wed, 08 Oct 2014 14:41:53 GMT
X-Cache: HIT
X-Cache-Hits: 5
$ GET -de "http://test:8080/varnish.vcl.html?q2=2&q1=1&q3=3" |egrep
"(Cache|Age|ETag|Expires|Date)"
Date: Wed, 08 Oct 2014 14:25:28 GMT
Age: 994
ETag: W/"3d838b6-b4f1-504e9efa40148"
Expires: Wed, 08 Oct 2014 18:25:28 GMT
Client-Date: Wed, 08 Oct 2014 14:42:02 GMT
X-Cache: HIT
X-Cache-Hits: 6
19

100 # Only cache GET or HEAD requests. This makes sure the POST requests are always passed.
101 if (req.method != "GET" && req.method != "HEAD") {
102 return (pass);
103 }
104
105 # Some generic URL manipulation, useful for all templates that follow
106 # First remove the Google Analytics added parameters, useless for our backend
107 if (req.url ~ "(?|&)
(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
108 set req.url = regsuball(req.url, "&
(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_-.%25]+)", "");
109 set req.url = regsuball(req.url, "?
(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_-.%25]+)", "?");
110 set req.url = regsub(req.url, "?&", "?");
111 set req.url = regsub(req.url, "?$", "");
112 }
113
114 # Strip hash, server doesn't need it.
115 if (req.url ~ "#") {
116 set req.url = regsub(req.url, "#.*$", "");
117 }
118
119 # Strip a trailing ? if it exists
120 if (req.url ~ "?$") {
121 set req.url = regsub(req.url, "?$", "");
122 }
20

124 # Some generic cookie manipulation, useful for all templates that follow
125 # Remove the "has_js" cookie
126 set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
127
128 # Remove any Google Analytics based cookies
129 set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
130 set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
131 set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
132 set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
133 set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
134
135 # Remove the Quant Capital cookies (added by some plugin, all __qca)
136 set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
137
138 # Remove the AddThis cookies
139 set req.http.Cookie = regsuball(req.http.Cookie, "__atuvc=[^;]+(; )?", "");
140
141 # Remove a ";" prefix in the cookie if present
142 set req.http.Cookie = regsuball(req.http.Cookie, "^;s*", "");
143
144 # Are there cookies left with only spaces or that are empty?
145 if (req.http.cookie ~ "^s*$") {
146 unset req.http.cookie;
147 }
21

149 # Normalize Accept-Encoding header
150 # straight from the manual: https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
151 # TODO: Test if it's still needed, Varnish 4 now does this by itself if http_gzip_support =
on
152 # https://www.varnish-cache.org/docs/trunk/users-guide/compression.html
153 # https://www.varnish-cache.org/docs/trunk/phk/gzip.html
154 if (req.http.Accept-Encoding) {
155 if (req.url ~ ".(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
156 # No point in compressing these
157 unset req.http.Accept-Encoding;
158 } elsif (req.http.Accept-Encoding ~ "gzip") {
159 set req.http.Accept-Encoding = "gzip";
160 } elsif (req.http.Accept-Encoding ~ "deflate") {
161 set req.http.Accept-Encoding = "deflate";
162 } else {
163 # unkown algorithm
164 unset req.http.Accept-Encoding;
165 }
166 }
167
168 if (req.http.Cache-Control ~ "(?i)no-cache") {
169 #if (req.http.Cache-Control ~ "(?i)no-cache" && client.ip ~ editors) { # create the acl
editors if you want to restrict the Ctrl-F5
170 # http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh
171 # Ignore requests via proxy caches and badly behaved crawlers
172 # like msnbot that send no-cache with every request.
173 if (! (req.http.Via || req.http.User-Agent ~ "(?i)bot" || req.http.X-Purge)) {
174 #set req.hash_always_miss = true; # Doesn't seems to refresh the object in
the cache
175 return(purge); # Couple this with restart in vcl_purge and X-Purge header
to avoid loops
176 }
177 }
22

179 # Large static files are delivered directly to the end-user without
180 # waiting for Varnish to fully read the file first.
181 # Varnish 4 fully supports Streaming, so set do_stream in vcl_backend_response()
182 if (req.url ~ "^[^?]*.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av])
(?.*)?$") {
183 unset req.http.Cookie;
184 return (hash);
185 }
186
187 # Remove all cookies for static files
188 # A valid discussion could be held on this line: do you really need to cache static files
that don't cause load? Only if you have memory left.
189 # Sure, there's disk I/O, but chances are your OS will already have these files in their
buffers (thus memory).
190 # Before you blindly enable this, have a read here: http://mattiasgeniar.
be/2012/11/28/stop-caching-static-files/
191 if (req.url ~ "^[^?]*.
(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|pdf|png|rtf|swf|txt|woff|xml)(?.*)?$") {
192 unset req.http.Cookie;
193 return (hash);
194 }
195
196 # Send Surrogate-Capability headers to announce ESI support to backend
197 set req.http.Surrogate-Capability = "key=ESI/1.0";
198
199 if (req.http.Authorization) {
200 # Not cacheable by default
201 return (pass);
202 }
203
204 return (hash);
205 }
23

207 sub vcl_pipe {
208 # Called upon entering pipe mode. In this mode, the request is passed on to the backend, and any
further data from both the client and backend is passed on unaltered until either end closes the
connection. Basically, Varnish will degrade into a simple TCP proxy, shuffling bytes back and forth.
For a connection in pipe mode, no other VCL subroutine will ever get called after vcl_pipe.
209
210 # Note that only the first request to the backend will have
211 # X-Forwarded-For set. If you use X-Forwarded-For and want to
212 # have it set for all requests, make sure to have:
213 # set bereq.http.connection = "close";
214 # here. It is not set by default as it might break some broken web
215 # applications, like IIS with NTLM authentication.
216
217 #set bereq.http.Connection = "Close";
218
219 # Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-
websockets.html)
220 if (req.http.upgrade) {
221 set bereq.http.upgrade = req.http.upgrade;
222 }
223
224 return (pipe);
225 }
226
227 sub vcl_pass {
228 # Called upon entering pass mode. In this mode, the request is passed on to the backend, and the
backend's response is passed on to the client, but is not entered into the cache. Subsequent requests
submitted over the same client connection are handled normally.
229
230 # return (pass);
231 }
24

233 # The data on which the hashing will take place
234 sub vcl_hash {
235 # Called after vcl_recv to create a hash value for the request. This is used as a key to look up
the object in Varnish.
236
237 hash_data(req.url);
238
239 if (req.http.host) {
240 hash_data(req.http.host);
241 } else {
242 hash_data(server.ip);
243 }
244
245 # hash cookies for requests that have them
246 if (req.http.Cookie) {
247 hash_data(req.http.Cookie);
248 }
249 }
250
251 sub vcl_hit {
252 # Called when a cache lookup is successful.
253
254 if (obj.ttl >= 0s) {
255 # A pure unadultered hit, deliver it
256 return (deliver);
257 }
25

259 # https://www.varnish-cache.org/docs/trunk/users-guide/vcl-grace.html
260 # When several clients are requesting the same page Varnish will send one request to the
backend and place the others on hold while fetching one copy from the backend. In some products this
is called request coalescing and Varnish does this automatically.
261 # If you are serving thousands of hits per second the queue of waiting requests can get
huge. There are two potential problems - one is a thundering herd problem - suddenly releasing a
thousand threads to serve content might send the load sky high. Secondly - nobody likes to wait. To
deal with this we can instruct Varnish to keep the objects in cache beyond their TTL and to serve the
waiting requests somewhat stale content.
262
269 # We have no fresh fish. Lets look at the stale ones.
270 if (std.healthy(req.backend_hint)) {
271 # Backend is healthy. Limit age to 10s.
272 if (obj.ttl + 10s > 0s) {
273 #set req.http.grace = "normal(limited)";
275 } else {
276 # No candidate for grace. Fetch a fresh object.
277 return(fetch);
278 }
279 } else {
280 # backend is sick - use full grace
281 if (obj.ttl + obj.grace > 0s) {
282 #set req.http.grace = "full";
284 } else {
285 # no graced object.
286 return (fetch);
287 }
288 }
289
290
291 # fetch & deliver once we get the result
292 return (fetch); # Dead code, keep as a safeguard
293 } 26

295 sub vcl_miss {
296 # Called after a cache lookup if the requested document was not found in the cache. Its purpose is
to decide whether or not to attempt to retrieve the document from the backend, and which backend to
use.
297
298 return (fetch);
299 }
300
301 # Handle the HTTP request coming from our backend
302 sub vcl_backend_response {
303 # Called after the response headers has been successfully retrieved from the backend.
304
305 # Pause ESI request and remove Surrogate-Control header
306 if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
307 unset beresp.http.Surrogate-Control;
308 set beresp.do_esi = true;
309 }
310
311 # Enable cache for all static files
312 # The same argument as the static caches from above: monitor your cache size, if you get
data nuked out of it, consider giving up the static file cache.
313 # Before you blindly enable this, have a read here: http://mattiasgeniar.
be/2012/11/28/stop-caching-static-files/
314 if (bereq.url ~ "^[^?]*.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]
|pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|xml|zip)(?.*)?$") {
315 unset beresp.http.set-cookie;
316 }
27

319 # Large static files are delivered directly to the end-user without
320 # waiting for Varnish to fully read the file first.
321 # Varnish 4 fully supports Streaming, so use streaming here to avoid locking.
322 if (bereq.url ~ "^[^?]*.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av])
(?.*)?$") {
323 unset beresp.http.set-cookie;
324 set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize
blocks (128k by default) if
325 # the backend doesn't send a Content-Length header,
so only enable it for big objects
326 set beresp.do_gzip = false; # Don't try to compress it for storage
327 }
329 # Sometimes, a 301 or 302 redirect formed via Apache's mod_rewrite can mess with the HTTP
port that is being passed along.
330 # This often happens with simple rewrite rules in a scenario where Varnish runs on :80 and
Apache on :8080 on the same box.
331 # A redirect can then often redirect the end-user to a URL on :8080, where it should be :
80.
332 # This may need finetuning on your setup.
333 #
334 # To prevent accidental replace, we only filter the 301/302 redirects for now.
335 if (beresp.status == 301 || beresp.status == 302) {
336 set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
337 }
339 # Disable queue coalescence for uncacheable objects
340 if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
341 set beresp.uncacheable = true; # hit_for_pass
342 set beresp.ttl = 120s; # Remeber this
344 }
346 # Allow stale content, in case the backend goes down.
347 # make Varnish keep all objects for 6 hours beyond their TTL
348 set beresp.grace = 6h;
351 } 28

353 # The routine when we deliver the HTTP request to the user
354 # Last chance to modify headers that are sent to the client
355 sub vcl_deliver {
356 # Called before a cached object is delivered to the client.
357
358 if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits,
disable when not needed
359 set resp.http.X-Cache = "HIT";
360 } else {
361 set resp.http.X-Cache = "MISS";
362 }
363 # Please note that obj.hits behaviour changed in 4.0, now it counts per objecthead, not per
object
364 # and obj.hits may not be reset in some cases where bans are in use. See bug 1492 for
details.
365 # So take hits with a grain of salt
366 set resp.http.X-Cache-Hits = obj.hits;
368 # Remove some headers: PHP version
369 unset resp.http.X-Powered-By;
371 # Remove some headers: Apache version & OS
372 unset resp.http.Server;
373 unset resp.http.X-Drupal-Cache;
374 unset resp.http.X-Varnish;
375 unset resp.http.Via;
376 unset resp.http.Link;
379 }
380
381 sub vcl_purge {
382 # restart request
383 set req.http.X-Purge = "Yes";
384 return(restart);
385 }
29

387 sub vcl_synth {
388 if (resp.status == 720) {
389 # We use this special error status 720 to force redirects with 301 (permanent)
redirects
390 # To use this, call the following from anywhere in vcl_recv: error 720 "http:
//host/new.html"
391 set resp.http.Location = resp.reason;
392 set resp.status = 301;
394 } elseif (resp.status == 721) {
395 # And we use error status 721 to force redirects with a 302 (temporary) redirect
396 # To use this, call the following from anywhere in vcl_recv: error 720 "http:
//host/new.html"
397 set resp.http.Location = resp.reason;
398 set resp.status = 302;
400 }
401
403 }
404
405
406 sub vcl_fini {
407 # Called when VCL is discarded only after all requests have exited the VCL. Typically used to
clean up VMODs.
408
409 return (ok);
410 }
30

sub vcl_recv {
if (req.url ~ "^/varnish/") {
# Lame example to generate content, look down in vcl_synth for status 820
return (synth(820, "Test"));
}
}
sub vcl_synth {
if (resp.status == 820) {
# Lame example showing how to generate synthetic content.
# Usually to do this you would use a vmod from https://www.varnish-cache.org/vmods
# to do things like memcache/redis/database/... lookups and return meaningful things
# expected url /varnish/parameter1/min/max/ -> /varnish/real_random_api/8/32/
set resp.http.Content-Type = "text/plain; charset=utf-8";
set req.http.X-parameter1 = regsub(req.url, "^/varnish/(.*?)/.*", "1");
set req.http.X-min = regsub(req.url, "^/varnish/.*?/(.*?)/.*", "1");
set req.http.X-max = regsub(req.url, "^/varnish/.*?/.*?/(.*?)/.*", "1");
synthetic("You are " + client.ip + {"
url: "} + req.url + {"
reason: "} + resp.reason + {"
1st parameter: "} + req.http.X-parameter1 + {"
Have your random number between "} + req.http.X-min + " and " + req.http.X-max + " : " + std.random
(std.real(req.http.X-min, 0), std.real(req.http.X-max, 100)) + {"
End
"});
set resp.status = 200; # Be careful to set it after using resp.reason or it'll overwrite it
return (deliver);
}
return (deliver);
}
31

Playing with vcl_synth
$ GET -E http://localhost:8080/varnish/reason/-8/42/
GET http://localhost:8080/varnish/reason/-8/42/
User-Agent: lwp-request/6.03 libwww-perl/6.05
200 OK
Connection: close
Date: Mon, 24 Nov 2014 15:26:12 GMT
Accept-Ranges: bytes
Server: Varnish
Content-Length: 133
Content-Type: text/plain; charset=utf-8
Client-Date: Mon, 24 Nov 2014 15:26:12 GMT
Client-Peer: 127.0.0.1:8080
Client-Response-Num: 1
X-Varnish: 42
You are 127.0.0.1
url: /varnish/reason/-8/42/
reason: Test
1st parameter: reason
Have a random number between -8 and 42 : -2.204
End
32

But that's not enough:
$ GET -e http://test/test1.html
200 OK
Connection: close
Date: Fri, 03 Oct 2014 16:37:59 GMT
ETag: "3d838cc-51-504816b16fe6a"
Server: Apache
Vary: Accept-Encoding
Content-Length: 81
Content-Type: text/html
Last-Modified: Fri, 03 Oct 2014 09:32:57 GMT
33

# http://httpd.apache.org/docs/2.0/mod/mod_expires.html
ExpiresActive On
ExpiresByType image/gif "access plus 4 days"
#<FilesMatch ".(gif|jpe?g|png)$">
# Header merge Cache-Control "public" # Default anyway
# ExpiresDefault "access plus 4 days"
#</FilesMatch>
ExpiresByType image/jpg "access plus 4 days"
ExpiresByType image/jpeg "access plus 4 days"
ExpiresByType image/png "access plus 4 days"
ExpiresByType image/x-icon "access plus 4 days"
ExpiresByType text/css "access plus 3 days"
ExpiresByType application/x-javascript "access plus 3 days"
ExpiresByType text/html "access plus 4 hours"
ExpiresByType application/xml "access plus 1 hours"
34

$ GET -e http://test/test1.html
200 OK
Connection: close
Date: Fri, 03 Oct 2014 17:37:42 GMT
ETag: "3d838cc-51-504816b16fe6a"
Server: Apache
Vary: Accept-Encoding
Content-Length: 81
Content-Type: text/html
Expires: Fri, 03 Oct 2014 21:37:42 GMT
Last-Modified: Fri, 03 Oct 2014 09:32:57 GMT
35

Numbers on faster hardware
(AMD FX(tm)-8120 Eight-Core Processor 32GB RAM)
/usr/sbin/ab2 -n 500000 -c 200 -k -H "Accept-encoding: gzip" "http:
//localhost/prueba.html"
Requests per second: 24318.71 [#/sec] (mean)
Time per request: 8.224 [ms] (mean)
Time per request: 0.041 [ms] (mean, across all concurrent
requests)
Transfer rate: 8161.06 [Kbytes/sec] received
/usr/sbin/ab2 -n 500000 -c 200 -k -H "Accept-encoding: gzip" "http:
//localhost:8080/prueba.html"
Requests per second: 25780.13 [#/sec] (mean)
Time per request: 7.758 [ms] (mean)
Time per request: 0.039 [ms] (mean, across all concurrent
requests)
Transfer rate: 8611.04 [Kbytes/sec] received 36

Resources
● https://www.varnish-cache.org/docs
○ https://www.varnish-software.com/static/book/VCL_Basics.html
○ https://www.varnish-cache.org/trac/wiki
● High Performance Browser Networking by Ilya Grigorik (http:
//chimera.labs.oreilly.com/books/1230000000545/index.html)
● Caching in HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616-
sec13.html)
● The only moment you should ever cache static files is if you have
memory to spare (http://mattiasgeniar.be/2012/11/28/stop-caching-static-
files/)
● How to Use Varnish to Speed up my Website (http://symfony.
com/doc/current/cookbook/cache/varnish.html)
● https://www.varnish-cache.org/vmods (redis, memcache, MySQL,
PostgreSQL, ratelimit, device detection, json, auth, ...)
37

Varnish presentation for the Symfony Zaragoza user group

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

En vedette

En vedette (7)

Similaire à Varnish presentation for the Symfony Zaragoza user group

Similaire à Varnish presentation for the Symfony Zaragoza user group (20)

Dernier

Dernier (20)

Varnish presentation for the Symfony Zaragoza user group