How to use Varnish cache with WordPress

I’ve been wanting to play around with a Web Cache/proxy in front of my web server, just to see how hard it would be and how well it would work.

Initially, I was going to try Squid, which is used far and wide.  Pretty quickly I found out about Varnish, which is supposed to be easy to use, and quick to install.   That may have been true, but getting it to work with WordPress was relatively painful for me.   However, once I got it working, it made some crazy improvements in the speed of this site.

I won’t go into too much detail about the more standard parts.   Should you need help with things like installing or finding packages, Varnish might not be for you.  I will also assume you have a fully functional WordPress installed on Apache 2.

Install Varnish

Configure your Varnish config file (/etc/varnish/default.vcl.)  Here is mine (also, check THIS post.)

backend default {
.host = "173.230.157.240";
.port = "8080";
}
 
sub vcl_recv {
# Normalize Content-Encoding
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|lzma|tbz)(\?.*|)$") {
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            remove req.http.Accept-Encoding;
        }
    }
# Remove cookies and query string for real static files
    if (req.url ~ "^/[^?]+\.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.*|)$") {
       unset req.http.cookie;
       set req.url = regsub(req.url, "\?.*$", "");
    }
# Remove cookies from front page
    if (req.url ~ "^/$") {
       unset req.http.cookie;
    }
}
sub vcl_fetch {
        if (req.url ~ "^/$") {
                unset obj.http.set-cookie;
        }
}

Regretfully I don’t remember where I got that particular  file, but I’m going to see if I can find it again.  It’s  perfect for what we need to do.  All you need to do is change your  IP to match your host.   By doing this, you’ll be pointing at your web server (called backend.)  We’re going to change your web server to listing on 8080, and varnish will answer all normal port 80 traffic.

Now to configure Apache.  Edit your /etc/httpd/conf/httpd.conf, and look for these lines, and add 8080 to the end (the majority were just 80.)

Listen 127.0.0.1:8080
Listen 173.230.157.240:8080
 
# for each of your Virtual host
NameVirtualHost *:8080

Now stop apache, and stop and start varnish, and start apache.

There are a few ways to see if it’s working.

varnishstat
varnishhist -d
varnishtop

Probably the most fun is to use ab (Apache Benchmark.)   I did this from my desktop client against my Linode 512MB slice.

First, let’s look at our web server directly (via 8080 we changed it to.)

ab -c 100 -n 1000 http://cainmanor.com:8080/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
 
Benchmarking cainmanor.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
 
Server Software:        Apache
Server Hostname:        cainmanor.com
Server Port:            8080
 
Document Path:          /
Document Length:        0 bytes
 
Concurrency Level:      100
Time taken for tests:   54.226 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      291000 bytes
HTML transferred:       0 bytes
Requests per second:    18.44 [#/sec] (mean)
Time per request:       5422.552 [ms] (mean)
Time per request:       54.226 [ms] (mean, across all concurrent requests)
Transfer rate:          5.24 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       28   35   6.4     33      72
Processing:    89 3852 7070.4   1924   48164
Waiting:       89 3852 7070.3   1923   48163
Total:        119 3887 7071.0   1957   48202
 
Percentage of the requests served within a certain time (ms)
  50%   1957
  66%   1997
  75%   2041
  80%   2091
  90%   4021
  95%  18134
  98%  35079
  99%  41195
 100%  48202 (longest request)

Now let’s go to port 80 directly

ab -c 100 -n 1000 http://cainmanor.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
 
Benchmarking cainmanor.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
 
Server Software:        Apache
Server Hostname:        cainmanor.com
Server Port:            80
 
Document Path:          /
Document Length:        18654 bytes
 
Concurrency Level:      100
Time taken for tests:   3.507 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      19276653 bytes
HTML transferred:       18960147 bytes
Requests per second:    285.17 [#/sec] (mean)
Time per request:       350.665 [ms] (mean)
Time per request:       3.507 [ms] (mean, across all concurrent requests)
Transfer rate:          5368.33 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       30   75 127.0     52    1052
Processing:   121  254 153.4    203    1104
Waiting:       33   78  86.4     54     984
Total:        167  329 189.5    262    1235
 
Percentage of the requests served within a certain time (ms)
  50%    262
  66%    299
  75%    317
  80%    343
  90%    481
  95%    767
  98%   1125
  99%   1143
 100%   1235 (longest request)

I also tried to use W3 Total Cache and memcached at the same time, but I got slightly slower response times.

These are impressive improvements.
Before – Time taken for tests: 54.226 seconds
After – Time taken for tests: 3.507 seconds

Have Fun

2 Comments

  1. The VCL is actually incomplete for a fully functional WordPress installation. Anyway, regarding W3 Total Cache:

    Mmemcache(d) is for multiple server configurations. The lowest backend response is realized by using W3TC with an opcode cache for object and database cache (to reduce execution time for misses) and disk enhanced for page cache. Then use HTML minify (and HTTP Compression obviously) to reduce response times and increase concurrency/throughput further.

  2. I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success.

Comments are closed.