I’ve been wanting to play around with a Web Cache/proxy in front of my web server, just to see how hard it would be and how well it would work.
Initially, I was going to try Squid, which is used far and wide. Pretty quickly I found out about Varnish, which is supposed to be easy to use, and quick to install. That may have been true, but getting it to work with WordPress was relatively painful for me. However, once I got it working, it made some crazy improvements in the speed of this site.
I won’t go into too much detail about the more standard parts. Should you need help with things like installing or finding packages, Varnish might not be for you. I will also assume you have a fully functional WordPress installed on Apache 2.
Install Varnish
Configure your Varnish config file (/etc/varnish/default.vcl.) Here is mine (also, check THIS post.)
backend default { .host = "173.230.157.240"; .port = "8080"; } sub vcl_recv { # Normalize Content-Encoding if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|lzma|tbz)(\?.*|)$") { remove req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { remove req.http.Accept-Encoding; } } # Remove cookies and query string for real static files if (req.url ~ "^/[^?]+\.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.*|)$") { unset req.http.cookie; set req.url = regsub(req.url, "\?.*$", ""); } # Remove cookies from front page if (req.url ~ "^/$") { unset req.http.cookie; } } sub vcl_fetch { if (req.url ~ "^/$") { unset obj.http.set-cookie; } } |
Regretfully I don’t remember where I got that particular file, but I’m going to see if I can find it again. It’s perfect for what we need to do. All you need to do is change your IP to match your host. By doing this, you’ll be pointing at your web server (called backend.) We’re going to change your web server to listing on 8080, and varnish will answer all normal port 80 traffic.
Now to configure Apache. Edit your /etc/httpd/conf/httpd.conf, and look for these lines, and add 8080 to the end (the majority were just 80.)
Listen 127.0.0.1:8080 Listen 173.230.157.240:8080 # for each of your Virtual host NameVirtualHost *:8080 |
Now stop apache, and stop and start varnish, and start apache.
There are a few ways to see if it’s working.
varnishstat
varnishhist -d
varnishtop |
Probably the most fun is to use ab (Apache Benchmark.) I did this from my desktop client against my Linode 512MB slice.
First, let’s look at our web server directly (via 8080 we changed it to.)
ab -c 100 -n 1000 http://cainmanor.com:8080/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking cainmanor.com (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Apache Server Hostname: cainmanor.com Server Port: 8080 Document Path: / Document Length: 0 bytes Concurrency Level: 100 Time taken for tests: 54.226 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Non-2xx responses: 1000 Total transferred: 291000 bytes HTML transferred: 0 bytes Requests per second: 18.44 [#/sec] (mean) Time per request: 5422.552 [ms] (mean) Time per request: 54.226 [ms] (mean, across all concurrent requests) Transfer rate: 5.24 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 28 35 6.4 33 72 Processing: 89 3852 7070.4 1924 48164 Waiting: 89 3852 7070.3 1923 48163 Total: 119 3887 7071.0 1957 48202 Percentage of the requests served within a certain time (ms) 50% 1957 66% 1997 75% 2041 80% 2091 90% 4021 95% 18134 98% 35079 99% 41195 100% 48202 (longest request) |
Now let’s go to port 80 directly
ab -c 100 -n 1000 http://cainmanor.com/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking cainmanor.com (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Apache Server Hostname: cainmanor.com Server Port: 80 Document Path: / Document Length: 18654 bytes Concurrency Level: 100 Time taken for tests: 3.507 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 19276653 bytes HTML transferred: 18960147 bytes Requests per second: 285.17 [#/sec] (mean) Time per request: 350.665 [ms] (mean) Time per request: 3.507 [ms] (mean, across all concurrent requests) Transfer rate: 5368.33 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 30 75 127.0 52 1052 Processing: 121 254 153.4 203 1104 Waiting: 33 78 86.4 54 984 Total: 167 329 189.5 262 1235 Percentage of the requests served within a certain time (ms) 50% 262 66% 299 75% 317 80% 343 90% 481 95% 767 98% 1125 99% 1143 100% 1235 (longest request) |
I also tried to use W3 Total Cache and memcached at the same time, but I got slightly slower response times.
These are impressive improvements.
Before – Time taken for tests: 54.226 seconds
After – Time taken for tests: 3.507 seconds
Have Fun
The VCL is actually incomplete for a fully functional WordPress installation. Anyway, regarding W3 Total Cache:
Mmemcache(d) is for multiple server configurations. The lowest backend response is realized by using W3TC with an opcode cache for object and database cache (to reduce execution time for misses) and disk enhanced for page cache. Then use HTML minify (and HTTP Compression obviously) to reduce response times and increase concurrency/throughput further.
I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success.