Warm up + Refresh WordPress Varnish 3 Cache + CloudFlare

Varnish Cache is the best speed improvement for your WordPress site. It acts as a reverse proxy and will cache all of your pages as HTML and serve them up quickly, bypassing the slow PHP backend (Apache or nginx). With the WordPress Varnish cache method outlined here, you will always be serving up fast Varnish cached versions of your page. Varnish has a clever feature: using hash_always_miss which lets Varnish continue to serve its cached version while it refreshes its cache. This means you can slowly refill the Varnish cache instead of purging it all at once and serving slow uncached WordPress posts to new users. I will assume you have shell or root access to your server (like with Digital Ocean) to install and edit files and have XML sitemaps enabled. Yoast SEO creates XML sitemaps for you and I assume most WordPress users are using this plugin. This method works woth Varnish 3 and Cloudflare which requires a few extra steps compared to the Cloudflare-less guide, yet still works with your WordPress posts, pages and categories.

I have a WordPress plugin which automates this, use the contact form to help test it and get a free copy

Warm up WordPress Varnish 3 Cache Behind CloudFlare

I will assume you already have Varnish installed and configured to work with WordPress, my configuration will be posted in the near future.

CloudFlare includes both the origin IP and the CloudFlare IP in the header so we need to extract the IP as a string and convert it to IP format using a vmod. The extracted IP is used to test if the hash_always_miss requests comes from an authorized IP (i.e. your web server).

Three WordPress Varnish cache bash scripts are included

Manual refresh one URL – script prompts for the URL

Automated Full site refresh – script refreshes your whole site

Manual Multiple URL refresh – paste a list of URLs to be refreshed

Enable Real IP from CloudFlare in Varnish

Prepare Varnish 3 for CloudFlare

You probably already have Varnish installed, however you will still need to prepare a Varnish source to build vmods which enable additional functionality for Varnish.

Note that if you are on a 64-bit system this will probably install Varnish 4 despite the 3.0 label. It is better to use the Varnish 4 CloudFlare guide here.

Configuring Varnish for Smart Refreshing

Open your Varnish vcl file, usually default.vcl

sudo nano /etc/varnish/default.vcl

Add the Varnish ipcast vmod functionality in your default.vcl by adding this at the top of the file. Ipcast converts strings to IP addresses so you can match the real IP to your editors access control list below.

import ipcast;

Add a section for editors, this is for security so that other machines cannot force refreshes on your page. Add it after the backend section but before the sub vcl_recv section. Change IP.of.Server to the source IP address that will be sending the curl refresh commands. I am just using the web server to run the script so it is the same IP as what CloudFlare points to for this domain.

acl editors {
"127.0.0.1";
"IP.of.Server";
}

Adjust your VCL file to include hash_always_miss in your sub vcl_recv section, you only need to add the red section. It looks for the header Cache-Control with no-cache (which we will send later using curl) and whether the sender is a member of the editors acl. Change Any.Valid.IP to any valid IP (e.g. 1.2.3.4) . Add the red section inside vcl_recv. This works because the first IP is always the real IP.

sub vcl_recv {
# set realIP by trimming CloudFlare IP which will be used for various checks
set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, "[, ].*$", "");
# check if the real IP is coming from your web server
if (ipcast.ip(req.http.X-Actual-IP, "Any.Valid.IP") ~ editors) {
# if the header is set to no-cache set always miss
if (req.http.Cache-Control ~ "no-cache" ) {
set req.hash_always_miss = true;
}
}
}

I have also set my Varnish time to live – how long Varnish should keep cached version of the page – to 1 year in the sub vcl_fetch function because I can refresh the cache using hash_always_miss instead of purging.

set beresp.ttl = 52w;

Test your adjusted Varnish default.vcl works

varnishd -C -f /etc/varnish/default.vcl

If you didn’t get any errors reload your Varnish configuration which doesn’t empty your Varnish cache like restarting the Varnish service does.

sudo service varnish reload

Manual Varnish Smart Refresh Purge for Single URL

Create a new script that will send the hash_always_miss request for a URL you paste into the script when prompted

Do a search or find and replace, make find your full URL http://www.yourwebsite.com and leave the replace blank

Highlight the whole column and copy it to the clipboard, you will paste it in the manual script below

sudo nano smartvarnishrefresh.sh

Change your site name and paste your WordPress URLs from the spreadsheet to refresh in the Varnish cache (inspired from GiantDorks). You should delete # Paste your URL sitemap here line. Adjust the sleep value to whatever you want, it is the small break taken before refreshing the subsequent URL.

Archives

Archives

DISCLAIMER

The information on HTPC Guides is for educational purposes and only condones obtaining public domain content. HTPC Guides is not responsible for content from any other site or provider. By using the links provided on this site you agree that neither this site nor its proprietor is in any way responsible for any damages or liability arising from use of external content.

Copyright

The information on this site is the intellectual property of the owner. Credit to other sources is provided where relevant. If you believe any information has not been sourced, please leave a comment and appropriate action will be taken.