Varnish Cache Explained

Varnish has become synonymous with high-performance HTTP acceleration. Varnish Cache is an open source reverse HTTP proxy accelerator that's great at reducing web server load on your server. Varnish is placed in front of the web server and acts as a proxy that establishes a bridge between the browser and the server.

The Varnish project was the brainchild of Danish technology architect  Poul-Henning Kamp. A well-known FreeBSD consultant, Kamp was the lead developer of Varnish.

The Varnish project was started by the online branch of the Norwegian tabloid newspaper Verdens Gang. But the software was so successful that it was eventually spun off into a separate company.

But don't worry, it's still free and completely open source software available under a two-clause BSD license. But if you want commercial support, it's available from Varnish Software. 

Currently, the support, management, and development of Varnish are taken care of under the banner of Varnish Software. 

How does Varnish Work?

Consider the following scenario. You walk into a busy coffee shop and order the house filter coffee. The person at the counter first checks whether there is a fresh pot ready to pour.

If there is, you get your cup immediately. If not, the order goes to the barista to brew a new pot. Varnish does the job of the person at the counter.

Varnish sits in front of your web server and the first time a visitor requests for a web page, Varnish gets the request and passes it to the backend server. The backend server processes the request and sends the response back to the browser.

Varnish, meanwhile 'caches' the rendered content and stores it in super fast RAM memory.

Any subsequent request for the same web page is served directly from the cached version (in RAM Memory!) without bothering the web server or Database.

Who Uses Varnish?

Many high-traffic news and content sites use Varnish.

Varnish offers large performance benefits for content-heavy, high traffic websites. Varnish can be used to cache both static and dynamic content.

There is a common misconception that Varnish is only for static content, but with a little bit of .VCL configuration tweaking it can work brilliantly for even dynamic websites and mobile applications. 

Why is Varnish so fast?

If you want to be able to serve thousands of concurrent requests per second without overloading your web server, Varnish is one of the best HTTP accelerators available.

Within the varnish configurations, you can set whether you want Varnish to store the cache on the hard disk or RAM memory.

To get the instant "click and it's there" feeling, you'll want to have that cache stored exclusively on your RAM.

You will see a much lower response time and a huge decrease in server resource usage with a properly configured Varnish setup.

Is Varnish Hard to Configure?

RHEL and its derivatives package Varnish.

Varnish comes with a base default configuration that can be used out of the box to get HTTP acceleration running with minimum fuss. Varnish itself does not terminate HTTPS, so on a modern site it sits behind a TLS terminator such as HAProxy or nginx.

Varnish provides you with a  Varnish Configuration Language (VCL) file, where you play with the configuration settings to fine tune its performance. The VCL file tells Varnish exactly how to handle the caching process. 

When and what to cache, when not to cache, and how long to keep the cached content in memory.

Mastering your .VCL skills can give you super powers allowing you to modify nearly every aspect of an HTTP request and response.

When you should use Varnish

1. If you've got a content-heavy website with frequent traffic spikes
2. Varnish is great for servers with heavy database load. By caching the most common web requests, your MySQL server can get some rest.
3. If you want to reduce your disk's activity prolonging your disk's life expectancy.
4. If one web server and database must serve thousands of concurrent visitors. 

When shouldn't you use Varnish?

There are plenty of easier ways to make great performance boosts like NGiNX micro caching that you can try out before resorting to Varnish.

For small websites with under 1,000 page views per day, you'll probably be just fine with an optimised LAMP Stack or NGiNX with PHP-FPM.

Because of the complexity of customising the .VCL file to your specific requirements, sometimes it's just not worth the effort to use Varnish. Getting Varnish to play well with dynamic content is tricky. Sure, you can use ESI 'hole punching,' to serve user-specific content, but is it really necessary?

Check the Low Lying Fruit First

Before Varnish, do the cheaper fixes: image optimisation, minified CSS and JavaScript, MySQL tuning.

Conclusion

Setting up Varnish to serve content properly and following your organisation's requirements can be tricky. Cache expiry rules, time to live, URL caching, and object caching requirements in Varnish are highly complex. As a result, many consultancy firms have been born out of the requirement to have “Varnish Implementations” done in their environment.

At EuroVPS we run Varnish in front of high-traffic sites, and our engineers can help you design the caching layer for yours.