Step 11: Enable Varnish. But steps 1 through 10 matter more. Varnish isn’t a magical solution to all your performance woes. In truth, it’s one of the more aptly-named software packages: you should only apply it only after you’ve finished sanding down the rough edges of your site’s performance. If you apply it too soon, you won’t have the smooth surface you were hoping for, and at best you’ll be covering over your site’s imperfections.

Here are some topics I run through before enabling Varnish on a Drupal site. Your mileage may vary, and your methods may vary as well. This list is designed for you to do a deep dive into finding out why your site is “slow,” and I like to perform some version of this list for every site launch I participate in.

1) Have you tried the built-in page cache?

  • Have you tried turning on the built-in page cache?
  • Do the pages cache as expected, or are you still getting uncached content delivered?
  • Are there any portions of your site that no longer work as expected with the built-in page cache enabled?

If turning on the built-in page cache causes headaches, then you likely have some architectural decisions to make.

For example: Drupal core comments are a fun way to “engage” with your visitors, but they’re going to be a real pain to work around if you want effective Varnish caching. But more importantly, do they really get any use on your site, aside from spammers? Could you get the same level of engagement from a third-party commenting system like Disqus, something that exists only on the client side, and thus doesn’t require anything from Drupal?

2) Have you done any PHP profiling?

  • Have you done any PHP profiling (using xhprof, others)?
  • Do you know which of your pages take the most time to load, and do you know why they take as long as they do?

Anytime you introduce custom code, you’re introducing something that has not been fully tested at scale.

Be mindful of trying to optimize too much — saving a few dozen milliseconds off of a page request will be nearly imperceptible and barely worth the effort. On the flip side, if you can shave half a second or more off, that’s worth even a significant refactor of your code.

3) Are there any views that take a long time to load?

  • Are there any views that take a long time to load?
  • Are they cacheable using a Views cache plugin, or would caching the view break it somehow?
  • And if so, why?

Writing your own Views cache plugin is easy, so if the built-in time-based cache is insufficient for your needs, feel free to implement your own. For example, if you have view output that is identical for logged-in and anonymous visitors, then your custom Views cache plugin can set a results and output key that does not include the logged-in user’s roles as part of the hash.

4) Are there any Panels panes that take a while to load?

  • Similar to the previous Views question, are there any Panels panes that take a while to load?
  • Are they cacheable? (I’m going to strongly urge you to answer yes to that question, even if it requires some extra thought.)

The built-in simple cache may be too simple for your needs, but there are some excellent Panels caching modules already available, and a Panels cache plugin is relatively simple to implement if you need a custom solution.

5) How about SSL?

  • Are there any SSL pages?
  • Are you using a module like Securepages to redirect certain paths to SSL?
  • What percentage of your traffic will be SSL?

Bottom line, unless you’re doing SSL termination, traffic coming in over port 443 is never going to be cached by Varnish. A page that renders slowly before you enable Varnish is going to be just as slow for your SSL traffic. Instead of allowing any page to be encrypted, use a module like Securepages to redirect your traffic back to port 80 (and back to Varnish) unless they’re on a specific set of pages.

6) Can users log in?

  • Do your users have the ability to log in, to create profiles, etc.?
  • Is there any value added to the UX if they’re logged in?
  • Is there any content that changes based on whether your visitors are authenticated or not?

Users are the enemy of a performant Drupal site. Having a user session automatically bypasses Varnish, and thus as long as your visitor is logged in — regardless of whether they’re viewing a members-only forum or the About Us page — they are not going to be served by Varnish. If you don’t absolutely need users, consider blocking them altogether.

If, however, having logged-in users is part of your build plan, take a minute to review alternative caching options. Views and Panels can cache content differently for logged-in and anonymous traffic. And if you have a custom menu callback, consider writing your own cache_get() and cache_set() logic to spare any expensive node_load() or database calls.

7) Are you running Pressflow?

  • Are you running Drupal 6?
  • Are you running the Pressflow fork of Drupal 6?

(Drupal 7 developers can skip this section altogether.)

Bottom line, if you’re looking to have Varnish cache your Drupal 6 site, using Pressflow is an absolute requirement. It simply will not work without it. Pressflow is a drop-in replacement for Drupal 6 core, and 99% of sites will function exactly the same — if not better — with Pressflow. Still, it doesn’t hurt to run some of your test coverage again, or, in lieu of test coverage, to test some of your custom functionality by hand.

Pressflow also includes some fun extras like path alias caching, which caches frequent path lookup database requests, as well as some core tweaks that help Drupal 6 in enterprise environments. A simple blog site wouldn’t see those benefits, but a large corporate site could.

8) Are you making any API calls?

  • Are you making any third-party API calls, like to a payment gateway, or to Twitter?
  • Are you running modules that periodically call out to get authentication tokens, like Mollom?

If you are, you’re completely at the mercy of the third-party’s API performance and how frequently you request data from it.

In load testing one of our client sites recently, I was getting terrible performance on a site that was sitting on some heavy-duty hardware, and it just didn’t seem right. I broke out the PHP profiler (xhprof), and discovered that the culprit was Mollom. A little known quirk of the Mollom module for Drupal is that when Mollom is in testing/development mode, it issues an API request on every page request as long as there’s at least one form on that page. That form doesn’t even have to be Mollom-enabled, it could just be the search form in the header.

In this specific case, I offered a patch to the module maintainer, and disabled testing/development mode in the module settings. But it was a lesson learned: any API call at all, no matter how benign, could have a measurable impact on your site’s performance, especially at scale.

9) Now we talk about optimizing your stack

Now it’s time to review hardware and server configuration. Note how I haven’t mentioned anything about the server environment until now.

There’s plenty of server jockeys out there who will insist that the first rule of getting Drupal to be performant is installing PHP-FPM, nginx, Memcache, and Varnish. They might even recommend Project Mercury, which is a (wonderful) install script for optimizing your entire stack.

As much as I love those tools, no matter how much stack optimization you do, you can still have a page on your site that issues 11,000 database calls that saturate your database server’s I/O, causing the page render to take over 40 seconds to complete, which ties up a PHP resource on your application server while it waits. Swapping out mod_php for PHP-FPM and tweaking my.cnf might take it from 40 seconds down to 35 seconds, but we’re still talking about a massive liability that, truly, has nothing to do with the stack and everything to do with how your site is built.

Remember that Varnish caches aren’t forever, and there will come a time when one of your visitors will hit that page completely uncached.

Am I suggesting that running Memcache isn’t an awesome way to optimize your cache bins? Absolutely not — and by all means, install it and use it, especially if you’re making use of Views and Panels caches, block caches, custom object caches, etc., or if your database layer isn’t on stellar hardware (or worse, if it’s on the same hardware). And the other optimizations are just as valuable. I’m simply arguing in favor of reviewing your Drupal build configuration first. Bottom line, check your custom code and your design decisions before you blame time-tested code like Apache and mod_php.

10) And now for something completely out of left field…

I’m going to suggest something completely out of the box, and I apologize if it ruffles the feathers of any die-hard Drupal developers out there, but: is Drupal the best solution for your site?

Could your site be rendered into static HTML files and still perform exactly the same?

Before you dismiss this question on spec, consider that you’re seeking to do exactly that, just with Varnish. All Varnish is going to do is render your Drupal pages into static files that it stores either on disk or in memory, and those static files are what get served to your visitors. Yes, with Varnish you do have the option of passing through your dynamic requests — AJAX calls, form submissions, etc. — to Drupal, but a 100%-effective Varnished site is going to be equivalent to a series of static HTML (or XML, JSON, etc.) files.

So, if your site is 99% static content, take a minute to ask yourself if the overhead of running a full-fledged content management system is worth the performance hit (and the administrative hassle). Consider also that rendering it into static files means you can upload them directly to a CDN, and you can skip the overhead of running your own servers. For some clients, the server infrastructure can be a significant budget line item, and it could be largely unnecessary.

If you’re seeking a way to automate the rendering of content into static files, look into projects like Jekyll and its derivatives. They’re not Drupal-based — they’re not even PHP-based — but they accept plain text files, Markdown, or HTML (really, any format you require), and plug the content into templates, and can support any metadata you need.

11) If you’ve made it this far, enable Varnish

If you’ve made it this far, feel free to enable Varnish — and enjoy a wildly-performant Drupal site. After all, you’ve taken the time to optimize top to bottom, and your site is moderately-performant when completely uncached. Adding Varnish on top of that smooth surface is going to mean so much more now.