Busting some caches with Django

In contrast to server-side code, client-side assets such as JavaScript files and static images are not directly deployed to where they are ultimately executed or displayed (i.e. the user’s browser). Rather, they are downloaded on demand whenever a browser retrieves a page for the first time.

For sake of efficiency, the browser saves the retrieved assets locally for future reference. This will result in a more fluid user experience as certain actions such as reloading a page won’t require downloading the very same files again.

As a consequence, changes to static assets are not guaranteed to be reflected on the client side without delay. After all, the browser might still be accessing previously cached versions of those assets. This can lead to an array of problems, ranging from mild display errors caused by an outdated CSS file to significant problems in functionality. For example, imagine some cached JavaScript code attempting to retrieve data from an API endpoint that doesn’t exist anymore on the server side.

To avoid these kinds of issues, the developer needs to ensure that a cache busting strategy is in place. Cache busting forces the browser to download a fresh copy of some static asset if it has changed since the last page visit. But how can we force the browser to do that?

Well, browsers decide whether or not to (re-)fetch a resource based on its file name. If the browser already knows a file name and if it hasn’t been too long since the last download, the browser will re-use the cached version. Therefore, we can trigger the download of a new asset version by giving it a name the browser has not yet encountered.

Django’s approach to naming different versions of an asset is to insert part of its MD5 hash into its name. For instance, if we have a CSS file called home.css, Django’s ManifestStaticFilesStorage will rename it to something like home.789f58f23e78.css when running Django’s built-in ./manage.py collectstatic command.

In addition to renaming the files, the ManifestStaticFilesStorage will generate a file called staticfiles.json. This file contains a mapping from the original file names to the hash-based names. Its purpose it to make the process of accessing the hash-based asset versions more efficient by not having to re-compute a static file’s MD5 hash when referencing it with the {% static %} template tag.

While working with the ManifestStaticFilesStorage, two things turned out not to fit my workflow. Firstly, running ./manage.py collectstatic frequently resulted in an error stating that an asset declared deep inside some vendor CSS (managed with npm) couldn’t be found. I am sure there are cases where this error would be useful information, but in my case it was more annoying than valuable.

Secondly, after running ./manage.py collectstatic the STATIC_ROOT directory would not only contain the renamed static files but also the original ones. To fix these two issues, I made some modifications to the ManifestStaticFilesStorage. Feel free to use the code for your setup too!

import os

from django.conf import settings
from django.contrib.staticfiles.storage import ManifestStaticFilesStorage


class CustomManifestStaticFilesStorage(ManifestStaticFilesStorage):
    def hashed_name(self, name, content=None, filename=None):
        try:
            return super().hashed_name(name, content, filename)
        except ValueError:
            return name

    def save_manifest(self):
        super().save_manifest()
        for path in self.hashed_files:
            os.remove(os.path.join(settings.STATIC_ROOT, path))