Service Workers of the World Unite

You have nothing to lose but your chains…unless you fuck it up, in which case your site will break in mysterious ways.


As of today, this little site is running its first Service Worker.

To review, a Service Worker is a mechanism in modern browsers that lives outside the usual DOM/window scope, and can intercept network requests.1

What what do with those requests is up to you. You can build a caching layer, add offline support, prefetch pages before the user clicks on them, or crazy stuff like distribute an entire SPA application as a zipped package and run the entire experience out of the cache.

As you can imagine, this is complicated. And as as I’ve mentioned before, I’m stupid. I’m sure I’ve made mistakes, and feedback by email is always welcome.

1.

In 2018 I started a brand new “Revenue Team” at The Atlantic focused on all the things that fund journalism, and adopted a crazy beagle-daushund puppy. Between the two, I don’t have a whole lot of time to do deep experimental dives on new technology.

And yet, I should. Even if I’m not going to take on a big side project, I can’t know if these new things are the right tool for the job unless I’ve taken the time to try them out in advance.

Ziggy plays guitar.

Does this site need a service worker? No. I’m only doing this to learn about how they work and see what I can do to performance with it.

After all, the reason to have a personal site is to try out new ideas in a safe spot where it doesn’t really matter if everything goes to hell.

2.

I’m using Mozilla’s Service Worker Cookbook and a few blog posts as guidance. The goal is to cache static assets as they’re downloaded, and serve them from cache whenever I can.

You can see the complete results at /sw.js, which may be newer than the version I’m describing today.

For my purposes, the main area of logic is handling the network requests, which works by listening on fetch events.

The cookbook has a Cache and Update example that looks like this:

self.addEventListener('fetch', function(evt) {
  console.log('The service worker is serving the asset.');
  evt.respondWith(fromCache(evt.request));
  evt.waitUntil(update(evt.request));
});

This example responds to the fetch with a response from the cache, and then updates the results in the cache when the request finishes. This let’s cached responses return instantly without creating a stale cache problem.

There are two pieces missing here:

  1. I only want to cache static assets, not HTML or images that aren’t shared between pages.
  2. Their example also prepopulates the cache using cache.addAll and a list of all the assets. I don’t necessarily know which assets I want to cache in advance.

I can address the first problem by wrapping the evt.respondWith logic in a check to ensure the service worker should touch this request.

// Determines which files are served from SW cache
function isStatic(evt) {
  // const origin = self.location.origin;
  const staticEnding =  evt.request.url.match(/(\.js|\.css|\.woff|\.woff2)$/);
  // const sameDomain = (evt.request.url.indexOf(origin) === 0);
  return staticEnding;
}

Originally, I was only caching JS, CSS, and fonts that come from this domain. You’ll notice the domain part is commented out for now. That’s because I’m testing having it cache Google Analytics as well. So far is seems to work.


The second problem is harder. I need to determine if a request is cached. If so, return the cached version and run an update in the background. If not, fetch it from the network.2

I broke the logic to download and cache a copy of the asset into a fetchAndUpdateCache. It’s used for both cache hits and misses. If there’s a cache hit, I run it in the background with a 1s delay.3 If not, its used directly to fulfill the request.

/**
 * Fetches a request and updates the cache
 * @param  {FetchRequest} request - The request being made
 * @param  {SWCache} cache - The cache to populate
 * @return {Promise} - The cache being updating in promise form
 */
function fetchAndUpdateCache(request, cache) {
  return new Promise((resolve) => {
    return fetch(request).then(function(response) {
      console.debug(`Updated ${ request.url } in the SW cache.`)
      const respCopy = response.clone();
      cache.put(request, respCopy);
      resolve(response);
    });
  })
}

/**
 * Get from cache or fallback to network
 * @param  {FetchRequest} request - A fetch request promise
 * @return {FetchResponse} - Either from cache or from fetch
 */
function fromCache(request) {
  return caches.open(CACHE).then(function (cache) {
    return cache.match(request).then(async function (matching) {
      if (matching) {
        // If we have it, return the cached version
        // but let the request go through and update the cache
        console.debug(`Serving ${ request.url } from cache.`);
        setTimeout(fetchAndUpdateCache.bind(null, request, cache), 1000);
        return matching;
      } else {
        // If we don't have it, fetch it
        console.debug(`Serving ${ request.url } from network.`);
        let resp = await fetchAndUpdateCache(request, cache);
        return resp;
      }
    });
  });
}

// On fetch, use cache but update the entry with the latest contents
// from the server.
self.addEventListener('fetch', function(evt) {
  if (isStatic(evt)) {
    // You can use `respondWith()` to answer immediately, without waiting for the
    // network response to reach the service worker...
    const resp = fromCache(evt.request);
    evt.respondWith(resp);
  }
});

Determining what’s in the cache works just like the examples, but after that, we wind up in a mess of Promises.

evt.respondWith takes a response object, so the fromCache function must resolve all promises/async madness and return a response.

During a cache hit, the cache.match passes a response when it resolves, so we can just return that.

During a cache miss, we need to use the fetchAndUpdateCache. I was able to make this work by having that method return a promise, which resolves itself with the response when fetch completes.

But now we have a function that returns a promise when needs to resolve in order to be returned. In other news, my head hurts.

function fromCache(request) {
  return caches.open(CACHE).then(function (cache) {
    return cache.match(request).then(async function (matching) {
      if (matching) {
        // This is fine
        console.debug(`Serving ${ request.url } from cache.`);
        return matching;
      } else {
        // This won't work because it returns
        // a promise that resolves with a response instead
        // of a response
        console.debug(`Serving ${ request.url } from network.`);
        return fetchAndUpdateCache(request, cache);
      }
    });
  });
}

At this point I checked to see well async/await is supported these days. Because any browser that supports Service Workers should support async/await as well, I can solve this problem cleanly with a await fetchAndUpdateCache(request, cache);


That’s when the console started throwing errors again. I’d set async function fromCache, but the await was living in one of its inner callback functions. I had to move the async keyword down to the callback for cache.match, where my await actually lived.

Then it started working.

3.

There’s a lot of trial on error in this process. The Chrome debugger is quirky inside ServiceWorkers, and lot’s of API’s I’m used to tapping into like requestAnimationFrame or window.location don’t exist in the service worker context.

What’s more, when you make mistakes, they behave in very strange ways. For a while my network fallback wasn’t returning a response to the respondWith. This meant on my first pageview everything worked because the service worker was installing and hadn’t intercepted any fetches yet. On the second pageview, my CSS broke, and on the third pageview everything was fine again because the network fallback had populated the cache.

I’ve found a few strategies help to wrangle all this:

  1. Put console.debug everywhere so you can easily see what’s being cached, updated, and passed through to the network as you move from page to page.
  2. Use incognito liberally. While you can remove a service worker from the Applications Tab of dev tools, I found I was often not sure if I’d fixed the bug or just not successfully updated the service worker. With a fresh incognito window, I could walk through the steps to reproduce the issue and be confident I was starting from a clean slate.

4.

This implementation is overkill for a small, lightweight site, but I think what I have is reusable on larger applications.

The next step will be to refactor it using more async/await, which should be a little cleaner and easier to understand than the nested promises structure from all the examples I was using.

The bigger takeaway is this: while I could make this work in a few hours, the promise of service workers and progressive web apps is the ability to make sites that work offline, complete with background syncing and all the features we expect native apps to have. This is a powerful tool, but it’s going to be very challenging for even highly experienced developers to build and maintain a reliable application like that.

That won’t fly.

The great thing about the web as a platform is how accessible it is to new developers. You can learn HTML and create something basic in a weekend. I want a web platform where junior developers can build simple offline projects within their first year of programming.

To get this technology to that level we’ll need better tooling and examples.

It might even be a good use case for a framework.


  1. It’s called a “Proxy,” but I find that phrasing is confusing when you’re first trying to understand what the thing is and how to use it. 

  2. Or disk cache, which will happen automatically. 

  3. The theory is to give uncached assets a chance to download before the cached ones upload. This might be an interesting idea for slow connections or cases of network dogpiling.