Journal tags: bug

Apple announced some stuff about artificial insemination at their WorldWide Developer Conference, none of which interests me one whit. But we did get a twitch of the webkit curtains to let us know what’s coming in Safari. That does interest me.

I’m really pleased to see that on desktop, websites that have been added to the dock will be able to intercept links for that domain:

Now, when a user clicks a link, if it matches the scope of a web app that the user has added to their Dock, that link will open in the web app instead of their default web browser.

Excellent! This means that if I click on a link to thesession.org from, say, my Mastodon site-in-the-dock, it will open in The Session site-in-the-dock. Make sure you’ve got the scope property set in your web app manifest.

I have a few different sites added to my dock: The Session, Mastodon, Google Calendar. Sure beats the bloat of Electron apps.

I have encountered a small bug. I’ll describe it here because I have no idea where to file it.

It’s to do with Spaces, Apple’s desktop management thingy. Maybe they don’t call it Spaces anymore. Maybe it’s called Mission Control now. Or Stage Manager. I can’t keep track.

Anyway, here are the steps to reproduce:

1. In Safari on Mac, go to a website like adactio.com
2. From either the File menu or the share icon, select Add to dock.
3. Click on the website’s icon in the dock to open it.
4. Using Apple’s desktop management (Spaces?) available through the F3 key, drag that window to a desktop other than desktop 1.
5. Right click on the site’s icon in the dock and select Options, then Assign To, then This Desktop.
6. Quit the app/website.
7. Return to desktop 1.

Expected behaviour: when I click on the icon in the dock to open the site, it will open in the desktop that it has been assigned to.

Observed behaviour: focus moves to the desktop that the site has been assigned to, but it actually opens in desktop 1.

If someone from Apple is reading, I hope that’s useful.

On the one hand, I hope this isn’t one of those bugs that only I’m experiencing because then I’ll feel foolish. On the other hand, I hope this is one of those bugs that only I’m experiencing because then others don’t have to put up with the buggy behaviour.

I’m quite sure this is of no interest to anyone but me, but I finally managed to fix a longstanding weird issue with my website.

I realise that me telling you about a bug specific to my website is like me telling you about a dream I had last night—fascinating for me; incredibly dull for you.

For some reason, my site was being brought to its knees anytime I syndicated a note to Mastodon. I rolled up my sleeves to try to figure out what the problem could be. I was fairly certain the problem was with my code—I’m not much of a back-end programmer.

My tech stack is classic LAMP: Linux, Apache, MySQL and PHP. When I post a note, it gets saved to my database. Then I make a curl request to the Mastodon API to syndicate the post over there. That’s when my CPU starts climbing and my server gets all “bad gateway!” on me.

After spending far too long pulling apart my PHP and curl code, I had to come to the conclusion that I was doing nothing wrong there.

I started watching which processes were making the server fall over. It was MySQL. That seemed odd, because I’m not doing anything too crazy with my database reads.

Then I realised that the problem wasn’t any particular query. The problem was volume. But it only happened when I posted a note to Mastodon.

That’s when I had a lightbulb moment about how the fediverse works.

When I post a note to Mastodon, it includes a link back to the original note to my site. At this point Mastodon does its federation magic and starts spreading the post to all the instances subscribed to my account. And every single one of them follows the link back to the note on my site …all at the same time.

This isn’t a problem when I syndicate my blog posts, because I’ve got a caching mechanism in place for those. I didn’t think I’d need any caching for little ol’ notes. I was wrong.

A simple solution would be not to include the link back to the original note. But I like the reminder that what you see on Mastodon is just a copy. So now I’ve got the same caching mechanism for my notes as I do for my journal (and I did my links while I was at it). Everything is hunky-dory. I can syndicate to Mastodon with impunity.

See? I told you it would only be of interest to me. Although I guess there’s a lesson here. Something something caching.

Two weeks ago, I wrote:

I woke up today to a very annoying new bug in Firefox. The browser shits the bed in an unpredictable fashion when rounding up single pixel line widths in SVG. That’s quite a problem on The Session where all the sheet music is rendered in SVG. Those thin lines in sheet music are kind of important.

Paul Rosen, who makes abcjs, the JavaScript library that renders sheet music on The Session, managed to get a fix out pretty quickly. But I use an older version of the library and updating it would introduce some side-effects that would take me a while to work around. So that option wasn’t available to me.

In this situation, when the problem is caused by a browser bug, the correct course of action is to file a bug with the browser. That had already been done. Now all I could do was twiddle my thumbs and wait for the next release of the browser, which would hopefully ship with the fix.

But I figured I may as well try to find a temporary workaround in the meantime.

At first, I looked at diving into the internals of the JavaScript—that’s where the instructions are given for drawing the SVGs.

But then I stopped and thought, “If the problem is with the rendering of the SVG, maybe CSS can help.”

I started messing around with SVG-specific CSS properties like stroke, fill, and so on. With dev tools open, I started targeting the paths that acted as bar lines in the sheet music, playing around with widths, opacities, and fills.

It was the debugging equivalent of throwing spaghetti at the wall. Remarkably, it actually worked.

I found a solution with this nonsensical bit of CSS:

stroke: currentColor;
stroke-opacity: 0;

For some reason, rather than making all the barlines disappear, this ensured they were visible.

It’s the worst kind of hacky fix—the kind where you have no idea why it works, but it does.

So I shipped it.

And at pretty much exactly the same time, a new version of Firefox dropped …with the bug fixed.

I can’t deny that there was a certain satisfaction in being able to work around a browser bug. But there’s much more satisfaction in deleting the hacky workaround when it’s no longer needed.

I documented a weird bug with web audio on iOS a while back:

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

I found a workaround but it was really hacky. By playing a one-second long silent mp3 file using audio, you could “kick” the sound into behaving. Then you can use the Web Audio API and it would play consistently.

Well, that’s all changed with the latest release of Mobile Safari. Now what happens is that the Web Audio stuff plays …for one second. And then stops.

I removed the hacky workaround and the Web Audio API started behaving itself again …but your device can’t be set to silent.

The good news is that the Web Audio behaviour seems to be consistent now. It only plays if the device isn’t muted. This restriction doesn’t apply to video and audio elements; they will still play even if your device is set to silent.

This descrepancy between the two different ways of playing audio is kind of odd, but at least now the Web Audio behaviour is predictable.

You can hear the Web Audio API in action by going to any tune on The Session and pressing the “play audio” button.

A while back I wrote a blog post called Web Audio API weirdness on iOS. I described a bug in Mobile Safari along with a hacky fix. I finished by saying:

If you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Recently Jonathan Aldrich posted a thread about the same bug. He included a link to my blog post. He also said:

Thanks so much for your post, this was a truly pernicious problem!

That warms the cockles of my heart. It’s very gratifying to know that documenting the bug (and the fix) helped someone out. Or, as I put it:

Yay for bugblogging!

Forgive the Germanic compound word, but in this case I think it fits.

Bugblogging doesn’t need to involve a solution. Just documenting a bug is a good thing to do. Recently I documented a bug with progressive web apps on iOS. Before that I documented a bug in Facebook Container for Firefox. When I documented some weird behaviour with the Web Share API in Safari on iOS, I wasn’t even sure it was a bug but Tess was pretty sure it was and filed a proper bug report.

I’ve benefited from other people bugblogging. Phil Nash wrote Service workers: beware Safari’s range request. That was exactly what I needed to solve a problem I’d been having. And then that post about Phil solving my problem helped Peter Rukavina solve a similar issue so he wrote Phil Nash and Jeremy Keith Save the Safari Video Playback Day.

Again, this warmed the cockles of my heart. Bugblogging is worth doing just for the reward of that feeling.

There’s a similar kind of blog post where, instead of writing about a bug, you write about a particular technique. In one way, this is the opposite of bugblogging because you’re writing about things working exactly as they should. But these posts have a similar feeling to bugblogging because they also result in a warm glow when someone finds them useful.

Here are some recent examples of these kinds of posts—tipblogging?—that I’ve found useful:

• Eric wrote about flexibly centering an element with side-sligned content using CSS.
• Rich documented how to subset a variable font on a Mac.
• Stephanie wrote about a CSS technique for animating in a newly added element.

All three are very handy tips. Thanks, Eric! Thanks, Rich! Thanks, Stephanie!

Dave recently wrote some good advice about what to do—and what not to do—when it comes to complaining about web browsers. I wrote something on this topic a little while back:

If there’s something about a web browser that you’re not happy with (or, indeed, if there’s something you’re really happy with), take the time to write it down and publish it

To summarise Dave’s advice, avoid conspiracy theories and snark; stick to specifics instead.

It’s very good advice that I should heed (especially the bit about avoiding snark). In that spirit, I’d like to document what I think is a bug on iOS.

I don’t need to name the specific browser, because there is basically only one browser allowed on iOS. That’s not snark; that’s a statement of fact.

This bug involves navigating from a progressive web app that has been installed on your home screen to an external web view.

To illustrate the bug, I’ll use the example of The Session. If you want to recreate the bug, you’ll need to have an account on The Session. Let me know if you want to set up a temporary account—I can take care of deleting it afterwards.

Here are the steps:

1. Navigate to thesession.org in Safari on an iOS device.
2. Add the site to your home screen.
3. Open the installed site from your home screen—it will launch in standalone mode.
4. Log in with your username and password.
5. Using the site menu, navigate to the links section of the site.
6. Click on any external link.
7. After the external link opens in a web view, tap on “Done” to close the web view.

Expected behaviour: you are returned to the page you were on with no change of state.

Actual behaviour: you are returned to the page you were on but you are logged out.

So the act of visiting an external link in a web view while in a progressive web app in standalone mode seems to cause a loss of cookie-based authentication.

This isn’t permanent. Clicking on any internal link restores the logged-in state.

It is surprising though. My mental model for opening an external link in a web view is that it sits “above” the progressive web app, which remains in stasis “behind” it. But the page must actually be reloading, either when the web view is opened or when the web view is closed. And that reload is behaving like a fetch event without credentials.

Anyway, that’s my bug report. It may already be listed somewhere on the WebKit Bugzilla but I lack the deductive skills to find it. I’m not even sure if that’s the right place for this kind of bug. It might be specific to the operating system rather than the rendering engine.

This isn’t a high priority bug, but it is one of those cumulatively annoying software paper cuts.

Hope this helps!

Safari has been subjected to a lot of ire recently. Most of that ire has been aimed at the proposed changes to the navigation bar in Safari on iOS—moving it from a fixed top position to a floaty bottom position right over the content you’re trying to interact with.

Courage.

It remains to be seen whether this change will actually ship. That’s why it’s in beta—to gather all the web’s hot takes first.

But while this very visible change is dominating the discussion, invisible changes can be even more important. Or in the case of Safari, the lack of changes.

Compared to other browsers, Safari lags far behind when it comes to shipping features. I’m not necessarily talking about cutting-edge features either. These are often standards that have been out for years. This creates a gap—albeit an invisible one—between Safari and other browsers.

Jorge Arango has noticed this gap:

I use Safari as my primary browser on all my devices. I like how Safari integrates with the rest of the OS, its speed, and privacy features. But, alas, I increasingly have issues rendering websites and applications on Safari.

That’s the perspective of an end-user. Developers who have to deal with the gap in features are more, um, strident in their opinions. Perry Sun wrote For developers, Apple’s Safari is crap and outdated:

Don’t get me wrong, Safari is very good web browser, delivering fast performance and solid privacy features.

But at the same time, the lack of support for key web technologies and APIs has been both perplexing and annoying at the same time.

Alas, that post also indulges in speculation about Apple’s motives which always feels a bit too much like a conspiracy theory to me. Baldur Bjarnason has more to say on that topic in his post Kremlinology and the motivational fallacy when blogging about Apple. He also points to a good example of critiquing Safari without speculating about motives: Dave’s post One-offs and low-expectations with Safari, which documents all the annoying paper cuts inflicted by Safari’s “quirks.”

Another deep dive that avoids speculating about motives comes from Tim Perry: Safari isn’t protecting the web, it’s killing it. I don’t agree with everything in it. I think that Apple—and Mozilla’s—objections to some device APIs are informed by a real concern about privacy and security. But I agree with his point that it’s not enough to just object; you’ve got to offer an alternative vision too.

That same post has a litany of uncontroversial features that shipped in Safari looong after they shipped in other browsers:

Again: these are not contentious features shipping by only Chrome, they’re features with wide support and no clear objections, but Safari is still not shipping them until years later. They’re also not shiny irrelevant features that “bloat the web” in any sense: each example I’ve included above primarily improving core webpage UX and performance. Safari is slowing that down progress here.

But perhaps most damning of all is how Safari deals with bugs.

A recent release of Safari shipped with a really bad Local Storage bug. The bug was fixed within a day. Yay! But the fix won’t ship until …who knows?

This is because browser updates are tied to operating system updates. Yes, this is just like the 90s when Microsoft claimed that Internet Explorer was intrinsically linked to Windows (a tactic that didn’t work out too well for them in the subsequent court case).

I don’t get it. I’m pretty sure that other Apple products ship updates and fixes independentally of OS releases. I’m sure I’ve received software updates for Keynote, Garage Band, and other pieces of software made by Apple.

And yet, of all the applications that need a speedy update cycle—a user agent for the World Wide Web—Apple’s version is needlessly delayed by the release cycle of the entire operating system.

I don’t want to speculate on why this might be. I don’t know the technical details. But I suspect that the root cause might not be technical in nature. Apple have always tied their browser updates to OS releases. If Google’s cardinal sin is avoiding anything “Not Invented Here”, Apple’s downfall is “We’ve always done it this way.”

Evergreen browsers update in the background, usually at regular intervals. Firefox is an evergreen browser. Chrome is an evergreen browser. Edge is an evergreen browser.

Safari is not an evergreen browser.

That’s frustrating when it comes to new features. It’s unforgivable when it comes to bugs.

At least on Apple’s desktop computers, users have the choice to switch to a different browser. But on Apple’s mobile devices, users have no choice but to use Safari’s rendering engine, bugs and all.

As I wrote when I had to deal with one of Safari’s bugs:

I wish that Apple would allow other rendering engines to be installed on iOS devices. But if that’s a hell-freezing-over prospect, I wish that Safari updates weren’t tied to operating system updates.

Firefox has a nifty extension—made by Mozilla—called Facebook Container. It does two things.

First of all, it sandboxes any of your activity while you’re on the facebook.com domain. The tab you’re in is isolated from all others.

Secondly, when you visit a site that loads a tracker from Facebook, the extension alerts you to its presence. For example, if a page has a share widget that would post to Facebook, a little fence icon appears over the widget warning you that Facebook will be able to track that activity.

It’s a nifty extension that I’ve been using for quite a while. Except now it’s gone completely haywire. That little fence icon is appearing all over the web wherever there’s a form with an email input. See, for example, the newsletter sign-up form in the footer of the Clearleft site. It’s happening on forms over on The Session too despite the rigourous-bordering-on-paranoid security restrictions in place there.

Hovering over the fence icon displays this text:

If you use your real email address here, Facebook may be able to track you.

That is, of course, false. It’s also really damaging. One of the worst things that you can do in the security space is to cry wolf. If a concerned user is told that they can ignore that warning, you’re lessening the impact of all warnings, even serious legitimate ones.

Sometimes false positives are an acceptable price to pay for overall increased security, but in this case, the rate of false positives can only decrease trust.

I tried to find out how to submit a bug report about this but I couldn’t work it out (and I certainly don’t want to file a bug report in a review) so I’m writing this in the hopes that somebody at Mozilla sees it.

What’s really worrying is that this might not be considered a bug. The release notes for the version of the extension that came out last week say:

Email fields will now show a prompt, alerting users about how Facebook can track users by their email address.

Like …all email fields? That’s ridiculous!

I thought the issue might’ve been fixed in the latest release that came out yesterday. The release notes say:

This release addresses fixes a issue from our last release – the email field prompt now only displays on sites where Facebook resources have been blocked.

But the behaviour is unfortunately still there, even on sites like The Session or Clearleft that wouldn’t touch Facebook resources with a barge pole. The fence icon continues to pop up all over the web.

I hope this gets sorted soon. I like the Facebook Container extension and I’d like to be able to recommend it to other people. Right now I’d recommed the opposite—don’t install this extension while it’s behaving so overzealously. If the current behaviour continues, I’ll be uninstalling this extension myself.

Update: It looks like a fix is being rolled out. Fingers crossed!

I think I’ve found some more strange service worker behaviour in Chrome.

It all started when I was checking out the very nice new redesign of WebPageTest. I figured while I was there, I’d run some of my sites through it. I passed in a URL from The Session. When the test finished, I noticed that the “screenshot” tab said that something was being logged to the console. That’s odd! And the file doing the logging was the service worker script.

I fired up Chrome (which isn’t my usual browser), and started navigating around The Session with dev tools open to see what appeared in the console. Sure enough, there was a failed fetch attempt being logged. The only time my service worker script logs anything is in the catch clause of fetching pages from the network. So Chrome was trying to fetch a web page, failing, and logging this error:

The service worker navigation preload request failed with a network error.

But all my pages were loading just fine. So where was the error coming from?

After a lot of spelunking and debugging, I think I’ve figured out what’s happening…

First of all, I’m making use of navigation preloads in my service worker. That’s all fine.

Secondly, the website is a progressive web app. It has a manifest file that specifies some metadata, including start_url. If someone adds the site to their home screen, this is the URL that will open.

Thirdly, Google recently announced that they’re tightening up the criteria for displaying install prompts for progressive web apps. If there’s no network connection, the site still needs to return a 200 OK response: either a cached copy of the URL or a custom offline page.

So here’s what I think is happening. When I navigate to a page on the site in Chrome, the service worker handles the navigation just fine. It also parses the manifest file I’ve linked to and checks to see if that start URL would load if there were no network connection. And that’s when the error gets logged.

I only noticed this behaviour because I had specified a query string on my start URL in the manifest file. Instead of a start_url value of /, I’ve set a start_url value of /?homescreen. And when the error shows up in the console, the URL being fetched is /?homescreen.

Crucially, I’m not seeing a warning in the console saying “Site cannot be installed: Page does not work offline.” So I think this is all fine. If I were actually offline, there would indeed be an error logged to the console and that start_url request would respond with my custom offline page. It’s just a bit confusing that the error is being logged when I’m online.

I thought I’d share this just in case anyone else is logging errors to the console in the catch clause of fetches and is seeing an error even when everything appears to be working fine. I think there’s nothing to worry about.

Update: Jake confirmed my diagnosis and agreed that the error is a bit confusing. The good news is that it’s changing. In Chrome Canary the error message has already been updated to:

DOMException: The service worker navigation preload request failed due to a network error. This may have been an actual network error, or caused by the browser simulating offline to see if the page works offline: see https://w3c.github.io/manifest/#installability-signals

Much better!

I told you about how I’m using the Web Audio API on The Session to generate synthesised audio of each tune setting. I also said:

Except for some weirdness on iOS that I had to fix.

Here’s that weirdness…

Let me start by saying that this isn’t anything to do with requiring a user interaction (the Web Audio API insists on some kind of user interaction to prevent developers from having auto-playing sound on websites). All of my code related to the Web Audio API is inside a click event handler. This is a different kind of weirdness.

First of all, I noticed that if you pressed play on the audio player when your iOS device is on mute, then you don’t hear any audio. Seems logical, right? Except if using the same device, still set to mute, you press play on a video or audio element, the sound plays just fine. You can confirm this by going to Huffduffer and pressing play on any of the audio elements there, even when your iOS device is set on mute.

So it seems that iOS has different criteria for the Web Audio API than it does for audio or video. Except it isn’t quite that straightforward.

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

Following my theory that the browser needs a “kick” to get into the right frame of mind for the Web Audio API, I resorted to a messy little hack.

In the event handler for the audio player, I generate the “kick” by playing a second of silence using the JavaScript equivalent of the audio element:

var audio = new Audio('1-second-of-silence.mp3');
audio.play();

I’m not proud of that. It’s so hacky that I’ve even wrapped the code in some user-agent sniffing on the server, and I never do user-agent sniffing!

Still, if you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Update: Time to remove this workaround. Mobile Safari has been updated.

It seems that some code that I wrote in Going Offline is haunted. It’s the trimCache function.

First, there was the issue of a typo. Or maybe it’s more of a brainfart than a typo, but either way, there’s a mistake in the syntax that was published in the book.

Now it turns out that there’s also a problem with my logic.

To recap, this is a function that takes two arguments: the name of a cache, and the maximum number of items that cache should hold.

function trimCache(cacheName, maxItems) {

First, we open up the cache:

caches.open(cacheName)
.then( cache => {

Then, we get the items (keys) in that cache:

cache.keys()
.then(keys => {

Now we compare the number of items (keys.length) to the maximum number of items allowed:

if (keys.length > maxItems) {

If there are too many items, delete the first item in the cache—that should be the oldest item:

cache.delete(keys[0])

And then run the function again:

.then(
    trimCache(cacheName, maxItems)
);

A-ha! See the problem?

Neither did I.

It turns out that, even though I’m using then, the function will be invoked immediately, instead of waiting until the first item has been deleted.

Trys helped me understand what was going on by making a useful analogy. You know when you use setTimeout, you can’t put a function—complete with parentheses—as the first argument?

window.setTimeout(doSomething(someValue), 1000);

In that example, doSomething(someValue) will be invoked immediately—not after 1000 milliseconds. Instead, you need to create an anonymous function like this:

window.setTimeout( function() {
    doSomething(someValue)
}, 1000);

Well, it’s the same in my trimCache function. Instead of this:

cache.delete(keys[0])
.then(
    trimCache(cacheName, maxItems)
);

I need to do this:

cache.delete(keys[0])
.then( function() {
    trimCache(cacheName, maxItems)
});

Or, if you prefer the more modern arrow function syntax:

cache.delete(keys[0])
.then( () => {
    trimCache(cacheName, maxItems)
});

Either way, I have to wrap the recursive function call in an anonymous function.

Here’s a gist with the updated trimCache function.

What’s annoying is that this mistake wasn’t throwing an error. Instead, it was causing a performance problem. I’m using this pattern right here on my own site, and whenever my cache of pages or images gets too big, the trimCaches function would get called …and then wouldn’t stop running.

I’m very glad that—witht the help of Trys at last week’s Homebrew Website Club Brighton—I was finally able to get to the bottom of this. If you’re using the trimCache function in your service worker, please update the code accordingly.

Management regrets the error.

Alright, so I’ve already talked about some gotchas when debugging service worker issues. But what if you don’t even realise the problem has anything to do with your service worker?

This is not a hypothetical situation. I encountered this very thing myself. Gather ‘round the campfire, children…

One of the latest case studies on the Clearleft site is a nice write-up by Luke of designing a mobile app for Virgin Holidays. The case study includes a lovely video that demonstrates the log-in flow. I implemented that using a video element (with a poster image). Nice and straightforward. Super easy. All good.

But I hadn’t done my due diligence in browser testing (I guess I didn’t even think of it in this case). Hana informed me that the video wasn’t working at all in Safari. The poster image appeared just fine, but when you clicked on it, the video didn’t load.

I ducked, ducked, and went, uncovering what appeared to be the root of the problem. It seems that Safari is fussy about having servers support something called “byte-range requests”.

I had put the video in question on an Amazon S3 server. I came to the conclusion that S3 mustn’t support these kinds of headers correctly, or something.

Now I had a diagnosis. The next step was figuring out a solution. I thought I might have to move the video off of S3 and onto a server that I could configure a bit more.

Luckily, I never got ‘round to even starting that process. That’s good. Because it turns out that my diagnosis was completely wrong.

I came across a recent post by Phil Nash called Service workers: beware Safari’s range request. The title immediately grabbed my attention. Safari: yes! Video: yes! But service workers …wait a minute!

There’s a section in Phil’s post entitled “Diagnosing the problem”, in which he says:

I first thought it could have something to do with the CDN I’m using. There were some false positives regarding streaming video through a CDN that resulted in some extra research that was ultimately fruitless.

That described my situation exactly. Except Phil went further and nailed down the real cause of the problem:

Nginx was serving correct responses to Range requests. So was the CDN. The only other problem? The service worker. And this broke the video in Safari.

Doh! I hadn’t even thought about service workers!

Phil came up with a solution, and he has kindly shared his code.

I decided to go for a dumber solution:

if ( request.url.match(/\.(mp4)$/) ) {
  return;
}

That tells the service worker to just step out of the way when it comes to video requests. Now the video plays just fine in Safari. It’s a bit of a shame, because I’m kind of penalising all browsers for Safari’s bug, but the Clearleft site isn’t using much video at all, and in any case, it might be good not to fill up the cache with large video files.

But what’s more important than any particular solution is correctly identifying the problem. I’m quite sure I never would’ve been able to fix this issue if Phil hadn’t gone to the trouble of sharing his experience. I’m very, very grateful that he did.

That’s the bigger lesson here: if you solve a problem—even if you think it’s hardly worth mentioning—please, please share your solution. It could make all the difference for someone out there.

I quite enjoy a good bug hunt. Just yesterday, myself and Cassie were doing some bugfixing together. As always, the first step was to try to reproduce the problem and then isolate it. Which reminds me…

There’ve been a few occasions when I’ve been trying to debug service worker issues. The problem is rarely in reproducing the issue—it’s isolating the cause that can be frustrating. I try changing a bit of code here, and a bit of code there, in an attempt to zero in on the problem, butwith no luck. Before long, I’m tearing my hair out staring at code that appears to have nothing wrong with it.

And that’s when I remember: browser extensions.

I’m currently using Firefox as my browser, and I have extensions installed to stop tracking and surveillance (these technologies are usually referred to as “ad blockers”, but that’s a bit of a misnomer—the issue isn’t with the ads; it’s with the invasive tracking).

If you think about how a service worker does its magic, it’s as if it’s sitting in the browser, waiting to intercept any requests to a particular domain. It’s like the service worker is the first port of call for any requests the browser makes. But then you add a browser extension. The browser extension is also waiting to intercept certain network requests. Now the extension is the first port of call, and the service worker is relegated to be next in line.

This, apparently, can cause issues (presumably depending on how the browser extension has been coded). In some situations, network requests that should work just fine start to fail, executing the catch clauses of fetch statements in your service worker.

So if you’ve been trying to debug a service worker issue, and you can’t seem to figure out what the problem might be, it’s not necessarily an issue with your code, or even an issue with the browser.

From now on when I’m troubleshooting service worker quirks, I’m going to introduce a step zero, before I even start reproducing or isolating the bug. I’m going to ask myself, “Are there any browser extensions installed?”

I realise that sounds as basic as asking “Are you sure the computer is switched on?” but there’s nothing wrong with having a checklist of basic questions to ask before moving on to the more complicated task of debugging.

I’m going to make a checklist. Then I’m going to use it …every time.

Whenever I create a fetch event inside a service worker, my code roughly follows the same pattern. There’s a then clause which gets executed if the fetch is successful, and a catch clause in case anything goes wrong:

fetch( request)
.then( fetchResponse => {
    // Yay! It worked.
})
.catch( fetchError => {
    // Boo! It failed.
});

In my book—Going Offline—I’m at pains to point out that those arguments being passed into each clause are yours to name. In this example I’ve called them fetchResponse and fetchError but you can call them anything you want.

I always do something with the fetchResponse inside the then clause—either I want to return the response or put it in a cache.

But I rarely do anything with fetchError. Because of that, I’ve sometimes made the mistake of leaving it out completely:

fetch( request)
.then( fetchResponse => {
    // Yay! It worked.
})
.catch( () => {
    // Boo! It failed.
});

Don’t do that. I think there’s some talk of making the error argument optional, but for now, some browsers will get upset if it’s not there.

So always include that argument, whether you call it fetchError or anything else. And seeing as it’s an error, this might be a legitimate case for outputing it to the browser’s console, even in production code.

And yes, you can output to the console from a service worker. Even though a service worker can’t access anything relating to the document object, you can still make use of window.console, known to its friends as console for short.

My muscle memory when it comes to sending something to the console is to use console.log:

fetch( request)
.then( fetchResponse => {
    return fetchResponse;
})
.catch( fetchError => {
    console.log(fetchError);
});

But in this case, the console.error method is more appropriate:

fetch( request)
.then( fetchResponse => {
    return fetchResponse;
})
.catch( fetchError => {
    console.error(fetchError);
});

Now when there’s a connectivity problem, anyone with a console window open will see the error displayed bold and red.

If that seems a bit strident to you, there’s always console.warn which will still make the output stand out, but without being quite so alarmist:

fetch( request)
.then( fetchResponse => {
    return fetchResponse;
})
.catch( fetchError => {
    console.warn(fetchError);
});

That said, in this case, console.error feels like the right choice. After all, it is technically an error.

Paul Yabsley wrote to let me know about an error in Going Offline. It’s rather embarrassing because it’s code that I’m using in the service worker for adactio.com but for some reason I messed it up in the book.

It’s the trimCache function in Chapter 7: Tidying Up. That’s the reusable piece of code that recursively reduces the number of items in a specified cache (cacheName) to a specified amount (maxItems). On page 95 and 96 I describe the process of creating the function which, in the book, ends up like this:

 function trimCache(cacheName, maxItems) {
   cacheName.open( cache => {
     cache.keys()
     .then( items => {
       if (items.length > maxItems) {
         cache.delete(items[0])
         .then(
           trimCache(cacheName, maxItems)
         ); // end delete then
       } // end if
     }); // end keys then
   }); // end open
 } // end function

See the problem? It’s right there at the start when I try to open the cache like this:

cacheName.open( cache => {

That won’t work. The open method only works on the caches object—I should be passing the name of the cache into the caches.open method. So the code should look like this:

caches.open( cacheName )
.then( cache => {

Everything else remains the same. The corrected trimCache function is here:

function trimCache(cacheName, maxItems) {
  caches.open(cacheName)
  .then( cache => {
    cache.keys()
    .then(items => {
      if (items.length > maxItems) {
        cache.delete(items[0])
        .then(
          trimCache(cacheName, maxItems)
        ); // end delete then
      } // end if
    }); // end keys then
  }); // end open then
} // end function

Sorry about that! I must’ve had some kind of brainfart when I was writing (and describing) that one line of code.

You may want to deface your copy of Going Offline by taking a pen to that code example. Normally I consider the practice of writing in books to be barbarism, but in this case …go for it.

Update: There was another error in the code for trimCache! Here’s the fix.

When I wrote about implementing my first Service Worker, I finished with this plea:

And remember, please share your code and your gotchas: it’s early days for Service Workers so every implementation counts.

Well, I ran into a gotcha that was really frustrating but thanks to the generosity of others, I was able to sort it out.

It was all because of an issue in Chrome. Here’s the problem…

Let’s say you’ve got a Service Worker running that takes care of any requests to your site. Now on that site, you’ve got a URL that receives POST data, does something with it, and then redirects to another URL. That’s a fairly common situation—it’s how I handle webmentions here on adactio.com, and it’s how I handle most add/edit/delete actions over on The Session to help prevent duplicate form submissions.

Anyway, it turns out that Chrome’s Service Worker implementation would get confused by that. Instead of redirecting, it showed the offline page instead. The fetch wasn’t resolving.

I described the situation to Jake, but rather than just try and explain it in 140 characters, I built a test case.

There’s a Chromium issue filed on this, and it will get fixed, but it in the meantime, it was really bugging me recently when I was rolling out a new feature on The Session. Matthew pointed out that the Chromium bug report also contained a workaround that he’s been using on traintimes.org.uk. Adrian also posted his expanded workaround in there too. That turned out to be exactly what I needed.

I think the problem is that the redirect means that a body is included in the GET request, which is what’s throwing the Service Worker. So I need to create a duplicate request without the body:

request = new Request(url, {
    method: 'GET',
    headers: request.headers,
    mode: request.mode == 'navigate' ? 'cors' : request.mode,
    credentials: request.credentials,
    redirect: request.redirect
});

So here’s what I had in my Service Worker before:

// For HTML requests, try the network first, fall back to the cache, finally the offline page
if (request.headers.get('Accept').indexOf('text/html') !== -1) {
    event.respondWith(
        fetch(request)
            .then( response => {
                // NETWORK
                // Stash a copy of this page in the pages cache
                let copy = response.clone();
                stashInCache(pagesCacheName, request, copy);
                return response;
            })
            .catch( () => {
                // CACHE or FALLBACK
                return caches.match(request)
                    .then( response => response || caches.match('/offline') );
                })
        );
    return;
}

And here’s what I have now:

// For HTML requests, try the network first, fall back to the cache, finally the offline page
if (request.headers.get('Accept').indexOf('text/html') !== -1) {
    request = new Request(url, {
        method: 'GET',
        headers: request.headers,
        mode: request.mode == 'navigate' ? 'cors' : request.mode,
        credentials: request.credentials,
        redirect: request.redirect
    });
    event.respondWith(
        fetch(request)
            .then( response => {
                // NETWORK
                // Stash a copy of this page in the pages cache
                let copy = response.clone();
                stashInCache(pagesCacheName, request, copy);
                return response;
            })
            .catch( () => {
                // CACHE or FALLBACK
                return caches.match(request)
                    .then( response => response || caches.match('/offline') );
                })
        );
    return;
}

Now the test case is working just fine in Chrome.

On the off-chance that someone out there is struggling with the same issue, I hope that this is useful.

Share what you learn.

Last Christmas I gave you my bug report. Well, more of a whinge really. Scott put together a much better bug report and test page:

When the meta viewport tag is set to content=”width=device-width,initial-scale=1”, or any value that allows user-scaling, changing the device to landscape orientation causes the page to scale larger than 1.0. As a result, a portion of the page is cropped off the right, and the user must double-tap (sometimes more than once) to get the page to zoom properly into view.

Yes, it’s the old orientation and scale bug in Mobile Safari.

I’m pleased to report that as of iOS version 6, this bug seems to have finally been squashed. Hallelujah!

Given the relatively rapid upgrade path for iPhone, iPod Touch and iPad users, it won’t be long until we can remove our clever solutions for working around this problem.

Stand down, hackers, stand down. This bug has been taken care of.

Dear Apple Claus,

I’ve been a very good boy this year so I hope you don’t me asking for a little present. What I’d really like for Christmas is for you to fix that strange orientation scaling bug in Mobile Safari.

Just in case you’ve forgotten about it, my friend Scott—who has been a very, very good boy this year (what with that whole Boston Globe thing)—put together a test page quite a while back to demonstrate the problem.

Basically, if I set meta name="viewport" content="width=device-width, initial-scale=1.0" then it means a pixel should be equal to a pixel: in portrait view, the width should be 320 pixels; in landscape view the width should be 480 pixels. But in Mobile Safari, if I move from portrait to landscape, the width jumps to a value larger than 480 pixels, which means the hapless user must double tap to bring the scale down to 1:1.

Now, admittedly, I could just set meta name="viewport" content="width=device-width" and leave it at that (or I could additionally declare minimum-scale=1.0). But then when the user changes from portrait to landscape, although it doesn’t have the same over-zooming behaviour, it does scale up. That means I’m not getting the full 480 pixels (it’s effectively still a 320 pixel wide display, even in landscape).

I could make the bug disappear by adding maximum-scale=1.0 or user-scaleable=no but that’s the cure that kills the patient. I also did some hacking with Shi Chuan but what we come up with still feels fairly clunky.

So that’s why I’m writing to you, Father Applemas. Won’t you fix this bug for me?

My friend PPK thinks you won’t fix this bug because it would trigger a reflow (and repaint) of the page …but I know that can’t be the reason because the bug doesn’t occur when going from landscape to portrait!

Also—and this is the really strange part—If I’m looking at a web page on my iPhone/Pod in a custom browser (like the Twitter app), rather than using Mobile Safari, then the bug doesn’t occur.

I don’t get, Apple Claus. Why have one behaviour for webviews in other people’s apps and a different behaviour for your own app?

Anyway, if you could see your way to granting this boy’s wish, it would make for a webby Christmas.

Hugs and kisses,

Jeremy

P.S. By this time next year, it would be lovely to have access to the camera (and other device APIs) from the browser …but I’m getting ahead of myself.

Update: the bug has been fixed in iOS 6.

John Allsopp and the fine folks at Westciv have released a really neat bookmarklet called XRAY. Drag it to your toolbar, visit any website and click on the bookmarklet to reveal a hovering DOM inspector. Clicking on any element in the currently loaded document will reveal not just its position in the DOM tree but also all the styles being applied to it.

Now I now what you’re going to say: You can do this already in Firebug! Yes, you can but Firebug is a browser-specific plugin. XRAY will work on any Mozilla-based browser or Safari (though it doesn’t yet work in Internet Explorer).

Much as I love tools like Firebug and YSlow, I always feel uneasy about being locked into a specific browser—regardless of whether that browser comes from the Microsoft Death Star or the Mozilla Rebel Alliance. That seem uneasiness also stops me from getting too excited about Greasemonkey scripts. They’re great but I wish that the same functionality was available to all browsers.

Anyway, that’s why I like bookmarklets (sorry Tantek, I can’t really get behind the term “favelet”). XRAY is particularly lovely example of the craft. For another lovely examplar, be sure to grab the microformats bookmarklet from fellow Brightonian Remy Sharp.