Journal tags: ipad

I documented a weird bug with web audio on iOS a while back:

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

I found a workaround but it was really hacky. By playing a one-second long silent mp3 file using audio, you could “kick” the sound into behaving. Then you can use the Web Audio API and it would play consistently.

Well, that’s all changed with the latest release of Mobile Safari. Now what happens is that the Web Audio stuff plays …for one second. And then stops.

I removed the hacky workaround and the Web Audio API started behaving itself again …but your device can’t be set to silent.

The good news is that the Web Audio behaviour seems to be consistent now. It only plays if the device isn’t muted. This restriction doesn’t apply to video and audio elements; they will still play even if your device is set to silent.

This descrepancy between the two different ways of playing audio is kind of odd, but at least now the Web Audio behaviour is predictable.

You can hear the Web Audio API in action by going to any tune on The Session and pressing the “play audio” button.

I told you about how I’m using the Web Audio API on The Session to generate synthesised audio of each tune setting. I also said:

Except for some weirdness on iOS that I had to fix.

Here’s that weirdness…

Let me start by saying that this isn’t anything to do with requiring a user interaction (the Web Audio API insists on some kind of user interaction to prevent developers from having auto-playing sound on websites). All of my code related to the Web Audio API is inside a click event handler. This is a different kind of weirdness.

First of all, I noticed that if you pressed play on the audio player when your iOS device is on mute, then you don’t hear any audio. Seems logical, right? Except if using the same device, still set to mute, you press play on a video or audio element, the sound plays just fine. You can confirm this by going to Huffduffer and pressing play on any of the audio elements there, even when your iOS device is set on mute.

So it seems that iOS has different criteria for the Web Audio API than it does for audio or video. Except it isn’t quite that straightforward.

On some pages of The Session, as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using the video element). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing a video or audio element “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

Following my theory that the browser needs a “kick” to get into the right frame of mind for the Web Audio API, I resorted to a messy little hack.

In the event handler for the audio player, I generate the “kick” by playing a second of silence using the JavaScript equivalent of the audio element:

var audio = new Audio('1-second-of-silence.mp3');
audio.play();

I’m not proud of that. It’s so hacky that I’ve even wrapped the code in some user-agent sniffing on the server, and I never do user-agent sniffing!

Still, if you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Update: Time to remove this workaround. Mobile Safari has been updated.

I started getting some emails recently from people having issues using The Session. The issues sounded similar—an interactive component that wasn’t, well …interacting.

When I asked what device or browser they were using, the answer came back the same: Safari on iPad. But not a new iPad. These were older iPads running older operating systems.

Now, remember, even if I wanted to recommend that they use a different browser, that’s not an option:

Safari is the only browser on iOS devices.

I don’t mean it’s the only browser that ships with iOS devices. I mean it’s the only browser that can be installed on iOS devices.

You can install something called Chrome. You can install something called Firefox. Those aren’t different web browsers. Under the hood they’re using Safari’s rendering engine. They have to.

It gets worse. Not only is there no choice when it comes to rendering engines on iOS, but the rendering engine is also tied to the operating system.

If you’re on an old Apple laptop, you can at least install an up-to-date version of Firefox or Chrome. But you can’t install an up-to-date version of Safari. An up-to-date version of Safari requires an up-to-date version of the operating system.

It’s the same on iOS devices—you can’t install a newer version of Safari without installing a newer version of iOS. But unlike the laptop scenario, you can’t install any version of Firefox of Chrome.

It’s disgraceful.

It’s particularly frustrating when an older device can’t upgrade its operating system. Upgrades for Operating system generally have some hardware requirements. If your device doesn’t meet those requirements, you can’t upgrade your operating system. That wouldn’t matter so much except for the Safari issue. Without an upgraded operating system, your web browsing experience stagnates unnecessarily.

For want of a nail…

• A website feature isn’t working so
• you need to upgrade your browser which means
• you need to upgrade your operating sytem but
• you can’t upgrade your operating system so
• you need to buy a new device.

Apple doesn’t allow other browsers to be installed on iOS devices so people have to buy new devices if they want to use the web. Handy for Apple. Bad for users. Really bad for the planet.

It’s particularly galling when it comes to iPads. Those are exactly the kind of casual-use devices that shouldn’t need to be caught in the wasteful cycle of being used for a while before getting thrown away. I mean, I get why you might want to have a relatively modern phone—a device that’s constantly with you that you use all the time—but an iPad is the perfect device to just have lying around. You shouldn’t feel pressured to have the latest model if the older version still does the job:

An older tablet makes a great tableside companion in your living room, an effective e-book reader, or a light-duty device for reading mail or checking your favorite websites.

Hang on, though. There’s another angle to this. Why should a website demand an up-to-date browser? If the website has been built using the tried and tested approach of progressive enhancement, then everyone should be able to achieve their goals regardless of what browser or device or operating system they’re using.

On The Session, I’m using progressive enhancement and feature detection everywhere I can. If, for example, I’ve got some JavaScript that’s going to use querySelectorAll and addEventListener, I’ll first test that those methods are available.

if (!document.querySelectorAll || !window.addEventListener) {
  // doesn't cut the mustard.
  return;
}

I try not to assume that anything is supported. So why was I getting emails from people with older iPads describing an interaction that wasn’t working? A JavaScript error was being thrown somewhere and—because of JavaScript’s brittle error-handling—that was causing all the subsequent JavaScript to fail.

I tracked the problem down to a function that was using some DOM methods—matches and closest—as well as the relatively recent JavaScript forEach method. But I had polyfills in place for all of those. Here’s the polyfill I’m using for matches and closest. And here’s the polyfill I’m using for forEach.

Then I spotted the problem. I was using forEach to loop through the results of querySelectorAll. But the polyfill works on arrays. Technically, the output of querySelectorAll isn’t an array. It looks like an array, it quacks like an array, but it’s actually a node list.

So I added this polyfill from Chris Ferdinandi.

That did the trick. I checked with the people with those older iPads and everything is now working just fine.

For the record, here’s the small collection of polyfills I’m using. Polyfills are supposed to be temporary. At some stage, as everyone upgrades their browsers, I should be able to remove them. But as long as some people are stuck with using an older browser, I have to keep those polyfills around.

I wish that Apple would allow other rendering engines to be installed on iOS devices. But if that’s a hell-freezing-over prospect, I wish that Safari updates weren’t tied to operating system updates.

Apple may argue that their browser rendering engine and their operating system are deeply intertwingled. That line of defence worked out great for Microsoft in the ‘90s.

Apple launched the iPad five years ago. A few months after its release, I bought one of those first-generation iPads. I used it for a while before concluding that, much as I had suspected, it wasn’t the right device for me. But it was the perfect device for my mother. So I gave the iPad to my mother.

Sure enough, it worked out great for her. But now it’s getting quite long in the tooth. So now I’ve given her a new iPad for her birthday; one that’s lighter, faster, and—crucially—comes with a camera. It’s a Facetime device.

Paul Irish, Divya Manian and Shi Chuan launched Mobile Boilerplate recently—a mobile companion site to HTML5 Boilerplate.

There’s some good stuff in there but I was a little surprised to see that the meta viewport element included values for minimum-scale=1.0, maximum-scale=1.0, user-scalable=no:

Setting user-scalable=no is pretty much the same as setting minimum-scale=1.0, maximum-scale=1.0. In any case, I’m not keen on it. Like Roger, I don’t think we should take away the user’s right to pinch and zoom to make content larger. That’s why my usual viewport declaration is:

Yes, I know that most native apps don’t allow you to zoom but I see no reason to replicate that failing on the web.

But there’s a problem. Allowing users to scale content for comfort would be fine if it weren’t for a bug in Mobile Safari:

When the meta viewport tag is set to content="width=device-width,initial-scale=1", or any value that allows user-scaling, changing the device to landscape orientation causes the page to scale larger than 1.0. As a result, a portion of the page is cropped off the right, and the user must double-tap (sometimes more than once) to get the page to zoom properly into view.

This is really annoying so Shi Chuan set about fixing the problem.

His initial solution was to keep minimum-scale=1.0, maximum-scale=1.0 in the meta viewport element but then to change it using JavaScript once the user initiates a gesture (the gesturestart event is triggered as soon as two fingers are on the screen). At the point, the content attribute of the meta viewport element gets updated to read minimum-scale=0.25, maximum-scale=1.6, the default values:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
  document.addEventListener("gesturestart", gestureStart, false);
  function gestureStart() {
    for (i=0; i<metas.length; i++) {
      if (metas[i].name == "viewport") {
        metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
      }
    }
  }
}

That works nicely but I wasn’t too keen on the dependency between the markup and the script. If, for whatever reason, the script doesn’t get executed, users are stuck with an unzoomable page.

I suggested that the script should also set the initial value to minimum-scale=1.0, maximum-scale=1.0:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
  for (i=0; i<metas.length; i++) {
    if (metas[i].name == "viewport") {
      metas[i].content = "width=device-width, minimum-scale=1.0, maximum-scale=1.0";
    }
  }
  document.addEventListener("gesturestart", gestureStart, false);
}
function gestureStart() {
  for (i=0; i<metas.length; i++) {
    if (metas[i].name == "viewport") {
      metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
    }
  }
}

Now the markup still contains the robust accessible default:

…while the script takes care of initially setting the scale values and also updating them when a gesture is detected. Here’s what’s happening:

1. By default, the page is scaleable because the initial meta viewport declaration doesn’t set a minimum-scale or maximum-scale.
2. Once the script loads, the page is no longer scalable because both minimum-scale and maximum-scale have been set to 1.0. If the device is switched from portrait to landscape, the resizing bug won’t be triggered because scaling is disabled.
3. When the gesturestart event is detected—indicating that the user might be trying to scale the page—the minimum-scale and maximum-scale values are updated to allow scaling. At this point, if the device is switched from portrait to landscape, the resizing bug will occur because the page is now scaleable.

Jason Weaver points out that you should probably detect for iPad too. That’s a pretty straightforward update:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i))

Mathias Bynens updated the code to use querySelectorAll which is supported in Mobile Safari. Here’s the code I’m currently using:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i)) {
  var viewportmeta = document.querySelector('meta[name="viewport"]');
  if (viewportmeta) {
    viewportmeta.content = 'width=device-width, minimum-scale=1.0, maximum-scale=1.0';
    document.body.addEventListener('gesturestart', function() {
      viewportmeta.content = 'width=device-width, minimum-scale=0.25, maximum-scale=1.6';
    }, false);
  }
}

You can try it out on Huffduffer, Salter Cane, Principia Gastronomica and right here on Adactio.

Right now there’s still a little sluggishness between the initial pinch-zoom gesture and the scaling; the scale values (0.25 - 1.6) don’t seem to take effect immediately. A second pinch-zoom gesture is often required. If you have any ideas for improving the event capturing and propagation, dive in there.

Update: the bug has been fixed in iOS 6.

When I was in Boston for An Event Apart back in May, I bought an iPad.

That makes it sound like a very casual act. Back then, iPads were in very short supply. When I asked an employee at the Apple Store in Boston if they had any iPads in stock, he didn’t answer Hmm… let me check, or Not at the moment. He answered Oh, God no!

Fortunately, Malarkey was far more persistent than I and managed, through means fair or foul, to track down the only two iPads for sale in all of Beantown.

I used the device for two months and then, as predicted, I gave it to my mother. It was definitely the right decision. She loves it. She’s using it all the time, surfing the web wherever she pleases.

I liked using the iPad, but I didn’t love it. Once the novelty wore off, I found I had few good reasons to use it rather than use my Macbook.

Some observations from a Summer spent in the company of the iPad…

The iPad is almost exactly the wrong weight. It’s just that little bit too heavy to comfortably hold in one hand. It’s no coincidence that all of iPad demonstrations show it resting on a raised leg. I found that if I was using the iPad for any length of time, I would adopt a more and more relaxed, nay… slouchy position. Before I knew it, I was supine on the sofa, one leg raised with the iPad resting against it, like a Roman emperor waiting to be fed individually-peeled grapes.

The iPad is a great device for reading. I rediscovered the power of RSS. The iPad is also a great device for browsing the web. The problem is, I don’t really browse the web that much. Instead, I browse, I bookmark, I quote, I huffduff, I copy, I paste, I tweet, I share. Using an iPad made me realise that the web is very much a read/write medium for me. While it’s possible to do all those things on the iPad, it isn’t easy.

The tell-tale moment came when I was reading something on the iPad—supine on the sofa, of course—that I wanted to post to Delicious. Rather than fill in a form using the iPad’s on-screen keyboard, I found myself putting the iPad down and reaching for my Macbook just to accomplish that one little task. That’s when I knew it wasn’t the ideal device for me.

Don’t get me wrong: I’m not joining in the chorus of those silly enough to say that the iPad can’t be used for creating, editing and updating. That conclusion would be far too absolute and Boolean. But I think the iPad favours consumption more than creation. And that’s okay. That still makes it a great device for many people—such as my mother.

There’s one big fly in the ointment of the iPad as a device for non-geeks—such as my mother—and that’s the out-of-the-box experience. John Gruber pointed this out in his review of the iPad:

One thing that is very iPhone-like about iPad is that when you first take it out of the box, it wants to be plugged into your Mac or PC via USB and sync with iTunes. … It creates an impression that the iPad does not stand on its own. It’s a child that still needs a parent. But it’s not a young child. It’s more like a teenager. It’s close. So close that it feels like it ought to be able to stand on its own.

My mother has an old G3 Ruby iMac which doesn’t have USB 2, so I had to set up the iPad on my machine before giving it to her. It’s such a shame that this step is even necessary. I love the idea of a portable touchscreen device that you can simply turn on for the first time, connect to a WiFi network, and start surfing the web.

Perhaps later versions of the iPad will support account synching via MobileMe. Perhaps later versions of the iPad will support multiple accounts, which would make it a great family household device. Perhaps later versions of the iPad will have a front-facing camera and support FaceTime. Perhaps later versions of the iPad will be slimmer and lighter.

Perhaps I will get a later version of the iPad.

Before Apple launched the iPad, I managed to refrain from adding to the deluge of speculation and rumour. Now that the much-anticipated tablet has been unveiled, I can’t resist jotting down my thoughts.

Now, this is just my reaction to a piece of technology. I feel a need to clarify that because discourse on the internet has a strange way of getting warped. Someone says I like Italian food, and someone else responds with Why do you hate Mexican food? Someone says I enjoyed watching Avatar, and someone else hears Everyone should enjoy watching Avatar. So bear in mind that this is just my personal reaction. I’m not saying that everyone should share my feelings. ‘Twould be a very dull world indeed in which we all felt the same.

I didn’t watch Steve Jobs unveiling the iPad—I was busy learning at a Skillswap event—but when I was reading up about it afterwards, I thought to myself I’m probably going to get an iPad…

Actually, at this point I need to take care of something:

Mum, if you’re reading this, could you stop now please? Thanks. Love you.

Anyway, as I was saying, I thought to myself I’m probably going to get an iPad …for my mother.

Honestly, there isn’t much on offer in the iPad that I don’t already have in my Macbook. I don’t think it is the device for me. But it is most definitely the device for my mother. I don’t mean “a theoretical persona such as one’s mother,” I mean my mother.

My mother is currently using a G3 Ruby iMac that used to belong to me. When she started using this machine, she had never used a keyboard, much less a computer. I am very, very glad that her first computer was a Mac and that she’s never had to deal with the world of pain that is Windows, but even a Mac has a learning curve for someone who’s never used a computer before.

I remember explaining what the cursor was and how the mouse controlled it. When I said “move it up”, she lifted up the mouse off the table. Thinking about it, the mouse isn’t as straightforward as we think: moving the mouse left and right does map to moving the cursor left and right, but moving the mouse forward and backward maps to moving the cursor up and down. Both the cursor and the mouse move on two-dimensional surfaces but only half the movements of the mouse correspond directly to movements of the cursor.

In computer years, a G3 iMac is ancient. It’s amazing that it still runs at all. I’ve been thinking for a while now about what would make a suitable replacement. A newer iMac would be good but they’re a little pricey for something that’s going to be used for web surfing, email, some digital photography and little else. A laptop would be nice. Now that my mother has WiFi, there’s no need for her to have to remain in one place to use her computer. But laptops are fiddly things with fiddly trackpads.

The iPad strikes me as the Goldilocks solution. It’s just right. If the European pricing follows the general Apple conversion rate, the iPad should be pretty darn affordable. It would be nice if it came with an iSight for iChatting; that might well get added in a later version. Web surfing, email and photo browsing are all not just possible, but likely to be pleasurable. That’s because the multitouch control mechanism is likely to feel far more intuitive than either a mouse or a trackpad. (Caveat: I haven’t used an iPad. Take my opinion, and the opinions of anyone else who hasn’t actually used one, with a heaped tablespoon of salt.)

So I’m probably going to get an iPad, but for someone else. If it came with nothing more than a WiFi connection and a web browser, it would still be a worthwhile device for my mother. In fact, the idea of using a computing device based around a browser is what’s driving the Google Chrome OS. Google’s vision is one wherein the file system and the hard drive are far less important than the web browser and the web server.

That’s why I’m slightly mystified about the App Store grumblings. Yes, it’s a closed system that Apple controls completely. But the same devices that support the App Store also come with a very advanced web browser. Personally, I think that if a device is capable of running HTML, CSS and JavaScript, I don’t think it can be described as “closed”.

Don’t like the closed nature of the App Store? Don’t use it. Use the web instead. That’s the point that PPK was making, albeit a bit stridently. Admittedly, if you want to make money directly from an app, you might have a harder time of it on the web than on the App Store. Make your app distribution bed and lie in it.

I’ve already seen people on Twitter sharing some ideas for the uses to which the iPad could be put:

• displaying sheet music on a music stand,
• showing recipes on a kitchen worktop,
• playing scrabble, sudoku and crosswords,
• reading comics,
• reading magazines a la Mag+,
• reading books in a way that doesn’t involve the silly page-turning visual metaphor built into the iBooks app.

All of those are great ideas and all of them can be implemented on the web. Remember that Mobile Safari already has excellent support for canvas, audio, video and offline storage. No App Store required. As Simon St. Laurent puts it, web developers can rule the iPad.

I understand the concerns of my fellow geeks who see the read-only nature of the iPad as restrictive compared to the read-write nature of laptop and desktop computers. Rafe Colburn asks Is the iPad the harbinger of doom for personal computing?:

I think that it’s a real possibility that in 10 years, general purpose computers will be seen as being strictly for developers and hobbyists.

Alex Payne foresees a tinkerer’s sunset:

The thing that bothers me most about the iPad is this: if I had an iPad rather than a real computer as a kid, I’d never be a programmer today.

While I understand and to a certain extent, share these forebodings, I’m cautiously optimistic that these fears won’t be realised. The iPad isn’t going to replace laptop or desktop computers; it’s a different kind of machine for a different kind of user.

Frasier Spiers welcomes the glimpse that the iPad offers us of information processing dissolving into behaviour when he writes:

If the iPad and its successor devices free these people to focus on what they do best, it will dramatically change people’s perceptions of computing from something to fear to something to engage enthusiastically with. I find it hard to believe that the loss of background processing isn’t a price worth paying to have a computer that isn’t frightening anymore.

Nik agrees:

Yes, it’s an entirely prescriptive way of computing - one that the hackers, tinkerers and geeks will find alien and protest about its lack of openness. But here’s the thing: for the people who the iPad is aimed at it really doesn’t matter that this experience is prescriptive.

I think he’s right. The iPad isn’t for geeks but I can foresee geeks, like me, buying iPads for members of their family …if for no other reason than to reverse the trend of the holiday season becoming the tech support season.

I’m not usually one for predictions, but I think I’ll try my hand at one now. The iPad will be the best-selling device to be purchased as a gift for Christmas 2010.