Journal tags: hackday

32

Audio

I spent the last couple of weekends rolling out a new feature on The Session. It involves playing audio in a web page. No big deal these days, right? But the history involves some old file formats…

The first venerable format is ABC notation. File extension: .abc, mime type: text/vnd.abc. It’s an ingenious text format for musical notation using ASCII. The metadata of the piece of music is defined in JSON-like key/value pairs. Then the contents are encoded with letters: A, B, C, etc. Uppercase and lowercase denote different octaves. Numbers can be used for note lengths.

The format was created by Chris Walshaw in 1997 when dial-up was the norm. With ABC, people were able to swap tunes on email lists or bulletin boards without transferring weighty image or sound files. If you had ABC software on your computer, you could convert that lightweight text file into sheet music …or audio.

That brings me to the second old format: midi files. File extension: .mid, mime-type: audio/midi. Like ABC, it’s a lightweight format for encoding the instructions for music instead of the music itself.

Think of it like SVG: instead of storing the final pixels of an image, SVG stores the instructions for drawing the image instead. The instructions in a midi file are like “play this note for this long on this instrument.” Again, as with ABC, you need some software to turn the instructions into sound.

There was a time when lots of software could play midi files. Quicktime on the Mac, for example. You could even embed midi files in web pages. I mean literally embed them …with the embed element. No Geocities page was complete without an autoplaying midi file.

On The Session, people submit tunes in ABC format. Then, using the amazing ABCJS JavaScript library, the ABC is turned into SVG on the fly! For years I’ve also offered midi files, generated on the server from the ABC notation.

But times have changed. These days it’s hard to find software that plays midi files. Quicktime doesn’t do it anymore. And you’d need to go to the app store on iOS to find a midi file player. It’s time to phase out the midi files on The Session.

I still want to provide automatically-generated audio though. Fortunately ABCJS gives me a way to do this. But instead of using the old technology of midi files, it uses a more modern browser feature: the Web Audio API.

The end result sounds like a midi file, but the underlying technique is more like a synthesiser. There’s a separate mp3 file for each note. The JavaScript figures out how long each “sample” needs to be played for, strings them all together, and outputs them with Web Audio. So you’ve got cutting-edge browser technology recreating a much older file format. Paul Rosen—the creator of ABCJS—has a presentation explaining how it all works under the hood.

Not only is there a separate short mp3 file for each note in seven octaves, but if you want the sound of a different instrument, you need samples for all seven octaves in that instrument. They’re called soundfonts.

Paul provides soundfonts for ABCJS. It’s a repo that was forked from this repo from Benjamin Gleitzman. And here’s where it gets small worldy…

The reason why Benjamin has a repo of soundfonts is because he needed to create midi-like audio in the browser. He wanted to do this for a project on September 28th and 29th, 2013 …at Science Hack Day San Francisco!

I was there too—working on my own audio-related hack—and I remember the excellent (and winning) hack that Benjamin worked on. It was called Symphony of Satellites and it’s still online along with the promo video. Here’s Benjamin’s post-hackday write-up from seven years ago.

It’s rare that the worlds of the web and Irish music cross over. When I got to meet Paul—creator of ABCJS—at a web conference a couple of years ago it kind of blew my mind. Last weekend when I set out to dabble with a feature on The Session, I certainly didn’t expect to stumble on a connection to Science Hack Day! (Aside: the first Science Hack Day was ten years ago—yowzers!)

Anyway, I was able to get that audio playback working on The Session. Except for some weirdness on iOS that I had to fix. But that’s a hack for another day.

Something for the weekend

Your weekends are valuable. Spend them wisely. I have some suggestion on how you might spend next weekend, October 19th and 20th, depending on where you are in the world.

If you’re in the bay area, or anywhere near San Francisco, I highly recommend that you go to Science Hack Day—two days of science, hacking, and fun. This will be the last one in San Francisco so don’t miss your chance.

If you’re in the south of England, or anywhere near Brighton, come along to Indie Web Camp. Saturday will feature discussions on owning your data. Sunday will be a day of doing. I’ve written about previous Indie Web Camps before, and I really can’t recommend it highly enough!

Do me a favour and register for a spot—it’s free—so I’ve got some idea of numbers. Looking forward to seeing you there!

Far afield

I spoke at Responsive Field Day here in Portland on Friday. It was an excellent event. All the talks were top notch.

The day flew by, with each talk clocking in at just 20 minutes, in batches of three followed by a quick panel discussion. It was a great format …but I knew it would be. See, Responsive Field Day was basically Responsive Day Out relocated to Portland.

Jason told me last year how inspired he was by the podcast recordings from Responsive Day Out and how much he and Lyza wanted to do a Responsive Day Out in Portland. I said “Go for it!” although I advised changing to the name to something a bit more American (having a “day out” at the seaside feels very British—a “field day” works perfectly as the US equivalent). Well, Jason, Lyza, and everyone at Cloud Four should feel very proud of their Responsive Field Day—it was wonderful.

As the day unfolded on Friday, I found myself being quite moved. It was genuinely touching to see my conference template replicated not only in format, but also in spirit. It was affordable (“Every expense spared!” was my motto), inclusive, diverse, and fast-paced. It was a lovely, lovely feeling to think that I had, in some small way, provided some inspiration for such a great event.

Jessica pointed out that isn’t the first time I’ve set up an event template for others to follow. When I organised the first Science Hack Day in London a few years ago, I never could have predicted how amazingly far Ariel would take the event. Fifty Science Hack Days in multiple countries—fifty! I am in awe of Ariel’s dedication. And every time I see pictures or video from a Science Hack Day in some far-flung location I’ve never been to, and I see the logo festooning the venue …I get such a warm fuzzy glow.

Y’know, when you’re making something—whether it’s an event, a website, a book, or anything else—it’s hard to imagine what kind of lifespan it might have. It’s probably just as well. I think it would be paralysing and overwhelming to even contemplate in advance. But in retrospect …it sure feels nice.

Habitasteroids

Science Hack Day San Francisco was held in the Github offices last weekend. It was brilliant!

Hacking begins Hacking Science hacker & grumpy cat enthusiast, Keri Bean Launch pad

This was the fifth Science Hack Day in San Francisco and the 40th worldwide. That’s truly incredible. I mean, I literally can’t believe it. When I organised the very first Science Hack Day back in 2010, I had no idea how far it would go. But Ariel has been indefatigable in making it a truly global event. She is amazing. And at this year’s San Francisco event, she outdid herself in putting together a fantastic cross-section of scientists, designers, and developers: paleontology, marine biology, geology, astronomy, particle physics, and many, many more disciplines were represented in the truly diverse attendees.

Saturday breakfast with the Science Hack Day community! The Science Hack Day girls! Stargazing on GitHub's roof Demos begin!

After an inspiring set of lightning talks on the first day, ideas started getting bounced around and the hacking began to take shape. I had a vague idea for—yet another—space-related hack. What clinched it was picking the brains of NASA’s Keri Bean. She’d help me get hold of the dataset I needed for my silly little hack.

So here’s the background…

There are many possibilities for human habitats in space: Stanford tori, O’Neill cylinders, Bernal spheres. Another idea, explored in science fiction, is hollowing out asteroids (Larry Niven’s bubbleworlds). Kim Stanley Robinson explores this idea in depth in his book 2312, where he describes the process of building an asteroid terrarium. The website of the book has a delightful walkthrough of the engineering processes involved. It’s not entirely implausible.

I wanted to make that idea approachable, so I thought about the kinds of people we might want to have living with us on the interior shell of a rotating hollowed-out asteroid. How about the people you follow on Twitter?

The only question that remains then is: which asteroid is the right one for you and your Twitter friends? Keri tracked down the motherlode of asteroid data and I started hacking the simplest of mashups—Twitter meets space rocks.

Here’s the result…

Habitasteroids!

Habitasteroids

Give it your Twitter username and it will tell you exactly which one of the asteroids in the main belt is right for you (I considered adding an enterprise option that would tell you where you could store your social network in the cloud …the Oort cloud, that is).

Be default, your asteroid will have the population density of Earth, which is quite generously. But if you want a more sparsely-populated habitat—say, the population density of Australia—or a more densely-populated world—with something like the population density of Japan—then you will be assigned a larger or smaller asteroid accordingly.

You’ll also be told by how much you should increase or decrease the rotation of the asteroid to get one gee of centrifugal force on the interior. Figuring out the equations for calculating centrifugal force almost broke me, but luckily I had help from a rocket scientist and a particle physicist …I’m not even kidding. And I should point out that the calculations take some liberties—I’m assuming a spherical body, which is quite a stretch, given the lumpy nature of most asteroids.

At 13:37 on the second day, the demos began. Keri and I were first up.

Jeremy wants to colonize an asteroid Habitasteroids

Give Habitasteroids a whirl for yourself. It’s a silly little thing, but I quite like how it turned out.

Speaking of silly things …at some point in the proceedings, Keri put the call out for asteroid data to her fellow space enthusiasts on Twitter. They responded with asteroid-related puns.

They have nice asteroids though: @brianwolven, @lukedones, @paix120, @LGalache, @motorbikematt, @brx0.

Oh, and while Habitasteroids might be a silly little hack, WRANGLER just might work.

WRANGLER: Capture and De-Spin of Asteroids and Space Debris

America

I’ve just come back from a multi-hop trip to the States, spanning three cities in just over two weeks.

It started with an all-too-brief trip to San Francisco for Science Hack Day, which—as I’ve already described—was excellent. It was a shame that it was such a flying visit and I didn’t get to see many people. But then again, I’ll be back in December for An Event Apart San Francisco.

It was An Event Apart that took me to my second destination: Austin, Texas. The conference was great, as always. But was really nice was having some time afterwards to explore the town. Being in Austin when it’s not South by Southwest is an enjoyable experience that I can heartily recommend.

Christopher and Ari took me out to Lockhart to experience Smitty’s barbecue—a place with a convoluted family drama and really, really excellent smoked meat. I never really “got” Texas BBQ until now. I always thought I liked the sauced-based variety, but now I understand: if the BBQ is good enough, you don’t need the sauce.

For the rest of my stay, Sam was an excellent host, showing me around her town until it was time for me to take off for New York city.

To start with, I was in Manhattan. I was going to be speaking at Future Of Web Design right downtown on 42nd street, and I showed up a few days early to rendezvous with Jessica and do some touristing.

We perfected the cheapskate’s guide to Manhattan, exploring the New York Public Library, having Tiff show us around the New York Times, and wrangling a tour of the MoMA from Ben Fino-Radin, who’s doing some fascinating work with the digital collection.

I gave my FOWD talk, which went fine once the technical glitches were sorted out (I went through three microphones in five minutes). The conference was in a cinema, which meant my slides were giganormous. That was nice, but the event had an odd kind of vibe. Maybe it was the venue, or maybe it was the two-track format …I really don’t like two-track conferences; I constantly feel like I’m missing out on something.

I skipped out on the second day of the conference to make my way over the bridge to Brooklyn in time for my third trip to Brooklyn Beta.

This year, they tried something quite different. For the first two days, there was a regular Brooklyn Beta: 300 lovely people gathered together at the Invisible Dog, ostensibly to listen to talks but in reality to hang out and chat. It was joyous.

Then on the third and final day, those 300 people decamped to Brooklyn’s Navy Yard to join a further 1000 people. There we heard more talks and had more chats.

Alas, the acoustics in the hangar-like space battled against the speakers. That’s why I made sure to grab a seat near the front for the afternoon talks. I found myself with a front-row seat for a series of startup stories and app tales. Then, without warning, the tech talks were replaced with stand-up comics. The comedians were very, very good (Reggie Watts!) …but I found it hard to pay attention because I realised I was in a living nightmare: somehow I was in the front-row seat of a stand-up comedy show. I spent the entire time thinking “Please don’t pick on me, please don’t pick on me, please don’t…” I couldn’t sneak out either, because that would’ve only drawn attention to myself.

But apart from confronting me with my worst fears, Brooklyn Beta was great …I’m just not sure it scales well from 300 to 1300.

And with that, my American sojourn came to an end. I’m glad that the stars aligned in such a way that I was able to hit up four events in my 16 day trip:

Radio Free Earth

Back at the first San Francisco Science Hack Day I wanted to do some kind of mashup involving the speed of light and the distance of stars:

I wanted to build a visualisation based on Matt’s brilliant light cone idea, but I found it far too daunting to try to find data in a usable format and come up with a way of drawing a customisable geocentric starmap of our corner of the galaxy. So I put that idea on the back burner…

At this year’s San Francisco Science Hack Day, I came back to that idea. I wanted some kind of mashup that demonstrated the connection between the time that light has travelled from distant stars, and the events that would have been happening on this planet at that moment. So, for example, a star would be labelled with “the battle of Hastings” or “the sack of Rome” or “Columbus’s voyage to America”. To do that, I’d need two datasets; the distance of stars, and the dates of historical events (leaving aside any Gregorian/Julian fuzziness).

For wont of a better hack, Chloe agreed to help me out. We set to work finding a good dataset of stellar objects. It turned out that a lot of the best datasets from NASA were either about our local solar neighbourhood, or else really distant galaxies and stars that are emitting prehistoric light.

The best dataset we could find was the Near Star Catalogue from Uranometria but the most distant star in that collection was only 70 or 80 light years away. That meant that we could only mash it up with historical events from the twentieth century. We figured we could maybe choose important scientific dates from the past 70 or 80 years, but to be honest, we really weren’t feeling it.

We had reached this impasse when it was time for the Science Hack Day planetarium show. It was terrific: we were treated to a panoramic tour of space, beginning with low Earth orbit and expanding all the way out to the cosmic microwave background radiation. At one point, the presenter outlined the reach of Earth’s radiosphere. That’s the distance that ionosphere-penetrating radio and television signals from Earth, travelling at the speed of light, have reached. “It extends about 70 light years out”, said the presenter.

This was perfect! That was exactly the dataset of stars that we had. It was a time for a pivot. Instead of the lofty goal of mapping historical events to the night sky, what if we tried to do something more trivial and fun? We could demonstrate how far classic television shows have travelled. Has Star Trek reached Altair? Is Sirius receiving I Love Lucy yet?

No, not TV shows …music! Now we were onto something. We would show how far the songs of planet Earth had travelled through space and which stars were currently receiving which hits.

Chloe remembered there being an API from Billboard, who have collected data on chart-topping songs since the 1940s. But that API appears to be gone, and the Echonest API doesn’t have chart dates. So instead, Chloe set to work screen-scraping Wikipedia for number one hits of the 40s, 50s, 60s, 70s …you get the picture. It was a lot of finding and replacing, but in the end we had a JSON file with every number one for the past 70 years.

Meanwhile, I was putting together the logic. Our list of stars had the distances in parsecs. So I needed to convert the date of a number one hit song into the number of parsecs that song had travelled, and then find the last star that it has passed.

We were tempted—for developer convenience—to just write all the logic in JavaScript, especially as our data was in JSON. But even though it was just a hack, I couldn’t bring myself to write something that relied on JavaScript to render the content. So I wrote some really crappy PHP instead.

By the end of the first day, the functionality was in place: you could enter a date, and find out what was number one on that date, and which star is just now receiving that song.

After the sleepover (more like a wakeover) in the aquarium, we started to style the interface. I say “we” …Chloe wrote the CSS while I made unhelpful remarks.

For the icing on the cake, Chloe used her previous experience with the Rdio API to add playback of short snippets of each song (when it’s available).

Here’s the (more or less) finished hack:

Radio Free Earth.

Basically, it’s a simple mashup of music and space …which is why I spent the whole time thinking “What would Matt do?”

Just keep hitting that button to hear a hit from planet Earth and see which lucky star is currently receiving the signal.*

Science!

*I know, I know: the inverse-square law means it’s practically impossible that the signal would be in any state to be received, but hey, it’s a hack.

Science Hack Day San Francisco

When I organised the first ever Science Hack Day in London in 2010, I made sure to write about how I organised the event. That’s because I wanted to encourage other people to organise their own Science Hack Days:

If I can do it, anyone can. And anyone should.

Later that year, Ariel organised a Science Hack Day in Palo Alto at the Institute For The Future. It was magnificent. Since then, Ariel has become a tireless champion and global instigator of Science Hack Day, spreading the idea, encouraging new events all over the world, and where possible, travelling to them. I just got the ball rolling—she has really run with it.

She organised another Science Hack Day in San Francisco for last weekend and I was lucky enough to attend—it coincided nicely with my travel plans to the States for An Event Apart in Austin. Once again, it was absolutely brilliant. There were tons of ingenious hacks, and the attendees were a wonderfully diverse bunch: some developers and designers, but also plenty of scientists and students, many (perhaps most) from out of town.

Hacking Hacking Hacking Lunch outdoors

But best of all was the venue: The California Academy of Sciences. It’s a fantastic museum, and after 5pm—when the public left—we had the place to ourselves. Penguins, crocodiles, a rainforest, an aquarium …it’s got it all. I didn’t get a chance to do all of the activities that were provided—I was too busy hacking or helping out—like stargazing on the roof, or getting a tour of the archives. But I did make it to the private planetarium show, which was wonderful.

Hacking Hacking

The Science Hackers spent the night, unrolling their sleeping bags in all the nooks and crannies of the aquarium and the African hall. It was like being a big kid. Mind you, the fun of sleeping over in such a great venue was somewhat tempered by the fact that trying to sleep in a sleeping bag on just a yoga mat on a hard floor is pretty uncomfortable. I was quite exhausted by day two of the event, but I powered through on the wave of infectious enthusiasm exhibited by all the attendees.

Sleeping over Sleeping over

Then when it came time to demo all the hacks …well, I was blown away. So much cool stuff.

Ariel and her team really outdid themselves. I’m so happy I was able to make it to the event. If you get the chance to attend a Science Hack Day, take it. And if there isn’t one happening near you, why not organise one? Ariel has put together a handy checklist to get you started so you can get excited and make things with science.

I’m still quite amazed that this was the 24th Science Hack Day! When I organised the first one three years ago, I had no idea that it could spread so far, but thanks to Ariel, it has become a truly special phenomenon.

Stargazing Planetarium

CERN and the line-mode browser

I remember when Mark took me aside to tell me about the project he was working on with CERN to restore the first ever website to its original URL. Needless to say, I was extremely pleased. After all, cool URIs don’t change.

Then, more recently, Mark told me about a follow-up project they were planning: to recreate the experience of using one of the first web browsers. He asked if I could help organise a hackday-style gathering to accomplish this. I jumped at the chance.

Together with Dan Noyes from the web team at CERN, we assembled a dozen people. Half of them were invited experts and half were chosen from applications. Because I was ostensibly involved in organising the event (although I didn’t really do much), I got a free pass.

And so we gathered in a war-room at CERN on the 18th and 19th of September to hack away at recreating the experience of using the line-mode browser in a modern browser—browserception!

Hacking

Just to be clear, the line-mode browser wasn’t the world’s first web browser. That honour goes to Tim Berners-Lee’s WorldWideWeb programme. But whereas WorldWideWeb only ran on NeXT machines, the line-mode browser worked cross-platform and was, therefore, instrumental in demonstrating the power of the web as a universally-accessible medium.

Just being at CERN was amazing. It’s like double Disneyland for me: not only is it home to the greatest experiment conducted in the history of our species, it’s also the birthplace of the web itself. We all felt quite humbled to be there.

That feeling was amplified when we had a very special guest show up at the start of the event—Robert Cailliau, co-conspirator with Tim Berners-Lee in getting the web off the ground. He gave us a history lesson in the early days of the web; the ideas, the people, and the principles that drove it. At one point, he passed around his notarised copy of the document that put the web into the public domain twenty years ago. “Be careful with that,” he half-joked. “CERN don’t know where their copy is anymore.”

Robert

We had another guest from the early days of the web with us for the duration of our hacking: an IBM RS/6000 machine from the early nineties running the line-mode browser.

Jeremy Brian

I drew on my experience from hack farm and Science Hack Day to get the ball rolling. I was acutely aware that some of us felt pretty unsure about what we could contribute, so I suggested splitting into two teams: one to work on building the line-mode browser simulator, and the other to work on telling the story of its history. That seemed to work out pretty well.

Geeking out Planning Prepping Geeking out

Remy and Brian L. did the really hard work, implementing a simulation of the line-mode browser in Node.js. Lea, Kimberly and John made sure the output looked and felt right. Meanwhile Brian S. had the crazy idea of trying to recreate the font from the IBM machine …by taking a photo of the screen and drawing the glyphs from the photo! Of course Mark jumped on that.

Craig headed up the story-telling side of things with Martin and Angela. My contribution consisted of writing some explanatory words and doing a bit of image optimisation. It would be easy to feel inadequate in the company of such talented developers, but as the hacking went on, it was clear that all those little contributions really add up.

We made a thing

It will probably move from its current URL—line-mode.cern.ch—to a permanent home. In the meantime, why not grab the code and install a copy locally?

  1. Install Node.js.
  2. Clone the github repo to your machine.
  3. Open up the Terminal, navigate to your cloned copy, and type: node .
  4. Open localhost:8000 in a browser.

You can read more about the project but I’m guessing what you’ll really want to do is fire up the line-mode browser. By default it loads a copy of the first ever web page, but you can also navigate other websites by changing this query string:

line-mode.cern.ch/www/proxy?url=http://adactio.com

Or, if you’re running it locally:

localhost:8000/www/proxy?url=http://adactio.com

You can also grab a bookmarklet from the resources page. Drag it to your bookmarks bar, pull up whatever page you want to view, and hit the bookmarklet to see it line-mode style.

dConstruct Huffduffer Adactio Clearleft

All the little details—the font, the animation, the sound—add up to an experience that I find quite immersive. In some ways, it’s a silly little project, but it’s also trying to convey an important message. I really love the final result. I feel incredibly honoured to have been involved—in a small way—in its creation.

Oh, and we also got to go down into the heart of the Large Hadron Collider to see the LHCb experiment. That was, in the truest sense of the word, awesome.

Dan LHCb Amongst the machines LHCb

CERN dev days

I went to CERN last year. It was amazing.

Don’t you wish that you could go to the birthplace of the web and the home of the most ambitious science project in the history of humanity? Well now you can!

On September 19th and 20th, a small group of developers will get together at CERN to hack on a project to recreate the first line-mode web browser. You can be part of that group. Fill out this form to apply. You’ll get a bursary to cover travel and accommodation. What are you waiting for?

In case you’re thinking “but what could I possibly contribute?” …welcome to my world. Through some clerical error, I’ve managed to get myself on the roster, but I have no idea how I’ll be able to help. Perhaps I can provide some experience from Hack Farm, which was a similar kind of gathering. Although Hack Farm never had a Giant. Hadron. Collider!

Do you know CSS? JavaScript? Node? Anything web-related? Get your application in before Monday, July 15th.

See you in Geneva.

Space by Botwest

I had a whole day of good talks yesterday at South By Southwest yesterday …and none of them were in the Austin Convention Center. In a very real sense, the good stuff at this event is getting pushed to the periphery.

The day started off in the Driskill Hotel with the New Aesthetic panel that James assembled. It was great, like a mini-conference packed into one hour with wonderfully dense knowledge bombs lobbed from all concerned. Joanne McNeil gave us the literary background, Ben searched for meaning (and humour) in advertising trends, Russell looked at how machines are changing what we read and write, and Aaron …um, talked about the helium-balloon predator drone in the corner of the room.

With our brains primed for the intersections where humans and machines meet, it wasn’t hard to keep pattern-matching for it. In fact, the panel right afterwards on technology and fashion was filled with wonderful wearable expressions of the New Aesthetic.

Alas, I wasn’t able to attend that panel because I had to get to the green room to prepare for my own appearance on Get Excited and Make Things With Science with Ariel and Matt. It was a lot of fun and it was a real pleasure to be on a panel with such smart people.

I basically used the panel as an opportunity to geek out about some of my favourite science-related hacks and websites:

After that I stayed in the Driskill for a panel on robots and AI. One of the panelists was Bina48.

I heard had heard about Bina48 from a Radiolab episode.

Radiolab - Talking to Machines on Huffduffer

Jon Ronson described the strange experience of interviewing her—how the questions always tended to the profound and meaningful rather than trivial and chatty. Sure enough, once Bina was (literally) unveiled on the panel—a move that was wisely left till halfway through because, as the panelists said, “after that, you’re not going to pay attention to a word we say”—people started asking questions like “Do you dream?” and “What is the meaning of life?”

I asked her “Where were you before you were here?” She calmly answered that she was made in Texas. The New Aesthetic panelists would’ve loved her.

I was surprised by how much discussion of digital preservation there was on the robots/AI panel. Then again, the panel was hosted by a researcher from The Digital Beyond.

Bina48’s personality is based on the mind file of a real person containing exactly the kind of data that we are publishing every day to third-party sites. The question of what happens to that data was the subject of the final panel I attended, Saying Goodbye to Your Digital Self, featuring representatives from The Internet Archive, Archive Team, and Google’s Data Liberation Front.

Digital preservation is an incredibly important topic—one close to my heart—but the panel (in the Omni hotel) was, alas, sparsely attended.

Like I said, at this year’s South by Southwest, a lot of the good stuff was at the edges.

Speaking, not hacking

I spent last week in Belfast for the Build conference, so I did.

The fun kicked off with a workshop on responsive enhancement which was a lot of fun. Toby has written a report of the day outlining all of the elements that came together for a successful workshop.

The day of the conference itself was filled with inspiring, uplifting talks full of positive energy …except for mine. My talk—All Our Yesterdays—had an underlying sense of anger, especially when I spoke about the destruction of Geocities. If you heard the talk and you’d like to explore some of the resources I mentioned, here’s a grab-bag of links:

I thought I had delivered the talk reasonably well only to discover that my American friends in the audience misinterpreted my quote from Tim Berners-Lee as “Cool your eyes don’t change.”

Still, it was wonderfully surreal to be introduced by Jesse Thorn.

Build Jeremy Keith

My appearance at Build was an eleventh hour affair. Ethan was originally set to speak but he had to cancel. Andy asked me to step in. At first I didn’t think it would be possible. Last Thursday—the day of the conference—was the day I was supposed to fly to San Francisco for Science Hack Day. Luckily I was able to change my flight.

That’s why I was up at the crack of dawn the day after Build to catch an early-morning flight to Heathrow where I would have to dash from the lowest to the highest numbered terminal to get on my transatlantic hackrocket.

So you can imagine how my heart sank as I sat in the departure lounge of Belfast International Airport listening to the announcement of a delay to the first flight. First it was one hour. Then two.

When I did finally make it to Heathrow, there was no chance of making the flight to San Francisco. I was hoping that perhaps it too had been delayed by the foggy weather conditions but no, it took off right on time. Without me.

As my flight from Belfast was a completely separate booking rather than a connecting flight, I couldn’t get on a later flight unless I paid the full fare. So I simply accepted my fate.

C’est la vie, c’est it is.

It looks like Science Hack Day San Francisco—to the surprise of absolutely no-one—was a superb event. There’s a write-up on the open.NASA blog outlining some of the amazing hacks, including the cute (and responsive) Space Ipsum and the freakishly brilliant synesthesia mask: syneseizure.

Science Hack Day SF science hack day

Building

I never made it to the Build conference in Belfast last year or the year before. I think it clashed with previous commitments every time.

This was going to be the third year in a row that I was going to miss Build. I had already slapped my money down for the excellent Full Frontal conference which is on the very same day as Build but takes place right here in Brighton in the excellent Duke Of York’s cinema.

But fate had other plans for me.

Ethan was going to be speaking at Build but he’s had to pull out for personal reasons …so Andy asked me if I’d like to speak. I may be a poor substitute for Ethan and it’s a shame that I’m going to miss Full Frontal but I jumped at the chance to join the stellar line-up.

As well as speaking at the conference itself on November 10th, I’ll be leading a workshop on responsive design and progressive enhancement on the preceding Tuesday. The conference is sold out but there are places available for the workshop so grab yourself a slot if you fancy spending a day working on a content-first approach to planning and building websites.

If you can’t make it to Belfast, I’ll be giving the same workshop at Beyond Tellerrand in Düsseldorf on Sunday, November 20th and there are still some tickets available.

If you can make it to Belfast, I look forward to seeing you there. I’ll be flying my future friendly flag high, just like I’m doing on the front page of the Build website.

That attire would also be suitable for my post-Build plans. The day after the conference I’ll be travelling to San Francisco for Science Hack Day on the weekend of November 12th. If the last one is anything to go by, it’s going to be an unmissable excellent weekend—I highly recommend that you put your name down if you’re going to be in the neighbourhood.

Looking forward to seeing you in Belfast or Düsseldorf or San Francisco …or wherever.

Days of science and hacking

I’ve got some tasty science announcements. Of course anything I have to say is going to be upstaged by today’s revelation about Mars. Those NASA showboaters.

Anyway, here’s Ariel’s keynote from OSCON in Portland last week.

Science Hack Day took centre-stage. Ariel has put together instructions for creating a Science Hack Day in your city. The front page of the wiki now contains a list of places where people are hoping to coordinate an event.

If you want to help make a Science Hack Day happen in Portland, get in touch with rocketeer Nathan Bergey—the science hacker behind the brilliant ISS Notify lamp.

Live in Cape Town? Want a Science Hack Day? Get in touch with ex-pat Dutch astronomer Carolina Ödman.

If you want to help make a Science Hack Day happen in San Diego, get in touch with biologist Jun Yin—the science hacker behind wearable DNA.

It looks like there’s quite a lot of interest in organising a Science Hack Day in Dublin to coincide with the Dublin City Of Science event in 2012. Get in touch with Ellen Byrne if you want to be involved.

Those are just a few. There are many more.

One event that’s definitely going ahead is another Science Hack Day San Francisco on November 12th and 13th. If last year’s event at the Institute For The Future is anything to go by, this will be unmissable.

And you can go.

No, really. In a Steve Jobsian style “just one more thing” addendum to her OSCON keynote, Ariel announced that the Alfred P. Sloan Foundation is providing a grant to fund the Science Hack Day Ambassador Program:

Thanks to this generous support, 10 people interested in organizing a Science Hack Day from around the world will be selected to win a scholarship for a trip to Science Hack Day San Francisco, occurring November 12-13, 2011, where they’ll experience first-hand how Science Hack Day works and connect with a global community of organizers. This Science Hack Day Ambassador Program will award individuals who are motivated and planning to organize a Science Hack Day in their city.

To apply, start organising a Science Hack Day in your city and then fill in the application form. The scholarship will cover airfare, four nights in a hotel and $300 for expenses. Ten lucky ambassadors will be winging their way to the bay area this November.

It’s time to get excited and make things with science in your corner of this pale blue dot.

Hacking History

I spent the weekend at The Guardian offices in London at History Hack Day. It was rather excellent. You’d think I’d get used to the wonderful nature of these kinds of events, but I once again I experienced the same level of amazement that I experienced the first time I went to hack day.

The weekend kicked off in the traditional way with some quickfire talks. Some lovely people from The British Museum, The British Library and The National Archives talked about their datasets, evangelists from Yahoo and Google talked about YQL and Fusion Tables, and Max Gadney and Matthew Sheret got us thinking in the right directions.

Matthew Sheret was particularly inspiring, equating hackers with time travellers, and encouraging us to find and explore the stories within the data of history. The assembled geeks certainly took that message to heart.

Ben Griffiths told the story of his great-uncle, who died returning from a bomber raid on Bremen in 1941. Using data to put the death in context, Ben approached the story of the lost bomber with sensitivity.

Simon created geStation, a timeline of when railway stations opened in the UK. On the face of it, it sounds like just another mashup of datetimes and lat-long coordinates. But when you run it, you can see the story of the industrial revolution emerge on the map.

Similarly, Gareth Lloyd and Tom Martin used Wikipedia data to show the emerging shape of the world over time in their video A History of the World in 100 Seconds, a reference to the BBC’s History of the World in 100 Objects for which Cristiano built a thoroughly excellent mobile app to help you explore the collection at British Museum.

Brian used the Tropo API to make a telephone service that will find a passenger on the Titanic who was the same age and sex as you, and then tell you if they made it onto a lifeboat or not. Hearing this over the phone makes the story more personal somehow. Call +1 (804) 316-9215 in the US, +44 2035 142721 in the UK, or +990009369991481398 on Skype to try it for yourself.

Audioboo / did you die on the Titanic? on Huffduffer

I was so impressed with the Tropo API that I spent most of History Hack Day working on a little something for Huffduffer …more on that later.

My contribution to the hack day was very modest, but it was one of the few to involve something non-digital. It’s called London On A Stick.

A pile of USB sticks had been donated to History Hack Day, but nobody was making much use of them so I thought they could be used as fodder for Dead Drops. I took five USB sticks and placed a picture from The National Archives on Flickr Commons on each one. Each picture was taken somewhere in London and has been geotagged.

Zeppelin over St. Paul's

I slapped sticky notes on the USB sticks with the location of the picture. Then I asked for volunteers to go out and place the sticks at the locations of the pictures: Paddington, Trafalgar Square, Upper Lambeth, St. Paul’s and Tower Bridge. Not being a Londoner myself, I’m relying on the natives to take up the challenge. You can find the locations at icanhaz.com/londononastick. I ducked out of History Hack Day a bit early to get back to Brighton so I have no idea if the five sticks were claimed.

Although my contribution to History Hack Day was very modest, I had a really good time. Matt did a great job putting on an excellent event.

It was an eye-opening weekend. This hack day put the “story” back into history.

Spacelift

My sojourn up the western seaboard of the United States has come to an end. It began in San Diego with the final An Event Apart of the year, which was superb as always. From there, I travelled up to San Francisco for Cindy and Matt’s wedding celebration, followed by a few days in Seattle. The whole trip was rounded out back in California at the wonderfully titled Institute For The Future in Palo Alto. For that was the location of Science Hack Day San Francisco.

It was an amazing event. Ariel did a fantastic job—she put so much effort into making sure that everything was just right. I suspected it was going to be a lot of fun, but once again, I was blown away by the levels of ingenuity, enthusiasm and sheer brilliance on display.

In just 24 hours, the ingenious science hackers had created particle wind chime which converts particle collisions into music that Brian Eno would be proud of, grassroots aerial mapping with balloons which produced astonishing results (including an iPad app), as well as robots and LEDs a-plenty. The list of hacks is on the wiki.

Science Hack Day San Francisco Science Hack Day San Francisco Science Hack Day San Francisco Science Hack Day San Francisco

My own hack was modest in scope. Initially, I wanted to build a visualisation based on Matt’s brilliant idea, but I found it far too daunting to try to find data in a usable format and come up with a way of drawing a customisable geocentric starmap of our corner of the galaxy. So I put that idea on the back burner and decided to build something around my favourite piece of not-yet-existing technology: the .

The idea

Spacelift compares the cost efficiency of getting payloads into geosynchronous orbit using a space elevator compared to traditional rocketry. Basically, it’s a table. But I’ve tried to make it a pretty table with a bit of data visualisation to show at a glance how much more efficient a space elevator would be.

The payloads I’ve chosen are spacecraft. Beginning with the modest Voyager 1, it gets more and more ambitious with craft like an X-Wing or the Millennium Falcon before getting crazy with the USS Enterprise and the Battlestar Galactica.

So, for example, while you could get a TIE-fighter into the Clarke belt using a single , two , or three , it would cost considerably more than using a space elevator, where you’re basically just paying for the electricity.

If you click on the dollar amount for each transport mechanism, you’ll see the price calculated as a tower of pennies. Using a , for example, will cost you a tower of pennies 22 times larger than a space elevator, assuming a space elevator is at least 38,000 kilometres tall/long. Using a space elevator, on the other hand, requires spending a tower of pennies about the same height as itself. I don’t even bother trying to visualise the relative height differences for getting anything bigger than the Tantive IV into orbit as it would require close to infinite scrolling.

I’m fairly confident of the cost-per-kilogram values for the rockets while the is necessarily fuzzy, given that the mechanism doesn’t exist yet. But by far the trickiest info to track down was the mass of fictional spacecraft. There’s plenty of information on dimensions, speed and armaments, but very little on weight. Mathias saved the day with some diligent research that uncovered the motherlode.

Having such smart, helpful people around made the whole exercise a joy. It was quite a pleasure to walk over to a group of hackers, ask Is anyone here a rocket scientist? and have at least one person raise their hand. The constant presence of Cosmos playing on a loop just added to the atmosphere of exploration and fun.

Implementation

I’ve put the code on GitHub, ‘cause that’s what a real hacker would do. It’s my first GitHub repository. Be gentle with me.

There’s CSS3 and HTML5 a-plenty. I deliberately don’t use the IE shim to enable styling of HTML5 structural elements in lesser versions of Internet Explorer; there wouldn’t be much point delivering RGBa, opacity and text-shadow styles to a browser that can’t handle ‘em.

The background colour changes depending on the payload. I’m using a variation of the MD5 colour encoding popularised by Dopplr and documented by Brian in his excellent new book, Designing With Data.

I’m using League Gothic by The League Of Movable Type for the type—ya gotta have a condensed font for data visualisations, right?

There’s also a google-o-meter image from the Google chart API.

Needless to say, the layout is responsive and adapts to different viewing environments …including print.

Spacelift, printed

Using CSS transforms, each page of table of price comparisons becomes a handy page to print out and stick on the office wall to remind yourself why the human race needs a space elevator.

Making Science Hack Day happen

When I was at South by Southwest back in March, I got very inspired by the Open Science panel moderated by Tantek. You can listen to that panel for yourself:

Open Science: Create, Collaborate, Communicate on Huffduffer

Round about the 43 minute mark—when Natalie Villalobos is talking about all the fantastic APIs and datasets that are out there—is the moment when I leaned over and whispered to Riccardo:

We should have a Science Hack Day.

Three months later, on the 19th and 20th of June, Science Hack Day happened. I still can’t quite believe it.

The event was a great success, and more importantly, it was a lot of fun. But you can read all about that on the website, on Twitter and elsewhere. What I want to talk about is how Science Hack Day came together.

When I first blogged about the day, right after the Open Science panel at South by Southwest, I wrote:

I have no idea how to get a venue or sponsors.

The Venue

The first thing I did was to set up a wiki to get people brainstorming. Many of the proposed venues were places that had previously hosted BarCamps or other geek gatherings requiring plentiful bandwidth and/or sleepover capability. The Guardian offices came up quite early on. I hadn’t been there myself but I knew that it had been the venue for BarCamp London 6.

I got in touch with Matt McAlister at The Guardian who, in a previous life, helped organise the first ever UK Hack Day. He gave the green light and in no time at all, I was talking to Alex Hazell, Events Executive at The Guardian. Her experience in organising previous events was invaluable in figuring out how to approach potential sponsors.

The Sponsors

Here’s what you need for a hack day:

  • physical space for two days,
  • bandwidth for a large number of developers,
  • food and drink,
  • prizes.

The first two were taken care of. The second two require money. The prizes are less of a priority but food and drink is essential hack fuel. That’s where the sponsors come in.

Lunches and breakfasts could be provided by The Guardian catering staff but that requires paying for staffing costs as well as food. All in all, it’s about £2.5K for the number of people expected at Science Hack Day. That still left Saturday evening food. Pizza is the traditional dish of choice. £500 ought to cover it. Throw in another few hundred quid for prizes and that comes to about £3000.

Rather than approaching companies to sponsor the whole amount, Alex recommended that I try asking for around £500 from each company I got in touch with. That’s a good amount of money but it’s small enough that people might be able to approve it without having to go through lots of bureaucratic procedures within their companies.

That turned out to be a good policy and before long, quite a few companies had agreed to sponsor the event. Fortunately for me, The Guardian’s events team were able to bill them individually which meant I didn’t have to deal with that side of things. The Wellcome Trust, Nature, Thoughtworks, LCGOT and Elsevier were all invoiced separately for their contributions to the fund for canteen food and prizes.

The pizza money came from YDN. As Christian was going to be there anyway, it was convenient for him to pay for the pizza with his company card rather than chasing the paper trail of Yahoo’s invoicing system.

In retrospect, I probably could have got the lunches and breakfasts more cheaply by just getting sandwiches and pastries delivered but then I would have either needed sponsors that were capable of paying on the spot or I would have needed to pay for it myself and then reclaim it by invoicing sponsors.

Unrelated to Science Hack Day, my pal Murray from Global Radio got in touch to ask if I would come in to chat to his team about HTML5. I said I’d love to …if Global Radio fancied sponsoring Science Hack Day. I’m devious like that.

Murray got company approval and I left him to organise the booze for the event—£500 worth of booze. I thought that would be just about right. But I was thinking back to previous BarCamps, not hack days. It turns out that while beer is definitely required for lubricating late night games of Werewolf, it’s actually not all that important to hacking.

It seems obvious in hindsight, but what hackers need are stimulants, not depressants. Fortunately, there was plenty of tea and coffee on hand. By the end of the event, there was also a massive surplus of beer.

Ah well, you live and learn.

The final sponsor was BERG. Their contribution wasn’t as large as the others but it had the advantage of being given in cash. That meant I had a petty cash fund throughout the day for repeat trips to the nearest supermarket for crisps, chocolate, and fruit. The latter food group was important because it was one that I had overlooked.

So two valuable lessons learned were:

  • Less beer, more juice and caffeine.
  • Less junk food, more fruit.

But those are relatively minor points. On the whole, everything went incredibly well.

The Numbers

The total cost was around £4,000 but I reckon it would be possible to organise a two-day event like this for half that much by removing the staffing costs (and the drink costs) if you’re okay with sandwiches for lunch and you don’t mind not having many shiny prizes—we had Lego Mindstorms and telescopes for the winners.

About 120 people signed up on the wiki. Another 20 or so listed themselves as “maybe.” I figured there would be the usual drop-out rate so I was expecting 80 or 90 people to make it on the day. That turned out to be a fairly accurate assessment. By Sunday, the numbers were down to around 75 because of non-returners: people who went home, rather than sleeping over, and then didn’t come back in the next day. A lot of those people couldn’t make it back because of family commitments—Sunday was Father’s Day.

In the run-up to Science Hack Day, I wasn’t sure whether or not to create an EventBrite listing to manage attendance. But the wiki seemed to be working pretty well so I just stuck with that. Instead of emailing people, I used the Science Hack Day website and Twitter account to keep everyone up to date and to get some pre-event excitement going.

The Event

I had a lot of help and support. Jessica helped out for the duration of Science Hack Day—I couldn’t have done it without her. In the weeks and months before the event, Matt Locke made some valuable introductions Natalie Villalobos went above and beyond the call of duty in putting me in touch with potential sponsors. Through her connections, I had the unbelievable honour of playing a specially recorded message at Science Hack Day from SETI’s Jill Tarter—the real-life inspiration for Ellie Arroway in Carl Sagan’s Contact:

Jill Tarter’s message to Science Hack Day on Vimeo

I’m still pinching myself about that. In fact, I’m still pinching myself about the whole event. How did I end up organising something like this?

I’m so glad that, straight after being inspired by the Open Science panel at South by Southwest, I blogged my crazy idea to create a Science Hack Day. By planting a flag in the sand like that, it stopped being just another bit of wishful thinking and starting to become real.

If I can do it, anyone can. And anyone should. There’s already a wiki page for other cities where Science Hack Days might be organised. You can help put one of those events together.

Get excited and make things …with science!

Science-Hackers

Londoning

The @media conference has been a constant star in the UK web standards community since 2005. This year, the baton was smoothly passed to those awesome aussies, John and Maxine of Web Directions, creating the hybridly-titled Web Directions @media that took place in London last week.

Before the conference proper, there were two days of workshops in the Hogwartsian location of the superbly-named Goodenough College. I spent a day on Getting semantic with microformats and HTML5. I think it went pretty well. I came to the conclusion that it’s easier to explain everything about microformats than to explain the new outline algorithm in HTML5. Luckily everyone had pocket books on hand. It took quite a while to fold them all but I think they helped.

The conference itself was of the usual high standards, although there was only so much I could see, given its dual-track nature. Everything was recorded though so the podcast will help take the sting out.

For my part, I moderated the now-traditional Hot Topics panel. It wasn’t quite as controversial as recent years but it was a thoroughly enjoyable discussion. My heartfelt thanks to the panelists, John, Hannah, Simon and Christian.

Now I’m back in Brighton but I’ll be heading back up to London on the weekend for Science Hack Day. It’s all set to be an excellent event. All of the requisite pieces are in place: bandwidth, food, drink, prizes, and plenty of smart people in a superb venue. If you haven’t added your name to the list of who’s coming, do it now.

Keep an eye on the website for more details in the run-up to the weekend—subscribe to the RSS feed. You can also follow @sciencehackday on Twitter but then you won’t get all the juicy details about playing with pollution data, documenting fictional APIs and developing synthetic biology.

T minus five days. I’ll see you on the launch pad.

A site for Science Hack Day

I spent the weekend immersing myself in HTML5 and CSS3.

I gave Principia Gastronomica a bit of a fine-tuning under the hood. I decided to ditch all the background images I was using to get rounded corners and drop shadows, and just use border-radius and box-shadow instead. Internet Explorer gets the same content with more pointy corners and without the illusion of depth.

I also launched a brand new site: ScienceHackDay.org. It’s a small little site designed to quickly dispense relevant information on the upcoming geektastic Science Hack Day in London on June 19th and 20th.

The front page is also a blog. I’ll be posting updates about the event there and I’ll also be highlighting science projects and hack ideas. Subscribe to the feed if you want get updates.

I couldn’t be bothered installing or writing blogging software for the site. It seemed like overkill for such a quick’n’nimble project so I’m just blogging in HTML using . It’s converted to Atom using Luke Arno’s handy service. I was going to use the transcoder on Microformatic.com but I think it uses which means it won’t work on the new HTML5 elements.

The underlying structure is a combination of microformats, HTML5 structural elements and ARIA roles. The styling is over-the-top CSS3. There’s border-radius, opacity and RGBa a-plenty. Setting colours using opacity and alpha transparency makes it a breeze to customise the colours on each page, so the schedule, location and not-so-FAQ pages all have a slightly different tint.

Internet Explorer doesn’t get any of this colourful or roundy goodness. It does however, get the typography and layout, much like the print stylesheet view. I wouldn’t do that for client work but I’m perfectly happy to do it for a weekend project.

Check out the site and if you decide that Science Hack Day looks like your kind of event, add your name on the wiki.

You can also follow @sciencehackday on Twitter to get bite-sized updates as they happen.

June

by Camper Van Beethoven—one of my all-time favourite albums—features the song June wherein David Lowery asks:

Are you weary of the lengthening days?
Do you secretly wish for November’s rain?

Not in the least, David, not in the least. In fact, I’m really looking forward to June, both for its lengthening days and its avalanche of geeky goodness. To whit:

Web Directions @media comes to London from June 8th to 11th. It is pretty much guaranteed to be awesome.

I’ll be fanning the flames once again for the Hot Topics panel. I’ll also be running a workshop on getting Semantic with microformats and HTML5. I love the fact that there are two workshops on HTML5 and yet there’s absolutely no overlap between the two—I’ll leave you to decide whether that’s a testament to the breadth of the HTML5 spec or an indication of just how much is encompassed by the word ‘HTML5’.

The price for the conference goes up on May 15th so you’d better get in there and grab a ticket now if you haven’t already. And just between you and me, if you use the promo code KEITH then you can get a whole hundred squid off the asking price.

If you’re already in London for @media Web Directions, consider sticking around that weekend for the BarCamp. Nothing has been announced other than the dates but .

A week later, on June 19th, is when the geekery really hits the fan: Science Hack Day at The Guardian offices in London! If you haven’t already done so, add your name to the list of potential attendees. Trust me: you won’t want to miss this.

Needless to say, I’ll be updating both here and on the wiki as the event comes together. And it is coming together very nicely: we have a great venue, we have plenty of bandwidth, and we have lots of interested geeks, hackers and programmers. The only thing I need to make sure I can get covered is the hackfuel: food and drink.

The total cost for food and drink will probably be somewhere between £2,000 and £3,000 but I’m hoping to spread that cost amongst a bunch of sponsors. I think £500 should be a nice sweet spot for sponsorship.

If you work for someone—or know of someone—that wants to support a fine event such as Science Hack Day and would consider £500 a small price to pay for the resulting , please get in touch and let me know.

Science hack space

While I was up in London, I arranged to pop by The Guardian offices at 90 York Way, the venue for Science Hack Day on June 19th and 20th.

Now I’m really excited about the event. The space is absolutely fantastic! There’s going to be loads of room and plenty of bandwidth, all in a very nifty building.

Location: sorted. Connectivity: sorted. Dates: sorted. Now the big issue is sorting out sponsorship: it takes money to feed all the geeks with hackfuel. If you have any ideas about potential sponsors, let me know.

And, if you haven’t already, add you name on the wiki.