A blog by Stuart Langridge

as days pass by

scratched tallies on the prison wall

What to do about hotlinking

Hotlinking, in the context I want to discuss here, is the act of using a resource on your website by linking to it on someone else's website. This might be any resource: a script, an image, anything that is referenced by URL.

It's a bit of an anti-social practice, to be honest. Essentially, you're offloading the responsibility for the bandwidth of serving that resource to someone else, but it's your site and your users who get the benefit of that. That's not all that nice.

Now, if the "other person's website" is a CDN -- that is, a site deliberately set up in order to serve resources to someone else -- then that's different. There are many CDNs, and using resources served from them is not a bad thing. That's not what I'm talking about. But if you're including something direct from someone else's not-a-CDN site, then... what, if anything, should the owner of that site do about it?

I've got a fairly popular, small, piece of JavaScript: sorttable.js, which makes an HTML table be sortable by clicking on the headers. It's existed for a long time now (the very first version was written twenty years ago!) and I get an email about it once a week or so from people looking to customise how it works or ask questions about how to do a thing they want. It's open source, and I encourage people to use it; it's deliberately designed to be simple1, because the target audience is really people who aren't hugely experienced with web development and who can add sortability to their HTML tables with a couple of lines of code.

The instructions for sorttable are pretty clear: download the library, then put it in your web space and include it. However, some sites skip that first step, and instead just link directly to the copy on my website with a <script> element. Having looked at my bandwidth usage recently, this happens quite a lot2, and on some quite high-profile sites. I'm not going to name and shame anyone3, but I'd quite like to encourage people to not do that, if there's a way to do it. So I've been thinking about ways that I might discourage hotlinking the script directly, while doing so in a reasonable and humane fashion. I'm also interested in suggestions: hit me up on Mastodon at @sil@mastodon.social or Twitter4 as @sil.

Move the script to a different URL

This is the obvious thing to do: I move the script and update my page to link to the new location, so anyone coming to my page to get the script will be wholly unaffected and unaware I did it. I do not want to do this, for two big reasons: it's kicking the can down the road, and it's unfriendly.

It's can-kicking because it doesn't actually solve the problem: if I do nothing else to discourage the practice of hotlinking, then a few years from now I'll have people hotlinking to the new location and I'll have to do it again. OK, that's not exactly a lot of work, but it's still not a great answer.

But more importantly, it's unfriendly. If I do that, I'll be deliberately breaking everyone who's hotlinking the script. You might think that they deserve it, but it's not actually them who feel the effect; it's their users. And their users didn't do it. One of the big motives behind the web's general underlying principle of "don't break the web" is that it's not reasonable to punish a site's users for the bad actions of the site's creators. This applies to browsers, to libraries, to websites, the whole lot. I would like to find a less harsh method than this.

Move the script to a different dynamic URL

That is: do the above, but link to a URL which changes automatically every month or every minute or something. The reason that I don't want to do this (apart from the unfriendly one from above, which still applies even though this fixes the can-kicking) is that this requires server collusion; I'd need to make my main page be dynamic in some way, so that links to the script also update along with the script name change. This involves faffery with cron jobs, or turning the existing static HTML page into a server-generated page, both of which are annoying. I know how to do this, but it feels like an inelegant solution; this isn't really a technical problem, it's a social one, where developers are doing an anti-social thing. Attempting to solve social problems with technical measures is pretty much always a bad idea, and so it is in this case.

Contact the highest-profile site developers about it

I'm leaning in this direction. I'm OK with smaller sites hotlinking (well, I'm not really, but I'm prepared to handwave it; I made the script and made it easy to use exactly to help people, and if a small part of that general donation to the universe includes me providing bandwidth for it, then I can live with that). The issue here is that it's not always easy to tell who those heavy-bandwidth-consuming sites are. It relies on the referrer being provided, which it isn't always. It's also a bit more work on my part, because I would want to send an email saying "hey, Site X developers, you're hotlinking my script as you can see on page sitex.example.com/sometable.html and it would be nice if you didn't do that", but I have no good way of identifying those pages; the document referrer isn't always that specific. If I send an email saying "you're hotlinking my script somewhere, who knows where, please don't do that" then the site developers are quite likely to put this request at the very bottom of their list, and I don't blame them.

Move the script and maliciously break the old one

This is: I move the script somewhere else and update my links, and then I change the previous URL to be the same script but it does something like barf a complaint into the console log, or (in extreme cases based on suggestions I've had) pops up an alert box or does something equally obnoxious. Obviously, I don't wanna do this.

Legal-ish things

That is: contact the highest profile users, but instead of being conciliatory, be threatening. "You're hotlinking this, stop doing it, or pay the Hotlink Licence Fee which is one cent per user per day" or similar. I think the people who suggest this sort of thing (and the previous malicious approach) must have had another website do something terrible to them in a previous life or something and now are out for revenge. I liked John Wick as much as the next poorly-socialised revenge-fantasy tech nerd, but he's not a good model for collaborative software development, y'know?

Put the page (or whole site) behind a CDN

I could put the site behind Cloudflare (or perhaps a better, less troubling CDN) and then not worry about it; it's not my bandwidth then, it's theirs, and they're fine with it. This used to be the case, but recently I moved web hosts5 and stepped away from Cloudflare in so doing. While this would work... it feels like giving up, a bit. I'm not actually solving the problem, I'm just giving it to someone else who is OK with it.

Live with it

This isn't overrunning my bandwidth allocation or anything. I'm not actually affected by this. My complaint isn't important; it's more a sort of distaste for the process. I'd like to make this better, rather than ignoring it, even if ignoring it doesn't mean much, as long as I'm not put to more inconvenience by fixing it. We want things to be better, after all, not simply tolerable.

So... what do you think, gentle reader? What would you do about it? Answers on a postcard.

  1. and will stay simple; I'd rather sorttable were simple and relatively bulletproof than comprehensive and complicated. This also explains why it's not written in very "modern" JS style; the best assurance I have that it works in old browsers that are hard to test in now is that it DID work in them and I haven't changed it much
  2. in the last two weeks I've had about 200,000 hits on sorttable.js from sites that hotlink it, which ain't nothin'
  3. yet, at least, so don't ask
  4. if you must
  5. to the excellent Mythic Beasts, who are way better than the previous hosts
I'm currently available for hire, to help you plan, architect, and build new systems, and for technical writing and articles. You can take a look at some projects I've worked on and some of my writing. If you'd like to talk about your upcoming project, do get in touch.

More in the discussion (powered by webmentions)