Journal tags: syndication

13

Federation syndication

I’m quite sure this is of no interest to anyone but me, but I finally managed to fix a longstanding weird issue with my website.

I realise that me telling you about a bug specific to my website is like me telling you about a dream I had last night—fascinating for me; incredibly dull for you.

For some reason, my site was being brought to its knees anytime I syndicated a note to Mastodon. I rolled up my sleeves to try to figure out what the problem could be. I was fairly certain the problem was with my code—I’m not much of a back-end programmer.

My tech stack is classic LAMP: Linux, Apache, MySQL and PHP. When I post a note, it gets saved to my database. Then I make a curl request to the Mastodon API to syndicate the post over there. That’s when my CPU starts climbing and my server gets all “bad gateway!” on me.

After spending far too long pulling apart my PHP and curl code, I had to come to the conclusion that I was doing nothing wrong there.

I started watching which processes were making the server fall over. It was MySQL. That seemed odd, because I’m not doing anything too crazy with my database reads.

Then I realised that the problem wasn’t any particular query. The problem was volume. But it only happened when I posted a note to Mastodon.

That’s when I had a lightbulb moment about how the fediverse works.

When I post a note to Mastodon, it includes a link back to the original note to my site. At this point Mastodon does its federation magic and starts spreading the post to all the instances subscribed to my account. And every single one of them follows the link back to the note on my site …all at the same time.

This isn’t a problem when I syndicate my blog posts, because I’ve got a caching mechanism in place for those. I didn’t think I’d need any caching for little ol’ notes. I was wrong.

A simple solution would be not to include the link back to the original note. But I like the reminder that what you see on Mastodon is just a copy. So now I’ve got the same caching mechanism for my notes as I do for my journal (and I did my links while I was at it). Everything is hunky-dory. I can syndicate to Mastodon with impunity.

See? I told you it would only be of interest to me. Although I guess there’s a lesson here. Something something caching.

adactio.com on Mastodon

I’ve been on Mastodon since 2017, but now my website is on there too. I’m @adactio@mastodon.social. My website is @adactio.com@adactio.com—search for adactio.com in your Mastodon client of choice.

What’s the difference? Well, with my mastodon.social account, I’m syndicating stuff—most of my notes, and all of my journal—and including a link back to the original source here on my site. With the adactio.com account, it is the original source.

I thought about migrating over to my adactio.com account from my mastodon.social account, but I actually like having a separate profile. I browse Mastodon a lot more than I post, and browsing is a lot easier to do with a regular account.

If you’d like your website to be available on Mastodon, Bridgy Fed is the magic tool that makes it available. You’ll need to be able to send webmentions, and you’ll need to configure some .well-known directives. If you’ve you’ve already got webmention-sending set up, it’s all quite straightforward (though you will also need to update your HTML to include a link back to fed.brid.gy in each entry you want syndicated).

For some reason the syndication didn’t seem to be working at first, but then when I followed @adactio.com@adactio.com from my @adactio@mastodon.social account, it started working. Maybe there needs to be at least one follower.

Also, my links don’t seem to be showing up on Mastodon even though it looks like everything is posting okay. Not sure what that is about.

Anyway, if you want to follow me on Mastodon, you now have a choice. There’s me, @adactio@mastodon.social, or there’s my website, @adactio.com@adactio.com.

The syndicate

Social networks come and social networks go.

Right now, there’s a whole bunch of social networks coming (Blewski, Freds, Mastication) and one big one going, thanks to Elongate.

Me? I watch all of this unfold like Doctor Manhattan on Mars. I have no great connection to any of these places. They’re all just syndication endpoints to me.

I used to have a checkbox in my posting interface that said “Twitter”. If I wanted to add a copy of one of my notes to Twitter, I’d enable that toggle.

I have, of course, now removed that checkbox. Twitter is dead to me (and it should be dead to you too).

I used to have another checkbox next to that one that said “Flickr”. If I was adding a photo to one of my notes, I could toggle that to send a copy to my Flickr account.

Alas, that no longer works. Flickr only allows you to post 1000 photos before requiring a pro account. Fair enough. I’ve actually posted 20 times that amount since 2005, but I let my pro membership lapse a while back.

So now I’ve removed the “Flickr” checkbox too.

Instead I’ve now got a checkbox labelled “Mastodon” that sends a copy of a note to my Mastodon account.

When I publish a blog post like the one you’re reading now here on my journal, there’s yet another checkbox that says “Medium”. Toggling that checkbox sends a copy of my post to my page on Ev’s blog.

At least it used to. At some point that stopped working too. I was going to start debugging my code, but when I went to the documentation for the Medium API, I saw this:

This repository has been archived by the owner on Mar 2, 2023. It is now read-only.

I guessed I missed the memo. I guess Medium also missed the memo, because developers.medium.com is still live. It proudly proclaims:

Medium’s Publishing API makes it easy for you to plug into the Medium network, create your content on Medium from anywhere you write, and expand your audience and your influence.

Not a word of that is accurate.

That page also has a link to the Medium engineering blog. Surely the announcement of the API deprecation would be published there?

Crickets.

Moving on…

I have an account on Bluesky. I don’t know why.

I was idly wondering about sending copies of my notes there when I came across a straightforward solution: micro.blog.

That’s yet another place where I have an account. They make syndication very straightfoward. You can go to your account and point to a feed from your own website.

That’s it. Syndication enabled.

It gets better. Micro.blog can also cross-post to other services. One of those services is Bluesky. I gave permission to micro.blog to syndicate to Bluesky so now my notes show up there too.

It’s like dominoes falling: I post something on my website which updates my RSS feed which gets picked up by micro.blog which passes it on to Bluesky.

I noticed that one of the other services that micro.blog can post to is Medium. Hmmm …would that still work given the abandonment of the API?

I gave permission to micro.blog to cross-post to Medium when my feed of blog posts is updated. It seems to have worked!

We’ll see how long it lasts. We’ll see how long any of them last. Today’s social media darlings are tomorrow’s Friendster and MySpace.

When the current crop of services wither and die, my own website will still remain in full bloom.

That fediverse feeling

Right now, Twitter feels like Dunkirk beach in May 1940. And look, here comes a plucky armada of web servers running Mastodon instances!

Others have written some guides to getting started on Mastodon:

There are also tools like Twitodon to help you migrate from Twitter to Mastodon.

Getting on board isn’t completely frictionless. Understanding how Mastodon works can be confusing. But then again, so was Twitter fifteen years ago.

Right now, many Mastodon instances are struggling with the influx of new sign-ups. But this is temporary. And actually, it’s also very reminiscent of the early unreliable days of Twitter.

I don’t want to go into the technical details of Mastodon and the fediverse—even though those details are fascinating and impressive. What I’m really struck by is the vibe.

In a nutshell, I’m loving it! It feels …nice.

I was fully expecting Mastodon to be full of meta-discussions about Mastodon, but in the past few weeks I’ve enjoyed people posting about stone circles, astronomy, and—obviously—cats and dogs.

The process of finding people to follow has been slow, but in a good way. I’ve enjoyed seeking people out. It’s been easier to find the techy folks, but I’ve also been finding scientists, journalists, and artists.

On the one hand, the niceness of the experience isn’t down to technical architecture; it’s all about the social norms. On the other hand, those social norms are very much directed by technical decisions. The folks working on the fediverse for the past few years have made very thoughtful design decisions to amplify niceness and discourage nastiness. It’s all very gratifying to experience!

Personally, I’m posting to Mastodon via my own website. As much as I’m really enjoying Mastodon, I still firmly believe that nothing beats having control of your own content on your domain.

But I also totally get that not everyone has the same set of priorities as me. And frankly, it’s unrealistic to expect everyone to have their own domain name.

It’s like there’s a spectrum of ownership. On one end, there’s publishing on your own website. On the other end, there’s publishing on silos like Twitter, Facebook, Medium, Instagram, and MySpace.

Publishing on Mastodon feels much closer to the website end of the spectrum than it does to the silo end of the spectrum. If something bad happens to the Mastodon instance you’re on, you can up and move to a different instance, taking your social graph with you.

In a way, it’s like delegating domain ownership to someone you trust. If you don’t have the time, energy, resources, or interest in having your own domain, but you trust someone who’s running a Mastodon instance, it’s the next best thing to publishing on your own website.

Simon described it well when he said Mastodon is just blogs:

A Mastodon server (often called an instance) is just a shared blog host. Kind of like putting your personal blog in a folder on a domain on shared hosting with some of your friends.

Want to go it alone? You can do that: run your own dedicated Mastodon instance on your own domain.

And rather than compare Mastodon to Twitter, Simon makes a comparison with RSS:

Do you still miss Google Reader, almost a decade after it was shut down? It’s back!

A Mastodon server is a feed reader, shared by everyone who uses that server.

Lots of other folks are feeling the same excitement in the air that I’m getting:

Bastian wrote:

Real conversations. Real people. Interesting content. A feeling of a warm welcoming group. No algorithm to mess around with our timelines. No troll army to destory every tiny bit of peace. Yes, Mastodon is rough around the edges. Many parts are not intuitive. But this roughness somehow added to the positive experience for me.

This could really work!

Brent Simmons wrote:

The web is wide open again, for the first time in what feels like forever.

I concur! Though, like Paul, I love not being beholden to either Twitter or Mastodon:

I love not feeling bound to any particular social network. This website, my website, is the one true home for all the stuff I’ve felt compelled to write down or point a camera at over the years. When a social network disappears, goes out of fashion or becomes inhospitable, I can happily move on with little anguish.

But like I said, I don’t expect everyone to have the time, means, or inclination to do that. Mastodon definitely feels like it shares the same indie web spirit though.

Personally, I recommend experiencing Mastodon through the website rather than a native app. Mastodon instances are progressive web apps so you can add them to your phone’s home screen.

You can find me on Mastodon as @adactio@mastodon.social

I’m not too bothered about what instance I’m on. It really only makes a difference to my local timeline. And if I do end up finding an instance I prefer, then I know that migrating will be quite straightforward, by design. Perhaps I should be on an instance with a focus on front-end development or the indie web. I still haven’t found much of an Irish traditional music community on the fediverse. I’m wondering if maybe I should start a Mastodon instance for that.

While I’m a citizen of mastodon.social, I’m doing my bit by chipping in some money to support it: sponsorship levels on Patreon start at just $1 a month. And while I can’t offer much technical assistance, I opened my first Mastodon pull request with a suggested improvement for the documentation.

I’m really impressed with the quality of the software. It isn’t perfect but considering that it’s an open source project, it’s better than most VC-backed services with more and better-paid staff. As Giles said, comparing it to Twitter:

I’m using Mastodon now and it’s not the same, but it’s not shit either. It’s different. It takes a bit of adjustment. And I’m enjoying it.

Most of all, I love, love, love that Mastodon demonstrates that things can be different. For too long we’ve been told that behavioural advertising was an intrinsic part of being online, that social networks must inevitably be monolithic centralised beasts, that we have to relinquish control to corporations in order to be online. The fediverse is showing us a better way. And this isn’t just a proof of concept either. It’s here now. It’s here to stay, if you want it.

Syndicating to Mastodon

I’ve been contemplating a checkbox. The label for this checkbox reads:

This is a bot account

Let me back up…

In what seems like decades ago, but was in fact just a few weeks, Elon Musk bought Twitter and began burning it to the ground. His admirers insist he’s playing some form of four-dimensional chess, but to the rest of us, his actions are indistinguishable from a spoilt rich kid not understanding what a social network is.

It wasn’t giving me much cause for anguish personally. For the past eight years, I’ve only used Twitter as a syndication endpoint for my own notes. But I understand that’s a very privileged position to be in. Most people on Twitter don’t have the same luxury of independence. It’s genuinely maddening and saddening to see their years of sharing destroyed by one cruel idiot.

Lots of people started moving to Mastodon. I figured I should do the same for my syndicated notes.

At first, I signed up for an account on mastodon.cloud. No particular reason. But that’s where I saw this very insightful post from Anil Dash:

When it came time to reckon with social media’s failings, nobody ran to the “web3” platforms. Nobody asked “can I get paid per message”? Nobody asked about the blockchain. The community of people who’ve been quietly doing this work for years (decades!) ended up being the ones who welcomed everyone over, as always.

I was getting my account all set up and beginning to follow some other folks, when I realised that I actually already had an exisiting account over on mastodon.social. Doh! Turns out that I signed up back in 2017 to kick the tyres, but never did much else because there weren’t many other people around back then. Oh, how times have changed!

Anyway, I thought I had really screwed up by having two accounts but this turned out to be an opportunity to experience some of the thoughtfulness in Mastodon’s design. The process of migrating from one Mastodon account to another—on a completely different instance—was very smooth! It was clear that this wasn’t an afterthought. This is an essential part of the fediverse and the design of the migration flow reflects that.

This gives me enormous peace of mind. If I ever want to switch to a different instance and still keep my network intact, I know it won’t be a problem. Mastodon is like the opposite of the roach-motel mentality that permeates most VC-backed so-called social networks.

As I played around some more—reading, following, exploring—my feelings of fondness only grew stronger. I like this place a lot!

I definitely wanted to syndicate my notes to Mastodon. At first, I implemented a straightforward RSS-to-Mastodon syndication using IFTTT (IF This, Then That), thanks to Matthias’s excellent tutorial.

But that didn’t feel quite right. When I syndicate to Twitter, I make a conscious choice each time. There’s a “Twitter” toggle that I can enable or disable in my posting interface. Mastodon deserved the same level of thoughtfulness.

So I switched off the IFTTT recipe and started exploring the Mastodon API. It’s going to sound like a humblebrag when I tell you that I got cross-posting working in almost no time at all, but that’s not a testament to my coding prowess (I’m really not very good), but rather a testament to the Mastodon API, which was a joy to work with.

  1. On your Mastodon instance, go to /settings/applications.
  2. Click on New Application.
  3. Fill in the details about your website and select write:statuses (and probably write:media) from the Scopes list.
  4. Copy Your access token to use in API calls.
  5. Write some sloppy code (in my case, PHP that uses CURL).

I did hit a wall when it came to posting images. That took me a while to get working, and I couldn’t figure out why. Was it something at Mastodon’s end while it was struggling under the influx of new users? As it turns out, no. It was entirely down to me being an idiot. (You know that situation where you’re working on a problem for ages and you’ve become convinced it’s an extremely gnarly rocket-science problem, but then turns out to be something stupid like a typo? Yeah. That.)

Then there’s the whole question of how to receive replies, likes, and reboosts from Mastodon here on my own site. Luckily, that was super easy, thanks to Brid.gy. One click and I was done. I love Brid.gy!

Take this note, for example. There’s a version on Twitter and a version on Mastodon. The original version on my own site gets responses from both places.

If I’m replying to a response on Twitter, I do not syndicate that to Mastodon.

Likewise, if I’m replying to a response on Mastodon, I do not syndicate that to Twitter.

Oh, one thing worth mentioning: if you’re sending a reply to something on Mastodon using the API, there’s an in_reply_to_id field for you to provide. But you should also include the full @username@instance of the person you’re replying to at the beginning of the message to ensure that it’s displayed as a reply rather than showing up as a regular post. Note the difference between this note on my site and its syndicated version on Mastodon.

Anyway, now I’m posting to Mastodon, but I’m doing it through the the interface of my own website. Which brings me to that checkbox in Mastodon’s profile settings:

This is a bot account

The help text reads:

Signal to others that the account mainly performs automated actions and might not be monitored

If I were doing the automatic cross-posting from RSS, I’d definitely tick that box. But as I’m making a conscious decision whenever I syndicate to Mastodon, I think I’m going to leave that checkbox unticked.

My cross-posting is not automated and I’m very much monitoring my Mastodon account …because I’m enjoying my Mastodon experience more than I’ve enjoyed anything online for quite some time. Highly recommended!

Image previews with the FileReader API

I added a “notes” section to this website eight years ago. I set it up so that notes could be syndicated to Twitter. Ever since then, that’s the only way I post to Twitter.

A few months later I added photos to my notes. Again, this would get syndicated to Twitter.

Something’s bothered me for a long time though. I initially thought that if I posted a photo, then the accompanying text would serve as a decription of the image. It could effectively act as the alt text for the image, I thought. But in practice it didn’t work out that way. The text was often a commentary on the image, which isn’t the same as a description of the contents.

I needed a way to store alt text for images. To make it more complicated, it was possible for one note to have multiple images. So even though a note was one line in my database, I somehow needed a separate string of text with the description of each image in a single note.

I eventually settled on using the file system instead of the database. The images themselves are stored in separate folders, so I figured I could have an accompanying alt.txt file in each folder.

Take this note from yesterday as an example. Different sizes of the image are stored in the folder /images/uploaded/19077. Here’s a small version of the image and here’s the original. In that same folder is the alt text.

This means I’m reading a file every time I need the alt text instead of reading from a database, which probably isn’t the most performant way of doing it, but it seems to be working okay.

Here’s another example:

In order to add the alt text to the image, I needed to update my posting interface. By default it’s a little textarea, followed by a file upload input, followed by a toggle (a checkbox under the hood) to choose whether or not to syndicate the note to Twitter.

The interface now updates automatically as soon as I use that input type="file" to choose any images for the note. Using the FileReader API, I show a preview of the selected images right after the file input.

Here’s the code if you ever need to do something similar. I’ve abstracted it somewhat in that gist—you should be able to drop it into any page that includes input type="file" accept="image/*" and it will automatically generate the previews.

I was pleasantly surprised at how easy this was. The FileReader API worked just as expected without any gotchas. I think I always assumed that this would be quite complex to do because once upon a time, it was quite complex (or impossible) to do. But now it’s wonderfully straightforward. Story of the web.

My own version of the script does a little bit more; it also generates another little textarea right after each image preview, which is where I write the accompanying alt text.

I’ve also updated my server-side script that handles the syndication to Twitter. I’m using the /media/metadata/create method to provide the alt text. But for some reason it’s not working. I can’t figure out why. I’ll keep working on it.

In the meantime, if you’re looking at an image I’ve posted on Twitter and you’re judging me for its lack of alt text, my apologies. But each tweet of mine includes a link back to the original note on this site and you will most definitely find the alt text for the image there.

Feeds

A little while back, Marcus Herrmann wrote about making RSS more visible again with a /feeds page. Here’s his feeds page. Here’s Remy’s.

Seems like a good idea to me. I’ve made mine:

adactio.com/feeds

As well as linking to the usual RSS feeds (blog posts, links, notes), it’s also got an explanation of how you can subscribe to a customised RSS feed using tags.

Then, earlier today, I was chatting with Matt on Twitter and he asked:

btw do you share your blogroll anywhere?

So now I’ve added another URL:

adactio.com/feeds/subscriptions

That’s got a link to my OPML file, exported from my feed reader, and a list of the (current) RSS feeds that I’m subscribed to.

I like the idea of blogrolls making a comeback. And webrings.

Reading

At the beginning of the year, Remy wrote about extracting Goodreads metadata so he could create his end-of-year reading list. More recently, Mark Llobrera wrote about how he created a visualisation of his reading history. In his case, he’s using JSON to store the information.

This kind of JSON storage is exactly what Tom Critchlow proposes in his post, Library JSON - A Proposal for a Decentralized Goodreads:

Thinking through building some kind of “web of books” I realized that we could use something similar to RSS to build a kind of decentralized GoodReads powered by indie sites and an underlying easy to parse format.

His proposal looks kind of similar to what Mark came up with. There’s a title, an author, an image, and some kind of date for when you started and/or finished reading the book.

Matt then points out that RSS gets close to the data format being suggested and asks how about using RSS?:

Rather than inventing a new format, my suggestion is that this is RSS plus an extension to deal with books. This is analogous to how the podcast feeds are specified: they are RSS plus custom tags.

Like Matt, I’m in favour of re-using existing wheels rather than inventing new ones, mostly to avoid a 927 situation.

But all of these proposals—whether JSON or RSS—involve the creation of a separate file, and yet the information is originally published in HTML. Along the lines of Matt’s idea, I could imagine extending the h-entry collection of class names to allow for books (or films, or other media). It already handles images (with u-photo). I think the missing fields are the date-related ones: when you start and finish reading. Those fields are present in a different microformat, h-event in the form of dt-start and dt-end. Maybe they could be combined:


<article class="h-entry h-event h-review">
<h1 class="p-name p-item">Book title</h1>
<img class="u-photo" src="image.jpg" alt="Book cover.">
<p class="p-summary h-card">Book author</p>
<time class="dt-start" datetime="YYYY-MM-DD">Start date</time>
<time class="dt-end" datetime="YYYY-MM-DD">End date</time>
<div class="e-content">Remarks</div>
<data class="p-rating" value="5">★★★★★</data>
<time class="dt-published" datetime="YYYY-MM-DDThh:mm">Date of this post</time>
</article>

That markup is simultaneously a post (h-entry) and an event (h-event) and you can even throw in h-card for the book author (as well as h-review if you like to rate the books you read). It can be converted to RSS and also converted to .ics for calendars—those parsers are already out there. It’s ready for aggregation and it’s ready for visualisation.

I publish very minimal reading posts here on adactio.com. What little data is there isn’t very structured—I don’t even separate the book title from the author. But maybe I’ll have a little play around with turning these h-entries into combined h-entry/event posts.

Posting to my site

I was idly thinking about the different ways I can post to adactio.com. I decided to count the ways.

Admin interface

This is the classic CMS approach. In my case the CMS is a crufty hand-rolled affair using PHP and MySQL that I wrote years ago. I log in to an admin interface and fill in a form, putting the text of my posts into a textarea. In truth, I usually write in a desktop text editor first, and then paste that into the textarea. That’s what I’m doing now—copying and pasting Markdown from the Typed app.

Directly from my site

If I’m logged in, I get a stripped down posting interface in the notes section of my site.

Notes posting interface

Bookmarklet

This is how I post links. When I’m at a URL I want to bookmark, I hit the “Bookmark it” bookmarklet in my browser’s bookmarks bar. That pops open a version of the admin interface tailored specifically for links. I really, really like bookmarklets. The one big downside is that they don’t work on mobile.

Text message

This is something I knocked together at Indie Web Camp Brighton 2015 using the Twilio API. It’s handy for posting notes if I’m travelling somewhere and data is at a premium. But I don’t use it that often.

Instagram

Thanks to Aaron’s OwnYourGram service—and the fact that my site has a micropub endpoint—I can post images from Instagram to my site. This used to happen instantaneously but Instagram changed their API rules for the worse. Between that and their shitty “algorithmic” timeline, I find myself using the service less and less. At this point I’m only on their for the doggos.

Swarm

Like OwnYourGram, Aaron’s OwnYourSwarm allows me to post check-ins and photos from the Swarm app to my site. Again, micropub makes it all possible.

OwnYourGram and OwnYourSwarm are very similar and could probably be abstracted into a generic service for posting from third-party apps to micropub endpoints. I’d quite like to post my check-ins on Untappd to my site.

Other people’s admin interfaces

Thanks to rel="me" and IndieAuth, I can log into other people’s posting interfaces using my own website as the log-in, and post to my micropub endpoint, like this. Quill is a good example of this. I don’t use it that much, but I really should—the editor interface is quite Medium-like in its design.

Anyway, those are the different ways I can update my website that I can think of right now.

Syndication

In terms of output, I’ve got a few different ways of syndicating what I post here:

Just so you know, if you comment on one of my posts on Facebook, I probably won’t see it. But if you reply to a copy of one of posts on Twitter or Instagram, it will show up over here on adactio.com thanks to the magic of Brid.gy and webmention.

Owning my words

When I wrote a few words about progressive enhancement recently, I linked to Karolina’s great article The Web Isn’t Uniform. I was a little reluctant to link to it, not because of the content—which is great—but because of its location on Ev’s blog. I much prefer to link directly to people’s own websites (I have a hunch that those resources tend to last longer too) but I understand that Medium offers a nice low barrier to publishing.

That low barrier comes at a price. It means you have to put up with anyone and everyone weighing in with their own hot takes. The way the site works is that anyone who writes a comment on your article is effectively writing their own article—you don’t get to have any editorial control over what kind of stuff appears together with your words. There is very little in the way of community management once a piece is published.

Karolina’s piece attracted some particularly unsavoury snark—tech bros disagreeing in their brash bullying way. I linked to a few comments, leaving out the worst of the snark, but I couldn’t resist editorialising:

Ah, Medium! Where the opinions of self-entitled dudes flow like rain from the tech heavens.

I knew even when I was writing it that it was unproductive, itself a snarky remark. Two wrongs don’t make a right. But I wanted to acknowledge that not only was bad behaviour happening, but that I was seeing it, and I wasn’t ignoring it. I guess it was mostly intended for Karolina—I wanted to extend some kind of acknowledgment that the cumulative weight of those sneering drive-by reckons is a burden that no one should have to put up with.

I knew that when I wrote about Medium being “where the opinions of self-entitled dudes flow like rain from the tech heavens” that I would (rightly) get pushback, and sure enough, I did …on Medium. Not on Twitter or anywhere else, just Medium.

I syndicate my posts to Ev’s blog, so the free-for-all approach to commenting doesn’t bother me that much. The canonical URL for my words remains on my site under my control. But for people posting directly to Medium and then having to put up with other people casually shitting all over their words, it must feel quite disempowering.

I have a similar feeling with Twitter. I syndicate my notes there and if the service disappeared tomorrow, I wouldn’t shed any tears. There’s something very comforting in knowing that any snarky nasty responses to my words are only being thrown at copies. I know a lot of my friends are disheartened about the way that Twitter has changed in recent years. I wish I could articulate how much better it feels to only use Twitter (or Medium or Facebook) as a syndication tool, like RSS.

There is an equal and opposite reaction too. I think it’s easier to fling off some thoughtless remarks when you’re doing it on someone else’s site. I bet you that the discourse on Ev’s blog would be of a much higher quality if you could only respond from your own site. I find I’m more careful with my words when I publish here on adactio.com. I’m taking ownership of what I say.

And when I do lapse and write snarky words like “Ah, Medium! Where the opinions of self-entitled dudes flow like rain from the tech heavens.”, at least I’m owning my own snark. Still, I will endeavour to keep my snark levels down …but that doesn’t mean I’m going to turn a blind eye to bad behaviour.

Syndicating to Medium

When I brainpuked my thoughts on Google’s AMP project, I finished up by saying it was one more option for the Indie Web approach to syndication:

When I publish something on adactio.com in HTML, it already gets syndicated to different places. This is the Indie Web idea of POSSE: Publish (on your) Own Site, Syndicate Elsewhere. As well as providing RSS feeds, I’ve also got Twitter bots that syndicate to Twitter. An If This, Then That script pushes posts to Facebook. And if I publish a photo, it goes to Flickr. Now that Medium is finally providing a publishing API, I’ll probably start syndicating articles there as well. The more, the merrier.

Until Medium provided an API, I didn’t see much point in Medium. Let me clarify: I didn’t see much point in it for me. I’ve already got a website where I can publish whatever I like. For someone who doesn’t have their own website, I guess Medium—like Facebook, Twitter, Tumblr, etc.—provides a place to publish. I think this is what people mean when they use the word “platform” in a digital—rather than a North Sea oil drilling—sense.

Publishing exclusively on somebody else’s site works pretty well right up until the day the platform turns out to be a trap door and disappears from under you.

But I’m really puzzled by people who already have their own website choosing to publish on Medium instead. A shiny content farm is still a content farm.

“It’s the reach!” I’m told. That makes me sad. The whole point of the World Wide Web is that everybody has an equal opportunity to share their thoughts. You don’t need to ask anyone for permission. The gatekeepers of the previous century—record labels, book publishers, film producers—can’t stop you from making whatever you want and putting it out there for the world to see. And thanks to the principle of net neutrality baked into the design of TCP/IP, no one gets preferential treatment.

Notice that I said “people who already have their own website choosing to publish on Medium instead.” That last bit is important. Using Medium to publish copies of what you’ve already published on your own site gives you the best of both worlds: ownership and reach. That’s what Kevin does, for example. And Jeffrey. Until recently that was quite a pain in the ass, requiring a manual copy’n’paste process.

Back when Medium first launched, Dave Winer said:

Let me enter the URL of something I write in my own space, and have it appear here as a first class citizen. Indistinguishable to readers from something written here.

It still isn’t quite that simple, but now that Medium has a publishing API, it’s relatively straightforward to syndicate copies of your posts to Medium at the moment you publish on your own site.

Here’s what I did…

First of all, I signed up for a Medium account. For the longest time, even this simple step was off-limits for me because Medium used to require authentication using Twitter. By itself, that’s not a problem. The problem was that Medium demanded write permissions for my Twitter account. Just say no.

Now it’s possible to sign up for Medium using email so that rudeness is less of an issue (although I’d really like to see Medium stop being so demanding when it comes to Twitter permissions, especially as the interface copy bends over backwards to promise that Medium would never post to Twitter on my behalf …so why ask for permission to do just that?).

Once I had a Medium account, I needed two pieces of secret information in order to use the API.

The first piece is an access token.

I went to my settings on Medium and scrolled all the way to the bottom to the heading “Integration tokens”. I entered a description (“Syndication from adactio.com”) and pressed the “Get integration token” button.

Now I could use that token to get the second piece of information: my user ID.

I opened up a browser tab and went to this URL: https://api.medium.com/v1/me?accessToken= …adding my new secret integration token to the end.

That returns a JSON response. One of the fields in the JSON object has the name “id”. The value of that field is my user ID on Medium.

With those two pieces of information, I could make an authenticated POST request using cURL. Here’s the PHP code I’m using. It’s probably terrible but please feel free to use it, copy it, fork it, or do anything else you want with it.

When I run that code, I get a JSON response back from Medium’s API. Assuming I get a successful response, I can store the URL of the Medium copy and link out to it from here. That copy on Medium has a corresponding link rel="canonical" in the head of the document pointing back here to adactio.com.

That’s pretty much it. I added a checkbox to my posting interface so that sending a copy of a post to Medium is just a toggle away. I’ll tick that checkbox when I post this. You could be reading this on my site or you could be reading the copy on Medium.

The code I wrote is pretty similar to how I post notes to Twitter and photos to Flickr. In fact, posting to Medium is more straightforward: Flickr requires three bits of secret information; Twitter requires four.

What would make this cross-posting with Medium really interesting would be if it could work in both directions. Then I’d be able to use the (very nice) writing interface on Medium to publish on adactio.com.

That’s not so far-fetched. I’ve already got a micropub endpoint here on my site (here’s the code). That’s how I’m able to use Instagram to post photos to my own site (using OwnYourGram). I let Instagram keep a copy of my photo. I’d be happy to let Medium keep a copy of my post.

We could make history:

We need to break out of the model where all these systems are monolithic and standalone. There’s art in each individual system, but there’s a much greater art in the union of all the systems we create.

AMPed up

Apple has Apple News. Facebook has Instant Articles. Now Google has AMP: Accelerated Mobile Pages.

The big players sure are going to a lot of effort to reinvent RSS.

That may sound like a flippant remark, but it’s not too far from the truth. In the case of Apple News, its current incarnation appears to be quite literally an RSS reader, at least until the unveiling of the forthcoming Apple News Format.

Google’s AMP project looks a little bit different to the offerings from Facebook and Apple. Rather than creating a proprietary format from scratch, it mandates a subset of HTML …with some proprietary elements thrown in (or, to use the more diplomatic parlance of the extensible web, custom elements).

The idea is that alongside the regular HTML version of your document, you provide a corresponding AMP HTML version. Because the AMP HTML version will be leaner and meaner, user agents can then grab the AMP HTML version and present that to the end user for a faster browsing experience.

So if an RSS feed is an alternate representation of a homepage or a listing of articles, then an AMP document is an alternate representation of a single article.

Now, my own personal take on providing alternate representations of documents is “Sure. Why not?” Here on adactio.com I provide RSS feeds. On The Session I provide RSS, JSON, and XML. And on Huffduffer I provide RSS, Atom, JSON, and XSPF, adding:

If you would like to see another format supported, share your idea.

Also, each individual item on Huffduffer has a corresponding oEmbed version (and, in theory, an RDF version)—an alternate representation of that item …in principle, not that different from AMP. The big difference with AMP is that it’s using HTML (of sorts) for its format.

All of this sounds pretty reasonable: provide an alternate representation of your canonical HTML pages so that user-agents (Twitter, Google, browsers) can render a faster-loading version …much like an RSS reader.

So should you start providing AMP versions of your pages? My initial reaction is “Sure. Why not?”

The AMP Project website comes with a list of frequently asked questions, which of course, nobody has asked. My own list of invented frequently asked questions might look a little different.

Will this kill advertising?

We live in hope.

Alas, AMP pages will still be able to carry advertising, but in a restricted form. No more scripts that track your movement across the web …unless the script is from an authorised provider, like say, Google.

But it looks like the worst performance offenders won’t be able to get their grubby little scripts into AMP pages. This is a good thing.

Won’t this kill journalism?

Of all the horrid myths currently in circulation, the two that piss me off the most are:

  1. Journalism requires advertising to survive.
  2. Advertising requires invasive JavaScript.

Put the two together and you get the gist of most of the chicken-littling articles currently in circulation: “Journalism requires invasive JavaScript to survive.”

I could argue against the first claim, but let’s leave that for another day. Let’s suppose for now that, sure, journalism requires advertising to survive. Fine.

It’s that second point that is fundamentally wrong. The idea that the current state of advertising is the only way of advertising is incredibly short-sighted and misguided. Invasive JavaScript is not a requirement for showing me an ad. Setting a cookie is not a requirement for showing me an ad. Knowing where I live, who my friends are, what my income level is, and where I’ve been on the web …none of these are requirements for showing me an ad.

It is entirely possible to advertise to me and treat me with respect at the same time. The Deck already does this.

And you know what? Ad networks had their chance. They had their chance to treat us with respect with the Do Not Track initiative. We asked them to respect our wishes. They told us get screwed.

Now those same ad providers are crying because we’re installing ad blockers. They can get screwed.

Anyway.

It is entirely possible to advertise within AMP pages …just not using blocking JavaScript.

For a nicely nuanced take on what AMP could mean for journalism, see Joshua Benton’s article on Nieman Lab—Get AMP’d: Here’s what publishers need to know about Google’s new plan to speed up your website.

Why not just make faster web pages?

Excellent question!

For a site like adactio.com, the difference between the regular HTML version of an article and the corresponding AMP version of the same article is pretty small. It’s a shame that I can’t just say “Hey, the current version of the article is the AMP version”, but that would require that I only use a subset of HTML and that I add some required guff to my page (including an unnecessary JavaScript file).

But for most of the news sites out there, the difference between their regular HTML pages and the corresponding AMP versions will be pretty significant. That’s because the regular HTML versions are bloated with third-party scripts, oversized assets, and cruft around the actual content.

Now it is in theory possible for these news sites to get rid of all those things, and I sincerely hope that they will. But that’s a big political struggle. I am rooting for developers—like the good folks at VOX—who have to battle against bosses who honestly think that journalism requires invasive JavaScript. Best of luck.

Along comes Google saying “If you want to play in our sandbox, you’re going to have to abide by our rules.” Those rules include performance best practices (for the most part—I take issue with some of the requirements, and I’ll go into that in more detail in a moment).

Now when the boss says “Slap a three megabyte JavaScript library on it so we can show a carousel”, the developers can only respond with “Google says No.”

When the boss says “Slap a ton of third-party trackers on it so we can monetise those eyeballs”, the developers can only respond with “Google says No.”

Google have used their influence like this before and it has brought them accusations of monopolistic abuse. Some people got very upset when they began labelling (and later ranking) mobile-friendly pages. Personally, I’ve got no issue with that.

In this particular case, Google aren’t mandating what you can and can’t do on your regular HTML pages; only what you can and can’t do on the corresponding AMP page.

Which brings up another question…

Will the AMP web kill the open web?

If we all start creating AMP versions of our pages, and those pages are faster than our regular HTML versions, won’t everyone just see the AMP versions without ever seeing the “full” versions?

Tim articulates a legitimate concern:

This promise of improved distribution for pages using AMP HTML shifts the incentive. AMP isn’t encouraging better performance on the web; AMP is encouraging the use of their specific tool to build a version of a web page. It doesn’t feel like something helping the open web so much as it feels like something bringing a little bit of the walled garden mentality of native development onto the web.

That troubles me. Using a very specific tool to build a tailored version of my page in order to “reach everyone” doesn’t fit any definition of the “open web” that I’ve ever heard.

Fair point. But I also remember that a lot of people were upset by RSS. They didn’t like that users could go for months at a time without visiting the actual website, and yet they were reading every article. They were reading every article in non-browser user agents in a format that wasn’t HTML. On paper that sounds like the antithesis of the open web, but in practice there was always something very webby about RSS, and RSS feed readers—it put the power back in the hands of the end users.

Some people chose not to play ball. They only put snippets in their RSS feeds, not the full articles. Maybe some publishers will do the same with the AMP versions of their articles: “To read more, click here…”

But I remember what generally tended to happen to the publishers who refused to put the full content in their RSS feeds. We unsubscribed.

Still, I share the concern that any one company—whether it’s Facebook, Apple, or Google—should wield so much power over how we publish on the web. I don’t think you have to be a conspiracy theorist to view the AMP project as an attempt to replace the existing web with an alternate web, more tightly controlled by Google (albeit a faster, more performant, tightly-controlled web).

My hope is that the current will flow in both directions. As well as publishers creating AMP versions of their pages in order to appease Google, perhaps they will start to ask “Why can’t our regular pages be this fast?” By showing that there is life beyond big bloated invasive web pages, perhaps the AMP project will work as a demo of what the whole web could be.

I’ve been playing around with the AMP HTML spec. It has some issues. The good news is that it’s open source and the project owners seem receptive to feedback.

JavaScript

No external JavaScript is allowed in an AMP HTML document. This covers third-party libraries, advertising and tracking scripts. This is A-okay with me.

The reasons given for this ban are related to performance and I agree with them completely. Big bloated JavaScript libraries are one of the biggest performance killers on the web. I’m happy to leave them at the door (although weirdly, web fonts—another big performance killer—are allowed in).

But then there’s a bit of an about-face. In order to have a valid AMP HTML page, you must include a piece of third-party JavaScript. In this case, the third party is Google and the JavaScript file is what handles the loading of assets.

This seems a bit strange to me; on the one hand claiming that third-party JavaScript is bad for performance and on the other, requiring some third-party JavaScript. As Justin says:

For me this is loading one thing too many… the AMP JS library. Surely the document itself is going to be faster than loading a library to try and make it load faster.

On the plus side, this third-party JavaScript is loaded asynchronously. It seems to mostly be there to handle the rendering of embedded content: images, videos, audio, etc.

Embedded content

If you want audio, video, or images on your page, you must use propriet… elements like amp-audio, amp-video, and amp-img. In the case of images, I can see how this is a way of getting around the browser’s lookahead pre-parser (although responsive images also solve this problem). In the case of audio and video, the standard audio and video elements already come with a way of specifying preloading behaviour using the preload attribute. Very odd.

Justin again:

I’m not sure if this is solving anything at the moment that we’re not already fixing with something like responsive images.

To use amp-img for images within the flow of a document, you’ll need to specify the dimensions of the image. This makes sense from a rendering point of view—knowing the width and height ahead of time avoids repaints and reflows. Alas, in many of the cases here on adactio.com, I don’t know the dimensions of the images I’m including. So any of my AMP HTML pages that include images will be invalid.

Overall, the way that AMP HTML handles embedded content looks like a whole lot of wheel reinvention. I like the idea of providing custom elements as an option for authors. I hate the idea of making them a requirement.

Metadata

If you want to provide metadata about your document, AMP HTML currently requires the use of Google’s Schema.org vocabulary. This has a big whiff of vendor lock-in to it. I’ve flagged this up as an issue and Aaron is pushing a change so hopefully this will be resolved soon.

Accessibility

In its initial release, the AMP HTML spec came with some nasty surprises for accessibility. The biggest is probably the requirement to include this in your viewport meta element:

maximum-scale=1,user-scalable=no

Yowzers! That’s some slap in the face to decent web developers everywhere. Fortunately this has been flagged up and I’m hoping it will be fixed soon.

If it doesn’t get fixed, it’s quite a non-starter. It beggars belief that Google would mandate to authors that they must make their pages inaccessible to pinch/zoom. I would hope that many developers would rebel against such a draconian injunction. If that happens, it’ll be interesting to see what becomes of those theoretically badly-formed AMP HTML documents. Technically, they will fail validation, but for very good reason. Will those accessible documents be rejected?

Please get involved on this issue if this is important to you (hint: this should be important to you).

There are a few smaller issues. Initially the :focus pseudo-class was disallowed in author CSS, but that’s being fixed.

Currently AMP HTML documents must have this line:

<style>body {opacity: 0}</style><noscript><style>body {opacity: 1}</style></noscript>

shudders

That’s a horrible conflation of JavaScript availability and CSS. It’s being fixed though, and soon all the opacity jiggery-pokery will only happen via JavaScript, which will be a big improvement: it should either all happen in CSS or all happen in JavaScript, but not the current mixture of the two.

Discovery

The AMP HTML version of your page is not the canonical version. You can specify where the real HTML version of your document is by using rel="canonical". Great!

But how do you link from your canonical page out to the AMP HTML version? Currently you’re supposed to use rel="amphtml". No, they haven’t checked the registry. Again. I’ll go in and add it.

In the meantime, I’m also requesting that the amphtml value can be combined with the alternate value, seeing as rel values can be space separated:

rel="alternate amphtml" type="text/html"

See? Not that different to RSS:

rel="alterate" type="application/rss+xml"

POSSE

When I publish something on adactio.com in HTML, it already gets syndicated to different places. This is the Indie Web idea of POSSE: Publish (on your) Own Site, Syndicate Elsewhere. As well as providing RSS feeds, I’ve also got Twitter bots that syndicate to Twitter. An If This, Then That script pushes posts to Facebook. And if I publish a photo, it goes to Flickr. Now that Medium is finally providing a publishing API, I’ll probably start syndicating articles there as well. The more, the merrier.

From that perspective, providing AMP HTML pages feels like just one more syndication option. If it were the only option, and I felt compelled to provide AMP versions of my content, I’d be very concerned. But for now, I’ll give it a whirl and see how it goes.

Here’s a bit of PHP I’m using to convert a regular piece of HTML into AMP HTML—it’s horrible code; it uses regular expressions on HTML which, as we all know, will summon the Elder Gods.

Home-grown and Delicious

I’ve been using Delicious since 2005—back when it was del.icio.us. I have over 2,000 bookmarks stored there. I moved to Magnolia for a while but we all know how that ended.

Back then I wrote:

Really, I should be keeping my links here on adactio.com, maybe pinging Delicious or some other social bookmarking site as a back-up.

Recently Delicious updated its bookmarklet-conjured interface, not for the better. I thought that I could get used to the changes, but I found them getting more annoying over time. Once again, I began to toy with the idea of self-hosting my bookmarks. I even exported all my data into a big XML file.

The very next day, some of Yahoo’s shit hit the web’s fan. Delicious, it was revealed, was to be sunsetted. As someone who doesn’t randomly choose to use meteorological phenomena as verbs, I didn’t know what that meant, but it didn’t sound good.

As the twittersphere erupted in anger and indignation, I was able to share my recently-acquired knowledge:

curl https://{your username}:{your password}@api.del.icio.us/v1/posts/all to get an XML file of your Delicious bookmarks.

A lot of people immediately migrated to Pinboard, which looks like an excellent service (and happens to be the work of Maciej Ceglowski, one of the best bloggers ever to put pixels to screen).

After all that, it turns out that “sunsetting” doesn’t mean “shooting in the head”, it means something more like “flogging off”, as clarified on the Delicious blog. But the damage had been done and, anyway, I had already made up my mind to bring my bookmarks in-house, so I began a fun weekend of hacking.

Setting up a new section of the site for links and importing my Delicious bookmarks was pretty straightforward. Creating a bookmarklet was pretty easy too—I already some experience of that with Huffduffer.

So now I’ll do my bookmarking right here on my own site. All’s well that ends well, right?

Well, not quite. Dom sounded a note of concern:

sigh. There goes the one thing I actually used delicious for, the social network. :(

Paul also pointed to the social aspect as the reason why he’s sticking with Delicious:

Personally, while I’ve always valued the site for its ability to store stuff, what’s always made Delicious most useful to me is its network pages in general, and mine in particular.

But it’s possible to have your Delicious cake and eat it at home. The Delicious API makes it quite easy to post links so I’ve added that into my own bookmarking code. Whenever I post a link here, it will also show up on my Delicious account. If you’re subscribed to my Delicious links, you should notice no change whatsoever.

This is exactly what Steven Pemberton was talking about when I liveblogged his XTech talk two years ago. Another Stephen, the good Mr. Hay, summed up the absurdity of the usual situation:

For a while we’ve posted our data all over the internet on all types of services. These services provide APIs so we can access the data we put into them, so that we can do things with that data. Read that again.

Now I’m hosting the canonical copies of my bookmarks, much like Tantek hosts the canonical copies of his tweets and syndicates them out to Twitter. Delicious gets to have my links as well, and I get to use Delicious as a tool for interacting with my data …only now I’m not limited to just what Delicious can offer me.

Once I had my new links section up and running, I started playing around with the Embedly API (I recently added the excellent oEmbed format to Huffduffer and I was impressed with its power). Whenever I bookmark a page with oEmbed support, I can pull content directly into my site. Take a look at the links I’ve tagged with “sci-fi” to see some examples of embedded Vimeo and Flickr content.

I definitely prefer this self-hosting-with-syndication way of doing things. I can use a service like Delicious without worrying about it going tits-up and taking all my data with it. The real challenge is going to be figuring out a way of applying that model to Twitter and Flickr. I’m curious to see which milestone I’ll hit first: 10,000 tweets or 10,000 photos. Either way, that’s a lot of my content on somebody else’s servers.