I had thought that NetNewsWire’s conditional GET support was rock-solid — and so my first reaction was to be very surprised to learn that it’s not!

My second reaction was to be appreciative — Rachel’s work here on setting up a test server and reporting on the results is really great. My goal has always been to make NetNewsWire a model net citizen, and learning where it’s not is super valuable. So: much respect and thanks to Rachel for this.

The Data

Let’s look at some data and try to figure out what’s happening. Here’s Rachel’s report for NetNewsWire.

Things to know: these are all requests for a NetNewsWire-specific feed, and the copy of NetNewsWire making these requests is on my personal laptop. That laptop is occasionally used for development, which can throw things off, but not often. You can even see in the data a gap lasting just over two weeks where there were no requests (I was on vacation).

(You can also see some anomalies from when I had it on my dev machine also — ignore every row where ip is v6, since that’s my dev machine.)

Another thing to know: this is testing direct feed-reading, as with the On My Mac (or iPhone/iPad) and iCloud accounts. With systems such as Feedly, Feedbin, and so on, we get the data from the sync system and not directly from the site.

Ignoring Timing Issues

Let’s set aside, at least for today, the timing issues. That situation could be improved, but it very much reflects that this is a desktop app with a command that allows you to refresh feeds manually, without having to wait for the next poll.

Conditional GET Issues

First, a refresher on how this should work.

When a server returns a Last-Modified header, the client should return that exact same string in follow-up requests in an If-Modified-Since header. The server then looks at the If-Modified-Since header and decides to either return a 200 plus the feed — if it has been modified since — or return a 304 Not Modified response and an empty body.

It’s the same story with the Etag header. The client should save it and return it in follow-up requests in an If-None-Match header.

This is great because it can save a ton of bandwidth, which is great for server and app alike. And NetNewsWire’s been doing this since the early 2000s.

But clearly there’s a bug! In some cases, NetNewsWire is not picking up and saving the changed Last-Modified and Etag headers. Sometimes it does, and sometimes it keeps using whatever it already had and ignores the new ones.

What could account for this? Let’s look at the logic.

Feed processing logic

Here’s what happens when a feed download completes without errors and the content is non-empty:

First we check the hash of the raw feed data against the hash of the raw feed data the last time it actually changed. If those hashes match, then the app stops processing, because the feed hasn’t changed: it’s exactly the same as last time.

This is an optimization that deals with the fact that many servers unfortunately don’t support conditional GET. It allows the app to skip feed parsing and updating the database. Saves a bunch of work. Good for battery life.

If the hashes don’t match, then processing continues: it parses the feed and then sends the parsed articles to the code that updates the database.

After that it updates and saves the hash of the raw feed data, and finally it stores the conditional GET info — it saves any Last-Modified and Etag header values to send with the next request.

This isn’t actually the code, but it’s what the logic looks like:

downloadDidComplete(httpResponse, feed, feedData)
	hash = feedData.md5
	if hash == feed.previousHash then return
	parsedFeed = parse(feedData)
	updateDatabase(feed, parsedFeed)
	feed.previousHash = hash
	feed.conditionalGetInfo = conditionalGetInfoFromResponse(httpResponse)

My theory

There’s a great chance you’ve already spotted what I think is the issue: it’s that optimization where we check the hash of the raw feed data and return if it matches the previous hash.

Here’s what I think has happened in some of the tests: the raw feed data was unchanged, but one or both of the Last-Modified and Etag header values did change.

NetNewsWire never picked up the changes to those headers, because that code didn’t run — it had already bailed when it saw that the raw feed data was unchanged.

The assumption I made when I wrote this code was that if the raw feed data was unchanged then of course the Last-Modified and Etag header values would be unchanged too, so there was no need to check to see if they were new.

And I think that in real-world situations this is probably true pretty much all the time, and it’s only in tests like this where my assumption wouldn’t be true.

But I can’t say that for sure! This is a real bug, and we’ll fix it and add a test or tests to make sure it doesn’t happen again.

Here’s what the new logic should look like:

downloadDidComplete(httpResponse, feedData, feed)
	hash = feedData.md5
	if hash != feed.previousHash {
		parsedFeed = parse(feedData)
		updateDatabase(feed, parsedFeed)
		feed.previousHash = hash
	}
	feed.conditionalGetInfo = conditionalGetInfoFromResponse(httpResponse)

With the above logic, conditionalGetInfo gets updated no matter what.

PS There could be other bugs

My theory does point to a bug that should get fixed. But is it the only bug? Is it even the bug that causes the issues in these tests?

Though I’m pretty confident that this is the bug — seems pretty obvious, right? — more investigation and testing is warranted.

It’s not a bug in your RSS reader if recent articles in this feed all suddenly appeared as unread. You may even have seeming duplicates.

Sorry about that! It’s due to my changing settings in my blog generator. Pages now have a .html suffix where before they had none. This change impacts permalinks, which also changes the guids in my RSS feed — and NetNewsWire and other RSS readers use the guid property to identify articles, which means these will show up as new articles.

(Note: I’ve created redirects so that old links pointing in will still work.)

I’ve been getting questions about NetNewsWire’s unavailability on Vision Pro. Why isn’t it there? When might it be there?

Here’s the scoop:

I consider it risky to support an app running on a device I don’t own. Imagine writing a Mac or iPhone app and not actually running it on one of those machines — you wouldn’t.

I realize that the app would act as if it were running on an iPad — but the Vision Pro is not really an iPad. It’s a device with very different interactions from the direct manipulation we’re used to on iPad. And the compatibility mode is a new thing because this device is a new thing — we don’t know how well it works and what the gotchas might be.

I could test it on the simulator, sure, but the simulator is a convenience for developers. It’s no replacement for running it on a real device.

And, yes, the app is open source, which could mean that other developers with a Vision Pro could help support it — but it’s important to remember that I’m the only person who has to support it. The NetNewsWire team is awesome, but I’m the one on the line for this.

So I want to be careful and go slowly, because if I made it available it would be extremely difficult to reverse the decision and take it away, even with an excellent reason.

I’m hoping that a consensus will form among developers that running apps on Vision Pro as iPad apps is fine, that it’s a cakewalk. If that happens, I’d go ahead and do it too. But it’s too soon for us to know that.

PS Why don’t I have a Vision Pro? I’m sure it’s an incredible technical achievement and an amazing experience — and pretty damn wonderful in just about every way — but it’s just not my thing. I like reading and writing mostly, plus making apps for reading and writing. My personal future of computing has been here for all these years — the Mac.

PPS Eventually the price will come down to where I’d consider buying one as a test device and for a little fun — but that may be a few years away. I’m hoping that we’ll find, sooner than that, that running as iPad on Vision Pro is a-okay.

I started using Apple computers — and writing code for them, starting with BASIC — 43 years ago, before the Macintosh, even, and I’ve made this my career. I’ve had all these decades to really, thoroughly delight in these incredible machines and software, and to give a little back with my own apps.

Apple’s positive effect on my life should not be underestimated. My Mom once (lovingly, teasingly) said to me that my alternate career, had all this never happened, was “criminal genius.” Which might have been fun too, but possibly more stressful than I might have liked. At any rate, Apple has saved me from a life of crime, and I should love Apple for that.

But I need to remember, now and again, that Apple is a corporation, and corporations aren’t people, and they can’t love you back. You wouldn’t love GE or Exxon or Comcast — and you shouldn’t love Apple. It’s not an exception to the rule: there are no exceptions.

Apple doesn’t care about you personally in the least tiny bit, and if you were in their way somehow, they would do whatever their might — effectively infinite compared to your own — enables them to deal with you.

Luckily, Apple has just provided us all with a reminder. Just like the sixth finger in an AI-rendered hand, Apple’s policies for Distributing apps in the U.S. that provide an external purchase link are startlingly graceless and a jarring, but not surprising, reminder that Apple is not a real person and not worthy of your love.

Tim Chambers, admin of the Mastodon server indieweb.social, which hosts my personal account and the NetNewsWire account, asked me if ActivityPub support is on the roadmap for NetNewsWire. Great question!

Tim was responding to Richard MacManus:

Are there any RSS Readers that are trying to integrate ActivityPub too? Pondering the glut of email newsletters in our inboxes these days, and wondering if we are due for an RSS Reader renaissance, except with a fediverse twist this time…

I hear this. It’s not the first time this has come up, and I’ve spent the past year or so thinking about it.

Since RSS is an open web thing that brings you stuff people write, and ActivityPub is also an open web thing that brings you stuff people write, it’s an obvious good idea to do both in the same app. Totally.

But the question was specifically about NetNewsWire.

We will make NetNewsWire work better with Mastodon-generated RSS feeds. There are some things we’re not doing yet that will make reading these a better experience.

But should NetNewsWire become a Mastodon client? That’s where the answer is not so obvious.

* * *

Let’s compare the NetNewsWire and Mastodon experiences.

NetNewsWire is a three-paned RSS reader. Sidebar (accounts, folders, feeds) on the left. Timeline (titles and/or first few lines) in the middle. Article on the right. See these screenshots to get the idea.

Each article has a read/unread status. Selecting an article marks it as read, and there are commands to mark all as read (and mark all as read above or below the selection).

This is not the only good model for an RSS reader, but it’s how NetNewsWire works — and it’s a bad fit for a Mastodon reading experience.

Whether you use Mastodon on the web or in an app you’re used to a single timeline per account (with perhaps side streams for mentions and such). All posts from everyone you follow appear in that single timeline. You don’t deal with read/unread status — instead you have a chronological position.

The timeline shows the entire post (usually), instead of an optional title and first few lines. There is no third article view, because the entire post is displayed in the timeline.

This experience isn’t unique to Mastodon, of course — that’s how X, Threads, Micro.blog, Bluesky, and other social media apps work.

But it’s not how NetNewsWire works, and if we try to mash that experience into the app — but have it on only when a Mastodon account is selected — we’d end up with a confusion that feels like two very different apps. Because it would be.

* * *

A second option: we could not change the experience and just assume that some people would like to use Mastodon in a NetNewsWire way instead of in the usual social media way. I’m extremely skeptical of this idea — I think it would be a pain, and I think most people who try it would drop it.

The third option, which I think is best, which we already do, is to support Mastodon via RSS. (Mastodon feeds are usually available as RSS — you can add .rss to the end of a URL to get a feed.)

There might be some Mastodon accounts you want to follow in your RSS reader instead of in your Mastodon client. Think of an account for a website that doesn’t have an RSS feed, but that reliably posts links to articles. Or a person you want to follow — but you don’t want to show up in their follower list. (Sounds a little antisocial, sure, but you can imagine good reasons.)

We can, and will, do more to make Mastodon-over-RSS a good experience in NetNewsWire. Right now we’re not picking up avatars or image attachments, which we can fix. There might be more things like that to do. We might add the ability to post to Mastodon, since it’s a natural thing to want to share stuff from your RSS reader to Mastodon.

But I don’t think we’ll make NetNewsWire a real Mastodon client.

* * *

It still seems like RSS and Mastodon could fit in the same app, though! If I were designing it, I’d start with the social media experience: the single timeline of posts. Very simple sidebar. No article view. No read/unread status — just position in the timeline.

You could add RSS feeds, but they’d be treated like Mastodon posts. Any article short enough would appear in full in the timeline, but most would probably have to be truncated. You’d open articles in your browser, just like you do now with social media apps (there’d be no third article pane).

Such an app could be a nice unified experience. Get your Mastodon, Threads, RSS feeds, Micro.blog and, hopefully, other services — anything that supports ActivityPub, RSS, or some other open format or API — all in one place, in a way that’s already familiar to everyone.

Sounds pretty great! But it’s not NetNewsWire.

Seattle Xcoders meetups are at 7 pm on the first, third, and fifth Thursdays of each month, lately at the Bale Breaker x Yonder Cider Taproom in Seattle, in Ballard.

We meet outside, usually in the uncovered area that has some benches and small gas fire pits. (Not actually pits. More like small coffee tables with rocks and fire on top.)

There are no presentations or anything like that — we’re not back to doing indoor meetings with slides. We’re just talking. (Seattle Xcoders has always had a strong social game, so this is no surprise.)

Everybody’s welcome! You don’t have to be a coder. Designers, support folks, testers, writers, and anybody interested in Apple-land apps are all encouraged to come hang out. Even managers.

If you’re interested, be sure to subscribe to the calendar, so you don’t miss any changes of date, time, or venue.

I myself don’t make it every time, but I do plan to be there this Thursday (June 29). See you there!

From time to time a NetNewsWire user lets me know that they’d be happy to pay for the app or add to a tip jar. The answer is always the same: we don’t take money, but here’s how to support NetNewsWire.

Sometimes, though, they insist! Which is flattering, and I take it to mean that they really like the app.

But I should explain why we won’t take even a dollar in tip money.

NetNewsWire Expenses Are Almost Nonexistent

We use free services like GitHub, and we aren’t asked to pay anything to the various syncing services (Feedbin, Feedly, iCloud, etc.) that the app works with.

My developer membership with Apple is a thing I’d have anyway. The website is hosted on the same account that hosts this blog, and it costs nothing extra.

The blog is hosted on Micro.blog, and Manton Reece very generously provides us a free account. The service that powers the Reader View is provided, also very generously and for free, by Feedbin.

I do pay for the smallest possible Linode instance to run a crash log catcher script; this costs $7.73 per month. If you add in periodic domain name renewals, we’d probably end up somewhere around $10 per month.

In other words: NetNewsWire is a really cheap hobby.

(Side note: you can see, though, how paying $100 a month for Twitter API access would blow this up. Not going to do that.)

Two Problems with Taking Money

NetNewsWire has no bank account and does no accounting. The organization Ranchero Software that hosts the NetNewsWire repo on GitHub does not exist as a legal entity — it’s not even a non-profit. (It’s real, though, in the sense that you can give a group of people a name.)

If we took money, even a little bit, I’d want to change all that. I’d file with the state of Washington, set up a bank account, start keeping track of income and expenses, separate NetNewsWire stuff from my stuff, start paying taxes. This would take up time I could have spent working on NetNewsWire itself. And I’d hate it.

So that’s problem one. Problem two would be — who gets the money? After we take out $10 a month for expenses, where does the excess go? There are a bunch of people who spend lots of hours every month working on the app — how would we divvy up the money we get? I can’t think of a fair way.

The Even Worse Problem with Taking Money

As bad as those first two problems are, problem three is worse. That’s the one where money could affect our decisions — or even just be perceived as possibly affecting our decisions.

Right now the app is made purely for love, and our goal is to make the best app we know how to make. Everybody who uses it — and everybody who volunteers to work on it — knows that nobody is making a single dime from it.

There is 100% trust that we never think about revenue when making decisions. There is quite enough of that in the tech world already, and it’s really, really nice for us to work on a thing where we never have to consider revenue (yes, we’re fortunate: I totally get that).

I think it’s also really nice for users to know that our decisions are always based on what’s best for the app and its users and never on what would make us more money.

That said: of course we think about popularity. It’s a factor. And I don’t mean for ego — I mean that we’re more likely to do a feature we think lots of people will use versus one we expect few people to use. But that’s just one part of the equation and not the driving force it would be if we were taking money.

Before You Think I’m Advocating Something

Just in case it’s not obvious, I’ll spell it out: this model is not the best model for every app. I’m not saying it is. It’s a model that I personally like and can do because I’ve done well enough in the for-profit world.

I especially want to mention indie developers — they do have to think about revenue in order to keep doing what they’re doing. They are all under-charging for their apps and putting in far more time and care than you even imagine. They’re doing the best work on the platforms we love and they take care of their customers better than larger companies. So whenever you think you might want to give us money, go buy something awesome from an indie instead.

Or — do this also! — give money to groups trying to lessen the cruelty of this world. They’re not hard to find. Here’s one.

For writers, artists, podcasters, journalists, and people who make things in public, Twitter was the one social networking site we all had to use.

It’s as if Twitter has been stretched out across the map of the internet, and whatever parts of the map it didn’t cover, it could still reach. Everything happened there. Even things that started on some blog, podcast, or real life still really happened on Twitter.

It was that way because we agreed to it. We may not have liked it all the time, but by our actions we were part of that consensus.

This has just changed, and now Twitter is only one of many nations on that map, no longer the indispensable one. And, worse, it’s showing clear signs of failing.

It Was Always a Bad Idea

The internet’s town square should never have been one specific website with its own specific rules and incentives. It should have been, and should be, the web itself.

Having one entity own and police that square could only deform the worldwide conversation, to disastrous ends, even with the smartest and most humane people at work.

Twitter’s new owner is certainly not one of those people. But it doesn’t matter: he unintentionally brought the change that needed to happen, the break in the consensus.

The Building Slowdown

I resented Twitter because it had become mandatory, because it had become the one place. And I resented it because it seemed to subdue the creative energy of the web.

You built a thing? How nice. Well, there’s Twitter, so we don’t need it. Or: how’s this going to replace Twitter? It’s not, so save your energy.

Everything you might build that had to do with communication, reading and writing and otherwise, was compared to Twitter or somehow in relation to Twitter, even when unasked for, and Twitter was the enormous factor in that equation.

A New Hope

With the fall of the Twitter consensus I am energized. I remember what it was like in the 2000s; I remember the liveliness and sparkle of those days on the web.

I don’t suggest dialing the clock back to old tech, but I hope we use the good parts from those days to help us build new and amazing things. There is so much to do, and so many grand problems to solve.

The web is wide open again, for the first time in what feels like forever.

DVD Extras

PS I outright stole the “stretched out across the map” thing from Franz Kafka’s “Letter to His Father.” It was just right for this. Come at me, Kafka.

PPS An earlier draft of this article had the line “Twitter was the island in the middle of the kitchen where we hung out, and now it’s a junk drawer of brands and nazis,” which I preserve because it’s true.