The trouble with app release notes
How we improved them and why our solution is now in Polaris
App release notes are a problem. They’re a problem we (all) created.
The challenge is twofold: app release notes are boring and we’ve conditioned folks not to read them. Done right, they can guide users with timely context; done wrong, they can alienate or annoy. App release notes exist in the mausoleum of important but boring words: terms and conditions, non-disclosure agreements, insurance documents. Stuff that makes an everyday, material difference to your life, but that almost no one reads — because they’re bad. But, they don’t have to be.
So, how do we fix them?
An extremely brief history
From 2016, many of our teams left developers to figure out app release notes for themselves. This didn’t really work for anyone: engineers had to spend time figuring out user-friendly language and users had to spend time figuring out what the nominally user-friendly language meant. Writers — myself included — were sometimes asked to help, but only ever at the last minute. Weekly releases had no process, no oversight, and no help. Sometimes we said things we shouldn’t: we made jokes that weren’t funny, referenced technology that no one understood, tried wordplay that alienated non-English speaking users, wrote jargon, and, yes, ok, we admit it, we even said “General bug fixes” once or twice.
What caused our problem
At Shopify, we have three apps that we support and six release note variants — Shopify mobile app (Android, iOS), Shopify POS (Android, iOS), and Shop (Android, iOS). The apps are owned by different teams, and there are separate teams for different devices. Not all teams have writers, and content designers at Shopify are embedded, so there isn’t a clear lateral structure for developers to get this kind of tertiary, non-product content reviewed. No one owns this content; and I only got involved by chance, having noticed a particularly worrying release note.
Because of these issues, our app release notes had problems. They were:
- Totally unscalable
- Very unfriendly for international markets
- Had no formal review structure
- Were no fun for anyone
We needed process: we needed to squash those pesky internal bugs. So I did what a content designer does.
Step 1: The audit
Our content audit had to answer a number of questions:
- What are we currently saying?
- Is there any consistency to our notes?
- Are there any patterns or recyclable syntax?
- Who writes them?
- Why are they like this?
- How long have they been like this?
- Is anything written down?
Here are a few examples of what we found:
8.53.0Mar 9, 2020
• We fixed an issue that was causing applied tags to not be displayed for draft orders.
8.54.0Mar 16, 2020
• The app has been put on a diet reducing the number of bits we take up on your device.
8.55.0Mar 30, 2020
• Introducing product filtering: narrow down your list of products by type, availability, and more.
Sometimes we were specific and took ownership. Sometimes we tried jokes about diets. Sometimes we did product introductions.
We had plenty of personality but no consistency — jokes, when they happened, could be offensive; product releases had nothing to contextualize their significance; bug fixes were being lost in a sea of “We fixed…”
Step 2: Competitor analysis
How does {this company} do it? How does {this other company} do it? Who is doing it well? Specifically, I wanted to discover if there were industry standards, and I wanted to understand what good looked like. It’s OK to review your own practice, and we had enough of it to review, but unless you can situate your findings in the broader context, you will struggle to figure out how well it worked, and whether you made any difference.
A competitor analysis can be a great way to test assumptions, discover better ideas, or gut-check your intuition.
We found good:
Specific: ✅
Personable: ✅
Intelligible: ✅
As a user, you understand when something changed, why it changed, and how it changed.
We found bad:
Specific: 🚫
Personable: 🚫
Intelligible: ✅
As a user, you can only ask: what exactly changed?
We found… intense:
Specific: ✅
Personable: ✅
Intelligible: 🤔
As a user, you feel loved, but maybe a bit confused. Long-term, you probably just feel like you really want to meet the person who writes these notes.
Our conclusion: Slack uses hierarchies, LinkedIn is consistent, Transit is genuinely funny. The ideal release note combines all three of these.
Step 3: Stakeholder management
On a project like this, stakeholder management is arguably the most important piece: you can draft the best guidance ever, but if you didn’t involve anyone, or no one knows you did it, or, worst, no one cares you did it—well, then, you’ve got a problem.
We want to create guidance and process: we needed our app dev teams to implement the changes and make sure they remained in place. We enlisted a bunch of folks: a director of engineering (to help implement the new process); a few devs (to make sure they agreed with our ideas); and senior staff content designers (to check I wasn’t just plain wrong!). A plurality of stakeholders can make management more time consuming, but if you assemble a diverse panel—seniority, tenure, background etc.—you can produce a far better product. Additionally, stakeholders become cheerleaders: your content cannot succeed if it isn’t implemented, and it won’t be socialized or actioned if your stakeholders weren’t along for the ride.
Given the audit results (mixed), stakeholder feedback (positive as hell), and what our competitors were doing (mixed), we decided our quickest and easiest win was language guidance and a clear process for shipping new release notes. Structure and guidance always!
Our solution: get out of the way
Deciding on the process was the easy part: engineers would write the notes and then would tag the content designer on call for a review. No one would be bottlenecked by whether I was at my desk or not. (I am always at my desk, obviously. Let’s make that clear. I’m usually playing Animal Crossing, but I am here.)
Creating the style guide was harder — there was lots to think about:
- Brand: Shopify has voice and tone guidelines. Shopify has a brand personality. App release notes should be in our voice: they’re written by us, about us. App release notes should be recognizably Shopify.
- Tone: Across our apps, neither our voice nor our tone were consistent — three apps, some with more personality than others, some with different audiences. Each app needs its own tone, but our apps affect people’s livelihoods. Our cardinal rule must always be clarity.
- Humor: Most people are funny. Everyone I work with is hilarious. But it’s hard to be funny when checkout is busted, or when the app is so bloated it crashes people’s phones, or when our last release had a critical flaw, or when you don’t know how your audience feels about something.
- Personality: the Shop app is not the same as the Shopify mobile app. One is for shopping, one is to make a living. Product differentiation is needed, but everything must first be recognizable as Shopify.
We needed to give the engineers enough process and guidance to write consistent messaging, but leave them enough scope to showcase a personality. And we needed to establish a codified process that meant a content strategist would always review the notes before they were shipped. We needed to give users a reason to actually care about release notes. We also needed to understand that users might not have time to read a brochure, but they probably also don’t have enough time to audit your new app release to check for new features.
Here are the rules we came up with:
- Be helpful and informative. Focus on important information that merchants need to know, without being overly technical or promotional.
- Use simple and plain language. Explain what changed and what the benefit is as efficiently and clearly as possible.
- Reflect the scale of the release. If it’s truly significant, tell people.
- Be specific. Never use generic descriptions like “Bug corrections and improvements.” If we fixed a problem or updated something, tell merchants what we fixed or updated.
- Be accessible to all. Merchants come from different backgrounds, locations, and levels of experience. We want everyone to understand us.
- Avoid being clever or congratulatory. We build products that affect people’s livelihoods: even if we played a role in their success, this isn’t about us.
- Avoid witticisms or humor. Always consider the audience’s perspective first. Shopify’s brand is not humorous and your joke might be stopping someone from getting back to work.
Enter Polaris
With finalized and approved guidance in place, the final question was whether to keep this guidance internal or publish it in our design system, Polaris. It wasn’t much of a discussion: transparency always creates better UX, and Shopify has a harmonious relationship with our partner ecosystem: if we can give some structure to the release notes for the ~4500 apps we have in our network then more to the good.
We say most of what they need to know right at the top of the guidance we published:
Let’s get out of the way.
For the engineers, we spent time articulating precisely the right hierarchy, style, and punctuation to use. However, our guides exist in context: they are guidelines, not rules to be slavishly followed.
So from here, we created patterns — repeatable but adaptable structures — that the engineers could use: updates should start “Update:” and bug fixes should start “Bug fix:”.
With patterns, we were also able to build various structures that could be referenced regardless of the kind of release notes that were being built: whether they included only an update, only a bug fix, both, multiple of each, or neither.
Our structures relied on our patterns:
The patterns, in turn, were created from humble beginnings: hierarchy, style, punctuation.
Maybe they’re not the sexiest words — but your users deserve to know what’s changed, how it changed, when it changed, and why to care. Release notes are one (of many) great touch points that we’ve overlooked for too long.
So, where we were:
We fixed a crash that occurred on some 32 bit older devices.
And now where we are:
Bug fix: we fixed a crash that sometimes affected iPhone 5 and 4th generation iPads.
Bonus game to play
A random sample of notes taken from across the iOS App Store. Use these to help convince your engineering team that fixing release notes is worthwhile: see if anyone can match the product to the release! (Products from the opening image in bold.)
The list is as published, punctuation included, parentheses excluded.
- We’ve made some performance updates and bug fixes to improve your experience. (League)
- This update includes bug fixes and performance improvements. (Workday)
- We made some stability tweaks and fixed some bugs. (Wealthsimple)
- We’re always making updates and improvements to make your experience better. (Audible)
- Performance enhancements and improved reliability. (Sonos)
- We’re always making changes and improvements. To make sure you don’t miss a thing, just keep your Updates turned on. (Spotify)
- The latest version contains bug fixes and performance improvements. (Instagram)
- We made improvements and squashed bugs. (Twitter)
- We update the app as often as possible to make it faster and more reliable for you. (Uber)
- You’re eligible for an upgrade! Now landing: bug fixes and improvements! (TripActions)
- We update the app regularly so we can make it better for you. Get the latest version for all of the available Facebook features. This version includes several bug fixes and performance improvements. Thanks for using Facebook! (Facebook) (248.1 MB)
- Further stability tweaks to make managing your listing and your travels as easy as possible. (AirBnB)
- Bug fixes and performance improvements. (Google Docs)
- Bug fixes and performance improvements (Fitbit)
- We update the app regularly so that we can make it better for you. Get the latest version for all of the available Messenger features. This version includes several bug fixes and performance improvements. Thanks for using Messenger! (Facebook Messenger)
- Bug fixes (Snapchat)
- Bug fixes, stability improvements, repairs to space-time continuum. (YouTube)
- Explore our latest effects and enjoy limitless fun! (TikTok)
- Han Solo made the Kessel Run in less than 12 parsecs, and we can make your streaming experience better with the tap of a button. Just update your app for the latest fixes. (Disney+)
- We’ve been hard at work making Discord better for you. For this version we’ve made a bunch of bug fixes and performance enhancements to make sure everything’s running as smooth as possible. (Discord)
- Pesky bugs have been exterminated. Continue as you were. (Tinder)
- We worked on various improvements and bug fixes (Venmo)
- Bye bye bugs! We’ve fixed up a few things for better flow so you can keep on being productive (and awesome). (Trello)
