Analyzing player choices in Episode 3 of Scarlet Hollow
In case you don’t know me (or the game I’m about to do a spoilers-heavy deep dive into), I’m Tony Howard-Arias, one of the co-creators of Scarlet Hollow, an episodic, choice-heavy horror adventure game with an unnervingly large amount of branching. The first three of our planned seven episodes are currently available to play in early access, and the first episode is available for free! I’ll be assuming that readers have played through that content throughout this post, so if spoilers are something you care about and you haven’t played our game yet, proceed with caution (or stop reading, play it, and then come back ;) )
Methodology and background
Since player demographics can change over time, and to give us the tools to take more interesting cross-sections of our data, we began by resurveying people about several major choices in the first two episodes of the game. Like previous player choice surveys, most of these questions reference something we call “the canon run,” which refers to the playthrough that a given player considers to be “the one that counts.” For some players, this is their first playthrough, and for others, it’s something they settle on after many runs.
Unlike other surveys, this time around, we also included questions about what choices players made across the sum total of all of their playthroughs. Since nearly 60% of our surveyed players had completed Episode 3 at least twice at the time they were surveyed, these additional questions gave us a chance to see what content people were interacting with.
Players could access the survey through a news post on our Steam page, via server announcements on our Discord, and from Tweets following the release of Episode 3. This creates a few caveats to our data-set:
- The timing of the survey means there was a bias towards players filling it out in the first week following Episode 3’s launch, or immediately following the Episode’s completion. Therefore, data influenced by play-count (how many times players finished, what content they saw across all playthroughs) is possibly skewed low.
- The places players could access the survey bias the sample towards the more devoted fans. (Possibly skewing the data in the opposite direction of the previous point.) We considered adding the survey to the end of Episode 3 to counteract this, but we wanted to devote that real estate towards pushing our players to join our community.
Total sample size at the time of this writing: 733.
The results
Starting with the high-level data, here’s trait pick rates for Episode 3—
And for the sake of comparison, here they are at the end of Episode 2.
As has been the case with previous surveys, trait preferences for the most part sorted themselves into a four-tiered system. There are a few differences, however —
- Keen Eye, Talk to Animals and Mystical all fell off by about 5%. Since they were our three most picked traits last time around, this is actually great, since it means more people are starting to see value in the lesser picked traits, either because of new content in Episode 3, or because of our ongoing series of between-episode balance patches.
- The bulk of those drop-offs were absorbed by Book Smart, which jumped 8 points and passed Mystical to become our third most popular trait. We’ve put a lot of effort into buffing Book Smart in particular, and Episode 3 was the trait’s big moment to shine so it’s good to see that play out.
- Powerful Build is still the least picked trait by far — it’s one of the ones we focused our attentions on for this month’s upcoming balance patch, so hopefully that’ll start to finally even things out.
We also asked our players to share with us what traits they’ve finished the first three episodes with.
In part due to our high replay-rate, these are a lot more balanced, with the biggest outliers being hot and street smart having outsized shares as bonus runs. On top of being our players’ least favorite trait, Powerful Build is also least-picked overall.
As additional baselines, we also have a breakdown on how players approached the major decisions in Episodes 1–3.
I’ve talked about it at length in previous posts, but for the most part, the big decisions in Episodes 1 and 2 didn’t quite lead to the splits we were hoping for. Episode 1 just barely passed the sniff test with a little of 20% of players (excluding those who could bypass the decision via Powerful Build) choosing to save Duke over Gretchen. This is actually a slight improvement over the last time we collected this information, likely because of Bo’s involvement in Episode 3. Hopefully this split continues to even out as more of the long-term consequences of the decision reveal themselves in the coming episodes!
Episode 2 had an even more dramatic split with only 11.4% of eligible players choosing to leave Alexis and Becka behind in the mines. This has skewed ever-so-slightly closer to a good split between releases, but it seems unlikely that needle will move anywhere close to even as the game continues.
We also surveyed what house players stayed at after the events of Episode 1, which continues to have a relatively even split, though that split continues to slant towards Tabitha over time.
Next, we’ve got some some of the lighter questions, including the fate of the boiled peanuts:
When given the opportunity, most players who’d taken them wound up deciding to abandon their oldest companion half-way through the game.
Believe it or not, the overwhelmingly majority of players decided they wanted to shower.
Moving on to biggest story decisions, we have some data on the “hangout” section in Episode 3, where players get the chance to spend one-on-one time with either their cousin Tabitha or one of the four romance options they’ve met so far. Street Smart players are also able to slip off before this to get some information on Wayne and the mines from Isaacs.
Since this is a Street Smart only option, we also wanted to look at this for players who selected it as a canon trait, and for players who played with it at all.
The most interesting thing here is the significant number of players who had Street Smart and didn’t realize this was an option, likely either because they didn’t meet Isaacs in Episode 2, or because they didn’t press the miners in his group on Wayne.
For the hangouts themselves, the biggest surprise is just how many people decided to stick with Tabitha — it’s definitely our favorite of those scenes, but we expected her to be a more controversial character with the fanbase (even if she’s our favorite) instead of a problematic fave that people are willing to lay down their lives for. Unsurprisingly, the pie chart lines up pretty well with the breakdown of people’s preferred romance options, too.
Stella is slightly underrepresented for hangout participation, likely for the same reason she’s the most popular romance option — players have already spent quite a bit more time with her than they have with any other characters, making them more likely to commit to her romance route, as well as more likely to branch out and spend a little more time with other characters, especially Tabitha. In fact, slicing the data to look at the hangout question specifically for Stellamancers, if they skipped Stella’s hangout it was almost exclusively to do Tabitha’s.
Oscar, Avery, and Kaneeka hangout rates all roughly perform on par with their romance participation, and then Tabitha likely gets an outsized boost from the none/Wayne/Reese categories.
And looking at the data, that’s true.
Speaking for a moment to the romance breakdown, something that came up quite a bit during the development of the first three episodes was that surely introducing Reese so late in the game would seriously impact his popularity as a love interest. We were pretty sure that wouldn’t be the case, and from this survey, it looks like there wasn’t anything to worry about, Reese is currently the second most popular love interest, and would likely be even higher up on the list if Wayne hadn’t been revealed. (We’re guessing there’s quite a bit of overlap between those two in particular.)
Everyone else is sitting at around 10%, which is a solidly even break-down. As time goes on, Stella continues to dip in popularity as well (dropping from 39% to 33%.) Anecdotally, some folks have fallen off of Stella specifically because they don’t want to step on their cousin’s toes by dating her ex. Likewise, expanding the dating pool, both by finally introducing players to Reese as well as surprise announcing Wayne as an extra romance option is always going to have a diluting effect.
Looking at cross-playthrough data, we see some interesting surges. Despite being barely ahead of Oscar and Avery on the canon list, Kaneeka has a more sizable lead across all playthroughs. The takeaway there might be that Oscar and Avery both have a more narrow appeal, but that the folks who take the time to explore those routes are more likely to be interested. Kaneeka’s hangout also has a more compulsive lead-in — you see her storm off earlier in the episode and have the option to check in on her — while Oscar and Avery’s hangouts are more passive.
Both Tabitha and Avery have almost the same split for ghost-hunting invitations, with 3/4 of players inviting them along. This high split isn’t surprising — getting to invite an extra companion along is always going to be viewed as extra content, even if we take palpable steps provide unique content for smaller parties. In the ghost realm, for instance, not having Tabitha means you get more interactions with Wayne. The effects of not inviting Avery are, generally speaking, less content in Episode 3, but that decision will have more lasting implications on the rest of the story.
Next up, the Big choice!
Episode 3 is the first time one of these major decisions has felt balanced to us. Excluding Book Smart players from the tally, 42.2% of players left the ghost behind, while the remaining 57.8% attempted to sacrifice years of their life to appease the spirit. Roughly a third of the players in the latter group were stopped by Tabitha.
There’s a lot of interesting stuff going on in this decision. For starters, let’s look at how these numbers change based on whether or not Tabitha is there.
Once you remove those pesky Book Smart players and slice the data by who you companion was in the haunt, we see two very different pictures. 72.5% of players who were accompanied by Wayne left the ghost behind to fester, while 70.3% of players who were accompanied by Tabitha attempted to make a sacrifice.
So the chips fell exactly where the other characters wanted them to — Wayne pushes you to leave the ghost behind, and nearly 3/4 of players listened to him, and in the other route, Sybil insists that Tabitha taking on the sacrifice to put the spirit to rest is the moral and correct thing to do. And nearly 3/4 of players in that route listened to her. Overall we couldn’t be happier with this breakdown. It’s still a decision in each route, with nearly 30% of players picking the other option, but those choices are also clearly shaped by the story.
It’ll be interesting to see if these numbers shift over time as more of the story is unraveled!
The persuasiveness of these characters also prompted us to go back and take another look at the major Episode 1/2 decisions, and lo and behold, the last character to speak before the decision is presented to the player correlates heavily with the split. In Episode 1, the last piece of information you’re given is that Gretchen is straining against her harness, and in Episode 2, the last piece of information you’re given is Rosalina pleading with you to save her friends. We’ve reversed that for the upcoming content patch in June, giving Duke the final say in Episode 1, and Tabitha/external narration the final say in Episode 2. Maybe that’ll sway things by a few percentage points! Who knows?
And finally, we thought it’d be neat to ask our players how scary they thought the game was!
The result was a nice bell curve that peaked at around a 7/10, which is exactly where we wanted things to be.
Finally, we did some fun little cross-sections of some of the earlier questions just to see how things would change.
We continue to see excellent retention and replay value too, with over half of Scarlet Hollow owners finishing Episode 3 at least once (Steam achievements) and over half of those surveyed finishing Episode 3 at least twice.
Hopefully this was a fun and informational read! If you haven’t had the chance, check out Scarlet Hollow on Steam, join our Discord, and/or follow us on Twitter.
Until next time!
