Finding a New Balance Between AI and Beneficial Bugs

Pharah, one of the heroes in Overwatch, has an emote where she says, “Rocket jump? That sounds dangerous.” Almost thirty years before Pharah said that, in Quake, some players discovered that shooting a rocket into the ground would damage them but also provided a jump boost, allowing them to reach higher places and thus having a combat advantage.

Using the word “bug” is almost like giving your interlocutor a prompt, inviting words like “stomp,” “crush,” or “wrangle” to fill in the blanks. But bugs are an integral part of games culture and the social networks that games build around themselves. The ability to shoot a rocket into the ground to jump higher has now become a staple feature of many first-person shooter games, and the popularity of that bug allowed Blizzard to gracefully wink at those players in the know, endearing them to a character that has otherwise little time to be developed.

In the field of Quality Assurance, bugs are defects of a larger or smaller severity, that can negatively affect the intended experience of the user or possibly even prevent them from experiencing the product in the first place. In extreme cases, such as security flaws, bugs can even have a lasting detrimental effect on players lives. Countless hours of hard work, effort and creativity are poured into minimizing the potential that a user might encounter these bugs, and countless people should be celebrated for their work in maximizing the user experience.

But in the field of video games, there must be some space left to sing the praises of bugs. They have been an integral part of developing the field, either through those bugs that have become genre-defining features or works of glitch art and the niche subculture of fan-made works of fiction exploring the terror that emerges when a beloved childhood video game is glitched to horrifying levels. Not to mention the culture of speedrunning and the competitive and artful purposeful exploitation of bugs, sometimes for the greater good. Sometimes, bugs are the ultimate “collectible” in a game. One might think of easter eggs in this vein, but easter eggs are intentional. The developer knows they are there, and like the rest of the game, they are a vehicle of communication between them and the players. But when players find a bug, they belong, for a moment, just to the players. They offer a window for them to appropriate a part of the game, congregate around it and share it with their friends and communities. If easter eggs and intended mechanics are the means through which designers and developers communicate with players, bugs are the means through which the game communicates with them. And it is at this bare interface between the game and the player that art, relationships and culture can evolve.

As participants in the field of video games, we owe a debt to these temporary breakpoints between the crafted, intended experience and the software’s inner workings. And more so, we must give our thanks to those folks working to ensure that we can see bugs in this positive manner, because without whom, the memorable glitches would not be culture-producing, but destructive. Quality assurance work asks much of its practitioners, particularly in the field of video games. It asks for perseverance ensuring that each part of the video game is not only functional, but also visually and auditory accurate throughout multiple configurations of hardware specs. It asks for creativity in trying to achieve tasks in ways not only unintended by the designers, but even implausible for the task to be performed. It asks for analytical skills in determining the spot where the defect occurs, its conditions and the impact that it may have when encountered by a player. It asks for QA professionals to repeatedly assume the roles of potential players, even when the game’s complexity leaves room for myriad player personas to fill its space. These calls are answered with hard work and time investment.

Modern video games strive for complexity, emergent gameplay and replayability, all packaged in and enhanced by flawless and pleasing graphics and sound design. Each of these aims comes with increased financial investment and increased demands on the makers of those games. Iteration loops become tighter, risks become higher and the requirement for quick responses becomes more stringent. In these conditions then, the responsibility of empowering game makers to achieve their goals should also increase. As the scale and depth of games increases, QA professionals will still be asked for perseverance, creativity, analytical skills and empathy, and so they must be allowed the space to exercise them. AI-driven QA tools can help do that by shortening the loop of performing a large battery of tests, by diversifying the use cases that they can analyze and act on, narrowing the possibility space where the breakage might have occurred, providing quick insight into ways of engaging with the game that might have been unintuitive and allowing an overview of the impact of the defect upon the experience at large.

Game design has always balanced the edge between player freedom and designed experience, which naturally puts pressure on the demands for testing. Increased player freedom adds pressure by exponentially increasing the action possibilities as well as their sequential combinations, which can sometimes be humanly impossible to parse in their entirety. For example, some years ago, I was playing Alan Wake. I came to a point in the game where I was not doing great, both in terms of health points and ammo. In this sorry state, I passed an auto-save checkpoint right before a fight. In my state, the fight was impossible to win. I was stuck in an auto-save respawn loop out of which I could not get out, and I could no longer progress in the game. I quit the game and have not returned to it since, nor have I played the sequels to Alan Wake. This is partly because of that bitter taste the experience left and because I feel like I have missed out on too much of the story and I would be lost. Having my experience safeguarded would have meant, in this case, that the unlikely sequences of behavior that I engaged in, with the same skill level that I had (that had me lose so much HP and ammo) would have had to be performed by a tester, at that specific checkpoint in the game. That tester would have had to do the same thing for each checkpoint under each variation of skill and action sequence. The conditions that led to my scenario would have taken a tester an incredible amount of time to find, if ever. But for me, that was the entirety of my experience with the game. That said, parsing such large amounts of data is all in a day’s work for AI. Such tools can provide testers with the capability to detect and call out occurrences that may have a 0.01% chance of happening but that affect 100% of a player’s experience.

So then where do we stand on bugs? Do we thank them, or do we stomp them? I think that actually is the wrong question to ask. What is needed is space and time for creativity, for understanding the potential of those bugs and celebrating the players and game makers who play in the meeting space between person and software. We will never be able to eliminate all bugs. Players will find them, and they will make them their own. But if QA professionals are not provided with the tools to diminish the presence of destructive bugs, we risk keeping those players away who might find the next rocket jump.

About the Author

Dr. Miruna Vozaru is a Game Analyst PhD who has worked in the industry as a researcher, teacher, level designer and level analyst. She is currently the Test Specialist at modl.ai where she is responsible for the company's wide range of QA needs, including game engine plugins, platform, games and AI agents.