Almost 10 years ago, the online phenomenon “Twitch Plays Pokémon” convened over a million people to play Pokémon Red at the same time, with each player’s keystrokes registering as commands for the one pixelated avatar. Now, like a Magikarp growing into a Gyarados, the evolution of technology begs a new question: can AI play Pokémon?
For the last few years, Seattle-based software engineer Peter Whidden has been training a reinforcement learning algorithm to navigate the classic first game of the Pokémon series — in that time, the AI has played more than 50,000 hours of the game. Whidden posted a 33-minute YouTube video telling the story of the AI’s development, and after nine days, the video has amassed 2.2 million views.
“What’s been super fun to see is how many people are engaging with it,” Whidden told TechCrunch. He uploaded the code he used to GitHub, along with instructions on how to operate and train the AI. “There’s a ton of people that seem really interested in actually doing this process of creating or designing.” One fan was able to apply his code to Pokémon Crystal, another retro Game Boy installment.
The AI’s reinforcement model is Pavlovian, giving the AI point-based incentives to level up Pokémon, explore new areas, win battles and beat gym leaders. Sometimes, these incentives don’t perfectly align with progression in the game, yet the failures of the AI are weirdly charming, which is probably why Whidden’s video has gone viral.
In one of the AI’s attempts, it simply stops to stare at the water in Pallet Town — the first place you visit in the game — and never moves. It gets stuck in an area with animated water, grass and NPCs who pace back and forth, meaning that every individual frame seems like a novel experience to the AI, even though it’s just sitting motionless without even getting its first Pokémon yet. But this AI isn’t in a rush to “catch ’em all.” It’s just enjoying the beauty of the Kanto region (or maybe it’s taking an ethical stance against forcing these cute little animals to battle each other… who can say).
“So, according to our own objective, just hanging out and admiring the scenery is more rewarding than exploring the rest of the world,” Whidden explains in the video. “This is a paradox that we encounter in real life: curiosity leads us to our most important discoveries, but at the same time, it makes us vulnerable to distractions and gets us into trouble.”
The AI somehow continues to pull on our heartstrings: Later, it experiences something akin to a traumatic event at the Pokémon Center. The AI’s success is measured in part by the total levels of all Pokémon in your party. But when an AI goes to the Pokémon Center and button smashes enough to deposit a Pokémon into storage, the sum of all levels reduces drastically, sending a strong negative signal to the AI. With both Pidgey and an unidentified creature nicknamed “AAAAAAAAAA” in its party, the sum of all levels was 25, but once Pidgey is deposited into the PC, the sum is only 12.
“It doesn’t have emotions like a human does, but a single event with an extreme reward value can still leave a lasting impact on its behavior,” Whidden narrates. “In this case, losing its Pokémon only one time is enough to form a negative association with the whole Pokémon Center, and the AI will avoid it entirely in all future games.”
Despite the AI’s ability to experience trauma and admire the pretty pixels of Pallet Town, it’s still just a computer. This AI isn’t able to read and interpret dialogue in the game, so in early iterations, the program would get stuck at an early crossroads in the game. When you reach the second town in Pokémon Red, you’re given an item to bring back to the Pokémon Professor in Pallet Town. But the AI was having a hard time backtracking to deliver the parcel, making it impossible to progress further. So, Whidden skipped ahead to make each game begin after delivering the package, and with Squirtle as the AI’s starter Pokémon, since the early game is generally easier with a water Pokémon at your service.
“In the video, the farthest that [the AI] reaches is Mt. Moon, between the first and second gym,” Whidden told TechCrunch. Caves are notoriously frustrating to navigate in early Pokémon games, even if you have an actual human brain. But Whidden recently tweaked some of the rewards in his code and tried a different learning algorithm, and finally, the AI managed to exit the cave and arrive in Cerulean City.
Other researchers have used reinforcement learning to study the use of AI in gaming, like with DeepMind’s AlphaGo, which was the first computer program to defeat a professional Go player. But Whidden’s video has garnered so much attention because he’s so adept at explaining unfamiliar concepts through a familiar medium: Pokémon.
Comment