(Bloomberg Businessweek) -- Artificial intelligence has come a long way in the 20 years since International Business Machines Corp.’s Deep Blue beat world champion Garry Kasparov in a six-game chess match, or even the six years since Watson trounced Ken Jennings on Jeopardy! Computers have beaten top human players at checkers, backgammon, poker, and go. Add to the list Super Smash Bros. Melee, a 2001 Nintendo Co. fighting game that lets you pit, say, Mario against Pikachu.
QuickTake Artificial Intelligence
Humanity has MIT researchers to thank for this defeat, chronicled in a paper they published in February, but Melee isn’t the only video game getting a lot of playtime from learning machines. AI software has cracked Super Mario Bros.; early Atari SA games such as Space Invaders; arcade mainstays Pac-Man and Mortal Kombat; even mobile favorite Angry Birds. Optimists say AI can help solve the world’s toughest problems, including cancer and climate change. So why are AI systems spending so much time gaming?
It’s all about data. Games allow AI software to tackle the kinds of complex logic problems found in the real world—uncertainty, negotiation, bluffing, cooperation—in carefully controlled environments, says Vlad Firoiu, who was part of the team that cracked Melee. Researchers can start their fledgling AI with relatively simple video game problems, run the tests thousands or millions of times, then gradually move on to more complex challenges as the system learns to handle the initial ones.
“With games, you can generate as much data as you like,” says Demis Hassabis, chief executive officer of DeepMind Technologies Ltd., the London AI company owned by Google parent Alphabet Inc. “You want to hit the sweet spot, not too hard and not too easy for your current algorithms.” Over the past three years, DeepMind has trained software to master Atari games and beat the world’s best players at go, a strategic board game popular in Asia.
Game environments are ideal for what’s known as reinforcement learning, says John Schulman, a researcher at nonprofit lab OpenAI, which is backed by the likes of Tesla Inc.’s Elon Musk, Y Combinator co-founder Sam Altman, and venture capitalist Peter Thiel. Reinforcement learning means the machines figure out proper sequences of actions through processes of trial and error. Again, the controlled environment helps: It’s much safer to teach an algorithm how to drive with a racing game than with your car. An engineer at Otto, the driverless-truck company owned by Uber Technologies Inc., has tried teaching AI software to play Grand Theft Auto V, as has a team from Intel Corp.
“We are very interested in what happens when you have several intelligent agents in the same environment. How do they interact? Can they learn to cooperate?” —Katja Hofmann, the Microsoft researcher who helped create Project Malmo
Hassabis says DeepMind has focused on games because it believes true general AI will have to understand its presence in a physical landscape. The simulated environments in games are a way to do this without having to build robots, which can be a pain in the ass, he says: “They are slow. They break. You can’t run faster than real time. You can’t run millions of them in parallel.”
Some influential executives have turned their noses up at game research. “We are not pursuing AI to beat humans at games,” Microsoft Corp. CEO Satya Nadella said at his company’s developers conference in September. Instead, he said, Microsoft had staked much of its future on AI as a means to help solve “the most pressing problems of our society and economy.” Chris Bishop, who heads Microsoft’s AI research lab in Cambridge, England, says making competitive games the benchmark for AI also reinforces fears that smart computers threaten people.
That said, Microsoft has created Project Malmo, an AI research environment based on the video game Minecraft, which it happens to own. Bishop says Minecraft doesn’t feed the man-vs.-machine narrative because it has no set objectives and isn’t necessarily competitive.
Knocks on the gaming model haven’t deterred other AI researchers. Besides Grand Theft Auto V, they’re using dozens of games as tests, including StarCraft II, Montezuma’s Revenge, and Freeciv, a free game based on Sid Meier’s Civilization series. But, as OpenAI’s Schulman says, the real trick is developing AI that can solve not one game but any game you give it. Then the technology might be ready for the game of life, and not the Milton Bradley version.
Game Theories
Researchers are training computers to beat dozens of video games. Different games help hone different skills.
Super Smash Bros. MeleeWho MIT researchersSkills Strategies for competing with multiple players and imperfect information
FreecivWho Arago, a German AI companySkills Memory, long-term planning, and compensating for imperfect information in turn-based play
Project Malmo (Minecraft)Who MicrosoftSkills Moving in 3D space from an embodied, first-person point of view, plus planning and cooperating with human players and other AI agents
LabWho DeepMindSkills Navigating a mazelike 3D world from a first-person perspective, remembering routes, accounting for multiple allies or rivals
Grand Theft Auto VWho Craig Quiter, a software engineer at Otto (the self-driving technology company Uber owns); Intel; Darmstadt UniversitySkills Driving—yes, driving, not murdering people
StarCraft: Brood WarWho Facebook and University of Oxford researchersSkills Navigating a complicated visual landscape; also dealing with imperfect information, memory, and long-term planning in real-time play rather than making players wait their turn
StarCraft IIWho DeepMind and othersSkills Planning and priority-setting akin to that in the earlier StarCraft game, plus active competitions involving some of the world’s best pro gamers
UniverseWho OpenAISkills Visualizing the pixels on a screen and manipulating a virtual keyboard and mouse
To contact the author of this story: Jeremy Kahn in London at jkahn21@bloomberg.net.
To contact the editor responsible for this story: Jeff Muskus at jmuskus@bloomberg.net.
©2017 Bloomberg L.P.
