The OpenAI team that’s been developing artificially intelligent agents who can play Dota 2 better than the pros decided to open access to amateur gamers to compete against its technology. The outcome has been predictably soul-crushing for anyone who is still enthusiastic about human intelligence. In 7,257 matches between humans and the OpenAI Five, the AI won 4,075 games, the humans abandoned (i.e., admitted defeat) 3,140 times, and a meager 42 games ended in the AI losing. That’s a win rate of 99.4 percent.
Fresh off their victory against reigning Dota 2 champion team OG, the OpenAI developers felt confident enough in the robustness of their agents to let laypeople interact with the AI. Aside from a competitive humans vs. AI mode, the OpenAI team also allowed players to participate in cooperative teams that mixed humans with AI. That’s potentially the most entertaining application for these vastly superior Dota AI players, and, indeed, people seemed to favor the cooperative option, playing more than 35,000 games in that mode.
Still, the stats illustrating the crushing superiority of the OpenAI agents are eye-opening. It took more than six hours of play before a human team claimed a victory in the 459th game. Only 22 out of 3,193 human teams ever claimed a victory against the AI. The total playing time between humans and the AI equated to 10.7 years, which is impressive until you learn that OpenAI Five generates this much data every 12 minutes of training by playing against itself.
Now that the amateur human challenge period is over, the OpenAI team will review the data gathered and “will use Dota 2 as a research platform for further development of powerful AI systems.”