S3 E14 OpenAI Research Scientist Noam Brown on Solving Poker and Diplomacy with AI

Summary

In the intriguing episode S3 E14 of the popular podcast, guest Noam Brown, a research scientist at OpenAI takes the listeners on an exploratory journey, centered around his commendable achievements in the development of ground-breaking Artificial Intelligence (AI) systems. As an AI expert, Brown's significant contributions include using AI to demonstrate human-like performance in games like Poker and Diplomacy.

Noam Brown conferred the importance of games as a testing ground for AI research, partly due to their clean rule structures and scoring systems that facilitate a direct comparison between humans and AI. He emphasized the complexities of poker, a game of imperfect information that distinguishes it from a game like Chess, which in turn necessitates different AI techniques to mirror human expertise.

Noam delved into the intricacies of AI techniques for Poker, employing regret minimization and self-play techniques using 'Deep counterfactual regret minimization' strategy. According to Noam, the goal for an AI playing poker is to achieve a Nash equilibrium, a strategy that guarantees in the long run that you will not lose in a two-player zero-sum game. In addition to developing a strategy, he noted the importance of the AI's ability to generalize between similar situations, evaluating the present state of affairs and planning the next move, a process akin to a type of 'search' a human player does.

The guest speaker intrigued the listeners by sharing the substantial analytical shift in AI learning methods. According to Noam, the addition of a search strategy to the final round of poker boosted the AI's performance by the equivalent of a 100,000 times increase in the size of the training model, significantly outperforming prior results which only saw 10-100 times improvements.

Brown's dialogue then shifts from Poker AI to a new challenge, the game of Diplomacy. This seven-player game, conducted through private conversations involves political strategy and negotiation, offering an interesting study in conducting negotiation through natural language and accounting for human socio-emotional behavior. The goal was to develop an AI that could handle the complexity of the game, plan individual and collective strategies, deal with potential betrayals, and win the game.

Noam Brown concluded the podcast by recalling his journey into the world of AI research. Initially working in finance, he soon discovered his real interest in research, leading to a career transition to the Federal Reserve as a research assistant. Despite AI not being a prominent field back in 2012, his interest in the subject led him to pursue a PhD in Computer Science, specializing in AI and Computational Game Theory. He shared his love for board games and low-stakes poker, which continues to fuel his passion and inspire his groundbreaking research work.