Collective AI: A Conversation with MIT’s Jeff Orkin

Improviso GameYou’re the director in an ultra-low-budget movie set around an alien encounter. You select scenes, summon special effects, and play the supporting cast members: G-Man, Scientist, Alien, and Little Girl. Your lead actor Ted is a struggling reporter who wakes up tied-up on an alien spaceship. What do you do? You improvise.

Improviso is the brainchild of Jeff Orkin, an artificial intelligence researcher working under Professor Deb Roy at MIT’s Media Lab. It is an outgrowth of an earlier game called The Restaurant Game, in which Orkin recorded 16,000 people playing the roles of customers and waitresses in a virtual restaurant. Improviso is built using the same code base and Torque 3D game engine from GarageGames. It currently runs on Windows, but a Mac OS X version is almost ready to release.

Improviso explores how ordinary people can engage in dramatic improv. Lasting about 15 minutes, each scene connects the lead actor, director, and supporting cast. These characters interact and assume roles to create cheesy-but-cool low-budget scripts. While many role-playing games ask players to step into the shoes of an orc, elf, wizard, dwarf (or some other fantastic character) to collect points, gold, mana (or some other reward), very few ask users to act, improvise, explore motives and language, and even go off-script.

The goal, according to Orkin, is to teach AI game bots to imitate the texture of human dialogue and interaction by collecting gameplay traces from real human users over the Internet. Orkin and Deb view their work as a promising first step towards collective AI-driven agents that can interact and converse with humans without requiring programming or specialists to handcraft behavior and dialogue.

Jeff OrkinOrkin leads the Improviso project in collaboration with the Singapore-MIT GAMBIT Game Lab, a joint effort of the Massachusetts Institute of Technology and the government of Singapore created to explore new directions for the development of games as a medium. Orkin’s Ph.D. thesis is an artificial-intelligence system that generates character behavior and dialogue based on the data captured from the improvised interactions of thousands of players.

Surf-D: What was your motivation in creating Improviso? How do the goals for Improviso differ from those for The Restaurant Game in furthering AI research?

Jeff: I’m really interested in characters that can play roles, interactively, based on lots of human demonstrations. Data collected with The Restaurant Game teach characters how to play a role in society, for example, a customer or waitress. Improviso collects data from people playing roles in a story — something that is completely divorced from our everyday experience.

When Improviso asks people to play the role of an Alien, or a government agent in Area 51, they can’t draw from their shared cultural knowledge as they can in The Restaurant Game. However, there’s likely some consistency in the data based on our shared pop-cultural knowledge of science fiction. We’ve all grown up with the same movies, so we know what happens when the aliens crash land on Earth — we know that they can phone home by hacking our electronic gadgets, we know that they are addicted to cat food, we know that some are shape shifters than can change their appearance, we know that their alien babies suction themselves to people’s faces, and so forth.

Surf-D: What has been done with the interaction traces from the Restaurant Game? How far along are you in building generalized script of typical restaurant behavior and dialogue?

Jeff: Early work focused on automatically learning behavior and dialogue, and we made progress toward that goal — we automated customers and waitresses that could interact with each other and with humans guided by statistical regularities, and they often did the right thing, but not always. Ultimately, I came to the conclusion that unsupervised machine learning would never be able to give game designers the control they need over the behavior. More recently I’ve been working on a human-machine collaborative system that drives behavior and dialogue based on gameplay traces that have been annotated with meta-data that better informs the AI system, and gives designers control over what is being learned.

Surf-D: Why did you select to partner with Singapore, and where do you see GAMBIT going?

Jeff: The GAMBIT Game Lab is a fantastic resource at MIT that explores research questions by building experimental games. Because my research is based in virtual worlds, working with GAMBIT was a natural fit. Developing games in an academic environment is challenging; collaborating with GAMBIT allowed me to work with a team of skilled artists and programmers to create something more ambitious than The Restaurant Game in terms of scale, and something that stands out stylistically.

Singapore funds GAMBIT because they recognize that the technical innovations coming out of MIT have the potential to differentiate games in the future, and exposing Singaporean students to cutting edge games-related research fosters creativity in the growing game industry in Singapore. GAMBIT will no doubt continue to make one-of-a-kind games, and to inspire students to innovate professionally in the game industry in Singapore and the U.S.

Surf-D: You have said that games are becoming a viable alternative to traditional academic publishing. How so?

Jeff: Academia has become very fragmented, and crowded with many sub-communities and conferences, which can make it difficult to cut through the noise and get new ideas out into the world. Games are a better way than traditional academic publishing to connect with, and share ideas with, the general public and with the game industry. I wouldn’t say they replace traditional publishing, but they can be an effective way to generate interest in reading the publications to get more detail.
For people working on games research, I think it is important to get experimental games out of the lab and into the hands of the public, just as film students are expected to make films and show them at festivals or online.

Surf-D: How do you view the future of Collective AI: do you see the “global we” of the Internet teaching future AIs how to behave like humans and pass the Turing test?

Jeff: Generating rich human-like interactions and dialogue is ultimately a content problem. I do think that today’s world — with massive amounts of cheap storage and fast network connections — can solve this content problem by harvesting the imaginations of game players around the world. However, my goal is not to pass the Turing test. The Turing test implies that we are trying to deceive people into confusing humans and AI characters.
When we play games with incredible graphics, we don’t confuse the games with reality, but the graphics fidelity allows us to suspend disbelief and immerse ourselves in the experience. Character behavior in games today rarely matches the fidelity of the visuals, and this is jarring — it detracts from the immersive experience. What I’m after with Collective AI is characters with behavioral fidelity high enough to suspend disbelief.

Surf-D: If you had unlimited funding, what follow-on research and/or game creation do you see yourself doing?

Jeff: Collective AI is really about democratizing the process of creating characters that can play roles in games and virtual worlds. There are many potential applications for socially intelligent characters, but the barrier to entry is too high due the labor intensive, highly technical ways characters are authored today. With unlimited funding, I would open the platform to other developers and researchers to record gameplay and to automate characters from shared data, in hopes of fostering more sophisticated characters in a wide range of applications, including new entertainment and educational experiences we’ve never seen before.

Leave a Reply