Creating Recipes with Artificial Intelligence

IBM’s cognitive computing team takes on a new frontier: meal planning

Now that Watson, IBM’s “Jeopardy!”-winning cognitive computing system, has left the game show circuit behind, it’s finding ample use for its talents in health care, finance, and other industries. Watson has quite a knack for combing through vast troves of data to find a fact or recommend a course of action based on knowledge and past experience.

And back at IBM’s Thomas J. Watson Research Center, in Yorktown Heights, N.Y., a research group led by electrical engineer and computer scientist Lav Varshney is pushing Watson to the next level. Varshney and his team don’t want a system that digs up a preexisting answer; they want to build a creative system, one that can come up with something new. And they want that something to be tasty.

Working with chefs at the Institute of Culinary Education in New York City, the IBM team is building a cognitive computing system that, given plenty of data about what foods most people already like, will be able to come up with inventive, delicious flavor combinations—far beyond the familiar pairings of tomato and basil, chocolate and strawberry, chicken and rosemary.

Recipe-generating programs already exist, but they are generally simple and case based, substituting new ingredients based on flavors that have worked well in other dishes. The IBM project is trying for something much harder: a system that comes up with radically new dishes based on combinations of ingredients and flavors that people would probably find appealing but may never have been put together in a dish before.

Here’s how it works. To begin, Varshney [left, in photo] and his team set up a large database of recipes from cookbooks, online recipe banks, and even the U.S. Navy (which, though not renowned for gourmet fare, makes some of its recipes publicly available). This collection is what’s known as an inspiration database, a pool of existing creative endeavors—recipes, in this case—that the system can draw from. “Then you can do a lot of transformations on those recipes,” Varshney says. “You can take two different recipes and mix them together; you can substitute ingredients; you can change the cooking processes.” The system takes those alterations as inputs, and then it knits together new recipes until it has—depending on available computing power—“a million or a billion or however many you want.”

But of those myriad recipes, many would likely prove unappetizing, and few would be both delectable and novel. So the system’s next and biggest challenge is to sort the wheat from the chaff—or perhaps the anise-infused wheat-berry salad from the nopal-cactus-and-halibut kebabs.

To do this, it first evaluates the taste-imparting molecules in each new combination of ingredients. Such flavor compounds might include, say, the fruity allyl propionate or coumarin, a chemical in green vegetables that has been described as “the taste of green.” The program then approximates what the newly meshed flavors taste like by comparing their component chemical structures with similar mixtures already in the database. To determine whether a new taste is appealing, the program is armed with a large body of research that has measured how much people like the flavors imparted by different combinations of compounds. If people tend to enjoy flavor combinations that are chemically similar to the new one, the computer can deduce that they’ll probably take pleasure in it, too.

To determine which flavor blends are truly novel, the system simply checks them against existing ones. Some mixtures will be entirely new, not found anywhere in the system’s database of recipes; others could be new within the context of a particular cuisine. For instance, they might introduce flavor pairings found in Indonesian food to an American-style dish.

Once the system has homed in on a promising recipe, Florian Pinel [right, in photo], a member of Varshney’s team who is both a software engineer and a professionally trained chef, will sometimes whip up a batch. “Last year we had a few things that were pretty good, like a turmeric-and-ginger grilled eggplant sandwich with a berry sauce,” Varshney recalls.

The system is still experimental, but Varshney is already envisioning real-world uses. Could it help improve school lunch menus? Definitely, he says. Could it create a range of recipes low in salt and fat for people with cardiovascular disease? No problem. What about dinners for a lactose-intolerant diabetic looking to incorporate specific seasonal produce into her diet? Eventually, Varshney believes. “Here you can create whatever you want under the constraints you set, rather than just having to work with what’s already there,” he declares.

But the biggest limitation of IBM’s approach may be that it oversimplifies the pleasures of eating. Our opinion of a meal is shaped by our feelings and memories, the people surrounding us, and the textures and presentation of the food in front of us—all things that are hard to capture in an algorithm.

Nevertheless, it’s clear that Varshney’s system is on track to create an inexhaustible recipe book of mouth-watering, never-before-tasted dishes. This artificial intelligence could very well prove that computers can outcompete humans at yet another cognitive craft we thought we alone had mastered.