Give This MIT Algorithm a Picture of Food and It'll Give You the Recipe

New software from MIT uses image recognition and a giant recipe database to tell you how to make the food in your food pics.

YouTubeMITCSAIL

By
Avery Thompson

Have you ever seen (or eaten) a delicious meal and wished you had the recipe to make it? Now all you have to do is take a picture and give it to an algorithm developed by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). Their algorithm can find a recipe for you based on nothing more than a single picture of the finished product.

Advertisement - Continue Reading Below

The researchers built their algorithm by combing through over a million different recipes collected from various recipe websites. They combined this recipe encyclopedia with image recognition software to identify the type of food in a photo and match it with its corresponding recipe.

In tests, the algorithm could identify the correct recipe 65 percent of the time. It was most successful with pastries and other baked goods because those were the majority of the recipes in the database and the ingredients were easiest to identify. The algorithm struggled with complex dishes like sushi rolls or dishes with ambiguous ingredients like smoothies.

Still, a 65 percent success rate isn't bad for a prototype, and the team believes that further refinement will improve the accuracy even further. In the near future, you could use the algorithm, dubbed "Pic2Recipe," to find recipes for all your meals.

The team is also considering next steps for the software, such as providing the option for users to specify some of the ingredients included in the recipe. The software could then return the precise amounts of each ingredient.

Advertisement - Continue Reading Below

"If you know what ingredients went into a dish but not the amount, you can take a photo, enter the ingredients, and run the model to find a similar recipe with known quantities, and then use that information to approximate your own meal," says lead study author Nick Hynes in a press release.

If you're interested in trying out Pic2Recipe for yourself, MIT built a demo webpage where you can upload your own photos for review.