Introduction

At the core, this is a cookbook application. In addition to maintaining a list of recipes, it allows the user to discover, create or share recipes. It also assists the user through the process of planning what to cook, shopping for the ingredients and the actual process of cooking.

Background

There are countless recipes available in the form of videos, blogs, embedded in apps and many other places. This is the process that started off the idea to create an app that is more than just being a cookbook.

It's Friday, these are the thoughts going through my head, I would like to have something spicy, filling and greasy (tired of the controlled diet I have been (hypothetically ) following week long).

I decide on Chicken Biryani . I have the option of either take-away or going to a restaurant. But having been there too many times (I know it conflicts with the controlled diet ). I would like to prepare that myself. A search online gives me the required instructions.

So I embark on to prepare Chicken Biryani at home.

(Potential Problem) There are too many cookbooks out there with the same or slightly varying information. Which one to follow?

After reading about preparing Chicken Biryani from various sources. I pick one to follow.

(Potential Problem) I read through the ingredients list and search in the Kitchen to make sure I have all the required ones.

(Potential Problem) In case I am out of stock of an ingredient, I need to find an alternative or know enough to skip it safely .

(Potential Problem) If I am missing a few main ones (like Chicken ), then I need to go shopping.

(Potential Problem) I need to prepare a list either in my head, paper or a shopping list app.

(Potential Problem) Once I am back home, I start cooking. I need to check through the steps a few times to make sure that I am following the said procedure.

We do have the technology and the devices to help us through the above said process. There are applications that do help us solve each of the identified problems, but the fact that each one is a different application and the integration between them doesn't exist is a major problem.

Please note that a few more identified problems have been omitted as they haven't been implemented yet.

Flavours tries to solve all the above identified problems.

What Can Flavours Do?

Contains collection of recipes

Suggests recipes based on your interests (includes a Learning algorithm to guess what you may like)

Allows you to create new recipes and publish them for others to use

Allows you to rate and give feedback on recipes

Helps create a cooking plan which contains what to cook and when

Helps create shopping lists based on ingredients from the recipes selected to cook

Keeps track of your groceries, so that common ingredients like salt are not added to your shopping list.

Reminders when you leave from work or home

Alerts when you are in the vicinity of your shopping center based on your groceries list

Walks you through the cooking process, step by step, with timers for cooking and plays your favorite music while you are at it

Take pictures of your creation, while you are cooking and after a recipe is done and upload to Facebook

Architecture

Flavours is available in two flavours as a Windows 8 store application and an Intel AppUp application. The services are developed using the ASP.NET Web API framework, with a SQL Server as the Database.

Windows Store application will be developed using XAML and C#. The Intel AppUp application is a WPF application.

Flavours uses Sensors

Multi Touch - Gesture driven activities are used to take user input. Creating new recipes relies on gestures and other sensor’s input rather than the traditional keyboard and mouse. Pinch an object anywhere in the application to pick-it-up.

GPS - Shopping reminders. User can choose to get alerts when in the vicinity of a shop. Users can also setup reminders to be triggered when they leave from work or home.

Ambient Light Sensor - Adjusts the contrast of the application based on the ambient light so that the application is pleasant to look under different light conditions.

Inclinometer - Tilt the right edge to navigate to the next step and tilt the left edge to navigate to the previous step.

Gyro and Accelerometer - On the recipe creation page. Shaking the device is taken as a step to perform shake on the ingredients. Rotating the device is understood as mixing the ingredients and fast rotation is taken as mixing thoroughly.

Flavours Explained

Flavours combines features of a cookbook, shopping list with reminders, kitchen timers and provides a fun way of creating and sharing new recipes.

Starting the app for the first time presents the user with four different groups (Featured Recipes, Cooking Plan, Shopping List and All Recipes). New sections will be added based on user's activity, Favourites section will be added.

The first time, the Featured Recipes are selected randomly and will update based on user’s activity. Cooking Plan contains recipes picked to be prepared later. These are selected recipes that the user plans to prepare at a selected time. Shopping List is populated based on the ingredients in the cooking plan.

Preparing a Recipe Step-by-Step

On viewing a single recipe, the user can choose 'Start Preparing', and the application switches into step by step directions mode. Inclinometer is used for navigating while cooking to avoid grease marks on the screen or keyboard. The user is presented with the following screen with instructions on using tilt for navigation. The images animate to show the direction of tilt required for moving forward and moving backward.

Content removed from these screenshots. The tilt process is a storyboard that is set to run forever, i.e., restarting on completed. These instructions are shown only when Inclinometer is identified.

Using Inclinometer data in our application requires the following steps:

Add a using statement for Windows.Devices.Sensors

Identify if Inclinometer is present and get an instance of the Inclinometer object. - Inclinometer.GetDefault() returns an inclinometer instance. If the result is null, it means that the inclinometer is not available. Based on the presence of the inclinometer, instructions for tilt can be toggled. When the inclinometer is not available, show a message saying ‘Flavours can be fun with Sensors. Unfortunately the required sensor has not been identified.’ which disappears in a few seconds without user interaction.

Set the ReportIntervalon the in milliseconds. This also drives the sensitivity (Refer to MSDN for detail on the sensitivity). While we can set the interval, it cannot be guaranteed, as the device driver has the final say.

Subscribe to the ReadingChangedevent to get a notification when the inclination has changed.

In the event handler, the argument is of type InclinometerReadingChangedEventArgsand contains information regarding the inclination in three degrees.

When leaving the page or to stop reading, unsubscribe from the event and set the report interval to 0 and discard the inclinometer object.

The Roll value of the tilt is saved in another variable (_prevRoll), when there is a difference beyond a threshold between the _prevRoll and _currentRoll,it is identified to be a tilt. Positive and negative values are important as they define the direction of the tilt. We get a positive value if tilted by lifting the left edge of the Ultrabook. (Based on documentation, needs to be tested).

Shopping List

On the main page, clicking/tapping on the shopping list header takes the user to a shopping list group page. Here, the user can pick a favorite shopping center and choose to be notified when around 5miles of the area. The user can add more markers like Work, Home, etc. and set up reminders for shopping when going to them or leaving them.

Note: The above screenshot is not a completed product. It hasn't been worked on with a designer's hat yet.

The user can delete shopping list items by swiping across the item. If the user chooses to add new item to the shopping list, he can do so too. This makes the app usable just as a shopping list application. When the user taps/clicks ‘add new item’ button in the app bar, a list of all available ingredients for cooking along with their images are shown, user can then pinch one of the item he/she wants to add to the shopping list and tap on the list to add it to the list. Hint text says ‘PINCH to pick-up, TAP to drop’.

Create a New Recipe

User can add recipes to the collection and share them with others or post on Facebook. From the main page, Clicking/Tapping on ‘add new recipe’ button in the app bar brings the user to this page.

The User provides Title and Description of the recipe which are the only two fields populated from on screen or a physical keyboard. The user is presented with a list of possible ingredients with images on the right. User can pick an ingredient using Pinch gesture and drop on the worktop area by Tapping on it. The following are generic steps in a recipe, and each step is identified by the user’s interaction with the sensors.

Chopping (Fine, Coarse, Vertical and Horizontal) - Say, the user picked a Lettuce from the list and drops on the work area (designated area on the screen). Info cards are used to represent that the items are placed on the work area. Using gestures, the user can draw the direction of the cut. Once completed, the user picks up (Pinch) the info-card from the worktop and places it aside (this is another designated area for the recipe steps), by tapping. This action notifies that a step needs to be added to the recipe procedure and an ingredient needs to be added to the required ingredients section of the recipe.

Mixing - User can pick existing items from the steps and drop on the work area. For example, if the user has already chopped a lettuce, cucumber and a tomato, they appear in the list of steps, the user can then pick them from the list of steps and place on the work area. A wobbling motion of the Ultrabook is recognized as a mixing step. When the user rotates it faster than a threshold, the step becomes ‘mix them thoroughly’.

Frying - User can pick a step from the procedure or pick new ingredients (that is either frying chopped or whole ingredients). Then gently tilting the Ultrabook forward while holding the area beside the trackpad, denotes frying the selected items. To explain it better, assume that the Ultrabook is a pan and the motion of tossing objects in the pan that is required to register it as a Frying operation.

Deep-frying - Action to be performed is similar to frying but the recognition threshold is higher, Requiring the user to move the Ultrabook faster.

Similar actions can be designated to other activities in the preparation process. Other identified actions include but may not be limited to Stirring, Cooking, Boiling, Grating, Grinding, Thaw, Grill, Cleaning, Microwave, Blending, Whisking, Oven, etc. For the steps in the procedure that cannot be directly interpreted from a gesture or sensor input, are provided with an icon which the user can drag on to the work area to perform. All actions that require sensors are confirmed before proceeding, giving the user a chance to cancel it if required.

Sharing Data

Recipe, ingredients and shopping lists data can be shared with other applications formatted as strings, JSON or XML. This will allow the user to continue using his/her favourite shopping list application while using the functionality of Flavours.

Sharing with other users can be done by publishing the recipe online. The app has a service (ASP.NET WEB API), that caters to the data needs of Flavours. Users can share recipes on Facebook as well.

Identifying User’s Interests Over Time

User’s activities such as selecting a recipe or going through the process of cooking a recipe are taken as inputs. Logistic classification algorithm is used to give each recipe a ‘may like rating’. The ingredients and preparation steps are used as parameters of the algorithm. User’s favorite recipes and prepared recipes form the training set. Each time the user prepares a recipe or marks one as favorite, the theta vector (constants in the algorithm that define the performance) are re-calculated with the new set of information.

The preparation process is given health rating and a factor for health is included into the suggestion process. For example, the user prefers deep fried food, but it is not a healthy choice when compared with salads, in that case both options may be presented so that the user can make a decision based on the calories and the health benefits.

Please note that more code snippets which explain sensor data calculations, will be added once the section for creating a recipe is implemented and tested. Rest of the application is normal code and hence omitted.

Which Color for Flavours?

The metro colors look good and I have been having a tough time picking one single color for the application. Here are a few icons, let me know which one you would like to see. The whole app uses the same color.

Blue - Currently set to this color.

Red - Stands out from the crowd among the tiles.

Dark Green (colors names for our identification only

Light Green

Orange

Code Usage

The following segment of code explains how Pinch gesture has been used for selection of an ingredient. This code is from Flavours User Experience Test project which was used to test the theory (that gestures can be used as events and the pinch feels okay to select) and ascertain that the gesture can be used for selection.

Generally a Tap / Click is used to select an item on the page. We have used Pinch to give the user a feeling of picking an ingredient and placing on the worktop area. Pinch gesture feels like picking grains off a surface.

To handle gestures in the application, we need to handle the Manipulation events and that is all that is required.

XAML Page

The XAML text in bold below shows the event handlers for Manipulation events.

ManipulationDelta gives the relative values for that particular delta. Which means that we won't be able to decide whether it is a Pinch or a Stretch by just a single Delta's values. Sum of individual deltas is used to decide if it is a Pinch or Stretch. Cumulative property can be used, but using a cumulative in the ManipulationDelta event can fire false positives when a single finger is used as well. If the Cumulative is used from ManipulationCompleted event then the UI is not responding immediately to the user's actions. UI responds after the fingers leave the surface, i.e., when the ManipulationCompleted event is fired.

Geolocation and Bing Maps Usage

Geolocation is used in Flavours to set up reminders based on the geographic location. When the user has a few items in the shopping list, Flavours automatically reminds the user when leaving the house or from work. Optionally other types of reminders can be set up, like remind me when I am within 5miles of the groceries shop.

This is how to use geolocation data from the device and use the positions in Bing Maps. Please note that the key provided is a test key generated for this app and may not be active when you use it.

When a position change is notified, we can check the distance from home, work or the shop and remind the user that he/she has items in their shopping list that need to be purchased.

Next

The project is currently being ported to WPF. I will post a follow up article about code sharing and porting the application to WPF.

Source Code

You can download the source code made available from the link at the top of the page. This is NOT the application's source code, while it is the code that has been used to test individual features as a proof of concept.

Note: Using Bing Maps SDK

In the project's build configuration, make sure that the Active Solution Platform is NOT 'Any CPU'. Any of the other configuration which defines the platform specifically works fine (that is ARM, x86 or x64).

License

Share

About the Author

I am passionate about software development and can't wait for the time when everything around us exposes an API for us to develop against.

I have been developing software using the .NET Framework since the last 10years. I hold an MSc in Advanced Software Engineering with a University Medal for Outstanding Academic Performance. I also picked up a few MCP and MCTS certs.

I follow Machine Learning and Artificial Intelligence closely and have completed a course online from Stanford University through Coursera by Andrew Ng.

I have developed and deployed a mobile application on iOS. The application is a telecommunications application allowing users to create a recording and send out voice message blasts to tens of thousands of contacts at once.

I play the role of an Architect and Software Developer. I have been delving in ASP.NET, WinForms (not anymore), WPF, Silverlight, ASP.NET MVC, Azure SDK and other .NET technologies around them like SharePoint. SPA with WEB API is currently the technology I am working on.

A Round 1 winner in App Innovation Contest 2012, but couldn't make it to the second round due to the lack of Code Signing Certificates. I will be re-writing parts of the code to convert the old App into a Windows 8 Store App soon.

I have been planning on releasing this app into the store for free and maintaining it would take some effort initially. It will need new features and lots of data (pictures and formatted procedures). So one way to solve that issue was to let the user create the content (the app comes with a set of recipes). Creation of content is a boring task for anyone(AFAIK), so I was planning on having a creation of Recipes page which contains gesture driven approach of creating new recipes. For ex: user selects ingredients from the right by dragging them into a bowl, can choose to draw on screen by touch (say drawing a few circles can mean 'mix these ingredients', more circles mean 'thoroughly', need to implement it to test the user experience if all goes well it will make it into the app)to represent mixing or pick 'mix' as a task. More similar to a game, so that it is fun to add new recipes as well. Then the user can add new pictures for each step while they are cooking.

This again hasn't been implemented yet and is still in Sketchflow at the moment. Going through a few designs, trying to keep it simple and fun. Since this feature is in its early stages, it may not make it for V1.

Thank you for reading the article, I appreciate it. With the volume of articles pouring in, it must be... say... fun .