C#, Azure, Xamarin, Web Services, Unity, Mixed Reality

In the last few years there has been great advancement in the field of neural networks and the Artificial Intelligence (AI). Microsoft has also developed a set of Cognitive Services that are a set on machine learning algorithms to solve problems in the field of Artificial Intelligence (AI). The Cognitive Services APIs are grouped into five categories:

Vision—analyze images and videos for content and other useful information.

Speech—tools to improve speech recognition and identify the speaker.

Language—understanding sentences and intent rather than just words.

Knowledge—tracks down research from scientific journals for you.

Search—applies machine learning to web searches.

In this tutorial, we see how identification is used with the FACE API. There is also an additional GitHub guide for character recognition through the Computer Vision API (see the end of the article).

It is important to set in the Camera of the scene the projection to Perspective and to be in the right position as shown in the picture. Position (0, -0.02, 0), Rotation (0, 0, 0), Scale (1, 1, 1). Essentially the point where you choose to display the camera is the point based on which the other objects in space will be initialized.

You will need two GameObjects, the first GameObject named “PhotoCaptureManager” will contain the scripts and will be responsible for capturing the photo from the HoloLens device and then for sending to Azure Cognitive service to process it. The second GameObject named “AudioSource” will contain audio Source components in order to play the voice from text-to-speech implementation. For example “He is pretty much 26 years old and looks happy”.

Before you start coding, you need to create an Azure account and save the service key in order to use in your code. Choose the billing package. With F0 free package you can create 20 calls a minute and 30k a month.