Tag: machine learning

Creating apps that work well with Augmented Reality requires some background knowledge of the image processing algorithms that work behind the scenes. One of the most fundamental concepts involves anchors. These rely on keypoints and their descriptors, detected in the recording of the real world.

In this last part, we bring the vital sign check list to life. Artificial Intelligence interprets assessments spoken in natural language. It extracts the relevant information and manages an up-to-date, browser-based checklist. Real-time communication is handled through Web Sockets with Socket.IO.

The example scenario focuses on a vital signs checklist in a hospital. The same concept applies to countless other use cases.

In this article series, we’re creating a sample app that interprets assessments from vital signs checks in hospitals. It filters out relevant information like the measured temperature or pupillary response. Yet, it’s easy to extend the scenario to any other area.

The vision: automatic checklists, filled out by simply listening to users explaining what they observe. The architecture of the sample app is based on a lightweight architecture: HTML5, Node.js + the LUIS service in the cloud.

Such an app would be incredibly useful in a hospital, where nurses need to perform and log countless vital sign checks with patients every day.

In part 1 of the article, I’ve explained the overall architecture of the service. In this part, we get hands-on and start implementing the Node.js-based backend. It will ultimately handle all the central messaging. It communicates both with the client user interface running in a browser, as well as the Microsoft LUIS language understanding service in the Azure Cloud.

Creating the Node Backend

Node.js is a great fit for such a service. It’s easy to setup and uses JavaScript for development. Also, the code runs locally for development, allowing rapid testing. But, it’s easy to deploy it to a dedicated server or the cloud later.

During the last few years, cognitive services became immensely powerful. Especially interesting is natural language understanding. Using the latest tools, training the computer to understand real spoken sentences and to extract information is reduced to a matter of minutes. We as humans no longer need to learn how to speak with a computer; it simply understands us.

I’ll show how to use the Language Understanding Cognitive Service (LUIS) from Microsoft. The aim is to build an automated check-list for nurses working at hospitals. Every morning, they record the vital sign of every patient. At the same time, they document the measurements on paper checklists.

With the new app developed in this article, the process is much easier. While checking the vital signs, nurses usually talk to the patients about their assessments. The “Vital Signs Checklist” app filters out the relevant data (e.g., the temperature or the pupillary response) and marks it in a checklist. Nurses no longer have to pick up a pen to manually record the information.

Mobile Apps are no longer simple tools and games. They have grown to amazingly complex systems. Which ingredients are necessary to successfully develop a next generation app?

You will need Artificial Intelligence / Machine Learning. Only the best performance will satisfy your customers – or do you want to wait more than a few seconds in a mobile app? How can you visualize your user interface with HoloLens?

At the #mdaw15, you will learn how to develop and plan such apps with modern frameworks. Join the next After-Work event in St. Pölten for free at https://mobility.builders/

What is a Mobile Developer After-Work Event?

#mdaw events are mixtures of conferences with more casual meet-ups. Different expert speakers approach an overall topic from diverse perspectives. Afterwards, there’s plenty of time for networking and discussion amongst attendees – with snacks and drinks, of course.

The events target mobile developers and decision makers. The goal is to dive deeper into relevant and already known topic areas, as well as to keep up to date with the constantly evolving and changing toolset of the mobile world.

In the meantime, we can look back to 14 previous #mdaw events. Topics so far included business apps, digital healthcare, Xamarin, user experience or a Refugee Hackathon. We’re organizing the events with technology partners like Microsoft, Oracle, IBM, the City of Vienna and many others.

The community has grown to around 500 attendees, and we continue to get more and more developers on board with each event!

Together with Helmut Krämer, I’ve founded the community in 2013 and am proud that it’s still around. We’ve had a tremendous impact on the Austrian developer community so far. With the upcoming #mdaw15, we’re extending the reach geographically and host our first event outside of Vienna, to reach even more developers!