Speech Recognition and Digital Assistants in LIMS

One question which tends to arise in the LIMS sales cycle is whether tablets or smartphones can be used. (The answer is yes.) The immediate follow-up question is how can we use it with gloves? Tablets and gloves don’t work well together.

One solution? LOTTIE – one of our LabVantage Lab projects.

Our Very Own LIMS ‘Skunkworks’LabVantage has its own skunkworks –the LabVantage Lab – where secretive people work on secret things in secret places. Well, maybe it isn’t that melodramatic.

The Rise of Digital AssistantsThe use of Digital Assistants such as Alexa, Echo, Cortana & Siri is exploding. The “wildfire” digital assistant market has penetrated the social consciousness – breaching our collective wallets to the tune of $4 billion a year.

The objective of a digital assistant is simple: get things done as efficiently as possible using a combination of voice input, natural language processing (NLP) and voice output.

Some more advanced assistants also provide visual feedback through the device itself, or via a phone or tablet app.

Meet LOTTIE.One of the experimental projects we have underway embeds the voice engine from Google Home in the LabVantage LIMS. LOTTIE – the LabVantage Open Talk Interactive Experience – is a helpful LIMS digital assistant the LV LABS team has put together.

This project has not been released to customers, and it may not be ready for release quite yet. But it’s the kind of thing our teams are working on to keep LIMS at the forefront of lab management.

LOTTIE was built using the GoogleHome system, which uses the Google API framework for voice input, speech and NLP. This API framework allows both the GoogleHome system and its speech recognition to be integrated into 3rd party devices and software.

One of these integrations is with the Chrome Browser, with its HTML5 Speech API (though both Microsoft Edge and Apple Safari browsers are close behind with implementing the HTML5 Speech API).

The LOTTIE assistant uses property-driven configuration to allow one or many LIMS commands to be executed by voice or click of a button.

It uses Natural Language processing combined with expression-based language to allow it to do things such as execute actions, calculate data, query data, enter results and navigate the LIMS.

It also hooks into the browser to allow other useful functions such as ‘search the internet,’ ‘get the weather’ or ‘convert units’ – and it’s fully customizable and extensible to allow other commands or skills to be defined.

Benefits of LOTTIE

· Hands-Free LIMS
As a hands-free system, users can speak sample results to the LIMS – even while holding on to a ladder up near a ceiling vent. This is a critical feature to have when touching the LIMS mobile device would compromise the cleanliness of the test, or smear the unit with material that might compromise the device.

· Task Automation
The LIMS assistant can have access to both LIMS data and instrumentation, and can take readings, enter results and analyze data by voice command.

· Reduced Risk of Error
The use of a digital assistant means less human error when data is transcribed.

· Improved Efficiency
Since the system can direct multiple tasks, data collection is more efficient.

· LIMS in Cleanrooms
A voice-controlled LIMS offers up a myriad of new potential uses, further extending capabilities to off-limits areas such as cleanrooms or other environments where hands can’t be used).

Prototyping the Future of LIMS Digital AssistantsThere are, of course, additional considerations with LIMS and the environments in which they tend to operate. LOTTIE gave us first-hand experience working with the in-built Chrome framework.

There are limitations, though – such as the requirement of an HTTPS connection to the LIMS, the need for the latest Chrome browser (not Edge, IE or Safari), a logged-in user session into LabVantage, and a decent quality microphone.

The natural evolution will be to work with a device such as GoogleHome, Alexa or HomePod to integrate skills using our REST API to enable querying & running LIMS commands independently of the LIMS.

Using such a setup would allow the use of the inbuilt NLP and enhanced 3D microphones of each device to process language and provide us a framework to do everything LOTTIE does – and more. (We’ll keep you posted!)

Lottie is a great example of the type of project we’re working on in our skunkworks – LV LABS. Many of these types of projects are initiated due to a user suggestion, or a discussion at a conference like CTEC. INSERT LINK So don’t be shy…if you’ve got an idea, let us know here! LINK TO CONTACT US

From Bench to CustomerLabVantage has long been an undisputed leader for new tech, and our pursuit to bring it into the lab is well-recognized.

The path for a technology like LOTTIE into a customer’s LIMS is rigorous. From seamless integration and the potential need to address regulatory issues to testing, and re-testing, even small changes to functionality demand a thorough vetting.

Our customer’s systems are, by necessity, not testing grounds for new products. Some adventurous & intrepid clients, however, want to play with newer features or add-ons, and they can be accommodated with the HUGE caveat that these prototypes are entirely experimental, may be untested and are notwarrantied.

Want to learn more? Contact LabVantage here. INSERT LINK

[The history of the term ‘skunkworks’ and its emergence at Lockheed in the 1940s is a fascinating story in its own right. Check out the wiki entry.]