In an attempt to make it easier for application developers to start working with the DeepSpeech model I’ve developed a GStreamer plugin, an IBus plugin and created some PPAs. To demonstrate what’s possible here’s a video of the IBus plugin providing speech recognition to any application under Linux:

GStreamer DeepSpeech Plugin

I’ve created a GStreamer element which can be placed into an audio pipeline, it will then report any recognised speech via bus messages. It automatically segments audio based on configurable silence thresholds making it suitable for continuous dictation.

Here’s a couple of example pipelines using gst-launch.

To perform speech recognition on a file, printing all bus messages to the terminal:

IBus Plugin

I’ve also created a proof of concept IBus plugin which allows speech recognition to be used as an input method for virtually any application. It uses the above GStreamer plugin to perform speech recognition and then commits the text to the currently focused input field whenever a bus message is received from the deepspeech element.

It’ll need a lot more work before it’s really useful, especially in terms of adding in various voice editing commands, but hopefully it’ll provide a useful starting point for something more complete.

PPAs

To make it extra easy to get started playing around with these projects I’ve also created a couple of PPAs for Ubuntu 17.10:

DeepSpeech PPA – This contains packages for libdeepspeech, libdeepspeech-dev, libtensorflow-cc and deepspeech-model (be warned, the model is around 1.3GB).

gst-deepspeech PPA – This contains packages for my GStreamer and IBus plugins (gstreamer1.0-deepspeech and ibus-deepspeech). Please note that you’ll also need the DeepSpeech PPA enabled to fulfil the dependencies of these packages.

April 21, 2014

Over the Easter weekend I finally got around to implementing a first prototype of an idea I’ve had for a while, which aims to bring some state of the art computer vision techniques to mobile devices.

Deep Vision uses the implementation of convolutional neural networks provided by libccv to classify images. So it’ll try to figure out whatever is the principal object in an image your provide it with.

At the moment it just has a sample classification database from the ImageNet project, containing 1000 assorted items, however in the future I’d like to see specific classifiers for different tasks (e.g. a classifier trained purely on different plants, so when you’re out for a hike and you want to know what something is you can just point your phone at it and find out.)

Unlike something like Google Goggles it’s doing all the classification on the phone itself without needing to upload the image to any external services.

This means that if your game is going to be purely QML based you can just grab the precompiled version, and run “make click-packages” and have packages built for both desktop and mobile use at the same time.

The template comes populated with an example application (one of the standard QML Box2D demos), which is found in the “src/” directory; so you’ll be able to see something running straight out of the box, then when you’re ready you can just replace this with your own game.

In the future I also plan to extend these templates to provide example packaging for multiple different QML + Box2D compatible mobile platforms (Sailfish, MeeGo, Blackberry, Android, etc.)

If you’re interested in seeing an example of the sort of thing you can achieve fairly easily with QML and Box2D I’ve also uploaded a video of one of my current work in progress projects:

December 30, 2013

Overview

Rockwatch allows your N9 to communicate with a Pebble smart watch. It makes it possible to install and manage Pebble apps, upgrade your Pebble’s firmware, receive notifications of e-mails and SMS messages and control your music from your Pebble.

New features in version 1.3

Support for incoming calls and caller ID.

Fixes incorrect time offsets when setting the watch’s time from the phone.

Download

November 12, 2013

Overview

CuteSpotify makes it possible to listen to your Spotify songs on Ubuntu Touch. It’s based on MeeSpot (a MeeGo Spotify client), which I’ve updated to make use of Qt5 and Ubuntu’s QML components. It’s still a little rough around the edges but most of the core functionality is in place now.

One particular problem to look out for is that because of the way Ubuntu Touch currently handles applications CuteSpotify has to be kept in the foreground and the phone has to be kept switched on for music to play (otherwise the application gets suspended). Approaches for handling applications that need to keep running are currently being debated, so hopefully that won’t be the case in future version of Ubuntu Touch.

As a temporary workaround, if your phone is in developer mode (achieved by connecting it up to the Ubuntu SDK) you can run “sudo service powerd stop” (default password is phablet) to stop the phone from going to sleep while you’re listening to music, then “sudo service powerd start” to resume normal power management when you’re done. (See popey’s comment for an alternative method).

Source

November 7, 2013

Overview

Eyrie is a music identification program that I originally created for the Nokia N9 but which I’ve now also ported to Ubuntu Touch. It works in a similar way to commercial applications like Shazam and SoundHound, listening to music through the phone’s microphone and then generating an acoustic fingerprint using the open source EchoPrint algorithm. Currently the EchoPrint database isn’t anywhere near as large as the various commercial offerings, so it won’t recognise everything, but it has reasonable coverage.

Installation

October 22, 2013

Overview

Rockwatch allows your N9 to communicate with a Pebble smart watch. It makes it possible to install and manage Pebble apps, upgrade your Pebble’s firmware, receive notifications of e-mails and SMS messages and control your music from your Pebble.

New features in version 1.1

Preliminary support for the HTTPebble protocol which allows watch faces and apps to communicate with HTTP servers via the phone’s network connection and also provides them with location information. For example this provides support for watch faces that display local weather information, like Fair Forecast.

The ability to set the watch’s time from the N9’s clock.

Upcoming features

In the next version I hope to add support for accessing the two-way SDK via DBUS, this will allow other N9 applications to communicate with watch apps and get feedback from the watch.

Download

October 16, 2013

Overview

Rockwatch allows your N9 to communicate with a Pebble smart watch. It makes it possible to install and manage Pebble apps, upgrade your Pebble’s firmware, receive notifications of e-mails and SMS messages and control your music from your Pebble.

Upcoming features

The next release will add:

Preliminary support for the HTTPebble protocol which allows watch faces and apps to communicate with HTTP servers via the phone’s network connection and also provides them with location information. For example this provides support for watch faces that display local weather information, like Fair Forecast.

Download

July 31, 2013

An article I wrote a little while back for DZone about native Tizen development has just been published, in it I work through the creation of an application to identify music (similar in principle to Eyrie, although greatly simplified). It involves a brief look at the native user interface builder, use of the native sound capture and HTTP APIs, a small amount of audio processing and instructions on how to incorporate a general 3rd party GNU/Linux library into a Tizen project.

April 19, 2013

Overview

Erudite makes it possible to use Amazon’s Cloud Reader on various mobile platforms which aren’t officially supported by Amazon. You can either read books online, or download them for reading offline. Your progress in a book is then also kept synchronised between your phone and other Kindle devices.

Symbian Belle support

Until recently Erudite only supported MeeGo Harmattan and Mer based phones (such as the Nokia N9), but now it’s also available for Symbian Belle phones as well. I’ll try and put together a build for Symbian Anna phones in the near future.

There’s a fairly comprehensive review of Erudite for Symbian over on All About Symbian: Erudite review.

Upcoming features

In the next release for both Symbian and MeeGo I’ll be focusing on orientation switching support, so users can optionally view their books in landscape mode, as well as investigating some apparent issues with very large books.