Building a specific Android application to handle a
WebRTC session on the smartphone and relay
commands to the controller via Bluetooth

Setting up a node.js server to serve an
HTML5 control page over HTTPS allowing visioconference and remote
control

The Telebot ready to be programmed

Therefore, the project will be a mix of Arduino, Android Java, and
Javascript (client-side and server-side). The source code is free
software, licensed under BSD 2-clause
license
and GPLv3 (Arduino
code). The complete source for the project is available on my
repository on GitHub.

Arduino programming

First, motor control is achieved through an extremely simple text
protocol over the Bluetooth serial, with one single-letter command and
one value per line. Motors are driven with
PWM using analogWrite.
In parallel, battery voltage is measured thanks to the probe on pin A0,
the value is scaled with the empirically-defined factor
batteryProbeFactor.

Before actually setting motor power, we have to add a correction in
order to keep the robot balanced: the principle is to move in the
direction the robot is leaning, like a
Segway. To this end, we
need to communicate with the MPU-6050 via the I2C bus to read the
measured values. I tried first to use the Arduino Wire library, but it
tends to hang after some time and does not seem to allow defining a
timeout. It is quite a drawback since it can crash the robot and as a
result make it fall down, so I prefer to use a software I2C
library.

The first job of the Android app is of course to connect to the base via
Bluetooth.

We list paired devices called "Telebot", which the HC-06 name has been
previously changed to (You only need to connect it directly to the UART
and send the AT command "AT+NAMETelebot"). Then we try to connect
the serial port service on each of them.

I have attempted for some time to make use of
OpenWebRTC to create a WebRTC-enabled
native Android application, but I could not obtain something reliably
interoperable with Firefox or Chromium, even after some debugging. For
this reason, I'm creating an Android application handling the robot
controls but relying on a Chrome instance to handle the actual WebRTC
session. It is kind of a hack, but it works.

In the app, the signaling channel is obvisously not used for WebRTC,
since it will be handled in a separate browser, but it is of the utmost
importance because it receives user commands to move the robot.

EDIT: I changed this behaviour in new
versions.
Now, user commands use an RTCDataChannel rather than the signaling
channel, allowing to reduce latency.

Since the robot uses a browser to handle the WebRTC session, the
Javascript client must handle active mode (for the user) and passive
mode (for the robot).

// Check WebRTC is availableif(!navigator.mediaDevices.getUserMedia){displayMessage("Browser not compatible");return;}// Get a local streamvarconstraints={audio:true,video:true};navigator.mediaDevices.getUserMedia(constraints).then(function(stream){localStream=stream;// Set self viewselfView.srcObject=stream;selfView.style.visibility="visible";selfView.onloadedmetadata=function(evt){selfView.play();};if(active){// If active, call button triggers peerJoin()callButton.onclick=function(){callButton.disabled=true;peerJoin();};}else{// If not active, call peerJoin() directlypeerJoin();}}).catch(function(err){logError(err);displayMessage("Media device not available");});

For its signaling channel, the Javascript client also use a modified
version of OpenWebRTC simple signaling
channel.
Either the arrows on the page or the keyboard can be used for user
input, and controls are sent over the channel just like WebRTC
signaling. Please check the
repository for the
complete source code.

// Send new controls to peerfunctionupdateControl(){if(controlContainer.style.display=="none")return;varleft=0;varright=0;if(controlUp){left+=1;right+=1;}if(controlDown){left+=-0.70;right+=-0.70;}if(controlLeft){left=Math.min(left-0.50,0);right=Math.max(right+0.30,0);}if(controlRight){left=Math.max(left+0.30,0);right=Math.min(right-0.50,0);}varpower=100;left=Math.round(Math.min(Math.max(left,-1),1)*power);right=Math.round(Math.min(Math.max(right,-1),1)*power);if(peer){peer.send(JSON.stringify({"control":{"left":left,"right":right}}));}}

I also installed coturnSTUN/TURN
server to help WebRTC session establishment when
NAT is
in use, i.e. in most of the use cases. I can't wait for ubiquitous
IPv6 network access to bring the death of NAT.

Thanks to the HTML5 interface, you can use the Telebot from any device
with a recent browser equipped for video and sound capture, even another
smartphone. However, it won't work out of the box on an iPhone, since
iOS does not have any major browser available with WebRTC support as of
today, even through this standard feature has been available on other
platforms for years...

The interface in Firefox for Android

Eventually, it works wonderfully! I have only added a microphone to the
smartphone to enhance sound capture.

They see me rollin’, they hatin’

EDIT: You can read about subsequent enhancements
here,
in particular the use of an RTCDataChannel for controls.