Status

The bridge application has been implemented in Java and works.
It has been tested in practice with the 'koppelting1' payload on 2017-8-26.

Next steps:

discuss/review with the people at habhub/ukhas/TTN

Introduction

This idea is about using TheThingsNetwork as a receiver for high-altitude balloon telemetry.

Receiving telemetry from high-altitude balloons is currently typically done on the 434 MHz band using RTTY modulation, with dedicated receivers listening for RTTY-modulated ASCII strings.
The operator of each receiver has to prepare his radio setup for receiving the telemetry, by tuning to the correct frequency at the correct time, setting up a dedicated software client that decodes the RTTY modulation and forwards the data to a central system over the internet.
The central system accepts data from many such receivers, performs de-duplication, keeps track of who received what and updates a nice graphical map of where each balloon is and where the receivers are. It can do a burst prediction, landing prediction based on a weather model

TheThingsNetwork consists of a world-wide network of LoRaWAN receivers that can accept data from nodes to receive low-bitrate telemetry packets and forward them on the internet.
It uses LoRa as a modulation scheme which allows for very small transmission power and is much less susceptible to slight tuning errors than the currently used RTTY modulation.
This means that telemetry packets can be received completely autonomously by TTN gateways, no need for a radio operator to make adjustments etc.

In short, the idea is:

you attach a LoRaWAN transmitter to the balloon, which been pre-configured with a set of keys generated by the TTN

the balloon broadcasts its telemetry every once in a while (say a few times per minute) and this is picked up by one or more TTN gateways which forward into the TTN infrastructure

the bridge software listens for packets received by the TTN and decodes the payload data into an id, latitude, longitude, altitude of the balloon

the bridge software converts each telemetry packet according to 'UKHAS' conventions and forward it to the habitat.habhub.org server which displays it on a convenient map, along with a burst/landing location prediction

the bridge software also forwards the TTN gateway locations, so they also appear on the map, along with their name / EUI and antenna altitude

the habitat.habhub.org server still sees the same messages like it would if there were many traditional receivers, so doesn't need any modification!

This way, the entire things network can be used to receive balloon telemetry!
There is no longer a need for radio operators to be present at their receiver at the exact time the balloon is launched, making manual adjustments, etc.
The Netherlands is already covered by many TTN gateways, greatly increasing the chance the balloon telemetry will be picked up.

See the table and example below to know which fields are required to construct the habitat ASCII sentence from the LoRa data payload sent by your tracker.

The habitat ASCII sentence typically looks like:

$$ttntest1,64,20:41:10,52.022100,4.693100,20.0,14.0,3.88*1C8A

Property mapping

Field value

SodaqOne raw

JSON

Cayenne

Remark

ttntest1

TTN node name

TTN node name

TTN node name

TTN node name

64

LoRaWAN FCNT

LoRaWAN FCNT

LoRaWAN FCNT

Frame count

20:41:10

field 0

TTN metadata time

TTN metadata time

TTN metadata time

52.022100

field 3

"lat"

latitude from GPS sensor

latitude (degrees)

4.693100

field 4

"lon"

longitude from GPS sensor

longitude (degrees)

20.0

field 5

"gpsalt"

altitude from GPS sensor

altitude (meters)

14.0

field 2

"temp"

value from TEMPERATURE sensor

temperature (degrees Celcius)

3.88

field 1

"vcc"

value from ANALOG_IN sensor

battery voltage (volts)

The field 1C8A is the CCITT-CRC16 over the characters between $$ and *, interpreted as bytes using US-ASCII encoding

Listener cache

The cache keeps track of when listener information / telemetry was sent last.
This makes it possible to avoid sending the listener information / telemetry with each payload telemetry document, reducing the load on the server.

Listener information / telemetry is sent only:

when this is the first time we are sending something for this listener

when it has been more than X minutes ago that we sent listener information / telemetry for this listener, where X is typically 10 minutes or so.

Helpful links

From a conversation on #highaltitude:

20:24 < adamgreig> there's a) a python library that's a lot easier to read
20:24 < adamgreig> but b) basically the gist is you just PUT to http://habitat.habhub.org/habitat/_design/payload_telemetry/_update/add_listener/<id> with some stuff
20:24 < adamgreig> http://habitat.readthedocs.io/en/latest/habitat/habitat/habitat/habitat.views.payload_telemetry.html#module-habitat.views.payload_telemetry
20:24 < adamgreig> so you have a new string, you take the sha256 hex digest of the base64 encoded raw data
20:25 < adamgreig> you PUT to that URL with that ID
20:25 < adamgreig> and you include that JSON struture with your callsign/details

Tracker configuration for the TTN-HAB bridge software

Roughly the following configuration is needed for your tracker to use the TTN-HAB bridge:

setup your LoRaWAN device for use with TheThingsNetwork

create a payload configuration document on the habhub.org webpage

download, configure and run the ttnhabbridge software

Setting up the tracker for TTN

Find out the LoRaWAN EUI of your tracker:

specifically for the SodaqOne, the EUI (LoRaWAN hardware address) is shown at startup of the tracker

perhaps just make up your own, if you don't use an RN2483?

Make up your mind about which binary payload format you are going to use:

I personally think the 'cayenne' payload format is very nice, because it's relatively flexible and is an actual standard. You need to put in at least the GPS coordinate in it. If you have it available, I would also put in the battery voltage and temperature in it.

Create the TTN application and register your tracker with TTN:

Create an account on TTN

Go to the TTN console and create a new application, or select an existing application you want to add the device to.

create a new node/device, this needs to be equal to the habhub payload name (but cannot use uppercase characters)

In the device settings screen:

under 'Device EUI', fill in the EUI of your LoraWAN tracker

under 'Activation method' choose ABP

disable option 'Frame Counter Checks'

click Save

Copy the network credentials back to your LoRaWAN tracker:

this means the "device address", "network key" and "application key"

in case you use a SodaqOne serial debug console, type the following:

copy the Network Session Key from the TTN console and type in the SodaqOne serial debug console 'key=<key>'

copy the App Session Key from the TTN console and type in the SodaqOne serial debug console 'app=<key>'

copy the device id from the TTN console and type in the SodaqOne serial debug console 'dev=<deviceid>' (is this right?)