Posts tagged mynewt

With all these sensors and platforms lying around, I wanted to just pick one and build a quick sensor demo. It should be easy, right?

The basic idea

As you may (or may not) know, I spent a lot of time with the Apache MyNewt project a year or so ago. It has enormous potential. It’s small, fast, and very lightweight. I even wrote a tutorial on developing an app with MyNewt. I’m sure this post is going to annoy the good folks working on that project, but that is surely not my intention.

In developing a new demo for my new-ish job, I have been having problems with the stability of the Arduino Bluetooth stack. I decided that maybe MyNewt was a better route for this particular device, so I began trying to develop the app for MyNewt. Here’s why I’m giving up.

This is a dead-simple app. It does 3 things:

Waits until it gets a connection over Bluetooth

When connected, it reads an I2C sensor

Sends the value from the sensor to the connected device

Simple. In Arduino, there is an I2C library, “Wire” library, and I can simply do a Wire.begintransmission(0x68) — the I2C address of the device — then Wire.write() as many bytes as I want, then Wire.endtransmission() and the write to the I2C device is done. Likewise, I can read from an I2C device’s registers with a Wire.requestFrom(I2CAddr, len) and get back the number of bytes I requested from the I2C Device address requested. Simple. Clean.

The MyNewt Approach

Unfortunately, in MyNewt, I pretty much have to start from scratch and build complex data structures, then figure out how to initialize them correctly, Here’s the “example” from another I2C sensor:

Notice that the “data_struct” takes the address out of the “sensor interface” structure, but then I have to pass them both to the i2c_master_write() call. I fooled around with this for a few days trying to adapt it to the sensor I have — and my sensor is much more simple — I simply write 4 bytes to the proper I2C address, then read back 4 bytes.

I’ll spare you the 3+ pages of C code in MyNewt it takes to get anywhere close to doing this. And then there’s the Bluetooth stack, and implementing a series of event handlers, etc. to deal with events, etc. It’s literally almost 1,000 lines of code (the main.c file is over 500 lines).

The problem

And this, I think, is the problem with MyNewt. In the year+ since I was involved it hasn’t evolved to be any more user-friendly. It remains deeply mired in the weeds. If what you want to do is write highly board-specific code that is not really reusable, and requires a very deep knowledge of the underlying hardware, RTOS, libraries, etc. then MyNewt is for you. If, on the other hand, you’re a developer that simply wants to get stuff done, it just isn’t. I would love to be able to use MyNewt for this demo. I really would. I think it has great potential, and it’s come a long way in its short lifetime. But in order for it to gain traction with a wider developer audience — like, say, the Arduino crowd — it’s going to have to start making things a lot easier for would-be developers.

Abstraction libraries — like Arduino’s Wire library — and wrapper libraries around the Bluetooth NimBLE libraries, etc. are a necessity I really do hope I can come back to MyNewt again and develop this demo app quickly. I just don’t have the time right now to dig around and write very low-level, board-specific, code in order to make this demo work. I’ll just have to continue to work around the bugs in Arduino.

I’ve been spending a lot of time over the last few months working on the Apache MyNewt IoT OS. If you don’t know what that is, then you really should! It’s a fairly new Apache Project, still in the ‘incubating’ phase, but they’re about to release their second beta of the 1.0 release (I think all the votes are in to approve it). Yes, there are some things still missing, and some rough edges, but it’s coming together nicely and if you’re looking for an Open Source IoT OS for your device, MyNewt may just what you’re looking for. It’s extremely small (the bootloader I built is all of 9Kb!), fast, and very flexible. I’ve written a couple of apps for it — a UART -based environmental sensor app and an Analog sensor app — that I’ll write up here separately. For today, I’ll start with the Analog sensor using the Analog-to-Digital (ADC) Converter. So let’s get started!

I based this demo on the Nordic Semi NRF52DK Developer Board which is a Cortex M4F-based Bluetooth SoC development board. It’s got a lot going for it, and the Nordic NRF52 chip is an extremely common IoT chip, so it’s ideal for development (in fact the next App I wrote is for an Arduino Primo board, which also uses the NRF52 SoC). For my Analog sensor I chose an eTape Liquid Level Sensor that I picked up from Adafruit. Why did I choose this sensor? Mostly because I bought it a while ago for some great reason that I can’t recall, so it was sitting on my desk! It’s a simple resistive-voltage analog sensor, and I already had it, so it was ideal. I also bought a 1000 ml graduated cylinder on eBay to mount it in, just to make things easy. Now that all that background is out of the way, let’s get started on building the application itself!

Let’s start at the beginning … It’s usually a good jumping off point. I’m not going to go through the entire setup of the MyNewt development environment, but there’s an excellent tutorial on getting set up here. I do my development on Mac OS Sierra, and I do not recommend going the Docker route, but other than that, get yourself setup and running with MyNewt and then come back.

Note: This tutorial will eventually be a part of the docs/tutorials in the official MyNewt release.

Easy! Nordic did not release the ADC drivers and such for the NRF52 as open-source, at least not under an Apache-friendly license, so they have been moved to an external repository in order to not run afoul of license and copyright issues. One thing that MyNewt does is keep lots of information in YAML files. And for each project you create, newt creates a directory in the targets directory to hold some of these configuration files. It also keeps one in your main development directory that tells the newt tool what repositories to include, etc. So here’s mine:

$ cat project.yml
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
project.name: "my_project"
project.repositories:
- apache-mynewt-core
- mynewt_stm32f3
- mynewt_arduino_zero
- mynewt_nordic
# Use github's distribution mechanism for core ASF libraries.
# This provides mirroring automatically for us.
#
repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core
repository.mynewt_arduino_zero:
type: github
vers: 0-latest
user: runtimeinc
repo: mynewt_arduino_zero
repository.mynewt_stm32f3:
type: github
vers: 0-latest
user: runtimeinc
repo: mynewt_stm32f3
repository.mynewt_nordic:
type: github
vers: 0-latest
user: runtimeinc
repo: mynewt_nordic

You don’t need all those repositories! The only two you need for this exercise are the apache-mynewt-core and the mynewt.nordic repositories. For the next example I’ll write up you’ll need the mynewt_arduino_primo repository as well.

Now that you have defined the needed repositories, it’s time to install everything so that you can get started.

Your version-strings will likely be different but you should get similar output.

For the app itself we’re going to extend the one of the included apps for a Bluetooth Peripheral Device so that we get the Bluetooth communications built in, so the first thing we’ll need to do is copy that app into our own app directory:

“But wait, mine is only different by the “@apache-mynewt-core” part, why do I need to add that?” you ask? Because when the app lived inside the mynewt-core repository, it could use a relative path. Now that we’ve removed it, it needs an explicit path, including the repository. (You’ll see this made even more clear in a minute.) So now we have an app, in it’s own package, called nr52_adc and it depends on a bunch of packages within mynewt-core.

Create two targets – one for the bootloader and one for the nrf52 board.

So we’ve created a target for our app, and told newt where to find that app. We’ve told newt what Board Support Package (BSP) to use for the app, and we’re going to build this for debugging. Later, we can change this to ‘optimized’ like the bootloader to make it smaller.

Now, if you haven’t installed a mynewt bootloader on your board yet, you’ll need to do that first. So let’s build both the targets, and install them.

So about 9Kb for the bootloader. I told you it was small! You can do the same thing for the app if you’re curious.

Now you have a BLE app, but really all you’ve done is change the name of the built-in bleprph app to nrf52_adc and load that. Not all that impressive, and it certainly won’t read an Analog Sensor right now. If you have LightBlue for iOS or Mac OS you can open that and you’ll see your ‘nimble’ BLE device show up. But we’re here to actually make a BLE App that sends actual readings from a sensor, so let’s do that next. In order to read an ADC sensor, and since the ADC package is in an external, licensed, repository, we’ll create a driver for it here in our app that will leverage the existing driver in the external repository. It adds another layer of
indirection, but it will also give us a look at building our own driver, so we’ll do it this way. It’s also possible to just put most of this in the main program, but where’s the fun in that?

Building a Driver

The first thing to do is to create the directory structure for your driver:

If you want to poke around in the Nordic-supplied driver files, you can figure out what all these are, and what they mean, but they are basically the default settings for the ADC, so we’re going with them. You’ll notice that these values are set via a `MYNEWT_VAL` macro. This macro is defined in the newt tool and it reads configuration details for the app from another YAML file, so we’ll define them shortly in a `syscfg.yml` file to be passed to the compiler at build time.

A few things need to be said about this part, as it is the most confusing. First, we’re using a default configuration for the ADC Channel via the `NRF_DRV_SAADC_DEFAULT_CHANNEL_CONFIG_SE` macro. The important bit here is that we’re actually using `AIN1`. I know what you’re thinking, “But we want ADC-0!” and that’s true. The board is actually labelled ‘A0, A1, A2’ etc., and the actual pin numbers are also listed on the board, which seems handy. At first. But it gets messy very quickly.

If you try to use AIN0, and then go poke around in the registers while this is running,

You’ll see that the pin for channel 0 is set to 1, which corresponds to AIN0, but that’s NOT the same as A0 — pin P0.03, the one we’re using. For that, you use AIN1, which would set the pin value to 2. Messy. Someone, somewhere, thought this made sense. I don’t want to meet that person.

The only other thing to note here is that we’re using the internal reference voltage, rather than setting our own. There’s nothing wrong with that, but since we are, we’ll have to crank up the gain a bit by using `NRF_SAADC_GAIN1_6`. I arrived at this conclusion via trial and error. Feel free to conduct your own trials on it.

Now, how do we actually read values? Well, we’ll use adc_read() for that.

This one’s pretty straight forward. We simply take a number of samples from the ADC, convert the result to a reading in millivolts, and return the result.

As i mentioned earlier, some of the values are derived from settings discovered at compile-time, so we’ll need some settings for our driver. In the `myadc` directory
you’ll need to add a `syscfg.yml` file:

Once that’s all done, you should have a working ADC Driver for your NRF52DK board. But what are you going to do to get the data out over Bluetooth? Well, we’ll get there! First, we need to create a task in the main app to go get those sensor readings, then we’ll worry about sending them out.

Publish it all via Bluetooth

Now that the driver is done, we’ll need to add calls to the main app’s `main.c` file, as well as a few other things. First, we’ll need to update the includes, and add a task for our ADC sampling.

We’ll need that `adc_task_handler()` function to exist, and that’s where we’ll initialize the ADC Device — since we now have a driver that can do that — and set the event handler. In the task’s while() loop, we’ll just make a call to `adc_sample()` to cause the ADC driver to sample the adc device.

This is where we actually read the ADC value and then update the BLE Characteristic for that value.

But wait, we haven’t defined those BLE services and characteristics yet! Right, so don’t try to build and run this app just yet or it will surely fail. I know, you’re impatient and want to see if your driver works, etc. and I’ve done this in such a way that you cant yet! Sorry.

Let’s get on to building those BLE services and characteristics then! Luckily, it’s very easy since we already have an app that does about 90% of the work for us! We just have to add a few things. Namely a Bluetooth Service for the readings, and then some Service Characteristics to supply the readings.

As with the BLE Peripheral app we started with, we will advertise a couple of values from our app. The first is not strictly necessary, but it will help us connect with an iOS/Mac OS app later. I defined a service and the characteristics in that service in `bleadc.h` as follows:

The first is the UUID of the service — I got this by randomly generating a UUID — followed by the 2 characteristics we are going to offer. The first characteristic is going to advertise the type of sensor we are advertising, and it will be a read-only characteristic. The second characteristic will be the sensor value itself, and we will allow connected devices to ‘subscribe’ to it in order to get constantly-updated values. I wanted to use 0xDEAD and 0xBEEF but that’s not how things worked out.

Note: You can choose any valid Characteristic UUIDs to go here. I’m using these values because I built a Bluetooth tool that uses them.

The value that I’ll be updating is also defined here as `gatt_adc_val`.

If we then go look at `gatt_srv.c` we can see the structure of the service and characteristic offering that we set up:

We’re just adding another Service, with 2 new Characteristics, to the existing application. We’ll need to fill in the function that will be called for this service, `gatt_srv_sns_access` next so that the service knows what to do.

You can see that when request is for the `ADC_SNS_TYPE`, we return the Sensor Type we defined earlier. If the request if for `ADC_SNS_VAL` we’ll return the `gatt_adc_val` value.

If you build, load and run this application now, you will see all those Services and Characteristics advertised, and you will even be able to read the “Sensor Type” String via the ADC_SNS_TYPE Characteristic. But you still won’t get any readings. Because you haven’t wired up the sensor yet!

Adding the Actual Sensor!

Ok, this is the last bit before everything comes together! Now that we have a fully functioning BLE App that we can subscribe to sensor values from, it’s time to actually wire up the sensor!

As previously mentioned, we’re going to be using an eTape Water Level Sensor. You can get one from Adafruit.

We’re going to use the sensor as a resistive sensor, and the setup is very simple. I’ll be using a ‘breadboard` to put this all together for illustrative purposes.

First, attach a jumper-wire from Vdd on the board to the breadboard.

Next, attach a jumper wire from pin P0.03 on the board to the breadboard. This will be our ADC-in.

The sensor should have come with a 560 ohm resistor, so plug that into the board between Vdd and ADC-in holes.

Finally, attach a jumper from GND on the board to your breadboard.

At this point your breadboard should look something like this:

Now attach one of the middle 2 leads from the sensor to ground on the breadboard and the other middle lead to the ADC-in on the breadboard. Your breadboard should now look like this:

And your eTape Sensor should look like this (at least if you have it mounted in a graduated cylinder as I do).

That concludes the hardware portion. Easy!

Guess what? Yep! At this point you should be able to build, create-image and load your application and see it properly sending readings! About bloody time!!

Congratulations, you’ve now completed both a hardware project and a software project by connecting a sensor to your device and using Mynewt to read data from that sensor and send it via Bluetooth to a connected device. That’s no small feat!

Seeing your Results

If you’re wondering how to actually view these sensor readings via Bluetooth, you have a couple of options. On Mac OS or iOS you can download the LightBlue app
This app lets you connect to, and interrogate, BLE devices like the one you just built, but it’s not really visually all that great.

If you used the BLE Service and Characteristic UUIDs used in this tutorial, you can also download and use a Mac OS MyNewt Sensor Reader App that I built. (Zip Archive) that allows you to graph your data, etc. An iOS version is in Beta testing and should be available soon.

Now, I was saying how small and fast this OS is, and the apps are. So here’s a little proof to go with that:

Here’s the ‘debug’ built version of the app:

136 -rwxr-xr-x 1 dsimmons staff 135188 Feb 8 10:55 nrf52_adc.img

It’s all of 135Kb. And here it is ‘optimized’

120 -rwxr-xr-x 1 dsimmons staff 118788 Feb 9 11:06 nrf52_adc.img

A whopping 119Kb. Together with the 9Kb bootloader, we’ve got an entire IoT OS and App in 128Kb. Wait, that fits on an Original Macintosh! That’s pretty nice!

Ok, I hope you enjoyed this little example. Next up I’ll write up another app I did that is an environmental Air Quality (CO2) sensor. It’s on an Arduino Primo, but maybe I should put them both on the same board … Hmmmm

I’ve been doing a lot of work on the Mynewt OS Project over at the Apache Software Foundation. It’s still ‘incubating’ so it’s not a full-fledged Apache project yet, but we’re making huge progress and it’s coming along nicely. In fact, we’re just releasing our first beta versions of the 1.0 release! If you’re an IoT hacker, and are looking for a small, fast, highly configurable (and open source!) RTOS, MyNewt it the ticket!

All of that being said, one of the things I’ve been working on — other than documentation — are demos. The Sensor APIs are a work in progress, but the progress has been fairly significant. So much so that I was able to build a demo attaching an Analog sensor to a Mynewt-powered device and start sending the sensor values out via Bluetooth!

The basic setup is a Nordic Semi NRF52dk Developer Kit Board, an eTape Liquid Level Sensor, and a cylinder full of water. The sensor goes in the water, the nrf52 board reads the sensor, and sends the sensor values out. I’ll write a separate post about the actual nrf52 setup, etc. shortly, but I wanted to write a bit about how I got to be able to read the values. TL;DR: I wrote a Mac OS X and an iOS App to do it!

One of the problems with sending data over Bluetooth is that, while sending the data is easy, letting the peripheral device know what kind of sensor it is tends to be a bit more problematic. I solved this problem by setting two bluetooth characteristic UUIDs for each sensor. The first, which I call the “Configuration UUID” is a 16-bit UUID (yeah, I know, I should use 128-bit UUIDs, but that’s a pain) that simply carries an ASCII string describing the sensor. The second is a paired UUID that I call the “Data UUID” and that is a NOTIFY characteristic that will carry the actual data. I call these “paired UUIDs” and I define them by giving the Configuration UUID a prefix of “0xDE” and the Data UUID a prefix of “0xBE” I pair them by giving them both the same ‘suffix’ so 0xDEAD and 0xBEAD are paired by the common ‘AD’ suffix.

Here’s what that looks like in the actual app:

You can change the Prefixes in the App, or just subscribe to all the NOTIFY characteristics found under the defined service Characteristic. You can click on the RSSI Signal Graph to add the raw RSSI value to the table, or click it again to remove it.

And the same app — well, mostly the same, it shares a lot of code! — also runs on iOS

It’s been a minute since I’ve posted anything new, and I apologize to anyone that follows along here and has missed my posting. I’ve been a bit busy lately with the MyNewt Operating System for IoT. It’s new, it’s cool, and it’s a lot of work right now.

That being said, I thought I’d take a minute to show off some of the new hardware I have to add to the Big Board of IoT on my desk. I’ll start with some stuff I’ve been pounding away at lately with MyNewt OS. No cool unboxing porn for you, as it’s all in heavy use right now.

First we have the trusty STMicro STM32F3DISCOVERY board. Cortex M3 MCU with a bunch of LEDs on the board but no built-in networking. No BLE, no WiFi, nothing.

Next, the one I’ve been working on the most lately, is the Nordic Semi NRF52DK Development Board. This one has BLE built in, and I’ve been doing Bluetooth testing etc. with it. Very capable little board and the Nordic NRF52 is a near-ubiquitous IoT workhorse.

You can see that the LED is lit on that one, and it’s wired up since it’s the one I’m currently using.

Next up is this Arduino M0 Pro. Another Cortex M0-based board, but again, without a radio.

And the one I haven’t gotten running yet, the Olimex STM32-e407. As I said, I haven’t even powered this one on as I seem to have misplaced my JTAG programmer, and the new one just arrived about an hour ago.

But now for the real excitement!! <drum roll please> I splurged a little and got myself a couple of Samsung ARTIK development kits. Now for these, I actually do have some unboxing porn, since they just arrived, and they came in these schwanky boxes.

First the ARTIK 5:

What did I say about the packaging? Sweet!

It’s a box in a box! And there’s a bag in the box!

That’s a big board! With lots of possibilities! Underneath there’s all sorts of cables and external antennas, etc. but I won’t bore you with pictures of that. Mostly because I want to get on to the next one.

The ARTIK 0.

Again with the fancy box!

But no anti-static bag? Huh? Tsk tsk Samsung! It really should be in a bag. Both pieces. This one is really quite exciting, so I put the module on the board and just HAD to power it up.

The little display comes on and everything. That’s about as far as I’ve gotten with it. Hopefully I’ll have some free time soon to play with these new Samsung boards, but I have to get some Mynewt stuff finished first!

I saw a cool animation someone did on the git activity of an open source project, so I decided to do one of my own. I’ve been contributing to an open source RTOS for IoT called MyNewt (pronounced minute, as in really small) so I decided I’d see what a visualization of the project looked like over the past year. It’s mesmerizing.

As you can see, the growth of the project over the past year is pretty nice, and watching it grow is really cool.

I’ll be posting more about this project soon, to be sure, but I wanted to show this now anyway. Enjoy!