Tag: azure

This post follows my first one on Internet-of-Things telemetry project based on Azure.
At that time I used a Netduino Plus 2 as remote sensor device: this time I use the awesome Seeedstudio LinkIt ONE board.
I strongly suggest to read the old one first, because most of the below content relies on concepts already discussed in the past.

The source of the project is hosted in the azure-veneziano GitHub repository.

The LinkIt ONE board impressions.

My viewpoint is the one from who does NOT like C/C++, does NOT have (decent) experience in such a languages, rather loves RAD environment. Even Netduino, which is based on the .Net Micro Framework, relies on a enjoyable IDE (Visual Studio). The .Net libraries together with the awesome debugger really makes the Netduino board the best ever.

Every time I must face a C/C++ program, I feel frustrated.
For this project, I had to “translate” the original Netduino C# sources to a Arduino/C/C++ “working” sketch. I say “working”, because my goal was just make the board working, not writing a decent program.
Why “frustrated”? Because I think it’s a nonsense wasting time against a hyper-verbose C/C++ project, than simply writing high-level languages like C# or Java. It’s a super-board, even faster and larger in terms of resources than a Netduino Plus 2. It’s not just a 8-bit AVR.

So, why the LinkIt ONE board should be “awesome”?
First of all, because the hardware. The board is a little masterpiece for hobbists, because is really full-featured, but also comes with a very reasonable price.
Secondly, I’d say the Arduino APIs approach, which simplifies the things for dumb+lazy programmers like me.
Honestly speaking, the APIs cover many of the board features, but I’d expect a better coverage. My impression is rather a thin layer which is wrapping the real OS running into the board. That’s pity, because the users may feel some limitation very early.

An example is the HTTP support, which is nonexistent.
A clearly claimed board “for prototyping Wearables and IoT devices” should expose at least the most common services, that a typical IoT project use.
I personally found annoying digging in the Internet for open source projects about very basic services like JSON manipulation, HTTP client and so other. Thankfully the Arduino-world is still huge, and there are plenty of users contributing with their own works.

To close this argument, most of the time I’ve spent on the porting was…about ME, and my inexperience on writing C/C++, searching for libraries on the Internet, and make them working in a decent way.
A normal C/C++ developer shouldn’t have any problem on writing apps for this board.

The software.

As said, except for the JSON and the HTTP libraries, almost everything was “cloned” from the original C# sources. That’s because many of you, maybe C/C++ skilled, will smile (laugh) by seeing how I’ve written the ported source.

The JSON library comes from Benoît Blanchon, who made a very cool job.
The library is well written, in my opinion, but you should bear in mind that the Benoît target was the “real” Arduino, as very poor on resources (and language features).
That is, the LinkIt board is a 32-bit ARM core, thus the memory usage is way higher than what described in the library Wiki.
Moreover, the Arduino default platform comes without “new” and “delete”, and the JSON library adds its own “new” operator overloading. However, that will fight with the LinkIt SDK, which is a C/C++ featured platform, and such operators are fully supported.
By the way, this board comes with 4MB of RAM. Far from being a problem allocating some KB of cache!

For the HTTP client, I chose this library.
It’s a minimalistic HTTP composer and parser, but way enough for what any hobby project requires. It comes also with the BASIC authentication.

There are some other difference from the original Netduino project.
First off, the TCP connection is made via Wi-Fi, because the LinkIt ONE does not comes with any Ethernet plug. However, this is a very straightful task, because the APIs/samples are very clear on how to set-up a wireless connection.
Another difference is on the GPS sensor monitoring, which is absent in the Netduino, but included in the standard LinkIt ONE package. Despite I used it mostly for fun, I believe it would be useful whereas you need to track the device position if it’s moveable.
Again, very easy to set-up.

The sketch initialization isn’t much different than the old Netduino’s one:

In this article I’ll show you how to refine the notification of an event by creating a detailed data report.
This post closes the basic part of the project. There will be other articles related to the Azure Veneziano project, but they are mostly additional components and enrichment to the base system.

The problem.

When the system alerts you about, for instance, the outside temperature which is getting higher the more is sunny, I believe there’s no need of any detail on.
However, the things could turn dramatically different if you receive a notification such as “your aquarium temperature is greater than 35°C”. Unless you have very special fishes, there are just two possibilities:

the fishes are in a serious danger, or

something is broken (e.g. the probe, the wiring, etc).

In the first case, there’s no other way than making some immediate action before the fishes die. In the second case, you could even tolerate the failure knowing that the system is unable to work properly, until it will be fixed.
However, if you’re at the mall, for instance, and you receive such a mail: what would you do? Better question: how to know what kind of problem is? Also, how the system evolved before facing the issue?
Of course you can add “redundancy” to the telemetry system, so that you’ll have more info (and that’s always a good thing). For instance, you could use two probes instead of just one. Since the fishes life is in danger, a probe more is actually a natural choice.
Anyway, if you receive a simple message like “the water temperature is 55°C”, you can’t understand where the problem is. A bit different if the message shows you the “evolution” of that temperature. If the evolution acts like a “step”, where the temperature rises to a prohibitive value in a few time, then it’s probably something broken in the hardware. Reasonably, the aquarium tank can’t get hotter in minutes or less.
All that depicts a scenery where a collection of values over time is an useful “attachment” to the alerting message. Here, the target is representing the data collected as both chart and table fashion.

Looking for the right library.

As for report I mean a simple document, which contains details on what happened. For this project, we’ll create a three-pages report with a couple of charts, and also a brief tabular history of the collected data.
Once again, I’d like to remember that this project is a kind of “sandbox” for something professional. Thus, I prefer to try “a bit of everything” in order to take practice with the environment: Azure at first, then several accessories. For this reason, I wanted the ability to create the report document in both Word- and PDF-format.
Around the Internet there is plenty of creation and conversion tools, but most of them are very expensive. In a professional context that would be feasible, but of course isn’t acceptable for any home/hobby target.

Finally, I bumped against the Spire.Doc Free-edition by e-iceblue.
They offer a complete suite of tools for many standard formats. Despite their regular price is off the hobbyist-pocket, they also offer the Free-Edition option. I tested only the Spire.Doc component (tailored for the Word documents), and the limitations are pretty acceptable. At first glance some limitation looks like a wall, but it’s easy to play around the APIs and to find the right trick!
Moreover, when I had some trouble with the library, I asked them an help by the forum, and the answer came very quickly.

How to create your own report.

The usage of the Spire.Doc library is very simple, however I created a series of support functions in order to specialize the code for the report creation.
It’s worthwhile to say that the generated report is meant as “attachment” for the notification mail, so the below code is called automatically when the logic sends a mail.
The only thing the logic should specify is the list of useful variables to detail in the report. That’s an obvious requirement, especially when you deal with many variables.

Then, since the cover is made up from a well-defined template, its creation is straightful immediately after the document model. I also used an extension-method pattern so that the various function invocation will shape as fluent-fashion.

What’s behind?
There’s nothing secret. Those functions are only a shortcut for easy manipulating a report, but anyone could create his/her own functions.
The document generation creates a Document instance, defines its properties as well as the available styles. Please, note that the library comes with several pre-defined styles, but I wanted to create my own:

So far, so well.
If you wonder what’s the result at this point, here is a snapshot:

Please, since I was running out of logo pictures of my “Home Company”, I turned for a picture of my boss, far serious than many CEOs all around the world.
Hope you love her!

Let’s turn page: here the work begins to get harder.
The second page should give a brief overview of what happened at the very beginning. At first glance, the reader should mean WHY the mail has been sent. That’s still pretty easy to do, because it’s just a bunch of “Paragraph” to insert into the current page.

Since some lines of text shouldn’t steal much space on the page, I want to place a couple of charts about the most recent evolution of the selected variables.
Later we’ll cover how the chart generation works.

It’s worthwhile noting that I didn’t use the Table APIs as the library exposes. Again, I wanted to play around the library to face its flexibility.
At the end, I used the “tabulation-technique” and I must admit that the Spire.Doc library is very easy yet very powerful to use.

The very last thing to do is obviously to save the composed document. As described earlier, there’s no a permanent place where the document is stored, rather it is streamed directly as attachment to the mail.
Here below there is the snippet for both the Word- and the PDF-formats, so that the mail will carry two identical reports, then the user can open with the favorite reader.

How does it appear on my phone?

I believe it’s funny trying to read a data report on a phone. I mean that even on a relatively small screen you can read the same things as you were on a PC. Well, it’s not as easy as it looks, but I really love it as a start point!

Nice enough!…

Some words about the PDF document creation.

As you know, the Azure Veneziano project has been built against a totally free environment. By the way, the free websites that Azure offers where this WebJob runs, come with some limitation. In this case, the Spire.Doc library require GDI/GDI+ for the PDF generation, and that’s unavailable/unsupported by the Azure free context.
I fully tested the PDF generation in a desktop application, and the result was always perfect. However, if you need the Azure-side PDF generation, I believe there are at least two choices:

move the WebJob to a Cloud Service/VM (or any paid context) as suggested here;

leverage some online services such as this one, which offers up to 500 documents per months fro free.

The chart generation.

There are plenty of chart libraries on the web, both free and commercial. However, I created my own charting library because I needed several features that they’re hard to find all around. More specifically, my charting library is tailored for our industrial supervisory control systems, where the requirements are very different from, for instance, financial applications.
For the Azure Veneziano project, I took a small fraction of this library, but far enough to render multi-plots, multi-axes, WPF charts.

The usage is pretty simple, although an user may find uselessly verbose the code. Again, that’s because the original library is full-featured, and many functions don’t come easy without a certain dose of source code.

The only “strange” thing is that we actually DO NOT HAVE a WPF application, but something like a Console application, “hidden” somewhere in the Azure cloud.
However, this isn’t surprising at all, because the above code instantiate an UserControl as container for the chart. Once the chart model setup is done, there’s a “fake” measuring+arranging pass, followed by a final rendering against the control’s face.
The very final step is to capture the visual of this usercontrol and save as a bitmap image (PNG format). This image is inserted in the document as usual

In this article I’ll show you how to setup some components of Windows Azure, in order to make the system working.

I won’t cover details such as “how to subscribe to the Azure platform” or similar. Please, consider the several posts around the web, that describes very well how to walk the first steps, as well the benefits coming from the subscription.
A good place to start is here.

The system structure more in depth.

In the previous article there is almost no description about the system structure, mainly because the post is focused on the device. However, since here the key-role is for Azure, it’s better to dig a bit in depth around what’s the target.

On the left there are a couple of Netduinoes as symbol of a generic, small device which interfaces with sensors, collects some data, then sends them to the Azure platform. This section is covered in the first part of the series.
The JSON-over-HTTP data sent by any device are managed by a “custom API” script within the Azure’s “Mobile Services” section. Basically a Node.JS JavaScript function which is called on every device’s HTTP request.
This script has two major tasks to do:

parse the incoming JSON data, then store them into a SQL database;

“wake-up” the webjob, because new data should be processed.

The database is a normal Azure SQL instance, where only two simple tables are necessary for this project. One is for holding the current variables state, that is every single datum instance incoming from any device. The other table depicts the “history” of the incoming data, that is the evolution of the state. This is very useful for analysis.

Finally, there is the “webjob”.
A webjob could be seen as a service or, more likely, as a console application. You can put (almost) anything into this .Net app, then it can started anytime. What I need is something like a endlessly running app, but in a “free-context” this service is shut-down after 20-30 minutes. That’s the way I used a trick to “wake it up” using kinda trigger from the script. Whenever new data are incoming the app is started, but can stay stopped whenever nothing happens.
The webjob task is just sending a mail upon a certain condition is met. In this article I won’t show anything sophisticated, than a very short plain-text mail. The primary goal here is setting up the Azure platform, and testing the infrastructure: in the next articles we’ll add several pieces in order to make this project very nice.

Looks nice, but…how much does cost all that?

Just two words about the cost of the Azure platform.
Entering into the Azure portal is much like as walking in Venezia: full of intriguing corners, each one different from others, and always full of surprises. The platform is really huge, but surprisingly simple to use.

I say that I was surprised, because you’ll be also surprised by realizing that many stuffs come for FREE. Unless you want to scale up (and get more professional) this project, your bill will stick to ZERO.

Setup the mobile service.

The Mobile Services are the most important components in order to interface any mobile device. The “mobile” term is rather oriented to devices like phones or small boards, but the services could be accessed even from a normal PC.
The first thing to do is create your own mobile service: this task couldn’t be more easy…

Type in your favorite service name, which has to be an unique identifier worldwide (as far I know).
About the database, ensure to pick the “Create a free 20 MB SQL database” (if you don’t have one yet), and the wizard will create automatically for you.
Two more parameters: select the closest region to you to host the service, then choose “JavaScript” as backend language for the management.

If you are creating a new database, you’ll face a second page in the wizard. Simply you have to specify the credentials to use to gain access to the database.

That’s all: within a few your brand new mobile service should be ready. The below sample view gives an overview about the service.

Please, notice that there are links where you can download sample apps/templates already configured with your own parameters!…Dumb-proof!

Also have a look at the bottom toolbar, where a “manage keys” button pops up some strange strings. Those strings are the ones that you should specify in the Netduino (and any other device) in order to gain access to the Azure Mobile Service.

The next task to do is about creating the database tables.
We need just three tables, and (even surprising) we don’t need to specify any column-schema: it will created automatically upon the JSON structure defined in the Netduino device software. This feature is by default, but you can disable it in the “configure” section, with the “dynamic schema” switch.

Table name

Purpose

tdevices

Each record is paired to a remote device and holds identification and status data of it.

tsensors

Each record is paired to a “variable” defined by a certain device somewhere and holds identification and status data of it.

thistory

Each record stores the value of a certain variable at the time it arrives on the server, or marks an event occurred. Think the table as a queue, where you can query the records in order to depict a certain variable’s value evolution over time.

Press “create” and enter “tsensors”, then ensure checked the “enable soft delete” and confirm. Repeat the same for both the “tdevices” and the “thistory” tables, and your task is over.
The “soft delete” feature marks a record as “deleted” and keeps it, instead of removing from the table. You should enable this feature when you deal with concurrency. I personally think it is useful even for a simple dubugging. The problem is that is up to you “cleaning” the obsolete records.

The last section to setup within the Mobile Service context is the “Custom API“, that is the code to run upon any incoming data request.
Simply select the “API” section, then press “create”.

The wizard will ask you the name of the new API, as well as the permission grants to access it.
Back to the Netduino code, the API’s name should be specified on any request.

Technically speaking, the name is the very last segment of the URI path which maps the request against Azure.

http://{your-service-name}.azure-mobile.net/api/{your-api-name}

At this point you can begin to type the script in.

The device-side entry-point for the data.

The handler for the incoming requests is just a JavaScript function. Better: one function per HTTP method. However, since the primary goal is pushing data from a device into the server, the method used is POST (CREATE, in the REST terminology) all the times.
The JavaScript environment comes with Node.Js, which is very easy yet compact to use. I’m NOT a JavaScript addict, but honestly I didn’t have much effort in coding what I wanted.
The “script” section of the API allows to edit your script as you were on Visual Studio. The only missing piece is the Intellisense, but for JavaScript I don’t need it actually.

As the data come in, the first thing is to look for the correspondent existent entry in the “tdevices” table, using the device’s identification as key. If the record does exist, it will be “updated”, otherwise a new entry will be added.
Upon an update, the logic here is comparing the incoming “configuration” version with the corresponding value stored in the table. If they don’t match, the “flush” flag is set, which serves to the next step to remove all the obsolete “sensor” entries.

When the operation on the “tdevices” table is over, begins the one on the “tsensors” and the “thistory” tables.
As in the previous snippet, first there is a selection of the records of “tsensors” marked as owned by the current device identifier. Then, if the “flush” flag is set, all the records are (marked as) deleted.
Finally, the data contained in the incoming message are scanned one item at once. For each variable, it looks for the corresponding entry in the recordset, then either update it or add a new record if wasn’t found.
Any item present in the message is also appended “as-is” to the “thistory” table.

The last but not least piece of script is for waking up the webjob.
Please, note that my usage of the webjob is rather uncommon, but I think it’s the best compromise. The trade is between the Azure “free-context” limitations, and the desired service availability. The result is a webjob configured as “running continuously”, but is shut down by the platform when there’s no external “stimulation”. The trick is to “wake up” the webjob only when necessary by invoking a fake call to its site.
Have a look at my question on StackOverflow on how to solve the problem.

At the end, it’s a trivial dummy read to the webjob deployment site. This read wakes up or keeps awaken the webjob.

Please, notice that all the “console” calls are useful only during the debugging stage: you should remove them when the system is stable enough.

If everything goes well, the Netduino should send some data to the Azure API, and the database should fill.
Here is an example of what the “tsensors” table may contain:

Creating and deploying the webjob.

To understand what a “webjob” is, I suggest to read the Scott Hanselman’s article.
Since a webjob is part of a web-site, you must create one first. Azure offers up to 10 web-sites for free, so that isn’t a problem. At the moment, I don’t use any “real” web-site (meaning pages), but I need the registration.
The procedure of registration, deployment and related task can be easily managed from within Visual Studio.

When I started the project I used Visual Studio Express 2013 for Web, and the Update 4 CTP allowed such a management. Since a few days, there’s another great alternative: Visual Studio 2013 Community, which comes out with Update 4 released, but offers also a lot of useful features.
The following snapshots were taken on the Express release, but should be similar on other editions.

Start Visual Studio and create a “Microsoft Azure Webjob” project, and give it the proper name.

As you may notice, the solution composition looks almost the same as a normal Console application.
In order to add the proper references, just choose the “Manage NuGet packages” from the project’s contextual menu.

Since this webjob will “run continuously”, but will be actually shut down often, the very first thing to add to the code is a procedure for detecting the shutting request, so that to exit the application gracefully.
This piece of code isn’t mine, so I invite to read the original article by Amit Apple about the trick.

At this point you might add some blocking code, and test what happens. As in the Amit’s article:

// Run as long as we didn't get a shutdown notification
while (isRunning)
{
// Here is my actual work
Console.WriteLine("Running and waiting " + DateTime.UtcNow);
Thread.Sleep(1000);
}
Console.WriteLine("Stopped " + DateTime.UtcNow);

Before deploying the webjob onto Azure, we should check the “webjob-publish-settings” file which is part of the project. Basically, we should adjust the file in order to instruct the server to run the webjob continuously. Here is an example:

Open the project’s contextual menu, and choose the “Publish as Azure Webjob” item. A wizard like this one will open:

We should specify the target web-site from this dialog:

If the web-site is not existent yet, we should create a new one:

When everything has been collected for the deployment, we can validate the connection, then proceed to the publication.

Once the webjob has been published, it should placed to run immediately. To test whether the shut down will happen gracefully, simply leave the system as is, and go to take a cup of coffee. After 20-30 minutes, you can check what really happened in the webjob’s log.

Please, note that it’s important that you leave any webjobs’ status page of the Azure portal during the test. It would hold alive the service without really shutting it down.

Enter in the “websites” category, then in the “Webjobs” section:

At this point you should see the status as “running” or being changing to. Click the link below the “LOGS” column, and a special page will open:

This mini-portal is a really nice diagnostic tool for the webjobs. You should able to trace both explicit “Console” logs and also exceptions. To reveal the proper flow of the webjob, you should check the timestamps, as well as the messages such as:

[11/03/2014 07:03:53 > bb4862: INFO] Stopped 11/3/2014 7:03:53 AM

The mail alert application.

Most of the material inherent to this article has been shown. However, I just would to close this part with a “concrete” sign of what the project should do. On the next article I’ll focus almost entirely on the webjob code, where the system could considered finished (many things will follow, though).

As described above, as soon a message from any device calls the API, the webjob is waken up (in case is stopped), and the data are pushed in the database.
The webjob task should pick those data out, and detect what is changed. However, the API and the webjob execution are almost asynchronous each other, so it’s better to leave the webjob running and polling for other “news”. On the other hands, when something changes by a remote point, it might be possible that something else will change too in a short time. This is another reason for leaving the webjob running until the platform shuts it down.

I don’t want to dig into details here: this will be argument for the next article. The only important thing is how the data are read periodically (about 10 seconds here) from the server. The data read are copied in a local in-memory model, for ease of interaction with the language.
At the end of each poll, the variable which are changed since the previous poll are marked with the corresponding flag. Immediately after, the program flow yields the execution of a custom logic, that is what the system should do upon a certain status.

Let’s say that this piece of code is “fixed”. Regardless what the system should react upon the status, this section will be always the same. For this reason there’s a special, well-defined area where we could write our own business logic.
Here is a very simple example:

If you remember, the “Analog0” and “Analog1” are two variables sent by the Netduino. When I turn the trimpots so that:

any of the two variables is detected as changed, and…

the “Analog0” value becomes greater than the “Analog1” value…

…then an e-mail message is created and sent to me…(!)

Here is what I see on my mailbox:

Conclusions.

This article looks long, but it isn’t actually so: there are a lot of picture because the Azure setup walkthrough.
Azure experts may say that a more straightforward solution would be using a Message-Hub instead of a tricky way to trigger a webjob. Well, yes and no. I didn’t find a way to “peek” what’s inside a queue without removing its content, as long as other problems to solve.
This is much more an experimental project built on the Azure “sandbox”, than a definitive optimal way to structure a telemetry system. However, I believe that’s a very good point to start, take practice, then refine your own project.

In the next article, I’ll show how to create a better (yet useful) mail alerting component.

This is the the first part of a series, where I’ll present a telemetry project as a classic “Internet of Things” (IoT) showcase. The project starts as very basic, but it’ll grow up in the next parts by adding several useful components.
The central-role is for Microsoft Azure, but other sections will space over several technologies.

The source of the project is hosted in the azure-veneziano GitHub repository.

Inspiration.

This project was born as a sandbox for digging into cloud technologies, which may applies to our control-systems. I wanted to walk almost every single corner of a real control-system (kinda SCADA, if you like), for understanding benefits and limitations of a full-centralized solution.
By the way, I was also inspired by my friend Laurent Ellerbach, who published a very well-written article on how to create your own garden sprinkler system. Overall, I loved the mixture of different components which can be “glued” (a.k.a. interconnected) together: it seems that we’re facing a milestone, where the flexibility offered by those technologies are greater than our fantasy.At the time of writing, Laurent is translating his article from French to English, so I’m waiting for the new link. In the meantime, here’s an equivalent presentation who held in Kiev, Ukraine, not long ago.

Why the name “Azure Veneziano”?

If any of you had the chance to visit my city, probably also saw in action some of the famous glass-makers of Murano. The “Blu Veneziano” is a particular tone of blue, which is often used for the glass.
I just wanted to honor Venezia, but also mention the “color” of the framework used, hence the name!

The system structure.

The system is structured as a producer-consumer, where:

the data producer is one (or more) “mobile devices”, which sample and sometime collect data from sensors;

the data broker, storage and business layer are deployed on Azure, where the main logic works;

the data consumers are both the logic and the final user (myself in this case), who monitor the system.

In this introductory article I’ll focus the first section, using a single Netduino Plus 2 board as data producer.

Netduino as the data producer.

In the IoT perspective, the Netduino plays the “Mobile device” role. Basically, it’s a subject which plays the role of a hardware-software thin-interface, so that the converted data can be sent to a server (Azure, in this case). Just think to a temperature sensor, which is wired to an ADC, and a logic gets the numeric value and sends to Azure. However, here I won’t detail a “real-sensor” system, rather a small simulation as anyone can do in minutes.
Moreover, since I introduced the project as “telemetry”, the data flow is only “outgoing” the Netduino. It means that there’s (still) no support to send “commands” to the board. Let’s stick to the simpler implementation possible.

The hardware.

The circuit is very easy.

Two trimpots: each one provide a voltage swinging from 0.0 to 3.3 V to the respective analog input. That is, the Netduino’s internal ADC will convert the voltage as a floating-point (Double) value, which ranges from 0.0 to 100.0 (for sake of readiness, meaning it as it were a percent).
There are also two toggle-switches. Each one is connected to a discrete (Boolean) input, which should be configured with an internal pull-up. When the switch is open, the pull-up resistor takes the input value to the “high” level (true). When the switch is closed to the ground, it takes the value to the “low” level, being its resistance lower than the pull-up. and two switches.
If you notice, there’s a low-value resistor in series to each switch: I used a 270 Ohms-valued, but it’s not critical at all. The purpose is just to protect the Netduino input from mistakes. Just imagine a wrong setting of the pin actually configured as an output: what if the output would set the high-level when the switch is closed to the ground? Probably the output won’t fry, but the stress on that port isn’t a good thing.

All those “virtual” sensors can be seen from a programmer perspective as two Double- and two Boolean-values. The funny thing is that I can modify their value with my fingers!

Again, no matter here what could be the real sensor. I’d like to overhaul the hardware section for those who don’t like/understand so much about electronics. There are many ready-to-use modules/shields to connect, which avoid (or minimize) the chance to deal with the hardware.

Some virtual ports and my…laziness.

Believe me, I’m lazy.
Despite I’m having a lot of fun by playing with those hardware/software things, I really don’t like to stay spinning all the time the trimpots or sliding the switches, but I need some data changing overtime. So, I created a kind of (software) virtual port.
This port will be detailed below, and its task is to mimic a “real” hardware port. From the data production perspective it’s not different from the real ports, but way easier to manage, especially in a testing/demo session.
This concept of the “virtual port” is very common even in the high-end systems. Just think to a diagnostic section of the device, which collects data from non-physical sources (e.g. memory usage, cpu usage, etc)

The software.

Since the goal is posting on a server the data read by the Netduino, we should carefully choose the best way to do it.
The simplest way to connect a Netduino Plus 2 to the rest of the world is using the Ethernet cable. That’s fine, at least for the prototype, because the goal is reach the Internet.
About the protocol, among the several protocols available to exchange data with Azure, I think the simplest yet well-known approach is using HTTP. Also bear in mind that there’s no any “special” protocol in the current Netduino/.Net Micro Framework implementation.
The software running in the board is very simple. It can be structured as follows:

the main application, as the primary logic of the device;

some hardware port wrappers as data-capturing helpers;

a HTTP-client optimized for Azure-mobile data exchange;

a JSON DOM with serialization/deserialization capabilities;

The data transfer is normal HTTP. As the time of writing, the .Net Micro-Framework still did not offer any HTTPS support, so the data are flowing unsecured.

The first part of the main application is about the ports definition. It’s not particularly different than the classic declaration, but the ports are “wrapped” with a custom piece of code.

The class derives from the original AnalogInput port, but exposes the “Sample” method to capture the ADC value (Read method). The purpose is similar to a classic Sample-and-Hold structure, but there is a compare algorithm which detect the new value’s variation.
Basically, a “tolerance” parameter (normalized) has to be defined for the port (default is 5%). When a new sample is performed, its value is compared in reference to the “old value”, plus the tolerance-window around the old-value itself. When the new value falls outside the window, the official port’s value is marked as “changed”, and the old-value replaced with the new one.
This trick is very useful, because allows to avoid useless (and false) changes of the value. Even a little noise on the power rail can produce a small instability over the ADC nominal sampled value. However, we need just a “concrete” variation.

The above class implements the IInputDouble interface as well. This interface comes also from another, more abstract interface.

Those interfaces yield a better abstraction over the different kinds of port: AnalogInput, InputPort and RampGenerator.

The RampGenerator as virtual port.

As mentioned earlier, there’s a “false-wrapper” because it does NOT wrap any port, but it WORKS as it were a standard port. The benefit become from the interfaces abstraction.
In order to PRODUCE data overtime for the demo, I wanted something automatic but also “well-known”. I may have used a random-number generator, but…how to detect an error or a wrong sequence over a random stream of numbers? Better to rely on a perfectly shaped wave, being periodic, so I can easily check the correct order of the samples on the server, but any missing/multiple datum as well.
As a periodic signal you can choose whatever you want. A sine is maybe the most famous periodic wave, but the goal is testing the system, not having something nice to see. A simple “triangle-wave” generator, is just a linear ramp rising-then-falling, indefinitely.

Here is how a triangle-wave looks in a scope (it’s a 100 Hz, just to give an idea).

Of course, I may have used a normal bench wave-generator as a physical signal source, as in the snapshot right above. That would have been more realistic, but the expected wave period would have been too short (i.e. too fast) and the “changes” with consequent message upload too frequent. A software-based signal generator is well suited for very-long periods, like many minutes.

The HTTP client.

As described above, the data are sent to the server via normal (unsecured) HTTP. The Netduino Plus 2 does not offer any HTTP client, but some primitives which help to create your own.
Without digging much into, the client is rather simple. If you know how a basic HTTP transaction works, then you’ll have no difficulty to understand what the code does.

The above code derived from an old project, but here are actually just few lines of code of that release. However, I want to mention the source for who’s interested in.

As the Azure Mobile Services offer, there are two kind of APIs which can be called: table- (Database) and custom-API-operations. Again, I’ll detail those features on the next article.
The key-role is for the OperateCore method, which is a private entry-point for both the table- and the custom-API-requests. All Azure needs is some special HTTP-headers, which should contain the identification keys for gaining access to the platform.
The request’s content is just a JSON document, that is simple plain-text.

The main application.

When the program starts, first creates an instance of the Azure Mobile HTTP-Client (Zumo), then wraps all the port references within an array, for ease of management.
Notice that there are also two “special” ports called “RampGenerator”. In this demo there are two wave-generators with a period of 1200 and 1800 seconds, respectively. Their ranges are also slightly different, but just for less confusion in the data verification.
The ability to fit all the ports in a single array, then treat them as they were an unique entity is the benefit offered by the interfaces abstraction.

After the initialization, the program runs in a loop forever, and about every second all the ports are sampled. Upon any “concrete” variation, a JSON message is wrapped up with the new values, then sent to the server.

The composition of the JSON message is maybe the simplest part, because the Linq-way of my Micro-JSON library.
The led toggling is just a visual heartbeat-monitor.

The message schema.

In my mind, there should be more than just a single board. Better: a more realistic system should connect several devices, even different from each other. Then, each device should provide its own data, and all the data incoming into the server would compose a big-bunch of “variables”.
For this reason, it’s important to distinguish the data originating source, and a kind of “device-identification”, unique in the system, is included in every message.
Moreover, I’d think that the set of variables exposed by a device could be changed any time. For example, I may add some new sensors, re-arrange the input ports, or even adjust some data type. All that means the “configuration is changed”, and the server should be informed about that. That’s because there’s a “version-identification” as well.

Then are the real sensors data. It’s just an array of Javascript objects, each one providing the port (sensor) name and its value.
However, the array will include only the port marked as “changed”. This trick yields at least two advantages:

the message length carries only the useful data;

the approach is rather “loose-coupled”: the server synchronizes automatically.

Each variable serialization is accomplished by the relative method declared in the IInput interface. Here is an example for the analog port:

Conclusions.

It’s easy to realize that this project is very basic, and there are many sections that could be improved. For example, there’s no any rescue of the program when an exception is thrown. However, I wanted to keep the application at a very introductory level.
It’s time to wire your own prototype, because in the next article we’ll see how to set-up the Azure platform for the data elaboration.