All is worthwhile if the soul is not small.

Menu

Tag Archives: sming

I bought one of these boards, from Aliexpress for under 4€. It took more or less a month to arrive. I’ve still haven’t too much time to do something with it, but here are my first impressions:

The Wemos D1 mini is, in my opinion a better buy than the ESP-01 and ESP-12, for many things:

– It has all the supporting electronics for power and serial communication:
– Included USB to serial converter CH340G, but with no TX and RX activity leds.
– It can be powered by the mini USB connection, or through a 5V pin, since it has a 5V to 3.3V linear converter
– It also has a 3.3V input/output pin.
– It comes with some modern version of NodeMcu, so it’s ready to be used with the ESPlorer application, just connect to USB and we are ready.
– It includes stack able pins that allows several ways of creating our devices.

But most important is that it can be flashed with alternative firmwares, like Sming, and that’s what I did. Sming applications that I have, like the Wifi Scanner worked just fine and detected several wifi networks on the vicinity.

Also, the Wemos D1 mini has a reset button, but there is no need to use it to flash the device. The RTS and DTR lines of the serial connection are properly mapped to the GPIO0 and RST pins of the ESP-8266 so flashing is automatic. No need to press reset or manipulate pins. I was able to flash the device with several different bit rates like 460800 and 921600. The esptool.py can have some issues detecting the board, from some of the tests that I did, but at the end it flashes the board with no issue.

So, being the double of the price of the ESP-01 and a bit less price difference from the ESP-12, but with the included power and USB-Serial electronics, that are needed to be budgeted and implemented with the other boards, and all on a nice small package, it is by far the best way to enter the ESP-8266 world.

The ST7735 display controller is one of the available display drivers found driving the 1.8 TFT 128×160 resolution display. This driver/TFT combo is found from several sources like Adafruit and eBay, and has software drivers for Arduino and other platforms.
Still these displays, despite the same name (1.8 TFT 128×160), can come with different controllers from the ST7735 chip, like the Samsung S6D02A1 normally found on displays labelled QDTech, and apparently, there are also versions based on the ILI9163C chip.

Anyway I only have one of these displays and it’s based on the ST7735 chip and so here’s how I’ve connected and used a 1.8 TFT with 128×160 resolution the ESP8266 based ESP-12 board using the Sming Framework. The board needs to be an ESP-12 or other that exposes the ESP8266 SPI hardware pins, and so the ESP-01 is out of the question.

Why Sming framework?
The main reason is that I can use Netbeans on my ESP8266 software build process without touching the Arduino IDE… Reason enough, I think 🙂

The Adafruit ST7735 library
The latest Adafruit ST7735 Library supports the Arduino Boards and Teensy boards, and at compile time, specific code segments are enabled/disabled based on the target architecture. For example, when compiling to an Arduino the __AVR__ compiler time variable is defined. As far as I’m aware, this variable is also set when using the Arduino IDE when targeting the ESP8266 board, so no issues here.

The Sming framework defines the following compile time variable: __ESP8266_EX__, and we can just hack the library to support the Sming framework, or adapt it to be used on all or platforms. Anyway, my version is available here, and as far as my tests go, it works fine: Sming Adafruit ST7735 ESP8266.

The hardware connections:
The Library for the Sming framework uses the hardware SPI pins, so the connections should be the following:

Fixed pins SPI pins (can’t be changed)

D13 – Serial data out -> MOSI/SDA

D14 – Clock

Pins that can be changed and defined on the library constructor:

D4 – CS -> Chip Select

D5 – DC -> Data/Command

Also there is the reset pin from the display, that we can connect to an ESP8266 pin, like RST, for example, but I’ve just connected the pin to VCC (3.3v).

Power

The display is powered up by 3.3V. To do that I needed to put a bit of solder on the back pads near the power regulator, that regulates VCC from 5V to 3.3V so that the regulator is out of the circuit. This is of course because the ESP8266 works at 3.3V, so no need for level conversion and other power source.

Software:

Based on this, and using my ST7735 library on Github, my initialization code looks like:

While doing some test with MQTT using the Sming Framework for the ESP8266 I had some issues that are worthy to be documented.
The Sming Framework uses the libemqtt library for MQTT support, and it seems it works fine. Connection, subscribing and publishing work out of the box, and one important thing, the keep alive mechanism also works. We can see how MQTT can be used with the Sming framework in here.

But what is this keep alive mechanism on the MQTT protocol? The MQTT protocol specification defines this:

It is the responsibility of the Client to ensure that the interval between Control Packets being sent does not exceed the Keep Alive value. In the absence of sending any other Control Packets, the Client MUST send a PINGREQ Packet.

This means that before a keep alive time is reached the client must do a request to the MQTT broker, and in case it doesn’t, it should send a PINGREQ.

For example, on a Mosquitto broker with logs enabled I can have this:

1450189246: New client connected from 192.168.1.200 as ESP8266 (c1, k10)

It shows that a client connected from 192.168.1.200 with the client identification ESP8266 and it defined as the keep alive interval 10s.

This means that at least before a ten second period is reached, the esp8266 MQTT client should send a PINGREQ or do something, like publishing to a topic.

The above requirements are probably the root of my problems and the fact that the ESP8266 chip/firmware can’t do too many things at once….

The test that I was doing was that I didn’t published nothing from the ESP8266 to the MQTT broker. All I was doing was to subscribe some MQTT topic and print the messages on the I2C 16×02 LCD that I have connected to the ESP8266.

Without any activity the esp8266 stayed connected to broker without any problems.

If I published something to the topic subscribed by the esp8266, it worked fine.

But if I published something massively to the topic subscribed, then the broker would drop the connection:

1450190071: Socket error on client ESP8266, disconnecting.

And that was a bit odd…

So at a first approach, I added a check timer to see if the MQTT connection was valid, and to reconnect if it wasn’t:

So every minute the connection to the broker is checked, and the reconnection, if needed, happens automatically.

But this solution doesn’t explain why the connection is lost when the burst of messages arrive at the topic.

Anyway to keep the long story short, the burst duration of messages where superior to the keep alive interval. Since that during that period I didn’t publish nothing, AND, the Mqtt library was busy to receive the incoming data burst, the broker assumed that the esp8266 was dead, and drop the connection.

Since the default keep alive was 10s, that value was too short to keep the connection stable under load.

The real solution is to increase the MQTT keep alive interval, to something that is bigger that any duration of probable data bursts. So what is needed is to specify a higher keep alive interval:

1450190936: New client connected from 192.168.1.200 as ESP8266 (c1, k180)

I can now burst messages to the esp8266 MQTT client without the connection being dropped.

So as long that incoming burst of messages are under 180 seconds ( which is easy) or some publishing from the ESP8266 happens, the connection stays up. Anyway, if it fails, the check timer will reconnect again to the broker.

Finally I had the time to finish out the developer board that I’m using to program the ESP-01 chips as I described on this link: ESP8266 Developer board.

One of the remaining doubts that I had was if the I2C bus would work at all on the developer board. The issue is that while GPIO2 pin is not connected to anything, it just has a week pull up resistor, the GPIO0 pin is also used to control the firmware flashing through the CP2102 USB-Serial DTR line as we can see on the below schema:

So the GPIO0 pin has two functions, as the SCL (clock line) and as the flashing enable line.

For the Sming framework, if not specified, the GPIO0 is the SCL line, and GPIO2 is the SDA line.

Well with or without the USB cable from the CP2102 connected to the computer, the I2C bus works fine as expected 🙂 and I can detect and control an I2C connected HD44780 lcd display, that worked right out of the box.

The HD44780 works at 5V and so its connected to the ESP8266 I2C bus through a voltage level shifter.

The Sming framework supports the MQTT protocol out of the box. Comparing the MQTT protocol implementation on Sming with Nodemcu Lua, it appears that Sming MQTT implementation is faster and more stable and at least, in a first sight, doesn’t require any buffering code and semaphores to be implemented as I did in MQTT on Nodemcu to work.

So, out of the box code, without any other supporting code, I can have two timers “shooting” out MQTT massages at 200ms/250ms out of the ESP8266 and at the same time receiving MQTT data without losing any message! And this with the Mosquitto broker running on a low power Synology NAS ARM based device… Off course WIFI conditions are fine, but it’s an impressive achievement.
With Lua, I had to implement buffering and semaphoring for publishing, otherwise, I would get exceptions and/or loose messages.

How to start:
The Sming framework already has one example for using the MQTT protocol named MqttClient_Hello.
It is a simple example but shows all that is needed to use the MQTT protocol on the Sming Framework based code.
The API is simple, and it’s based on MqttClient object, where we define the broker address and the callback function for when receiving subscribed topics messages.

As we can see the mqtt variable holds now a MQTT broker definition to be used, and also, defines the callback function that is called when a message arrives for a subscribed topic.

The sample function for displaying received messages is simple. It only prints the Topic name and message.
The function is the same for all subscribed topics, so in a real case scenario, some if/then/else logic is probably necessary.

The mqtt.subscribe subscribes the MQTT topic passed as parameter.
Off course we can do as in the NodeMcu example, that is to define an topic array and subscribe each of the array elements.

Subscribe and unsubscribe:
After connecting we can subscribe and unsubscribe with the mqtt.subscribe( topic ) and mqtt.unsubscribe( topic ) functions. In both cases the functions receive the topic name.

As standard with MQTT we can use the # and * specifiers in topic names to subscribe one or more or zero or more.

Publishing:
There are two publishing functions, one that is the standard without Quality of Service and retention defined, and another one where we define those parameters:

Testing:
My testing was done using the Mqtt-Spy tool for monitoring the messages published by the ESP8266, and a simple script that run on the broker machine that mass flood the subscribed topics…
Since I have the broker running on the Synology NAS, that as the ash shell, such script is as follows:

So with above code that publish on two topics at very short intervals and with the Mosquitto server loop based script that also publishes data subscribed by the Sming framework, I’ve not witness any drops or failures that I’m aware off. Impressive.

Off course in other environments the results can be different, but the code is simple and needs no work arounds to be effectively used.

This short post is just to document some functions of the file system API when using the Sming framework. The Sming framework has a simple API to access the SPIFF file system for opening, reading/writing and deleting files.

The file system can be used for holding web server files, configuration files, log files, anything that needs a file system…

So my quick notes:

Files at flashing time
As I notice in Sming – How to Start we can, at flash time of the firmware, to add files in the Spiffs file system. The only thing needed is to create a sub-drectory, add the files there and then refer the directory on the Makefile-user.mk: For a directory named files:

SPIFF_FILES = files

Also it seems that on the latest version, this directory is detected if exists, and if so, the file system is created.

One important thing is to use this version of the spiffy tool that comes with the release of Sming from Sming Hub. Just go to the /opt/Sming/Sming/spiffy directory and execute make to build and make available the spiffy binary. Other option is when building Sming libraries just execute make followed by make spiffy on /opt/Sming/Sming.
Don’t use spiffy from other sources as I previously was using. Otherwise it won’t work.

Using the Spiffs file system
If your application uses the Spiffy file system it must now must explicitly mount the file system at run time:

void init() {
spiffs_mount();
...
...
}

If the file system is not mounted we will get an error -10024 that means the SPIFFs file system is not available/configured…

Listing files
At the application level we can list the contents of the SPIFFs file system with the following function:

We can see here the fileList() function that returns a Vector of Strings with the available list of files. We can then get the file size of a file with the fileGetSize( filename ) function.

We don’t need to do nothing to start using these functions right away, except the inclusion of SmingCore.h.

Error List
Before going further, we need to know the meaning of the errors that the file system functions may return.
These errors are described here: spiffs.h
We can see if trying to open a file we get -10002 it means that the file was not found, or for another example, -10021 we are trying to write into a file open in read mode.

File Flags
The file flags are flags passed to fileOpen() API call to open files.

At least one of the following three MUST be present:

eFO_ReadOnly – Opens the file in READ ONLY mode

eFO_WriteOnly – Opens the file in WRITE ONLY mode

eFO_ReadWrite – Opens the file for reading and writing.

The remaining flags can be associated to change the fileOpen behaviour. For example:

file_t fh = fileOpen("config.json", eFO_ReadOnly)

The variable fh, if positive holds the file handle to be used by other functions for accessing/manipulating the opened file.
If negative, it holds the error, for example -10002, the file doesn’t exist.

So this flags:

eFO_CreateIfNotExist – If the file doesn’t exist it creates it.

eFO_Truncate – Before opening the file, it resets its contents to nothing.

eFO_CreateNewAlways – Identical it seems to the above, but first it deletes the file. It seems a bug workaround

eFO_Append – Opens the file and moves the file pointer to the end of the file.

Other functions
The API offers other functions, that in a IDE like NetBeans can be seen at code completion.
Worth mention are the setContent() function where all the logic above is done internally, for example:

setContent("log.txt", "System started");

The above instruction creates and writes on the log.txt file the content “System Started”.

There are other functions, like fileRename(), fileDelete(), getContent() that allow a quick and easy way to use the flash file system on the ESP8266 using the Sming framework.

After dumping Lua Nodemcu for doing any useful work/tinkering on the ESP8266(it works fine for simple things and that’s it), from the available options for programming the ESP8266, native C/C++ SDK programming, Arduino, Sming, Smart.js and Espruino, I’ve selected Sming. One of the main reasons for selecting Sming is the huge range of working examples comprising HTTP clients, servers, AJAX, sensors, MQTT, you name it. Also I can use Netbeans IDE to build the ESP8266 applications and since Netbeans offers code completion, It can help a lot with the apparent lack of API documentation.

Netbeans Sming Code Compltetion

Another advantage that is not stated enough is the SPIFFS file system support. Install spiffy and under your project directory, create a files sub-directory, put the files there, and those files are flashed onto the ESP8266 as the SPIFFS file system…

How to start
Install first the esp-open-sdk and then Sming. The Sming instructions are pretty clear, in my opinion, on the GitHub site. Also if using Netbeans instructions are here: Setting up Sming with Netbeans. Make sure that the include paths are configured. It will enable code completion and get rid of the “errors” on the IDE.

After that we can export the variables ESP_HOME and SMING_HOME with the esp-open-sdk and Sming paths, or change the Makefile-user.mk of each project to point to the correct locations.

Installing and configuring Spiffy.

The following instructions are deprecated! -> See below how to install from SmingHub.As usual, download spiffy from Github, compile, and make Sming aware of its location:

Goto now to the Sming/Sming subdirectory and edit the Makefile-project.mk and edit the SPIFFY ?= spiffy line to include the full path of spiffy. In my case:

# Full path spiffy command, if not in path.
SPIFFY ?= /opt/spiffy/build/spiffy

With this change when we clone one of the available projects we can just create a files subdirectory under the project directory to enable files that are put there to be flashed onto the ESP8266. Just make sure that you add to the project Makefile-user.mk file the line: SPIFF_FILES = files

Spiffy from SmingHub

The latest version of Sming that is available from Sming Hub already has spiffy available. We just need to build it.

And that’s it, spiffy binary is now available.
Another way to build is just to make spiffy at the /opt/Sming/Sming directory.

First project
For first project I recommend to clone/copy one of the provided examples to a directory OUT of the Sming folders. Why? Because in this case we won’t have any conflicts when doing a git pull to update the Sming framework, and we keep the original projects intact, so we can use them as reference.

After copying the project we can see that the file structure is simple, and coding isn’t that hard ( but it’s needed to have some C and C++ proficiency ) to build applications. Coding on Sming framework is very similar to programming for an Arduino function and library wise, but with some differences, namely the callback “way” of programming and the nonexistence of the setup()/loop() paradigm (it doesn’t makes sense on a callback way of programming).

The code folder structure should mimic the example projects, namely a root folder where the make files reside, and under this folder the app folder for application files (.cpp, .c), the include folder for include files, and a files folder (optional) for holding files for the spiff file system.
Still these folders can be changed by modifying the Makefile-user.mk file at the root directory, namely the MODULES entry and the SPIFF_FILES entry:

All Sming framework based programs must provide a init() function, that is called at ESP8266 startup. There is no setup() or loop() functions. The later function doesn’t make sense since for programming the ESP8266 we register a function for an event, and that function is called when the event triggers. No need for pooling and/or looping. So no loop() function. The Arduino sketch setup() function is in our Sming framework our init() function…

On the init() function we can setup a lot of things like, pin modes, serial port, and of course Wifi connectivity. Programming with an IDE that offers code completion allows us to see the available API and guess some of the functionality even without documentation ( note that most of the API is identical/similar to the Arduino).

So for programming with the Sming framework these are some of my notes that might be helpful:

Serial port
The always helpful serial port… allows the output of data and debugging information and keyboard input, at least when the device is connected to the computer.
The serial port usage from a coding stand point is identical to the Arduino library, but ESP8266 outputs some debugging information also for the serial port:

When the value of the input parameter flag is true, we can see a lot of information, including HTTP headers, and so on. Quite verbose, which means that our output might be lost in the middle of this debugging information.
Still with the above flag set as false we can still see some debug output that can be almost completely shut off with the following instruction: system_set_os_print(0);.

A final note regarding printing to the serial port and debugging, we can use the higher level Serial output instructions, but with Sming we also can use/mix the standard SDK instructions: this means that we can also use the functions debugf and os_printf.

System ready
We can on the init function register a callback for when the system is ready:

Wifi Connection
Almost all the examples have the setting up of a wifi connection, some as a client, others as access point.
The important thing is that the all the callback functionality either on success or failure is already implemented, and so it’s quite easy to check for success or failure.

Note that is way easier than Nodemcu Lua where we need to implement the connection check by ourselfs by doing a timer based pooling.
With Sming is quite easy, connectOK() is called when a successful connection is made, and connectFail() is called if the timeout (the second parameter) is reached and a connection isn’t made.

On connectFail() we can just again repeat the same instruction or list the available networks and try to connect to some of them.

Still, in my opinion, we should set up the WIFI parameters and connection initiation under the system ready callback.