Serial ports and C++

I’ve added a new article to my website. You can read it here. This article explains how to interface with serial ports from C++. Applications include communicating with an Arduino, the Fonera or any microcontroller. It presents four classes to access serial ports, starting from simple string read/write to advanced asyncronous I/O.

All the code is based on the Boost.asio library, so the code is portable across operating systems.

If you have comments, or find bugs, post them as comment to this blog post.

Like this:

Related

This entry was posted on September 14, 2009 at 3:39 pm and is filed under Articles. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.

Thank you for the bug report, I really appreciate it.
The bug was caused by an incorrect handling of the operation_aborted error in the readCompleted callback. In practice, after a timeout the read operation is aborted, and the next attempt at reading will cause the readCompleted callback to be called with operation_aborted error. Now this error no longer causes the system_error exception to be thrown, but rather a retry at reading.

In the end, I’ve uploaded on my website the version 1.01 of the serial code with the bug fixed. Please download the new version, and report here if works for you (it should, since I’ve compiled your code example and now it works for me).

I just tested the revision 1.01 writing the same string that I used with the SimpleSerial. Using the first example worked fine, but using the timeout version didn’t worked (don’t write and don’t read nothing). I think still some bug in this code, but I didn’t manage to found…

I was looking your asynchronous implementation and I just didn’t understand why did you use:

thread t(bind(&asio::io_service::run, &io_));
thread_.swap(t);

in every serial instance, instead just one

thread t(bind(&asio::io_service::run, &io_));

from outside the asyncserial class, sharing &io_ with all serial objects. I think Boost serialize all handlers, so this will avoid all the synchronization that you did. As well, i think you must explicit synchronization when you call io_service.run() from two different threads, that’s right?

I am not an expert c++ programmer and I am not as well an Boost specialist, it’s just a doubt !!! :)

It’s true, if you have many instances of this class since each contains a thread this could degrade performances, but the reason why I chose this design decision is because in the most common case an application opens only one serial port at a time, and in this case there will be only one thread.
Also one problem with the single io_service design is that the standard way of coding a singleton won’t be thread-safe until C++0x, though there’s certinly some workaround for this.
In uncommon cases where many serial ports are open at the same time it is always possible to use Boost.asio directly and have full control of the io_service.

The synchronization was used to keep only one call to async_write in progress at a time. The design uses a queue and a buffer. When data in the buffer is being transmitted with async_write an attempt to write something else will result in the additional data being queued until the async operation completes.
I don’t know if multiple calls to async_write are possible, and if it is it would be possible to instantiate a buffer for each write operation and pass it to async_write, but there will be the need to find a way to deallocate the buffer when the write completes to avoid memory leaks. I’ll look into this, since avoiding synchronization would increase performance and make the code simpler.

“In uncommon cases where many serial ports are open at the same time it is always possible to use Boost.asio directly and have full control of the io_service”

You are talking about using serial ports in different threads, or using multiple serial ports in a main while(true) poll?

I am designing a solution to use at least 8 serial ports and I did a quick test using a single io_service running in a single thread, shared with two serial ports and it worked, but I am afraid that is not a safe solution. I am thinking to use Boost.Asio directly as possible and because I will have a GUI and other processing that must be in a while(true) pooling, I need the io_service inside a thread.

Given the requirements of your application need ~8 open serial ports simultaneously I suggest you use asio directly in your application. If I were you first I’d try to call io_service.run() from only one background thread (while the main thread is free to handle the GUI and other processing) and see if the performance is acceptable. If it isn’t I’d look into thread pools and strands.
Anyway, I don’t think it’s necessary to call io_service.run() from more threads in this case. Serial ports are much slower than ethernet connections, and I think strands and thread pools have been introduced in asio only for server applications with hundreds of simultaneous connections…
In addition try to code the callback that is called when serial data is received in a way that returns as fast as possible (think like an interrupt routine) to further optimize your application.

First of all, I just want to say thanks for posting your serial wrapper classes. I have found them to be extremely useful. I can’t believe that there aren’t more C++ examples of cross-platform serial communication…

One thing that I have been struggling with is closing and re-opening connections. I noticed that you posted an update recently (Sept. 10, 2010) that was supposed to address this issue, but unfortunately it didn’t work for me (at least not at without some additional tweaking). If I wait =125 ms, everything works as expected. I’ve only tested this on Windows 7 (maybe it’s not necessary on Linux?).

Sorry for resurrecting an old post but I’m trying to understand the behaviour of the asynchronous read with my Arduino.

My Arduino sends short messages (20 characters or so, delimited by \n) 10 times a second – the readings from a tilt-compensated magnetometer. If I use read() it returns a vector of char which contains several of these short messages. Good so far!

What I don’t understand is *when* these messages are received. My code looks a bit like this

The sleep() is required for the Arduino/magnetometer to ‘wake up’. If I sleep() for 2 seconds the vector returned by read() is 200-300 long. If I sleep() for 4 seconds it is 600-800 long. If I sleep() for 36 seconds it is >7000 long. I don’t sleep() between read() & printing the contents of the vector.

So what’s happening? Does all of the data from the Arduino get cached/buffered somewhere from as soon as the serial object is instantiated, even before I call read(), then when I do actually call read() it simply dumps what is in this cache/buffer into a vector & return it?

If this were the case, it would explain why the vector always starts at the beginning of one of my messages – I assumed that it would ignore the incoming data until I actually called read() & at that point start caching/buffering up to 512 characters & then return it, so if I called read() midway through transmission of a message then vector[0] would be midway through a message, but that doesn’t seem to be the case?

When I call readStringUntil() will that start from the beginning of the vector & as such give me the *oldest* message? I’m only interested in the most recent message so if so I’ll have to change it somehow.

Yes, BufferedAsyncSerial, as the name implies, buffers everything that is sent to the port until the caller reads it. It’s designed like that so as not to lose any data even if the caller checks the port only periodically.
Note that the name has Async in it meaning it guarantees that read() does never block waiting for data, it just reads what has been previously stored in the buffer.
Also, read() may return data that is in the middle of a message, as it does not have any knowledge of string terminators. If you do not see this behaviour it’s because your lines are rather short so it’s unlikely that it will return a partial string.
According to your needs, you only need the latest measure, so you want to lose al but the last line, is it right?
If that’s what you want there are many ways to achieve this, the first that comes to mind is to use the CallbackAsyncSerial class. It allows to register a function to be called each time data arrives from the serial port. You could then parse that, get the magnetometer reading and store the result in a global variable or something like that. The rest of your code can then read that global variable and treat it as the most recent megnetometer reading.
If you want to try this route, then keep in mind two potential pitfalls:
-the callback is called from a thread owned by the serial port class,
so your global variable will need to be mutexed to prevent concurrent access. boost or C++0x mutex and lock objects will do just fine.
-the callback may be called with a partial line, or multiple lines depending on how the operating system internal buffers are, so you will need to design a parser that works even in those cases. A stringstream is a good starting point, actually.
If you find yourself in trouble, let me know ;)

Thanks very much for your response & for confirming the operation of read() in AsyncSerial. As you suspected the reason that I was always getting full messages was because of the short length of them – when I increased the frequency of read() operations to greater than the frequency with which the Arduino sent them I began to get partial messages (& other parts of my code segfaulted due to me not checking for this eventuality!).

The CallbackAsyncSerial class sounds perfect for my needs – I hadn’t actually looked at part 4 of the post/code yet! I have one question before I start trying to implement it this way though. The callback that you register is called each time data arrives, but what is the delimiter (if any)? From the example implementation in main.cpp of part 4 it’s obvious that the callback isn’t called on a character-by-character basis, so what causes it to decide that it has received enough to invoke the callback?

The buffering that is done at hardware and OS level is what decides the size of the chunks of data you’ll receive in the async callback.
For example, the FT232 chip, which is a common USB to serial converter performs internal buffering for increased throughput. This means that if you keep sending data fast enough the OS will see data coming in chunks whose size I don’t remember, but it’s something like 64bytes. Of course, if you fill up a buffer only partially and then stop sending data there’s a timeout that will flush partially full buffers so that the OS will see that data without having to wait an unspecified amount of time for the buffer to fill up.
Also, the OS is then free to perform an additional level of buffering prior to delivering the data to applications, so in the end the chunks of data received through the callback depend on a lot of hardware and OS implementation details you should’nt care about, or worse, depend on.
The advantage of the callback API is that it lets you see the data as soon as the OS allows it, being therefore good for realtime applications, but it invariably requires some tokenization code to be written to split that data in meaningful units, such as lines if the data format is ASCII.

Hi. I try to use the BufferedAsyncSerial as a member (BufferedAsyncSerial *m_serial) in a class and create it with new in the constructor. But it just don’t work. I tried all possible ways to open, close, reopen the port but I can’t get any text out. Using it as you did in your example its no problem. Can you give some hint?

Hi,
I’ve been using TimeoutSerial (V1.05) for a while with great success (thanks), but with a recent feature I’m now having issues. The device I’m connecting to may be on one of a number of baud rates so I’m implementing a ‘Detect’ function which simply Opens the port, issues a known command and waits for either a response or timeout before moving to the next.

I’m having issues with the timeout as I’ve seen mentioned in some early posters. I see in the comments that 1.05 fixed an issue in this area, but can’t see if you’ve posted any newer versions than 1.05 as gitorious.org is currently down for maintenance (yay!).

The process is as follows:

Open Port as baud rate A (successful)
readStringUntul(“\n”) <- Timeout
Close Port (successful)
Open Port as baud rate A <- Access is denied