ESD closes shop. What’s next in store for embedded programming?

Sunday, April 29th, 2012 by Miro Samek

The demise of the ESD Magazine marks the end of an era. In his recent post “Trends in Embedded Software Design“, the magazine insider Michael Barr commemorates this occasion by looking back at the early days and offering a look ahead at the new emerging trends. As we all enjoy predictions, I’d also like to add a couple of my own thoughts about the current status and the future developments in embedded systems programming.

It’s never been better to be an embedded software engineer!

When I joined the embedded field in the mid 1990s, I thought that I was late to the party. The really cool stuff, such as programming in C, C++, or Ada (as opposed to assembly), RTOS, RMA, or even modeling (remember ROOM?) have been already invented and applied to programming embedded systems. The Embedded Systems Programming magazine brought every month invaluable, relevant articles that advanced the art and taught the whole generation. But then, right around the time of the first Internet boom, something happened that slowed down the progress. The universities and colleges stopped teaching C in favor of Java and, the numbers of graduates with skills necessary to write embedded code dropped sharply, at least in the U.S. In 2005 the Embedded Systems Programming magazine has been renamed to Embedded Systems Design (ESD), and lost its focus on embedded software. After this transition, I’ve noticed that the ESD articles became far less relevant to my software work.

All this is ironic, because at the same time embedded software grew to be more important than ever. The whole society is already completely dependent on embedded software and looks to embedded systems for solutions to the toughest problems of our time: environment protection, energy production and delivery, transportation, and healthcare. With the Moore’s law marching unstoppably into its fifth decade, even hardware design starts looking more like software development. There is no doubt in my mind: the golden time of embedded systems programming is right now. The skyrocketing demand for embedded software, on one hand, and diminished supply of qualified embedded developers on the other hand, creates a perfect storm. It simply has never been better to be an embedded software engineer! And I don’t see this changing in the foreseeable future.

Future trends

Speaking about the future, it’s obvious that we need to develop more embedded code with less people in less time. The only way to achieve this is to increase programmer’s productivity. And the only know way to boost productivity is to increase the level of abstraction either by working with higher-level concepts or by building code from higher-level components (code reuse).

Trend 1: Real-time embedded frameworks

My first prediction is that embedded applications will increasingly be based on specialized real-time embedded frameworks, which will increasingly augment and eventually displace the traditional RTOSes.

The software development for the desktop, the web, or mobile devices has already moved to frameworks (.NET, Qt, Ruby on Rails, Android, Akka, etc.), far beyond the raw APIs of the underlying OSes. In contrast, the dominating approach in the embedded space is still based directly on the venerable RTOS (or the “superloop” at the lowest end). The main difference is that when you use an RTOS you write the main body of the application (such as the thread routines for all your tasks) and you call the RTOS services (e.g., a semaphore, or a time delay). When you use a framework, you reuse the main body and write the code that it calls. In other words, the control resides in the framework, not in your code, so the control is inverted compared to an application based on an RTOS.

The advantage, long discovered by the developers of “big” software, is that frameworks offer much higher level of architectural reuse and give conceptual integrity (often based on design patterns) to the applications. Frameworks also encapsulate the difficult or dangerous aspects of their application domains. For example, most programmers vastly underestimate the skills needed to use the various RTOS mechanisms, such as semaphores, mutexes, or even simple time delays, correctly and therefore developers vastly underestimate the true costs of using an RTOS. A generic, event-driven, real-time framework can encapsulate such low-level mechanisms, so the programmers can develop responsive, multitasking applications without even knowing what a semaphore is. The difference is like having to build a bridge each time you want to cross a river, and driving over a bridge that experts in this domain have built for you.

Real-time, embedded frameworks are not new and, in fact, have been extensively used for decades inside modeling tools capable of code generation. For example, a leading such tool, IBM Rhapsody, comes with an assortment of frameworks (OXF, IDF, SXF, MXF, MicroC, as well as third-party frameworks, such as RXF from Willert).

However, I don’t think that many people realize that frameworks of this sort can be used even without the big modeling tools and that they can be very small, about the same size as bare-bones RTOS kernels. For example, the open source QP/C or QP/C++ frameworks from Quantum Leaps (my company) require only 3-4KB of ROM and significantly less RAM than an RTOS.

I believe that in the future we will see proliferation of various real-time embedded frameworks, just as we see proliferation of RTOSes today.

Trend 2: Agile modeling and code generation

Another trend, closely related to frameworks is modeling and automatic code generation.

I realize of course, that big tools for modeling and code generation have been tried and failed to penetrate the market beyond a few percentage points. The main reason, as I argued in my previous post “Economics 101: UML in Embedded Systems“, is the poor ROI (return on investment) of such tools. The investment, in terms of tool’s cost, learning curve, maintenance, and constant “fighting the tool” is so large, that the returns must also be extremely high to make any economic sense.

As the case in point take for example xtUML (executable UML) tools, which are perhaps the highest-ceremony tools of this kind on the market. In xtUML, you create platform-independent models (PIMs), which need to be translated to platform-specific models (PSMs) by highly customizable “model compilers”. To preserve the illusion of complete “platform and technology independence” (whatever that means in embedded systems that by any definition of the term are specific to the task and technology), any code entered into the model is specified in “Object Action Language” (OAL). AOL functionally corresponds to crippled C, but has a different syntax. All this is calculated to keep you completely removed from the generated code, which you can only influence by tweaking the “model compiler”.

But what if you could skip the indirection level of AOL and use C or C++ directly for programming “action functions”? (Maybe you don’t care for the ability to change the generated code from C to, say, Java). What if you could replace the “model compiler” indirection layer with a real-time framework? What if the tool can turn the code generation “upside down” and give you direct access to the code and the freedom to actually do the physical design (see my previous post “Turning code generation upside down“)?

Obviously, the resulting tool will be far less ambitious and “lover level” than xtUML. But “lower level” is not necessarily pejorative. In fact, the main reason for popularity of C in embedded programming is that C is a relatively “low level” high-level programming language.

I predict that we will see more innovation in the area of “low level” and low-cost modeling and code generating tools (such as the free QM modeling tool from Quantum Leaps). I hope that this innovation will eventually bring us “modeling and code generation for the masses”.

4 Responses to “ESD closes shop. What’s next in store for embedded programming?”

I do agree with you that embedded software is it’s golden time. I even think it will still grow a lot in the coming years. I own an electronics company (IKALOGIC) and it’s in deed quite hard to find talented embedded software designers. As you said all student (or most of them in informatics institutes here in France) spend way too much time learning things like JAVA, and not enough time learning C…

What difference does it make what programming language they get at school? I have seen and hired loads of grads and the only ones that were already good at programming learned it by themselves. But even then, i think most programmers really start to learn the trade after school.

Another thing. Back in 1995 i worked for a programmers body shop that was then specialized in embedded. I was leading the transition from embedded to desktop development. The reason was simple: money. An engineer with years of experience who knew all corners of C and assembly was billed for 30% less then some kid from university with 4 weeks of Visual Basic training.
Of course there were many who could not care much about salary and simply loved their work, but others did move from embedded to let’s call it enterprise development. And they didn’t regret it a minute: they got paid better and they quickly became recognized specialists (it takes a good C programmer just a week or two to master Java for example). We had actually several great success cases of guys who really moved to the top after they moved away from embedded.

Imagine you are young right now. With or without your C courses, i think websites like http://www.state-machine.com with a design from a past century seem to try hard to scare off (young) people. Compare for example http://rubyonrails.org/: it is fast, geeky and social. It is ironic to see that both projects are basically the same: it is about making developing more fun and productive. Even more dramatic: ruby on rails is old school, it does web applications like dad does at Google.

So IMHO if young grads are not interested in embedded it is all self inflicted by the industry itself. Or to say it differently: if my imaginary son with high IQ would consider an embedded career i would beat the crap out of him and point him to the countless discussions on 32-bit taking over 8-bit of the past decade. I hope that will make him understand that his talent will be better appreciated in another sector.

If you are a young and talented you will find a much more fruitful and fun career in say green tech, that is cool now.

Jaap,
I think that you miss the boat completely. The embedded software market is growing much faster than any other area of programming. In fact, I think that the embedded revolution hasn’t even started yet and that the big things (like robotics, “internet of things”, “smart grid”, you name it) are yet to come. But you don’t need to take my word for it. Big companies, like Intel and TI are pulling their resources from desktops, servers, and even tablets or smart-phones to re-focus on embedded (e.g., see the recent article “TI steering OMAP to embedded” at http://www.eetimes.com/electronics-news/4397207/TI-steering-OMAP-toward-embedded).

I came out of college in 1994 with the ability to code Assembly, C, and Pascal as well as SQL and Lisp. I loved the hardware side of things and knew how processors and memory worked but the embedded systems employers only wanted engineers and I was a CS guy. So off I went into RAD Enterprise development. I made great money and wasted years and years of creative talent drowning in oceans of over-abstracted VB/T-SQL and Java/PLSQL. Dismal performance on highly abstracted systems seemed to be all the rage. During my last development work for a large corporation, I saw them (like many others) starting to move away from hugely expensive, unsupportable “enterprise” applications and beginning to adopt a smaller, faster attitude and approach. They were starting to do agile development on mobile platforms; still farting around with bytecode and virtual machines, but getting closer to clarity. If my high IQ son starts learning how a virtual machine (or CLR) works instead of diving into the actual function of the processor or micro-controller, I’ll beat the crap out of him and point him to documentation on processors, fpga’s and micro-controllers.