Getting Started with Bare Metal ESP32 Programming

The ESP32 modules sold by Espressif are very popular in the IoT and embedded development space. They are very cheap, they are quite fast, they include radios and peripherals for WiFi and Bluetooth communication, and in some ways they even appear to bridge the gap between MCU and CPU. And Espressif provides pre-built modules with built-in antennas and external Flash memory, both of which appear to be required for general-purpose ‘IoT’ application development. They can be a bit power-hungry when they are using their wireless communication modules though, and I haven’t found much information on how to develop ESP32 applications without using the heavyweight (but very functional and well-written) “ESP-IDF” toolchain which is distributed by Espressif.

Usually, avoiding bulky and proprietary HALs is a worthwhile goal in and of itself. But Espressif has actually released their ESP-IDF toolchain under a very permissive Apache license, and it looks like a well-thought-out system with solid ongoing support. So if you are looking at starting a new project with the ESP32, I would personally recommend using the ESP-IDF to save time and effort. But sometimes it is nice to learn about how chips work at a deeper level, and ESP-IDF projects are often quite large, and they can take a long time to build depending on your environment.

The large code size also discourages what appears to be one use case that the chip was designed for: to load new instructions into RAM every time that it reboots from an external ‘socket’. The current crop of ESP32 modules use a SPI Flash chip as that ‘socket’, but if you put them in a factory or a field you might want to use Ethernet, or RS-232, or who knows what. I’m not sure how extensible the chip’s ROM bootloader actually is yet, but let’s take a look at what it takes to get a simple C program running on the ESP32 without using the ESP-IDF build system.

Unlike the STM32 and MSP430 microcontrollers which I have written about previously, there are not many software tools available for the ESP32 core. The ESP32’s dual-core architecture uses two ‘Xtensa LX6’ CPU cores which Espressif licenses from Cadence, and I haven’t seen them in any other mainstream microcontrollers. It looks like a core that is intended to be customized for the needs of an application as a step between general-purpose microcontrollers and something like an ASIC, so maybe it is more common in application-specific environments than general-purpose ones. In this case, it looks like the specific application which Espressif chose is wireless communication, and apparently a lot of the WiFi and Bluetooth code is burned directly into the ESP32’s ROM.

The ESP32 also looks more like a proper CPU than many microcontroller cores, with a few hundred kilobytes of on-chip RAM, a 240MHz top speed, an MMU, and support for up to 8 process IDs (2 privileged / 6 unprivileged) per core. People used to make do with much less, but since the ESP32 is complex and somewhat unique, Espressif provides the only toolchain that I know about which can build code for it. That means that while this tutorial will not use the full ESP-IDF development environment, it will still use Espressif’s ports of GCC and OpenOCD for compilation and debugging, as well as their esptool utility for formatting and flashing the compiled code. The target hardware will be either the ESP32-WROVER-KIT board which includes a JTAG debugging chip, or any of the smaller generic ESP32 dev boards (such as the ESP32-DevKitC) combined with an FTDI C232HM cable.

And like most of my previous tutorials, the software presented here is all open-source and you should be able to build and run it on the platform of your choice. It won’t have any colorful LEDs this time – sorry – but the code is available on GitHub. I’m also still learning about this chip and there is a lot that I don’t know, so corrections and comments are definitely appreciated. So if you’re still interested after those disclaimers, let’s get started by building and installing the toolchain!

ESP32 Toolchain Setup

Espressif provides good documentation for installing their toolchain, as well as for building it from source. I am impressed with their documentation, and the source build was fairly painless on an ARM architecture. So either follow the ‘Setup Toolchain’ steps for a pre-built Windows/Mac/Linux toolchain, or follow the steps to build it from source. Once it is installed, you should be able to check which version you have with:

> xtensa-esp32-elf-gcc --version
xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Your toolchain’s version will probably differ depending on when you are. Next, follow the steps to install the ESP-IDF package. We won’t be using its build system, but we will be looking at its ‘hello world’ example, using some utilities that it ships with, and inspecting its bootloader code to figure out how the system reaches a main method. Standing on the shoulders of giants…

I put the ESP-IDF files under ~/esp-idf/, but they can go anywhere; I’ll use $(IDF_PATH)/... to refer to files in the ESP-IDF source code.

Inspecting ESP-IDF

Our first question should be, ‘what happens when the system boots?’ This is sort of explained in section 3 of the ESP32 Technical Reference Manual, but it’s not very clear about a lot of the specifics. What memory address does the system boot to? How do I map my code to that address if the only re-writable nonvolatile memory is on an external SPI Flash chip? If some DRAM and IRAM memory spaces refer to the same physical memory, what do I put in the linker script? And so on.

Fortunately, the ESP IDF has an Apache-licensed bootloader project, which we can look at to see what gets things started. The project is located under $(IDF_PATH)/components/bootloader/, and there is some supporting logic under $(IDF_PATH)/components/bootloader_support/. The actual bootloader only has one bootloader_start.c source file a few directories down, and you can see in that file that a function called call_start_cpu0 is the entry point of the whole program.

It sounds like the ESP32 has a ROM bootloader which cannot be overwritten, and that “first-stage” bootloader drops the PC into a preset memory address in the chip’s Instruction RAM. I’m not very clear on this, but I guess that the ROM bootloader also fetches a pre-set area of the external Flash memory to place into that RAM space. I’m not quite sure how it decides what that address is yet, but the documentation suggests that it is a setting in the Flash MMU peripheral (see the ‘IROM’ section). Whatever the case, the default ‘single-app’ projects place the application at 0x10000 and the bootloader at 0x1000.

So to get a ‘hello world’ C program going, we need to link our code such that it maps to the right memory space in IRAM, and then we need to upload that code to where the ESP32 expects its ‘second-stage’ bootloader to be on the SPI Flash chip. Finally, we will verify that everything worked by stepping through the program in GDB.

When you have the luxury of working source code, one easy way to look at memory addresses is to compile and inspect an ELF file, and in our case that means building a basic ESP-IDF project. The framework ships with a few examples, and the documentation goes over how to configure and build one, so go ahead and follow the steps to copy the ESP-IDF ‘hello world’ project and configure it. Once you get to the ‘Build and Flash’ step, you can stop and just run make. If the build seems slow, try make -j4 to use 4 threads.

Once the project finishes building, it will leave three .bin files in the build/ directory, and two .elf files. There is the bootloader which gets written to 0x1000 in the SPI Flash, a partition table at 0x8000 which tells the bootloader which images are available to boot, and since the default setting is to only build one application image, the ‘hello world’ application image gets written to 0x10000.

To find out where the chip boots to, let’s inspect the bootloader using an nm command: xtensa-esp32-elf-nm build/bootloader/bootloader.elf. It will spit out a list of where various parts of the program are placed in memory, like this:

So we can see that the entry call, call_start_cpu0, is located a bit after 0x40080000 and the bootloader expects to call a call_user_start_cpu0 function which is defined elsewhere as the entry point to the user’s application. You can also look in the linker script under $(IDF_PATH)/components/bootloader/subproject/main/esp32.bootloader.ld to see how the bootloader’s memory is laid out and why; it looks like the ‘program text’ section starts at 0x40080400 to leave room for a vector table, but a word of warning: I think that the esptool formatting which we will talk about later might also relocate some of these memory segments.

Once the project builds, run make flash monitor and watch as the demo runs and prints to the board’s Serial output. That should verify that your toolchain is working, but the simple test program that I’m going to go over won’t have GPIO or UART outputs – remember when I said that you should use the ESP-IDF for any real project? So let’s also go over how to step through an ESP32 program using GDB.

OpenOCD and GDB

Before we go any further, you’ll need a way to debug the project – our example is only going to increment a variable forever, so we’ll need to be able to check what that value is to make sure that it works. So go ahead and install Espressif’s port of OpenOCD – they have instructions available here. You’ll also need to connect an appropriate JTAG interface to the ESP32 in order to use OpenOCD; like I mentioned in the first few paragraphs of this post, you can use a C232HM cable or breakout board with the .cfg files that ship with the ESP32 port of OpenOCD. If you have an ESP32-WROVER-KIT board (the one with an LCD display on one side,) it already has one of those chips built in. Otherwise, you’ll need to connect your own.

If you’re using a different development board, you can connect a C232HM cable to the following ESP32 pins:

ESP32 Pin #

JTAG Signal / Wire Color

14

TMS (Brown)

12

TDI (Yellow)

GND

GND (Black)

13

TCK (Orange)

15

TDO (Green)

Espressif’s OpenOCD port has a bin/ directory with the OpenOCD binary, and a share/ directory with scripts for connecting to the ESP32. Once you have everything connected properly, you should be able to open an OpenOCD interface with the command from the Espressif documentation linked above. Depending on how you install it, it might look something like:

Once the connection is open, we can connect to GDB over local port 3333, and step through the ESP-IDF bootloader. At the time of writing, it looks like Espressif recommends creating a script with a handful of initialization commands, and passing that to GDB. So leave the OpenOCD connection open, and in a new window, create a new file in your project directory called gdbinit with the following commands:

You can debug the program normally from here; if you enter continue, the bootloader should halt at the start of its C entry method, and you can step through it with the usual commands like next, step, etc. Espressif has a guide for command-line GDB debugging which goes over some of those common GDB commands in more detail, if you need a refresher.

Writing a Standalone Application

Now that we know how to check whether a program is working without needing to use GPIO pins or a UART connection, let’s write a ‘hello world’ C program without using the ESP-IDF build system. The ESP32 doesn’t really have a concept of a vector table in Flash, because it doesn’t have any on-chip Flash memory. It looks like it expects you to relocate the interrupt vectors into RAM, but that is not required for a super-simple ‘hello world’ program so for the sake of simplicity I am going to gloss over interrupts for now.

We will need a linker script, but the Xtensa GCC toolchain looks about the same as the ARM Cortex-M/R GCC toolchain, so our linker script can look very similar to those of the STM32 chips which I have written about previously. The biggest difference is that the ESP32 only has no Flash memory – instead, we refer to IRAM (Instruction RAM) and DRAM (Data RAM). I’m still learning, but I think the main difference is that you can execute code from IRAM:

If you look at the memory map in the ESP32 Technical Reference Manual, you’ll notice that this only uses a subset of the available RAM on the chip. There are several different banks located at different memory addresses, and this simple test program doesn’t need very much memory, so for the sake of simplicity I only used the core ‘instruction’ and ‘data’ RAM segments used by the ESP-IDF bootloader. And we rely on that first-stage ROM bootloader which is permanently burned into the chip to pull our code out of the external SPI Flash chip and put it into the expected IRAM segment in the ESP32.

We will also need a main.c file containing a program to compile. For the sake of simplicity, I also included some minimal startup logic to copy bss/data segments into RAM with memset and memcpy:

With those files written, you can build the program with make – it should spit out a main.elf file. You can use a similar GDB command as we used to inspect the bootloader, as long as you have an OpenOCD connection attached to the chip:

xtensa-esp32-elf-gdb -x gdbinit main.elf

Flashing a ‘Hello World’ Application

To actually upload our test program, we need to format it for the ESP32 and then store it in the SPI Flash chip connected to the actual ESP32 within the module. We can do that with Espressif’s esptool utility. It should be part of the ESP-IDF download, but you can find more detailed instructions in the project’s GitHub repository. To format the ELF file into a binary image:

Note that you might need to specify a different port, depending on which system resource your ESP32 is connected to. And I think that this also depends on the partition table which we uploaded to 0x8000 in Flash as part of the test ‘hello world’ project. That sort of feels like cheating, but I haven’t quite figured out how the chip retrieves its code from Flash yet.

Anyways, once the image is flashed, you can use the same steps as above to connect with GDB – just replace build/bootloader/bootloader.elf with main.elf. You should be able to step through the program after it reaches its ‘main’ method, and observe that the poor sisyphus variable increments and overflows endlessly.

Conclusions

This isn’t actually useful for writing an ESP32 application; what are you going to do, write your own WiFi and Bluetooth drivers? But it is a fun learning exercise, and maybe it would be possible to access some of the ESP-IDF functionality from a ‘bare metal’ program like this, depending on how you built it.

Plus, I skipped some important things that I haven’t figured out yet, like where to put the interrupt vector table and what it should look like. But there you go – I hope this trivia about how the ESP32 works was interesting or helpful. And corrections are welcome as always; like I said, there’s a lot I still haven’t figured out. And you can find a repository with this example’s code on GitHub.

Comments (2):

Mahyar

July 1, 2019 at 4:41 am

This is an excellent write up, and a great start to making a tiny SDK. Also, thanks for the source code.

Related posts:

Several years ago, a company called Future Technology Devices International (FTDI) sold what may have been the most popular USB / Serial converter on the market at the time, called the FT232R. But this post is not about the FT232R, because that chip is now known for its sordid history. Year after year, FTDI enjoyed their successful chip’s market position – some would say that they rested too long on their laurels without innovating or reducing prices. Eventually, small microcontrollers advanced to the point where it was possible to program a cheap MCU to identify itself as an FT232R chip and do the same work, so a number of manufacturers with questionable ethics did just that. FTDI took issue with the blatant counterfeiting, but they were unable to resolve their dispute through the legal system to their satisfaction, possibly because most of the counterfeiters were overseas and difficult to definitively trace down. Eventually, they had the bright idea of publishing a driver update which caused the counterfeit chips to stop working when they were plugged into a machine with the newest drivers.

FTDI may have technically been within their rights to do that, but it turned out to be a mistake as far as the market was concerned – as a business case study, this shows why you should not target your customers in retaliation for the actions of a 3rd party. Not many of FTDI’s customers were aware that they had counterfeit chips in their supply lines – many companies don’t even do their own purchasing of individual components – so companies around the world started to get unexpected angry calls from customers whose toy/media device/etc mysteriously stopped working after being plugged into a Windows machine. You might say that this (and the ensuing returns) left a bad taste in their mouths, so while FTDI has since recanted, a large vacuum opened up in the USB / Serial converter market almost overnight.

Okay, that might be a bit of a dramatized and biased take, but I don’t like it when companies abuse their market positions. Chips like the CH340 and CH330 were already entering the low end of the market with ultra-affordable and easy-to-assemble solutions, but I haven’t seen them much outside of Chinese boards, possibly due to a lack of multilingual documentation or availability from Western distributors. So at least in the US, the most popular successor to the FT232R seems to have been Silicon Labs’ CP2102N.

It’s nice to have a cheap-and-cheerful way to put a USB plug which speaks UART onto your microcontroller boards, so in this post, I’ll review how to make a simple USB / UART converter using the CP2102N. The chip comes in 20-, 24-, and 28-pin variants – I’ll use the 24-pin one because it’s smaller than the 28-pin one and the 20-pin one looks like it has some weird corner pads that might be hard to solder. We’ll end up with a simple, small board that you can plug into a USB port to talk UART:

Drivers for the CP2102N are included in most popular OSes these days, including Linux distributions, so it’s mostly plug-and-play.

It’s worth noting that you can buy minimal CP2102N boards from AliExpress or TaoBao for about $1, but where’s the fun in that?

It has been about nine months since ST released their new STM32G0 line of microcontrollers to ordinary people like us, and recently they released some new chips in the same linup. It sounds like ST wants this new line of chips to compete with smaller 8-bit micros such as Microchip’s venerable AVR cores, and for that market, their first round of STM32G071xB chips might be too expensive and/or too difficult to assemble on circuit boards with low dimensional tolerances.

Previously, your best bet for an STM32 to run a low-cost / low-complexity application was probably one of the cheaper STM32F0 or STM32L0 chips, which are offered in 16- and 20-pin TSSOP packages with pins spaced 0.65mm apart. They work great, but they can be difficult to use for rapid prototyping. It’s hard to mill or etch your own circuit board with tight enough tolerances, and it’s not very easy to solder the chips by hand. Plus, the aging STM32F031F6 still costs $0.80 each at quantities of more than 10,000 or so, and that’s pretty expensive for the ultra-cheap microcontroller market.

Pinout and minimal circuit for an STM32G031J6 – you only really need one capacitor if you have a stable 3.3V source.

Enter the STM32G031J6: an STM32 chip which comes in a standard SOIC-8 package with 32KB Flash, 8KB RAM, a 64MHz speed limit, and a $0.60 bulk price tag (closer to $1.20-1.40 each if you’re only buying a few). That all compares favorably to small 8-pin AVR chips, and it looks like they might also use a bit less power at the same clock speeds. Power consumption is a tricky topic because it can vary a lot depending on things like how your application uses the chip’s peripherals or what voltage the chip runs off of. But the STM32G0 series claims to use less than 100uA/MHz, and that is significantly less than the 300uA/MHz indicated in the ATTiny datasheets. Also, these are 32-bit chips, so they have a larger address space and they can process more data per instruction than an 8-bit chip can.

Considering how easy STM32 chips are to work with, it seems like a no-brainer, right? So let’s see how easy it is to get set up with one of these chips and blink an LED.

As someone who likes both electronics and the outdoors, sometimes I get anxiety about a lack of electricity. It would be nice to go camping somewhere away from it all, and still be able to charge things and run some lights, a display, maybe a small cooler. I’m sure some of you are rolling your eyes at that, but I’ve also been wanting to play with adding aftermarket indicators to old cars, like backup sensors or blind spot warnings, and it’d be nice to run them off a separate battery to avoid the possibility of accidentally draining the car’s battery overnight.

Since low-power solar panels are fairly cheap these days, I figured that it might be worth buying a few to mount to my car’s roof. And since my car is technically a pickup, it was very easy to put the battery in the bed and run the wiring through the canopy’s front window:

I’ve secured the battery a bit more since taking these pictures, but this is the basic idea – it’s pretty simple.

If you have a different kind of car, I’d imagine that you could just as easily put the battery in your trunk, but you might need to drill a hole for the wires if you don’t want to leave one of your windows cracked open.

I guess that a lot of this guide won’t apply exactly to your situation, because you’ll have different dimensions to work with, different limitations, and probably different solar panels. But I hope that laying out each step that I took and what worked for me might be helpful – your basic approach could probably look very similar.

And before we go any further, please keep your expectations in check. These panels can only produce up to 100W in direct sunlight, which is nowhere near enough power for something like an electric vehicle. So read on if this sounds interesting, but the car still runs on gas. We’re not saving the world here.