While I understand where you coming from (Automake not working for some libraries), from an architectural PoV that does not really make sense. The Automake script is trying to take code and compile that into a standalone application that can be run on an OS; as the ESP32 does not have a traditional OS that can load and run standalone programs, the compiler is right in refusing to provide a binary for this. I'd say the issue here is that Automake does not take into account that libraries can be used for embedded purposes. (And this is not the only place it does that; from experience, once you get past the compiler stage, you'll get more errors because the Automake scripts thinks that host stuff is embedded stuff.)

How is ESP32 different here from every other architecture? E.g. AVR? Embedded ARM? They all allow creating an ELF file. It should always be possible to create an ELF file, even if that on its own is useless. I fail to see why xtensa-esp32-elf would need to be any different than e.g. arm-none-eabi-elf. What you need for embedded ARM is an image as well, just like for ESP32, but it still allows you to create ELF binaries as an intermediate step. As a matter of fact, even ESP-IDF does that: It creates an ELF as an intermediate step, which is then used as input for an image. The problem is that the toolchain is incomplete and essential parts of it moved to ESP-IDF, rather than part of the toolchain. As someone who has dealt with an insane amount of toolchains, worked on compilers, etc. let me make this absolutely clear: The current state of your toolchain is that it is broken.

I recommend you look at other toolchains that also target embedded on how to deal with this. arm-none-eabi-elf with newlib is a good start. Other toolchains like amiga-gcc face similar problems, where as part of the toolchain you need to build some glue libraries so you can link together a binary at the end. It seems you already do use newlib, so you need to integrate that into your toolchain build rather than ESP-IDF. That solves the problem of the missing crt and libc, at which point you should be able to get an ELF file, even if a useless one.

Hi js,
Thanks for the feedback. Yes, we realize this is a defect in the current toolchain, and we do have a plan to move parts of the code which is currently in IDF into newlib/libgloss. The two reasons why it's not done yet are as follows. First, we still consider the "syscalls" layer of IDF to be in a state of a flux. It's easy to fix something when it is part of IDF and then to ship the changes. Changing same thing in newlib, rebuilding the toolchain, and requiring users to update it is a much more difficult proposition. So we'd like to be sure things are somewhat stable before starting to move certain parts of IDF code into newlib, moving newlib out of IDF into the toolchain (currently the copy in IDF is being used...), making sure we can link something without including IDF, etc. The second reason is practical one: we have not yet seen significant use cases for which having a standalone toolchain is important. If you have some use cases (other than producing an elf file which, as you say, is useless on its own), please let us know.

I can understand that development in ESP-IDF is easier. But I don't think the two are mutually exclusive . You can build it as part of the toolchain, and then build a newer version as part of ESP-IDF and use that. Once you are happy with your changes and they are stable, you can just copy them back to the toolchain .

As for not having seen significant use cases: This is a hen egg problem, basically: People want to port libraries to ESP32, but the toolchain is broken. Since there are no libraries that would require a non-broken toolchain, the toolchain is not fixed. Since there is no non-broken toolchain, no libraries get ported .

You can see in this thread alone that there are several people trying to port libraries and currently cannot because of the broken toolchain. I am one of those, and I was actually asked by one of your users to port my library - so demand is there, it seems .

ESP_igrr wrote:The second reason is practical one: we have not yet seen significant use cases for which having a standalone toolchain is important. If you have some use cases (other than producing an elf file which, as you say, is useless on its own), please let us know.

I'm a bit out of my comfort zone when it comes to building compilers, but I think I might have a use case for exactly that. As you may or may not be aware, I'm using CLion as my IDE. I previously made a post about getting CLion to work with the extensa toolchain, but I have since learned that it only works if you first compile using idf.py and then use the same build directory in CLion.

As part of a new tool chain setup, CLion tests the configured compiler/linker with a very simple application:

First of all, great work on the CMake based build system so far. I just finished transitioning our own project and everything works like a charm.

However, I did notice that some crucial functionality is missing in idf.py, like write_flash and read_flash from esptool.py. We use read_flash to read back the contents of the chip and validate that flash encryption is enabled. We use write_flash to, for example, flash the SPIFFS image. Without being able to flash the SPIFFS image at the file system offset, we won't be able to fully move to the CMake based build system just yet.

Can you please let me know when you plan to release read_flash (for reading back the chip content) and write_flash (for flashing the SPIFFS image) in idf.py?

However, I did notice that some crucial functionality is missing in idf.py, like write_flash and read_flash from esptool.py. We use read_flash to read back the contents of the chip and validate that flash encryption is enabled. We use write_flash to, for example, flash the SPIFFS image. Without being able to flash the SPIFFS image at the file system offset, we won't be able to fully move to the CMake based build system just yet.

How are you doing this at the moment with the Make-based build system? If it's by calling esptool.py directly, this is still possible in the CMake based build system - in fact idf.py is calling esptool.py behind the scenes.

Or are you looking for more build system integration, so (for example) a flashing target knows to flash the SPIFFS partition automaticaly?

I left a comment on the PR as well, but Generic CMake support is almost ready for merging to master. This should enable interoperability between non-ESP-IDF CMake projects and IDF. The new support includes two example projects - one where an unmodified CMake library project is included as a component in an ESP-IDF project, and one where a generic CMake executable project pulls in ESP-IDF as a library subproject.

Who is online

About Us

Espressif Systems is a fabless semiconductor company providing cutting-edge low power WiFi SoCs and wireless solutions for wireless communications and Internet of Things applications. ESP8266EX and ESP32 are some of our products.