[security] Migrating from library version 2.1 to 2.2

Once library version 2.2 is released (it is currently in beta and available on the development branch on GitHub) users that were using signing or encryption will have to re-do personalization. This is because in 2.2, a checksum has been added to the security data in EEPROM to avoid usage of tampered or accidentally wiped data.
Migrating the data is not complicated as personalization in 2.2 has been rewritten to be simpler to use.

There are a few options here. The simplest is to just re-do personalization from scratch. The new SecurityPersonalizer will guide you through the process. Just execute it unmodified on the device and follow the instructions in the serial console.

If you want to reuse an existing AES or HMAC key , follow this procedure:

Execute the SecurityPersonalizer example without any modification on your device. Then take note of the EEPROM contents displayed. It can look something like this:

Copy the HMAC and/or SERIAL and/or AES key (if you use soft signing) or only the AES key (if you use ATSHA204 based signing) and put the values into the following lines in the SecurityPersonalizer.ino file (leave the default values of the HMAC and SERIAL for ATSHA204 based signing):

Execute the sketch again. Now the data will be rewritten to EEPROM and a checksum will be calculated and stored. For hardware based signing, the "old" HMAC key will remain in use, but the checksum will be updated for the AES key (it will also cover the software HMAC and SERIAL, but they are not used for hardware based signing).

If you do not use, nor plan to use, whitelisting, you can leave the SERIAL value as is, and execute the modified personalizer without further modifications on every devie. If you plan to use (or already use) whitelisting, you need to rewrite the proper serial on all devices as well (users of hardware based signing can ignore this, as the serial of the ATSHA204A is used and is fixed to each device.Note that if you change the SERIAL compared to what was stored previously, you also will need to update all whitelists that has an entry for that node.

EDIT: Obviously, do NOT copy your ATSHA204A HMAC key and store it in EEPROM. It would negate the benefit of the ATSHA readback protection.

One thing that I must admit is that I am a little sad that the option I was advocating for about a year ago hasn't really become the option I was hoping for. I suspect that's because I didn't now enough about the difference between signing and encryption, so I was unable to argue well enough what I was looking for.

Perhaps it can still be a feature request for 2.3: the ability to just put a password at the top of the scripts, and then automatically have encrypted communication on the network. Without signing. As this allows me (and other relative n00bs) to keep using my low-memory Nano's. Because when I tried to implement the current easy option I ran out of memory on almost all the nodes. Which prompted my question about the future of MySensors, and the implied need to upgrade to ARM based devices.

Again, I know what I am looking for is very poor security. But all I want is to have a really easy pathway (high usability) that lowers the incentive for my neighbour to snoop my network. To go from "hey, look, a node has popped up in Domoticz. Let's see how much power my neighbour uses" to "no node has popped up in Domoticz".

As I read in the Raspberry gateway thread, creating a simple, encrypted network has in a way become harder. Now all nodes need their own unique key. Hence the sadness, as it seems that upgrading my Nano network to an encrypted state without having to dive into technical details, code generation, signing process and allround effort is actually further away than it was before.

@alowhum I don't understand what you mean? You have MY_SIGNING_SIMPLE_PASSWD. It enables signing and encryption with out any further configurations. Why would you not want to enable signing? And the password option is also available on the raspberry pi port as documented here: https://www.mysensors.org/apidocs/group__RaspberryPiGateway.html
The documentation is unclear on the implications of the password flag, but it does enable encryption as well: --my-signing-password=<PASSWORD>

The simple version is really great, don't get me wrong. Absolutely great usability!

The reason I would like a non-signing option is that it saves a lot of ram. By only using encryption it should be possible to continue using Arduino Nano's as nodes.

I tried using the MY_SIGNING_SIMPLE_PASSWD option on my Nano's, and the result was that most of my sketches became too big or unstable.

There just isn't enough space/ram for both hardcore security and sensor libraries.

Thanks for explaining that I might be able to 'criple' the MY_SIGNING_SIMPLE_PASSWD option. I would suggest that this 'crippling' could perhaps become a feature in itself called "MY_ENCRYPTION_SIMPLE_PASSWD (without the signing).

@alowhum I will consider it for 2.2.1. But I also need to mind the complexity of the overall functionality. I don't want the security solution to grow more into a beast than it already is.
The functionality you seek is really simple to implement. But gives even more options for a user to decide on. And although that is for some a great thing it is not for everyone.
I will see if I can somehow structure the documentation to outline all the configuration settings and try to give each a elaborate description on pros and cons.

@sineverba I also believe the signal report flag is reversed nowadays, and is an opt-in feature and not an opt-out feature, using MY_SIGNAL_REPORT_ENABLED which defaults to "off". Hence it is not listed in the memory savings section of the documentation, but the documentation of MY_SIGNAL_REPORT_ENABLED does warn that it adds about 1k of flash use.

Simple encryption and simple signing are now two separate functions you can call at the top of your script by adding a line with a password: MY_ENCRYPTION_SIMPLE_PASSWD and MY_SIGNING_SIMPLE_PASSWD.

You can also just put "MY_SECURITY_SIMPLE_PASSWD" at the top of your script, and that will do both in one go. This used to be called the MY_SIGNING_SIMPLE_PASSWD option, which also did both.

MY_SIGNING_SIMPLE_PASSWD is now
called MY_SECURITY_SIMPLE_PASSWD.
MY_SIGNING_SIMPLE_PASSWD only affects signing,
and a new flag, MY_ENCRYPTION_SIMPLE_PASSWD
only affects encryption.
MY_SECURITY_SIMPLE_PASSWD enable both these flags.

@alowhum you are welcome. Just remember that simple in this context also mean weak. Storing the secrets in the sketch is a huge security implication on targets that does not support readout protection. Atmga328p among others.

@alowhum in general where are two options, DualOptiboot which require an external spi flash but is radio agnostic, or the mysbooloader which have no requirements on external components but might need to be recompiled to match your radio settings.

@gohan hence my comment "in general", and in this sence I believe the board in question is "Can Arduino nano's be updated OTA?" and a nano is atmega328p based, and I believe it supports both bootloader variants. Of course there are some devices that might not support both, or perhaps even any of them, but as most of this discussion relates to resource limited nodes, I think only atmega328p based devices are considered.

Simple encryption and simple signing are now two separate functions you can call at the top of your script by adding a line with a password: MY_ENCRYPTION_SIMPLE_PASSWD and MY_SIGNING_SIMPLE_PASSWD.

You can also just put "MY_SECURITY_SIMPLE_PASSWD" at the top of your script, and that will do both in one go. This used to be called the MY_SIGNING_SIMPLE_PASSWD option, which also did both.

MY_SIGNING_SIMPLE_PASSWD is now
called MY_SECURITY_SIMPLE_PASSWD.
MY_SIGNING_SIMPLE_PASSWD only affects signing,
and a new flag, MY_ENCRYPTION_SIMPLE_PASSWD
only affects encryption.
MY_SECURITY_SIMPLE_PASSWD enable both these flags.

I want to follow upon this: I use RFM69 for transport, they have an encryption engine in hardware. Will there be any difference in time to process the message between using:

MY_SECURITY_SIMPLE_PASSWORD with soft encryption done on the ATmega or

MY_SIGNING_SIMPLE_PASSWORD and encryption on the RFM or

just MY_SIGNING_SIMPLE_PASSWORD.

If I'm not mistaken a signed message is the full 32 byes anyway, so the actual "airtime" will not change, but maybe the processing time before that will.

And yes, I am aware of the implications in total system security. But none of my nodes are accessible from the outside of my house. I am not worried about someone reading the contents of my sketch.

@bilbolodz Not sure what you mean. Then you have to send personalization sketch OTA and that is really not recommended unless you can do that in a secure way. And the personalizer is not designed to use any radios so you cannot expect to be able to FOTA over a new sketch after personalization. You will have to "bring in" your devices if you want to redo personalization.

So it means that (If I want use signing) I'm stuck with mysensors version 2.1?
I'm not able compile new code (using 2.2) and upload it over OTA because I will loose signing right (without changeing EEPROM content)?
I think there should be a way to migrate sketch (which is using soft signing) from 2.1 to 2.2 with OTA. Secure sending of keys actually is NOT a problem because (as I understood) new in 2.2 is "only checksum" of EEPROM.
So if I have "2.1 mysensors library" personalized devices, it's possible to write a program which gets existing data from EEPROM calculate checksum and write it to EEPROM (maybe also migrating other structures if there were other changes). Such sketch can be in safe way transferred via OTA because it doesn't contain any secret information. Then I can upload new (using 2.2 library) version of my sketch and I will have working "2.2" devices without touching it (only OTA). What do you think about it?

Actually now I've realised (luckily) that in these "hard to get devices" I'm using hardware signing so it should be not a problem (?)

@bilbolodz Correct, checksum is on EEPROM data, so you can make a sketch to read it and calculate the checksum yourself. It is all open source so you can just see in the personalizer how it is calculated and replicate that

@bilbolodz That is fine. But as I have currently no means of verifying, I will not make a update to the guide. If you find a way to do it, you are free to post a guide if other need to make a similar thing.

@anticimex I have a question - I have two GWs: one with ATSHA204A and one soft signing.
I have noticed that when doing #define PERSONALIZE_ATSHA204A, AES key is saved to ATSHA204A and EEPROM. Is this correct? In what case I need a combination of HMAC + AES keys? I do not do whitelisting

@alexsh1 it is worth mentioning that if you personalize using 2.2.0, you are free to go back to earlier versions without re doing personalization. It is only when going from 2.1 to 2.2 it has to be redone due to the checksum. Of course, be aware that overwriting the checksum will cause signing to fail again.

@alexsh1 re-personalize the node that lost the checksum. Each node (including gw) has a unique personalization and checksum (possibly). Just because one node looses the checksum or gets a mismatch does not mean any other node has the same problem. Everything is stored in eeprom locally.