Verify fail error after upgrading to Mysensors 2 and adding signing

Hi,
I have a network of sensors and actors running mySensors. They were all running on MySensors Version 1.5 without problems. Today I decided to update to Version 2 (downloaded it directly from this site) and while I am at it to also add signing for security reasons.

What I did:

Run the SecurityPersonalizer on my new gateway and creat a HMAC and serial key

Burn the serial gateway sketch, disable the old gateway in domoticz and add the new one

Taking all my nodes and with each: use the clearEEPROM sketch, use the securityPersonalizer sketch with my gateway HMAC key and burn the new update 2.0 sketch

I ignored that one node and added others (my RGBW controllers) without a problem. Afterwards I retried the old nodes, but all senors had the same errors. One of the first sensors (that worked) had also stopped sending data, so I powercycled it and got the same kind of error on the gateway.
Any idea why this is happening, or how I can fix it? Would really love to use my sensors again

Ok it definitly is a signing error. When I remove the signing related defines everything works just fine. Any idea how this could work on some nodes but not on others? Anything I can do to reset this or something?
It seems like all the actuators (RGBW controllers) are working just fine, while the sensors don't. Is signing always handled at the receiving side? Then perhaps something's wrong with the gateway handling the messages?

PS I disabled the inclusion mode in my gateway sketch as I don't want to use additional buttons. Could not find much information about it. Is it only need with vera? Might that have anything to do with my problems?
Here is my gateway sketch, as it might matter for this problem:

I kept testing today but I just can't find the error. The gateway is a arduino nano with an nrf24L01 and a capacitor on it, the node is (for this test) powered by the serial converter (so no power issue possible imo).
Here is the serial ouput of the gateway:

Another thing to test is to see if your EEPROM has been corrupted/overwritten. You can configure the SecurityPersonalizer to not write/update any keys (disable the STORE-defines) and see what the stored data is (if for instance the HMAC key was somehow altered).
From what I can see the nonces and signatures are exchanged correctly, and that leaves only the HMAC key and whitelisting operations as "unknowns". Whitelisting problems usually mean an incorrect serial used at the sender side compared to what the receiver has in it's list. You see the message when a node identifies the sender as one present in it's whitelist and the node is configure to have a whitelist.
In your logs above, the GW (which is in your nodes whitelist) only sends one signed message to the node, and this show on the sender (GW) side as:0;255;3;0;9;Signature salted with serial
and at the receiver (node) as:Sender found in whitelist

To me, it looks a lot like your HMAC has somehow been altered. Do you write to EEPROM in your sketches?

Skipping whitelisting changes nothing. None of my nodes write to EEPROM.
So I guess the HMAC is somehow wrong. I did not check it as my nodes haven't had the headers soldered on for serial connections, I program them directly via USBasp. I did just add the headers for one node but only got gibberish on the console. I'll try to check again tomorrow.

@LastSamurai ok, eeprom corruption or overwrite is the only plausible explanation I have from looking at your logs. Everything is the same between sender and receiver except the resulting hmac (signature) and the only part involved at that stage is the hmac key. And Judy one bit wrong will result in a completely different signature. In 2.0.0 version of the library, more data is used u the library in eeprom so if there is code that writes to eeprom which does not take into account the area reserved by the library it might inadvertently overwrite parts of eeprom used by the library. The soft serial and hmac keys are part of this. You could also try to just dump the eeprom using your programmer to verify that it contain the secrets you specified with the personalizer.

@Anticimex I think you are right. I did upload the securityPersonalizer sketch but I guess it did not write anything. I just downloaded the EEPROM via AVRDUDESS and only got a whole bunch of 0xff's. Although I am not 100% sure if I did it right
My nodes have different fuse settings (running at 8MHz internal only and 1.8V brown out). 115200 serial doesn't seem to work reliably at that speed, so I normally changed the serial speed to 9600. Can there be any problem with that and the securityPersonalizer sketch? Can I just use a

#define MY_BAUD_RATE 9600

at the top of the sketch? Because when I tried it earlier I only got nonsense on the console. Ill try it again tomorrow though, perhaps I missed something.

I did just try it with another node and signing worked there without a problem. Even with the different serial speeds.
I noticed with that node that I did set the HMAC key, but not serial key for the node itselft (which then results in FFFFF.... I think). Might this cause the problems I mentioned? Might be some kind of bug (although one should really create a serial number for the nodes ; ) ).

@LastSamurai serial number is only used for whitelisting. If you use soft signing you need to configure the personalizer to store it and it should be unique for every sensor board. If any node (or gateway) require whitelisting, your node will use the serial as salt for the signature and the receiver will look up the serial in its local whitelist if it finds a match for the node ID of your node and do the responding salting to verify the signature.
I'm other words, if you use whitelisting, you have to match the whitelists with the serials of the nodes that communicate.
The signing documentation describes how to do this.

From your logs, however, I would say it is a hmac mismatch. The verbose debug will show if serials could have been the problem. Typically by asking if the sender is in the whitelist.