twistedit blog

So last week I spent quite a bit of time trying to upgrade ACI 1.1 to the latest version of 1.3 with no success. Luckily, I was in the initial stages of building out the initial fabric, because the failure that I incurred forced me to 'setup-clean-config.sh <filename>' on the leaf and spines, and 'erase-config setup' on the APICs. This forced me to go back through the bootstrapping process on the APICs, and a rediscovery of the fabric. I did this 3x, until I finally punted, and tried to upgrade to 1.2 (which was successful).

I wish now, that I would have recorded the error I was getting, but what would happen is the first APIC would try to upgrade, and simply fail. It was hung, and none of the others would start the upgrade as the 'token' was not being passed to them. I tried to reload the failed APIC, but with no success. After the reboot the 'controller level' was set to 1.3, and that APIC was still at 1.1, so it would not join the cluster, and the only message I got from it was 'the messaging layer was unable to deliver the stimulus (rejected, peer is not ready).' This brought me to the Cisco community article: https://supportforums.cisco.com/discussion/12984291/messaging-layer-was-unable-deliver-stimulus-rejected-peer-not-ready and my ultimate conclusion..that I was stuck!

we are also running into a few bug trying to upgrade from 1.1 to 2.0. and i can tell you it isnt easy. to start with i needed to load the fireware via an external source instead of uploading the firmware under the firmware repo. because we are getting an error a few second into the upload job.

so i needed to setup a http server or a ssh server where the file where located and let the ACI controller download it via the external source firmware a bit complicated :/

well i got the firmware upload, and the next bug hit us.. and that the one you posted about, i can tell you that if you are on a 1.1 ACI controller you have to go to 1.2 first and then you should be able to go to the version you want lol..