After a flawless upgrade to ESXi 5.1 of one of my HP Microservers N36L, upgrading the second N36L gave some issues. I am using VMware Update Manager for the upgrade, which failed at 93% with the following error: “Cannot execute upgrade script on host“.
A short investigation of both /var/log/vmware/vua.log on the ESXi host and C:\Users\All Users\VMware\VMware Update Manager\Logs on the Update Manager didn’t offer any clues. I ran into this VMware KB article which is about quite the same issue, although the article is talking about an upgrade from 4.1 to 5.0.
Unfortunately I didn’t have the error which is mentioned in the KB article on my systen: “OSError: [Errno 39] Directory not empty: /bootbank/state.XXXXXXX (where XXXXXXX is a number)”. This didn’t stop me investigating the /bootbank directory. The /bootbank/ directory didn’t had a state.XXXXXXX directory, but there was a /altbootbank/state.92793/ directory available on the system which contained a local.tgz. Local.tgz contains a system state backup of your ESXi server. More information about the two bootbanks is available in this VMware PDF:
“The ESXi system has two independent banks of memory, each of which stores a full system image, as a fail-safe for applying updates. When you upgrade the system, the new version is loaded into the inactive bank of memory, and the system is set to use the updated bank when it reboots. If any problem is detected during the boot process, the system automatically boots from the previously used bank of memory. You can also
intervene manually at boot time to choose which image to use for that boot, so you can back out of an update if necessary.”
So /altbootbank/ is used in a upgrade scenario, to have a fail safe option. I decided to move the local.tgz (in /altbootbank/state.92793/) one directory higher. This did the trick and the Update Manager successfully deployed the upgrade…Well, almost. At the end of the upgrade, the ESXi server reverted to an old network configuration. Probably to a networkconfiguration which was in the local.tgz? (Anybody additional information on this?).
After a manual reconfiguration of the network settings, everything is working fine now.