Oracle Blog

Tips and Tricks for the Solaris System Administrator

Live Upgrade on a Laptop ?

It is fascinating that one of the most interesting features in Solaris isn't
new - in fact it's been around for quite some time. I'm referring to Live Upgrade which allows the installation or upgrade of an alternate
boot environment while you are doing productive work in
another. And why would I want to do this, especially on a laptop ?

I can think of several reasons.

Live Upgrade can be completely driven from ISO disk images of installation media.
This means that I don't have to burn DVDs or CDs, or even better - no more late
night runs to CompUSA or Fry's before the big presentation in the morning.
And I won't miss the sound of the DVD reader grinding itself to death every time
I install Solaris.

It simplies running multiple versions of Solaris, significantly.
One of the most frequent questions we encounter in Installfests is which OS to run ?
Solaris 10, OpenSolaris Community Edition, Solaris Developer Express, the nightly
OpenSolaris BFU. And the answer is all of them. OK, maybe not for everyone - but
even if you are going to choose one OS, upgrading to the newest update or applying a
set of patches safely, while keeping the older environment as a fallback is a good
thing, especially if you are on the road.

And best of all, Live Upgrade is quite simple as long as you follow the rules - and
fortunately there are only a few.

The plan

As with most things, the critical first step is the plan - and the earlier the better. For Live Upgrade
this means the disk layout prior to installation. You will have to reserve some space for
your alternate boot environment(s) when you initially install Solaris. While not strictly true, the mobile laptop user with a single disk drive will find the task of relocating data and repartitioning your disk
after the fact rather complicated. In other words, you are better off planning in advance.

I use three - one for my customer facing
Solaris 10 environment, one for my Open Solaris Community Edition daily driver, and one for
whatever super-cool-gotta-have OpenSolaris project that is only available via BFU (today that is Xen, but it has been other things in the past and will likely be so in the future). And we
all know that BFU means Better Forget Upgrades - but I have found that Live Upgrade is a
simple way of preparing for a BFU. This third boot environment is also used as a backup
during major Solaris updates (yeah, I would really like a 4th and 5th, but at some point you
have to stop). Three is good.

How big do these need to be ?

That depends on how much stuff you carry around - specifically how
much /opt will grow. There are some techniques you can use to reduce the storage requirements (sharing
/opt directories across boot environments), but these tricks make other things more complicated (like zone creation). 6GB makes a good
root file system, perhaps a bit larger if you start installing packages from Blastwave.
Solaris Developer Express with the Studio development tools and Netbeans environment slightly
larger - 8GB to 10GB.

At first glance you may find the documentation a bit intimidating. Perhaps this is why more people aren't
adopting Live Upgrade in their environment. Live Upgrade can do all sorts of spectacular things such as installing from a
flasharchive, splitting and joining file systems, breaking Solaris Volume Manager mirrors - none of
which apply to our current situation. In fact when you strip Live Upgrade down to the few parts that we
will be using, it is amazingly simple. Here is an outline of the steps you will perform (after thoroughly reading
the documentation - I have to say that).

Check Infodoc 72099 to make sure all of the prerequisite patches are installed (and remember to
do this each time as Live Upgrade is an evolving product).

Create the new boot environment if it doesn't already exist. You can name your current
boot environment the very first time you do this.

Download the DVD ISO image for the new Solaris release or update

Mount the ISO disk image

Install the packages SUNWluu, SUNWlur (and if present, SUNWlucfg) from the installation media
(in Solaris_nn/Product) in your current boot environment. This is a critical step - incorrect Live Upgrade software is the most
common cause of failure.

Run luupgrade -u to upgrade your new boot environment

Check the upgrade logs in /var/sadm/system/logs in the new boot environment. The boot environment is
mounted as /a during the upgrade, or you can use lumount after the upgrade is complete.

Activate the new boot environment using luactivate

init 0 (also critical as K0 RC scripts are installed to perform important steps like updating your
bootloader configuration, syncing system files, and building a good boot archive.

It's really that simple. Let's see this all in action.

Preparing for Live Upgrade

I'm starting with a Solaris 10 6/06 system, freshly installed with my root file system
(aka the primary boot environment) in c0d0s3. From my disk partition table you will see that I
have two other boot environments available: c0d0s0 and c0d0s4. I plan on putting Solaris 10 11/06
in c0d0s0 and the latest OpenSolaris Community Edition (aka Nevada) in c0d0s4.

Since I've already read the documentation a few hundred times, I can proceed to patch analysis.

After comparing the patch list in Infodoc 72099 with what I have in Solaris 10 6/06 (showrev -p), I see that I am behind on
four patches: 119255, 118855, 112653, and 121003. I fire up our new best friend updatemanager
and see that I also need 121119 and 119255. These two will point updatemanager at the
new repository.

So I apply 121110-08 and 119255-27, and restart updatemanager. This lets me find the rest of
the patches that I will need.

I then apply 119255-32, 118855-19, 112653-04 and 121003-03. None of these required a reboot, so
we can proceed immediately to the next step.

Creating the new boot environment

Since I will be upgrading from media and not a flash archive, I have to create a complete new boot
environment, based on my current installation. This step will essentially clone the current
system. As a side note, there are lots of fascinating uses for this that we will explore later.

Since this is the first time I am using lucreate, I also get to name my current boot environment.
Future invocations of lucreate will be done without the -c argument.

# lucreate -c sol10u2 -n sol10u3 -m /:c0d0s0:ufs

This magic sequence told Live Upgrade to name the current boot environment sol10u2 and
to create a new one called sol10u3 with the new UFS root file system on /dev/dsk/c0d0s0.

At this point all of the system files from the current root file system are being copied
to the new boot environment. On this laptop and with a rather complete Solaris install, it
takes about 45 minutes.

Upgrading Live Upgrade

If you have not already done so, download and unpack the DVD ISO image for Solaris 10 11/06
(or whatever media you wish to use).

Perform a Live Upgrade

We will now run luupgrade(1m) to upgrade the new boot environment. Since Live Upgrade doesn't
support non-global zones you should delete all installed zones in the target boot environment.
This restriction is relaxed in the latest OpenSolaris snapshots. Since I'm starting with a clean
Solaris 10 6/06, there are no non-global zones, so I can proceed.

# luupgrade -u -n sol10u3 -s /mnt

This magic sequence says to run upgrade from the media source /mnt (mounted in an earlier step)
against the boot environment sol10u3. The process will go something like

Extract the miniroot from the source media

Mount the new boot environment as /a

Remove all of the packages from the new boot environment

Install new packages from the source media

Install a new boot archive

Unmount the new boot environment

On this laptop the process takes about 2 hours. During the upgrade you can watch the
process by doing something like

# tail -f /a/var/sadm/system/logs/upgrade_log

When the upgrade is complete, check the logs for any failed package installations. Some are
harmless, but all should at least be investigated. Also look at /var/sadm/system/data/upgrade_cleanup
for a list of files that had conflicts, and what Live Upgrade did to resolve them. Most of these
are harmless, but you should check to see if anything looks out of place.

Activating the new boot environment

The last step is to activate the new boot environment. This is all done by a K0 script, so after
activation it is imporant that you use shutdown or init 0. Do not use reboot as this bypasses the
normal shutdown process and the K0 scripts will not run. This is also an FAQ!

# luactivate sol10u3
# init 0

At this point some serious magic will be invoked. The K0 script will update your bootloader
configuration (in this case /boot/grub/menu.lst on the boot partition), a new boot archive
will be built (just in case), and commonly modified system files will be synced. This last
step is driven by /etc/lu/synclist.

Extremely important GNOME warning: GNOME configurations are not generally compatible
across releases. If you are switching between Solaris 10 and Nevada, OpenSolaris, or Solaris
Developer Express you are advised to maintain separate home directories so that your GNOME
configurations (.gnome, .gnome2, .gconf) don't collide. If you choose different Solaris users
then you are done. If you are just changing your home directory in /etc/passwd then Live
Upgrade will gladly send your old directory setting to your new boot environment - and that's
not at all good. Either remove /etc/passwd from the file synclist or remember to log in
via command line or failsafe session once you've activated and booted the new boot environment
and manually change /etc/passwd.

And that's really about it. Recovery is rather simple. After activation you will
have all of your boot environments available for selection. Skipping the activation
step isn't recommended as it bypasses various sanity checks and the file sync process,
but when recovering a broken boot environment I think I can deal with things like
old passwords.

Practice makes perfect

One more time, this time putting a nevada build 57 in c0d0s4. The only change here
is that nevada adds a new LU package: SUNWlucfg. The magic sequence looks something like

Upgrading existing Boot Environments

Once you have used all of your boot evironments your next upgrade will require you to
either delete one of them and start again or upgrade an existing boot environment. The differences are minor
(ludelete and lucreate versus lurename and lumake), but there are a couple of
things have to consider - if you don't then this will come back to haunt you
wnen you least expect it.

Since there is no livedowngrade, you can only upgrade from an older version to a newer version. The
implication is that you can't use Live Upgrade to install Solaris 10 from a Solaris 11 based system
(Nevada, Solaris Express Community Edition, Solaris Express Developer Edition). That's not exactly correct, you
can install a Solaris 10 from a flasharchive, but that's an installation, not an upgrade.

Now you know why I have three boot environments. I leave an old Solaris 10 around as a Live Upgrade source and then alternate the other two between whatever is the most interesting - today that is Solaris 10 11/06 as the legacy and Solaris 10 8/07 and Nevada in the ping pong set. Now that Solaris 10 8/07 has been released, it will become the legacy Solaris environment and I will go back
to alternating between Nevada builds with and without xVM.

Another consideration, and common source of problems, is that Live Upgrade expects that the target enviroment match the source environment. As an example, lets assume that I am driving a Live Upgrade from Solaris 10 to one of my Nevada ping pong environments. It seems attractive to just upgrade one of the boot environments as it is - but it doesn't match the source. Some of the time, even most of the time, this will seem to work - but there will be times when drivers will not get replaced and you will have a hard to debug mess. The simple solution is to remember to lumake before you luupgrade.

One last thing - remember to check back to Infodoc 72099 as it does change occasionally - generally as a result of a Solaris release. But do check back to make sure
your patch levels are correct.

One last example - my current boot environment is s10u3 and it is running Solaris 10 11/06. I have an nv69
and nv71 running Solaris Express Community Editions 69 and 71. I want to put Solaris 10 8/07 in the nv69
boot enviromnent and call it s10u4.
After installing patches 119255-42, 121429-08, 126539-01, and 125419-01, the magic sequence goes something like

And when a new Solaris Express Developer Edition or the next update of Solaris 10 is available, just repeat all of the steps
above. I hope that this helps you understand an amazing Solaris capability and that
you start using this on all of your systems, including your laptop.

Copying failsafe kernel from media.
Uncompressing miniroot
Creating miniroot device
miniroot filesystem is <ufs>
Mounting miniroot at </mnt/Solaris_11/Tools/Boot>
Validating the contents of the media </mnt>.
The media is a standard Solaris media.
The media contains an operating system upgrade image.
The media contains <Solaris> version <11>.
Constructing upgrade profile to use.
Locating the operating system upgrade program.
Checking for existence of previously scheduled Live Upgrade requests.
Creating upgrade profile for BE <sol-nv-b63-x86>.
ERROR: You must install the Zones tool patches to use this set of Live Upgrade
packages to manipulate the boot environment at </>.
ERROR: Unable to create ICF file for boot environment <sol-nv-b63-x86>.
ERROR: Unable to determine root slice for BE <sol-nv-b63-x86>.

This is because you have non-global zones either configured or installed. Detach your zones using zoneadm detach and then remove the configurations with zonecfg delete. Once your non-global zones are gone then you can use Live Upgrade. Once your luupgrade is complete your
non-global zones can be reattached (zonecfg create -a and zoneadm attach).
This step will no longer be required once all of the zone live upgrade pieces get delivered into Solaris 10.

About

Bob Netherton is a Principal Sales Consultant for the North American Commercial Hardware group, specializing in Solaris, Virtualization and Engineered Systems. Bob is also a contributing author of Solaris 10 Virtualization Essentials.

This blog will contain information about all three, but primarily focused on topics for Solaris system administrators.