Links

Looking for UNIX and IT expertise? Why not get in touch and see how we can help?

The Solaris disk management/virtualisation tools have gone through many name changes – ODS, SDS, SVM – but the basic tools have remained the same. With the introduction of Sun Cluster, Sun needed to come up with a way to share storage between cluster nodes. Obviously this functionality needed to be added to SVM, and they came up with the idea of metasets.

Normally, metadevices are local to a host. When encapsulating disk slices into metadevices, you first have to dedicate a disk slice to store the metadatabase (with them metadb command). You can then create replicas of the metadb, and scatter them across slices, to ensure they don’t get corrupt and you lose all your metadevice information.

metasets are, in a nutshell, a collection of metadevices that have their own metadb state databases.

This works well in a cluster – the metaset metadevices, along with their metadbs, can be moved around, existing on the node which needs to mount their filesystems.

It also makes things easy for use when mounting LUNs from a SAN. If we can encapsulate the LUNs within metadevices, in their own metaset, then if our host dies we can just re-import everything on another host. Think of it as very basic – but very quick and easy – disaster recovery.

Think of metasets as being very similar to disk groups in Veritas Volume Manager.

Creating the metaset is a simple process. First of all we define our metaset and add our host to it.

bash-3.00# metaset -s test -a -h avalon

Syntax is pretty straightforward:

-s is used to specify which metaset we’re using

-a is the add flag. Guess what -d does?

-h specifies the hostname which owns this metaset

All metasets are owned by at least one host (it’s how they track who can access them). If you’re in a cluster environment, multiple hosts will own the metaset, allowing the cluster software to move the metadevices between nodes.

For a single hosted metaset, however, we just need to add one host, and we need to make sure that it will automatically take ownership and import the metaset on boot.

All we have to do to make this happen is enable the autotake flag on the metaset:

bash-3.00# metaset -s test -A enable

And that completes the setup of the metaset. We then just select which LUNs we’re interested in, and add them in to the metaset:

Note that when we add devices to a metaset (disks or LUNs) we need to only specify the device name – not slices, and not s2 (the Solaris way to reference an entire disk by a single reserved slice).

Normally, when you create a metadevice, you are encapsulation a slice that already exists on disk. This means the data stays intact. This is not the case when importing a disk into a metaset.

The act of importing a disk re-partitions it. All existing partitions are deleted, with a tiny slice on s7 being created to store the metadb replica, and the rest given over to s0. Note that s2 – the usual way of addressing a disk in Solaris – is also removed.

We can query the metaset and have a look at it’s contents, to check everything is OK:

bash-3.00# metaset -s test

Set name = test, Set number = 5

Host Owner
avalon Yes (auto)

Drive Dbase

/dev/dsk/c7t60060E80141189000001118900001A10d0 Yes

/dev/dsk/c7t60060E80141189000001118900001A17d0 Yes

/dev/dsk/c7t60060E80141189000001118900001A18d0 Yes

/dev/dsk/c7t60060E80141189000001118900001A19d0 Yes

Once we’ve populated our metaset, we create metadevices as normal. The only extras we need when using the metainit command is that we need to specify which metaset we’re using, and that we’ll always be using s0.

Let’s create a single metadevice striped across all 4 LUNs in our metaset:

metainit works in the same way it’s always done – we need to specify the full path to the slice we’re using – but with the additional -s flag to tell metainit which metaset we want to add the metadevice to.

We can use the summary flag to metastat (sorry, Solaris 10 only) to show us the summary of what we’ve just configured:

metasets are an easy way to group together storage and filesystems in Solaris, especially where the storage is external to your host, and you’d like the flexibility of importing it to another host in the future – for example, as part of some DR work if the host fails.

Looking for UNIX and IT expertise? Why not get in touch and see how we can help?

I came across this particular issue for a client, and it turned out to be a harsh gotcha in Solaris 9.

Quick recap: SVM metasets are a group of disks (usually from a SAN) that have their own meta state databases. They grew out of Sun Cluster as a way to share storage between cluster nodes, using SVM, and have since become a really handy way of managing SAN volumes.

Anyway, Solaris 9 4/04 introduced the ability to have ‘autotake’ metasets. Basically, one host was the master, and it could automatically import and manage the metaset on boot. This was great, because it finally swept aside the last baggage of Sun Cluster, and meant you could have your metasets referenced in /etc/vfstab and mount them at boot – just like real disks.

And there was much rejoicing across the land.

In this particular case, there was a host running Solaris 9 (for client software reasons) which had many terabytes of SAN LUNs mounted as metasets. I say had because when it rebooted, the machine said it couldn’t autotake the disk set because it wasn’t the owner, before dropping to single user mode complaining it couldn’t check any of the filesystems.

Odd. A quick check from single user mode, and yes indeed – the metaset was configured for autotake, but the host wasn’t the owner. Comment the (many) filesystems out of /etc/vfstab, continue the boot, and check again once at run level 3. Hang on – now the host is the metaset owner.

Whisky Tango Foxtrot, over. A quick Google threw up far too many suggestions to hack the startup scripts so that the SVM daemons start before the filesystem mounts. Not a great idea.

A very quick dig through Sunsolve turned up Sun BugID 6276747 – “Auto-take fails to work with fabric disks”
Turns out that this is an issue with the Solaris 9 SAN Foundation Suite, and how the kernel initialises SAN fabric LUNs, as opposed to FC-AL LUNs.

Adding the following like to /etc/system:

set fcp:ssfcp_enable_auto_configuration = 1

Followed by a quick reboot later, and behold! metasets are imported and mounted correctly, no further problems. This appears to be purely an issue in Solaris 9, so apart from old client apps I’m hoping we can leave this one behind.

Looking for UNIX and IT expertise? Why not get in touch and see how we can help?

Solaris Volume Manager has also been variously known as Disksuite or ODS (but not the Solaris Volume Manager which was the rebadged and bundled Veritas Volume Manager!) and comes with lots of neat features. One of the best is simple method for expanding metadevices and filesystems – on a live system!

For my example today, I’m going to use a Sun T2000 which has 4 zones, all of which are running from a metadevice composed of several 10gb LUNs from a SAN. The object of this exercise is to add another 10gb to the zone filesystem, so the developers can fire up another zone.

The fact that we’re using a metaset from a SAN doesn’t matter – the key thing here is that, by encapsulating a disk (or LUN, or partition) and using a metadevice, we can quickly and easily expand live filesystems.

First of all, let’s have a look at the metaset on the host, which has been imaginatively called ‘zones’: