Administering Device Groups

As your cluster requirements change, you might
need to add, remove, or modify the device groups on your cluster. Sun Cluster provides
an interactive interface called clsetup that you can use
to make these changes. clsetup generates cluster commands.
Generated commands are shown in the examples at the end of some procedures.
The following table lists tasks for administering device groups and provides
links to the appropriate procedures in this section.

Caution –

Do not run metaset —s setname —f -t on a cluster node that is booted
outside the cluster if other nodes are active cluster members and at least
one of them owns the disk set.

Note –

Sun Cluster software automatically creates a raw-disk device group
for each disk and tape device in the cluster. However, cluster device groups
remain in an offline state until you access the groups as global devices.

Table 5–4 Task Map: Administering
Device Groups

Task

Instructions

Update the global-devices namespace without a reconfiguration reboot
by using the cldevice populate command

How to Update the Global-Devices Namespace

When
adding a new global device, manually update the global-devices namespace by
running the cldevice populate command.

Note –

The cldevice populate command does not have
any effect if the node that is running the command is not currently a cluster
member. The command also has no effect if the /global/.devices/node@nodeID file system is not mounted.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on any node of the cluster.

You
can run this command on all nodes in the cluster at the same time.

Reconfigure the namespace.

# cldevice populate

On each node, verify that the cldevice populate command
has been completed before you attempt to create any disksets.

The cldevice command calls itself remotely on all nodes, even when the
command is run from just one node. To determine whether the cldevice
populate command has completed processing, run the following command
on each node of the cluster.

# ps -ef | grep scgdevs

Example 5–21 Updating the Global-Devices Namespace

The following example shows the output generated by successfully running
the cldevice populate command.

Migrating the Global-Devices Namespace

You can create a namespace on a loopback file interface (lofi)
device, rather than creating a global-devices namespace on a dedicated partition.
This feature is useful if you are installing Sun Cluster software on systems
that are pre-installed with the Solaris 10 OS.

Note –

ZFS for root file systems is supported, with one significant exception.
If you use a dedicated partition of the boot disk for the global-devices file
system, you must use only UFS as its file system. The global-devices namespace
requires the proxy file system (PxFS) running on a UFS file system. However,
a UFS file system for the global-devices namespace can coexist with a ZFS
file system for the root (/) file system and other root file systems, for
example, /var or /home. Alternatively,
if you instead use a lofi device to host the global-devices namespace, there
is no limitation on the use of ZFS for root file systems.

The following procedures describe how to move an existing global-devices
namespace from a dedicated partition to a lofi device or the opposite:

A lofi device is now created on /.globaldevices and mounted as the global-devices file system.

Repeat these steps on other nodes whose global-devices namespace
you want to migrate from a partition to a lofi device.

From one node, populate the global-device namespaces.

# /usr/cluster/bin/cldevice populate

On each node, verify that the command has completed processing before
you perform any further actions on the cluster.

# ps -ef \ grep scgdevs

The global-devices namespace now resides on a lofi device.

How to Migrate the Global-Devices Namespace From a lofi Device to a Dedicated Partition

Become superuser on the global-cluster voting node whose namespace
location you want to change.

On a local disk of the node, create a new partition that meets
the following requirements:

Is at least 512 MByte in size

Uses the UFS file system

Add an entry to the /etc/vfstab file for
the new partition to be mounted as the global-devices file system.

Determine the current node's node ID.

# /usr/sbin/clinfo -nnode ID

Create the new entry in the /etc/vfstab file,
using the following format:

blockdevice rawdevice/global/.devices/node@nodeID ufs 2 no global

For example, if the partition that you choose to use is /dev/did/rdsk/d5s3, the new entry to add to the /etc/vfstab file
would then be as follows: /dev/did/dsk/d5s3 /dev/did/rdsk/d5s3
/global/.devices/node@3 ufs 2 no global

Unmount the global devices partition /global/.devices/node@nodeID.

Remove the lofi device that is associated with
the /.globaldevices file.

Adding and Registering Device Groups

How to Add and Register a Device Group (Solaris Volume Manager)

Use the metaset command to create a Solaris Volume Manager disk
set and register the disk set as a Sun Cluster device group. When you register
the disk set, the name that you assigned to the disk set is automatically
assigned to the device group.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Caution –

The name of the Sun Cluster device group that you create (Solaris
Volume Manager, Veritas Volume Manager, or raw-disk) must be the same as the
name of the replicated device group.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on one of the nodes connected to the disks where you are creating the disk
set.

SPARC: Solaris 9 only: Calculate
the number of names for Solstice DiskSuite metadevices or Solaris Volume Manager
volumes that you need for your configuration, and modify the /kernel/drv/md.conf file on each node. This step is not required if you are running
on Solaris 10.

Add the Solaris Volume Manager disk set and
register it as a device group with Sun Cluster. To create a multi-owner disk group,
use the –M option.

# metaset -sdiskset-a-M-hnodelist

-sdiskset

Specifies the disk set to be created.

-a-hnodelist

Adds the list of nodes that can master the disk set.

-M

Designates the disk group as multi-owner.

Note –

Running the metaset command to set up a Solstice
DiskSuite/Solaris Volume Manager device group on a cluster results in one
secondary by default, regardless of the number of nodes that are included
in that device group. You can change the desired number of secondary nodes
by using the clsetup utility after the device group has
been created. Refer to How to Set the Desired Number of Secondaries for a Device Group for more information about
disk failover.

If you are configuring a replicated device group, set the replication
property for the device group.

# cldevicegroup sync devicegroup

Verify that the device group
has been added.

The device group name matches the disk set name
that is specified with metaset.

# cldevicegroup list

List the DID mappings.

# cldevice show | grep Device

Choose drives that are shared by the cluster nodes that will
master or potentially master the disk set.

Use the full DID device name, which has the form /dev/did/rdsk/dN, when you add a drive to a disk set.

In the following example, the entries for DID device /dev/did/rdsk/d3 indicate that the drive is shared by phys-schost-1 and phys-schost-2.

Specifies the disk set name, which is the same as the device
group name.

-a

Adds the drive to the disk set.

Note –

Do not use the lower-level device name (cNtXdY) when you add a drive to a disk set.
Because the lower-level device name is a local name and not unique throughout
the cluster, using this name might prevent the metaset from being able to
switch over.

Verify the status of the disk set and drives.

# metaset -s setname

Example 5–22 Adding a Solaris Volume Manager Device Group

The following example shows the creation
of the disk set and device group with the disk drives /dev/did/rdsk/d1 and /dev/did/rdsk/d2 and verifies that the device group has been created.

How to Add and Register a Device Group (Raw-Disk)

Sun Cluster software supports the use of raw-disk device groups in addition
to other volume managers. When you initially configure Sun Cluster, device
groups are automatically configured for each raw device in the cluster. Use
this procedure to reconfigure these automatically created device groups for
use with Sun Cluster software.

Create a new device group of the raw-disk type for the following reasons:

You want to add more than one DID to the device group

You need to change the name of the device group

You want to create a list of device groups without using the
-v option of the cldg command

Caution –

If you are creating a device group on replicated devices, the
name of the device group that you create (Solaris Volume Manager, Veritas
Volume Manager, or raw-disk) must be the same as the name of the replicated
device group.

Identify the devices that you want to use and unconfigure any
predefined device groups.

The following commands remove the predefined
device groups for d7 and d8.

How to Add and Register a Replicated Device Group
(ZFS)

To replicate ZFS, you must create a named device group and list
the disks that belong to the zpool. A device can belong to only one device
group at a time, so if you already have a Sun Cluster device group that contains
the device, you must delete the group before you add that device to a new
ZFS device group.

The name of the Sun Cluster device group that you create (Solaris Volume
Manager, Veritas Volume Manager, or raw-disk) must be the same as the name
of the replicated device group.

Caution –

Full support for ZFS with third-party data-replication technologies
is pending. See the latest Sun Cluster Release Notes for updates on ZFS support.

Delete
the default device groups that correspond to the devices in the zpool.

For example, if you have a zpool called mypool that
contains two devices /dev/did/dsk/d2 and /dev/did/dsk/d13, you must delete the two default device groups called d2 and d13.

If the application resource group will run in local zones, create
a new resource group with the nodelist containing the appropriate local zones.
The global zones corresponding to the local zones must be in the nodelist
of the resource group created in Step #4. Set the +++ value
in the rg_affinities property from this resource group
to the resource group you created in Step #4.

Create an HAStoragePlus resource (hasp-rs)
for the zpool you created in Step #3 in the resource group that you created
in either Step #4 or #6. Set the resource_dependencies property
to the hasp-rs resource that you created in Step #5.

Do not
register the Oracle RAC shared disk groups with the cluster framework.

Maintaining Device Groups

You can perform a variety of administrative tasks for your device groups.

How to Remove and Unregister a Device Group (Solaris Volume Manager)

Device groups are Solaris Volume Manager disksets that have been registered with Sun Cluster.
To remove a Solaris Volume Manager device group, use the metaclear and metaset commands. These commands remove the device group with the
same name and unregister the disk group as a Sun Cluster device group.

Refer to the Solaris Volume Manager documentation for the steps to remove a disk set.

How to Remove a Node From All Device Groups

Use this procedure to remove a cluster
node from all device groups that list the node in their lists of potential
primaries.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on the node that you are removing as a potential primary of all device groups.

Determine the device group or groups of which the
node to be removed is a member.

Look for the node name in the Device group node list for each device group.

Verify that the node has been removed from the potential
primaries list of all device groups.

The command returns nothing
if the node is no longer listed as a potential primary of any device group.

# cldevicegroup list -v nodename

How to Remove a Node From a Device Group (Solaris Volume Manager)

Use this procedure to remove a cluster node from the list of potential
primaries of a Solaris Volume Manager device group. Repeat the metaset command
for each device group from which you want to remove the node.

Caution –

Do not run metaset —s setname —f
-t on a cluster node that is booted outside the cluster if other
nodes are active cluster members and at least one of them owns the disk set.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Verify that the node is still
a member of the device group and that the device group is a Solaris Volume Manager device
group.

Become superuser on the node
that currently owns the device group that you want to modify.

Delete the node's hostname from the device group.

# metaset-ssetname-d -hnodelist

-ssetname

Specifies the device group name.

-d

Deletes from the device group the nodes identified with -h.

-hnodelist

Specifies the node name of the node or nodes that will be
removed.

Note –

The update can take several minutes to complete.

If the command fails, add the -f (force) option to the
command.

# metaset-ssetname-d -f -hnodelist

Repeat Step 4 for each device group from which the node is being removed
as a potential primary.

Verify that the node has been
removed from the device group.

The device group name matches the
disk set name that is specified with metaset.

phys-schost-1% cldevicegroup list -v devicegroup

Example 5–23 Removing a Node From a Device Group (Solaris Volume Manager)

The following example shows the removal of the hostname phys-schost-2 from a device group configuration. This example eliminates phys-schost-2 as a potential primary for the designated device group.
Verify removal of the node by running the cldevicegroup show
command. Check that the removed node is no longer displayed in the screen
text.

How to Create More Than Three Disksets in a Cluster

If you are running Solaris 9 and intend to create more than three disksets
in the cluster, perform the following steps before you create the disksets.
You do not need to perform this procedure if you are running Solaris 10. Follow
these steps if you are installing disksets for the first time or if you are
adding more disksets to a fully configured cluster.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Ensure that the value of the md_nsets variable is high enough. The value should accommodate
the total number of disksets you intend to create in the cluster.

On any node of the cluster,
check the value of the md_nsets variable in the /kernel/drv/md.conf file.

If the number of disksets in
the cluster will be greater than the existing value of md_nsets minus
one, increase the value of md_nsets on each node.

The maximum permissible number of disksets is the value of md_nsets minus one. The maximum possible value of md_nsets
is 32.

Ensure that the /kernel/drv/md.conf file is identical on each node of the cluster.

Caution –

Failure to follow this guideline can result in serious Solaris Volume Manager errors
and possible loss of data.

From one node, shut down the
cluster.

# cluster shutdown -g0 -y

Reboot each node in the cluster.

On SPARC based systems, run the following command.

ok boot

On x86 based systems, run the following commands.

When
the GRUB menu is displayed, select the appropriate Solaris entry and press
Enter. The GRUB menu appears similar to the following:

You can run this command on all nodes in the cluster at the same
time.

From one node of the cluster,
run the cldevice populate command.

On each node, verify that the cldevice populate command has been completed before you attempt
to create any disksets.

The cldevice command
calls itself remotely on all nodes, even when the command is run from just
one node. To determine whether the cldevice populate command
has completed processing, run the following command on each node of the cluster.

# ps -ef | grep scgdevs

How to Create a New Disk Group When Encapsulating
Disks (Veritas Volume Manager)

You can convert nonroot disks to Sun Cluster device groups by encapsulating
the disks as VxVM disk groups, then registering the disk groups as Sun Cluster device
groups.

Disk encapsulation is only supported during initial creation of a VxVM disk
group. After a VxVM disk group is created and registered as a Sun Cluster device
group, only disks which can be initialized should be added to the disk group.

If you use VxVM to set up shared disk groups for Oracle RAC, use the
cluster functionality of VxVM as described in the Veritas Volume
Manager Administrator's Reference Guide.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on any node of the cluster.

If the disk being encapsulated
has file system entries in the /etc/vfstab file, make
sure that the mount at boot option is set to no.

Set back to yes after the disk is encapsulated
and registered as a Sun Cluster device group.

Encapsulate the disks.

Use vxdiskadm menus or the graphical user interface
to encapsulate the disks. VxVM requires two free partitions as well as
unassigned cylinders at the beginning or the end of the disk. Slice two must
also be set to the entire disk. See the vxdiskadm man page
for more information.

Shut down and restart the node.

The clnode evacuate command switches over all resource
groups and device groups including all non-voting nodes in a global cluster
from the specified node to a next-preferred node. Use the shutdown command
to shut down and restart the node.

# clnode evacuate node[,...]
# shutdown -g0 -y -i6

If necessary, switch all resource groups and device groups back
to the original node.

If the resource groups and device groups
were initially configured to fail back to the primary node, this step is not
necessary.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.administer RBAC authorization on any node of the cluster.

Determine the primary node for
the device group to which you are adding the new volume.

# cldevicegroup status

If the device group is offline,
bring the device group online.

# cldevicegroup switch -n nodename devicegroup

nodename

Specifies the name of the node to which to switch the device
group. This node becomes the new primary.

devicegroup

Specifies the device group to switch.

From the primary node (the node currently mastering
the device group), create the VxVM volume in the disk group.

Refer
to your Veritas Volume Manager documentation for the procedure used to create the VxVM volume.

Synchronize the VxVM disk
group changes to update the global namespace.

How to Assign a New Minor Number to a Device Group
(Veritas Volume Manager)

If device group
registration fails because of a minor number conflict with another disk group,
you must assign the new disk group a new, unused minor number. After assigning
the new minor number, rerun the procedure to register the disk group as a Sun Cluster device
group.

Become superuser on any node
of the cluster.

Determine the minor numbers
in use.

# ls -l /global/.devices/node@nodeid/dev/vx/dsk/*

Choose another multiple of 1000
not in use as the base minor number for the new disk group.

How to Register a Disk Group as a Device Group
(Veritas Volume Manager)

This procedure uses the clsetup utility
to register the associated VxVM disk group as a Sun Cluster device group.

Note –

After a device group has been registered with the cluster, never
import or export a VxVM disk group by using VxVM commands. If you make
a change to the VxVM disk group or volume, follow the procedure How to Register Disk Group Configuration Changes (Veritas Volume Manager) to register the device group configuration changes.
This procedure ensures that the global namespace is in the correct state.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Before You Begin

Ensure that the following prerequisites have been completed prior to
registering a VxVM device group:

Superuser privilege on a node in the cluster.

The name of the VxVM disk group to be registered as a device
group.

A preferred order of nodes to master the device group.

A desired number of secondary nodes for the device group.

When you define the preference order, you also specify whether the device
group should be switched back to the most preferred node if that node fails
and later returns to the cluster.

Nonprimary cluster nodes (spares) transition to secondary according
to the node preference order. The default number of secondaries for a device
group is normally set to one. This default setting minimizes performance degradation
that is caused by primary checkpointing of multiple secondary nodes during
normal operation. For example, in a four-node cluster, the default behavior
configures one primary, one secondary, and two spare nodes. See also How to Set the Desired Number of Secondaries for a Device Group.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on any node of the cluster.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with VxVM device
groups, type the number that corresponds to the option for device groups and
volumes.

The Device Groups Menu is displayed.

To register a VxVM device
group, type the number that corresponds to the option for registering a VxVM disk
group as a device group.

Follow the instructions and type the
name of the VxVM disk group to be registered as a Sun Cluster device group.

If this device group is replicated by using storage-based replication,
this name must match the replication group name.

If you use VxVM to
set up shared disk groups for Oracle Parallel Server/Oracle RAC, you do not
register the shared disk groups with the cluster framework. Use the cluster
functionality of VxVM as described in the Veritas Volume Manager
Administrator's Reference Guide.

If you encounter the following
error while attempting to register the device group, reminor the device group.

If you are configuring a replicated device group, set the replication
property for the device group.

# cldevicegroup sync devicegroup

Verify that the device group
is registered and online.

If the device group is properly registered,
information for the new device group is displayed when you use the following
command.

# cldevicegroup status devicegroup

Note –

If you change any configuration information for a VxVM disk
group or volume that is registered with the cluster, you must synchronize
the device group by using clsetup. Such configuration changes
include adding or removing volumes, as well as changing the group, owner,
or permissions of existing volumes. Reregistration after configuration changes
ensures that the global namespace is in the correct state. See How to Update the Global-Devices Namespace.

Example 5–25 Registering a Veritas Volume Manager Device Group

The following example shows the cldevicegroup command
generated by clsetup when it registers a VxVM device
group (dg1), and the verification step. This example assumes
that the VxVM disk group and volume were created previously.

How to Register Disk Group Configuration Changes
(Veritas Volume Manager)

When you
change any configuration information for a VxVM disk group or volume, you
need to register the configuration changes for the Sun Cluster device group. Registration
ensures that the global namespace is in the correct state.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on any node in the cluster.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with VxVM device
groups, type the number that corresponds to the option for device groups and
volumes.

The Device Groups Menu is displayed.

To register configuration changes,
type the number that corresponds to the option for synchronizing volume information
for a VxVM device group.

Follow the instructions and type the
name of the VxVM disk group that has changed configuration.

The following example shows the cldevicegroup command
generated by clsetup a changed VxVM device group (dg1) is registered. This example assumes that the VxVM disk group
and volume were created previously.

# clsetupcldevicegroup sync dg1

How to Convert a Local Disk Group to a Device Group
(VxVM)

Perform this procedure to change a local VxVM disk group to a globally
accessible VxVM device group.

Become superuser on a node of the cluster.

Start the clsetup utility

# clsetup

Unset the localonly property.

Choose the menu item, Device groups and volumes.

Choose the menu item, Reset a local VxVM disk group to a VxVM
device group.

Follow the instructions to unset the localonly property.

Specify the nodes that can master the disk group.

Return to the main menu in the clsetup utility.

Choose the menu item, Device groups and volumes.

Choose the menu item, Register a diskgroup.

Follow the instructions to specify the nodes that can master the
disk group.

When finished, quit the clsetup utility.

Verify that the device group is configured.

phys-schost# cldevicegroup show

How to Convert a Device Group to a Local Disk Group
(VxVM)

Perform this procedure to change a VxVM device group to a local VxVM disk
group that is not managed by Sun Cluster software. The local disk group
can have more than one node in its node list, but it can be mastered by only
one node at a time.

Become superuser on a node of the cluster.

Take the device group offline.

phys-schost# cldevicegroup offline devicegroup

Unregister the device group.

Start the clsetup utility.

phys-schost# clsetup

Choose the menu item, Device groups and volumes.

Choose the menu item, Unregister a VxVM disk group.

Follow the instructions to specify the VxVM disk group
that you are unregistering from Sun Cluster software.

Quit the clsetup utility.

Verify that the disk group is no longer
registered with Sun Cluster software.

phys-schost# cldevicegroup status

Command output should no longer show the device group that you unregistered.

Import the disk group.

phys-schost# vxdg import diskgroup

Set the localonly property of the disk group.

Start the clsetup utility.

phys-schost# clsetup

Choose the menu item, Device groups and volumes.

Choose the menu item, Set a VxVM disk group as a local disk group.

Follow the instructions to set the localonly property
and to specify the single node that is to exclusively master the disk group.

When finished, quit the clsetup utility.

Verify that the disk group is successfully configured as a local
disk group.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization on any node of the cluster.

Determine the primary node and
status for the device group.

# cldevicegroup status devicegroup

If the device group is offline,
bring it online.

# cldevicegroup online devicegroup

From the primary node (the node currently mastering
the device group), remove the VxVM volume in the disk group.

# vxedit -gdiskgroup-rf rmvolume

-gdiskgroup

Specifies the VxVM disk group that contains the volume.

-rfrmvolume

Removes the specified volume. The -r option
makes the operation recursive. The -f option is required
to remove an enabled volume.

Using the clsetup utility,
register the device group configuration changes to update the global namespace.

How to Remove and Unregister a Device Group (Veritas Volume Manager)

Removing a Sun Cluster device group
causes the corresponding VxVM disk group to be exported, not destroyed.
However, even though the VxVM disk group still exists, it cannot be used
in the cluster unless reregistered.

This procedure uses the clsetup utility to remove
a VxVM disk group and unregister it as a Sun Cluster device group.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.modify RBAC authorization
on any node of the cluster.

Take the device group offline.

# cldevicegroup offline devicegroup

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with VxVM device
groups, type the number that corresponds to the option for device groups and
volumes.

The Device Groups Menu is displayed.

To unregister a VxVM disk
group, type the number that corresponds to the option for unregistering a VxVM device
group.

Follow the instructions and type the name of the VxVM disk
group to be unregistered.

The following example shows the VxVM device group dg1 taken
offline, and the cldevicegroup command generated by clsetup when it removes and unregisters the device group.

# cldevicegroup offline dg1
# clsetupcldevicegroup delete dg1

How to Add a Node to a Device Group (Veritas Volume Manager)

This procedure adds a node to a device group
using the clsetup utility.

The prerequisites to add a node to a VxVM device group are:

Superuser privilege on a node in the cluster

The name of the VxVM device group to which the node will
be added

The name or node ID of the nodes to add

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with VxVM device
groups, type the number that corresponds to the option for device groups and
volumes.

The Device Groups Menu is displayed.

To add a node to a VxVM device
group, type the number that corresponds to the option for adding a node to
a VxVM device group.

Follow the instructions and type the device
group and node names.

Verify that the node has been
added.

Look for the device group information for the new disk
displayed by the following command.

# cldevicegroup show devicegroup

Example 5–28 Adding a Node to a Veritas Volume Manager Device Group

The following example shows the scconf command generated
by clsetup when it adds a node (phys-schost-3 )
to a VxVM device group (dg1 ), and the verification
step.

How to Remove a Node From a Device Group (Veritas Volume Manager)

Use this procedure to remove a cluster node
from the list of potential primaries of a Veritas Volume Manager (VxVM) device group (disk
group).

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Verify that the node is still
a member of the group and that the group is an VxVM device group.

Device
group type VxVM indicates a VxVM device group.

phys-schost-1% cldevicegroup show devicegroup

Become superuser or assume a
role that provides solaris.cluster.readand solaris.cluster.modify RBAC authorization on a current cluster member node.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To reconfigure a device group,
type the number that corresponds to the option for device groups and volumes.

To remove the node from the VxVM device
group, type the number that corresponds to the option for removing a node
from a VxVM device group.

Follow the prompts to remove the
cluster node from the device group. You are asked for information about the
following:

VxVM device group

Node name

Verify that the node has been
removed from the VxVM device group or groups.

# cldevicegroup show devicegroup

Example 5–29 Removing a Node From a Device Group (VxVM)

This example shows the removal of the node named phys-schost-1
from the dg1 VxVM device group.

How to Remove a Node From a Raw-Disk Device Group

Use this procedure to remove a cluster node from the list of potential
primaries of a raw-disk device group.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization on a node in the cluster other than
the node to remove.

Identify the device groups that are connected to
the node being removed, and determine which are raw-disk device groups.

# cldevicegroup show -n nodename -t rawdisk +

Disable the localonly property
of each Local_Disk raw-disk device group.

How to Change Device Group Properties

The method for establishing the primary ownership of a device
group is based on the setting of an ownership preference attribute called preferenced. If the attribute is not set, the primary owner of an
otherwise unowned device group is the first node that attempts to access a
disk in that group. However, if this attribute is set, you must specify the
preferred order in which nodes attempt to establish ownership.

If you disable the preferenced attribute, then the
failback attribute is also automatically disabled. However,
if you attempt to enable or re-enable the preferenced attribute,
you have the choice of enabling or disabling the failback attribute.

If the preferenced attribute is either enabled or
re-enabled, you are required to reestablish the order of nodes in the primary
ownership preference list.

This procedure uses clsetup to set or unset the preferenced attribute and the failback attribute
for Solaris Volume Manager or VxVM device groups.

Before You Begin

To perform this procedure, you need the name of the device group for
which you are changing attribute values.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with device groups,
type the number that corresponds to the option for device groups and volumes.

The Device Groups Menu is displayed.

To change key properties of
a device group, type the number that corresponds to the option for changing
key properties of a VxVM or Solaris Volume Manager device group).

The Change
Key Properties Menu is displayed.

To change a device group property,
type the number that corresponds to the option for changing the preferences
and/or failback properties.

Follow the instructions to set the preferenced and failback options for a device
group.

Verify that the device group
attributes have been changed.

Look for the device group information
displayed by the following command.

# cldevicegroup show -v devicegroup

Example 5–31 Changing Device Group Properties

The following example shows the cldevicegroup command
generated by clsetup when it sets the attribute values
for a device group (dg-schost-1).

How to Set the Desired Number of Secondaries for
a Device Group

The numsecondaries property specifies the
number of nodes within a device group that can master the group if the primary
node fails. The default number of secondaries for device services is one.
You can set the value to any integer between one and the number of operational
nonprimary provider nodes in the device group.

This setting is an important factor in balancing cluster performance
and availability. For example, increasing the desired number of secondaries
increases the device group's opportunity to survive multiple failures that
occur simultaneously within a cluster. Increasing the number of secondaries
also decreases performance regularly during normal operation. A smaller number
of secondaries typically results in better performance, but reduces availability.
However, a larger number of secondaries does not always result in greater
availability of the file system or device group in question. Refer to Chapter 3, Key Concepts for System Administrators and Application Developers, in Sun Cluster Concepts Guide for Solaris OS for more information.

If you change the numsecondaries property, secondary
nodes are added or removed from the device group if the change causes a mismatch
between the actual number of secondaries and the desired number.

This procedure uses the clsetup utility to set the numsecondaries property for all types of device groups. Refer to cldevicegroup(1CL) for information
about device group options when configuring any device group.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
role that provides solaris.cluster.read and solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.

# clsetup

The Main Menu is displayed.

To work with device groups,
select the option labeled Device groups and volumes.

The Device
Groups Menu is displayed.

To change key properties of
a device group, select the option labeled Change key properties of a device
group.

The Change Key Properties Menu is displayed.

To change the desired number
of secondaries, type the number that corresponds to the option for changing
the numsecondaries property.

Follow the instructions
and type the desired number of secondaries to be configured for the device
group. The corresponding cldevicegroup command is then
executed, a log is printed, and the utility returns to the previous menu.

If you change any configuration information for a VxVM disk
group or volume that is registered with the cluster, you must reregister the
device group by using clsetup. Such configuration changes
include adding or removing volumes, as well as changing the group, owner,
or permissions of existing volumes. Reregistration after configuration changes
ensures that the global namespace is in the correct state. See How to Update the Global-Devices Namespace.

Verify that the device group
attribute has been changed.

Look for the device group information
that is displayed by the following command.

The following example shows the cldevicegroup command
that is generated by clsetup when it configures the desired
number of secondaries for a device group (dg-schost-1).
This example assumes that the disk group and volume were created previously.

The following example shows the cldevicegroup command
that is generated by clsetup when it sets the desired number
of secondaries for a device group (dg-schost-1) to two.
See How to Set the Desired Number of Secondaries for a Device Group for information about changing the desired number
of secondaries after a device group is created.

How to List a Device Group Configuration

You do not need to be superuser to list
the configuration. However, you do need solaris.cluster.read authorization.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Use one method from the following list.

Sun Cluster Manager GUI

See the Sun Cluster Manager online help for more information.

cldevicegroup show

Use cldevicegroup show to list the configuration
for all device groups in the cluster.

cldevicegroup show devicegroup

Use cldevicegroup show devicegroup to
list the configuration of a single device group.

cldevicegroup status devicegroup

Use cldevicegroup status devicegroup to
determine the status of a single device group.

cldevicegroup status +

Use cldevicegroup status + to determine
the status of all device groups in the cluster.

Use the -v option with any of these commands to obtain
more detailed information.

How to Switch the Primary for a Device Group

This procedure can also be used to start (bring online) an inactive
device group.

You can also bring an inactive device group online or switch the primary
for a device group by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help
for more information.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Become superuser or assume a
profile that provides solaris.cluster.modify RBAC authorization
on any node of the cluster.

Use cldevicegroup switch to
switch the device group primary.

# cldevicegroup switch -n nodename devicegroup

-nnodename

Specifies the name of the node to switch to. This node become
the new primary.

devicegroup

Specifies the device group to switch.

Verify that the device group
has been switched to the new primary.

If the device group is properly
registered, information for the new device group is displayed when you use
the following command.

# cldevice status devicegroup

Example 5–37 Switching the Primary for a Device Group

The following example shows how to switch the primary for a device group
and verify the change.

How to Put a Device Group in Maintenance State

Putting a device group in maintenance state prevents that device
group from automatically being brought online whenever one of its devices
is accessed. You should put a device group in maintenance state when completing
repair procedures that require that all I/O activity be acquiesced until completion
of the repair. Putting a device group in maintenance state also helps prevent
data loss by ensuring that a device group is not brought online on one node
while the disk set or disk group is being repaired on another node.

Note –

Before a device group can be placed in maintenance state, all
access to its devices must be stopped, and all dependent file systems must
be unmounted.

The phys-schost# prompt
reflects a global-cluster prompt. Perform this procedure on a
global cluster.

This procedure provides the long forms of the Sun Cluster commands.
Most commands also have short forms. Except for the long and short forms of
the command names, the commands are identical. For a list of the commands
and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

Place the device group in maintenance
state.

If the device group is enabled, disable the device group.

# cldevicegroup disable devicegroup

Take the device group offline.

# cldevicegroup offline devicegroup

If the repair procedure being
performed requires ownership of a disk set or disk group, manually import
that disk set or disk group.

For Solaris Volume Manager:

# metaset -C take -f -s diskset

Caution –

If you are taking ownership of a Solaris Volume Manager disk set, you must use the metaset-C take command
when the device group is in maintenance state. Using metaset -t brings
the device group online as part of taking ownership. If you are importing
a VxVM disk group, you must use the -t flag when importing
the disk group. Using the -t flag prevents the disk group
from automatically being imported if this node is rebooted.

For Veritas Volume Manager:

# vxdg -t importdisk-group-name

Complete the repair procedure
that you need to perform.

Release ownership of the disk
set or disk group.

Caution –

Before taking the device group out of maintenance state, you
must release ownership of the disk set or disk group. Failure to release ownership
can result in data loss.

For Solaris Volume Manager:

# metaset -C release -s diskset

For Veritas Volume Manager:

# vxdg deport diskgroupname

Bring the device group online.

# cldevicegroup online devicegroup
# cldevicegroup enable devicegroup

Example 5–38 Putting a Device Group in Maintenance State

This example shows how to put device group dg-schost-1
in maintenance state, and remove the device group from maintenance state.