Menu

IPMP on Solaris 11

I have recently been testing the pre-release of Oracle Solaris 11, snv_167 to be precise. While Oracle has done well to update documentation surrounding their recent updates to the Solaris 11 networking and virtualization stack, it will still come as a shock to many experienced Solaris administrators that the old methods of configuring IPMP (IP multipathing) using configuration files is no more! Yes, I’m referring to the /etc/hostname.<interface> files that were so easily understood (and misunderstood in the case of IPMP).

Solaris 11 brings some well overdue changes to how we administer the range of networking. In particular, there are two new commands that effectively end the need for ever using /usr/sbin/ifconfig again. The first of these is /usr/sbin/ipadm which is now delivered by the system/network package. The ipadm command is very straight-forward and easy to use. It is intended by the developers to eliminate ifconfig and the hostname.<interface> files. I must commend them on their efforts since it not only accomplishes their goal, but also simplifies our administration of the network layer.

While there are many ways to configure IPMP, the full-breadth of options will not be discussed here. Instead, I must refer you to the full documentation, Oracle Solaris Administration: Network Interfaces and Network Virtualization guide. This post only covers IPMP in Active-Passive mode. However, in regards to documentation, be certain to read the man page for ipadm as the man page contains better examples than the guide. It seems the guide lacks updates and still refers to ifconfig over ipadm.

As background information for my testing, I have created an LDom and assigned it two interfaces (vnet0 and vnet1) which are bound to separate virtual switches (vsw0 and vsw1) on aggregates (aggr1 and aggr2 respectively) composed of separate physical network cards, each of which connect to a separate physical switch. The diagram below will help 😉

IPMP Configuration Diagram in an LDom using Link Aggregation

Now, to configure IPMP in Active-Passive mode using vnet0 and vnet1 I simply used the following example.

Now what actually occurs is quite interesting. The first two commands using the create-ip subcommand actually perform two tasks. Each command both “plumbs” or enables the network interface and creates an IP interface on that network interface. Next, the ipadm set-ifprop command marks the vnet1 IP interface as the “STANDBY” interface. The fourth command using ipadm create-ipmp, performs 3 separate actions. It creates the IPMP group named “ipmp0” and adds both IP interfaces (vnet0 and vnet1) to the IPMP group with the IPMP interface name of “ipmp0”. Finally, the ipadm create-addr command sets a static IPv4 address to the IPMP interface “ipmp0”. Note the “/v4” syntax appended to the IPMP interface name. This is required for IPv4. Refer to the ipadm man page if configuring IPMP on an IPv6 network. This results in the following active working configuration.

The first command above, using the show-addr subcommand, displays the IP address previously assigned to the IPMP interface “ipmp0”. The second command using the show-if subcommand displays all of the interfaces with their assigned “CLASS” and “CURRENT” associated flags. These are explained in detail in the ipadm man page. However, a little more explanation is warranted here on the “CURRENT” flags. IPMP interface “ipmp0” has “bm——–46” which stands for broadcast, multicast, IPv4, and IPv6 enabled, respectively. IP interfaces “vnet0” and “vnet1” both have flags enabled for broadcast, multicast, link-based probing, IPv4 and IPv6. IP interface “vnet1” has the additional flags of “s” and “i” for standby and inactive, respectively. These last two flags were enabled by the ipadm set-ifprop –p standby=on –m ip vnet1 command.

Finally, the second new command I mentioned toward the beginning of this post, /usr/sbin/ipmpstat. The ipmpstat command is much more simplified and aimed solely at IPMP configurations. There are several additional options besides the ones shown below, but those are referenced in the man page for ipmpstat.
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok 10.00s vnet0 (vnet1)
# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
vnet0 yes ipmp0 --mbM-- up ok ok
vnet1 no ipmp0 is----- up ok ok
#

The first command above displays quite simply the contents or IPMP interfaces for the IPMP group named “ipmp0”. The parentheses’ surrounding the IP interface named “vnet1” indicates that as a “STANDBY” interface. The second command, ipmpstat –i, displays roughly the same information, but with more detail in the “FLAGS” section. Note how “vnet1” shows the same inactive and standby flags as previously displayed using the ipadm show-if –o all command. However, “vnet0” displays a new flag of “M” for IPv6 multicast. This shows that all multicast communication (both IPv4 and IPv6) is sent via the “ACTIVE” IPMP interface.

There you have it, IPMP without configuration files! However, I must throw out the disclaimer for the geeky Solaris Admins out there. A file does exist at /etc/ipadm/ipadm.conf that contains any and all information for the active networking configuration created by ipadm. Also, as usual, Oracle advises not to edit the file directly.

Happy networking.

Advertisements

Share this:

Like this:

Brad Hudson is an Established Leader in IT Infrastructure, Engineering, Operations, and Customer Service with extensive experience developing and delivering infrastructure solutions, building and growing customer service organizations, leading change initiatives, providing data center management, and managing implementation activities.

Brad Hudson is an Established Leader in IT Infrastructure, Engineering, Operations, and Customer Service with extensive experience developing and delivering infrastructure solutions, building and growing customer service organizations, leading change initiatives, providing data center management, and managing implementation activities.[Read More...]