The work of engineers, when properly applied, can have a compounding effect on the work of others…

Solaris 11: Network Configuration Advanced

Solaris 11: Network Configuration Advanced

In my previous post we went over the basics of configuring a network interface in Solaris 11, without using the Network Auto Magic (NWAM) which is enabled by default. In this article we will go over some of the more advanced features which can be leveraged including VLANs, aggregation groups, and jumbo frames.

Configure VLAN Tagging

Create a VLAN, specify the VLAN id with the “-v”, then specify the interface with the “-l”, and finally create a name for the tagged interface (in this case user0 since this particular system is the user VLAN, if you have more descriptive names of your VLANs you can use them here).

For simplicity I will delete the IP configuration on the untagged interface.

# ipadm delete-addr bge0/v4

As we did in our previous article you can now create an IP configuration on top of the new tagged interface (which in this case is DHCP).

# ipadm create-addr -T dhcp user0/v4

Keep in mind if you change the VLAN you will most likely need to change your default route. The below options “-fp” f will flush meaning delete all current routes, while the p will make the new settings persistent. In this case afterwards we will only have 192.168.100.1 as a default gateway.

# route -fp add default 192.168.100.1

Delete a VLAN

If you have used the VLAN before you will need to “unwind” the configuration before deleting the VLAN.

# ipadm delete-addr user0/v4

# ipadm delete-if user0

Now that this is done you can delete the VLAN.

# dladm delete-vlan user0

Create an Aggregation Group

With this command we create a new aggregation group and assign interfaces to the aggregation group. If the interface is already in use then you will need to delete the interface before adding it to an aggregation group.

If you need to add or remove an interface from an aggregation group then the following commands will allow you to do that.

# dladm add-aggr -l bnx2 aggr0

# dladm remove-aggr -l bnx1 aggr0

You can also adjust the LACP Policy using the below command. Where L4 is L2, L3, L4, or any combination of them based on the desired behavior.

# dladm modify-aggr -P L4 aggr0

The LACP mode can be configured using the below command where active is either auto, active, or passive. Additionally if configuring active mode you must also configure a timer value of short or long, this option is not needed for auto or passive.

# dladm modify-aggr -L active -T short aggr0

Delete Aggregation Group

Delete the IP configuration from the aggregation group

# ipadm delete-addr aggr0/v4

Delete the Aggregated Interface

# ipadm delete-if aggr0

Delete the Aggregation Group

# dladm delete-aggr aggr0

Enable Jumbo Frames

Basically Jumbo Frames allow the system to reduce the network overhead by combining more data into a single TCP frame, this is analogous to renting a box truck when you move into a new house. If you had to use your Prius to move, you would spend much more time waiting to finish the process, as well as expending more resources. Now Jumbo Frames doesn’t mean that it will always help. If we step back to our analogy of moving into a new house, if you all of your stuff amounts to a single suitcase then renting a moving truck doesn’t do anything to make your trip more efficient. So if you are not send large amounts of data then Jumbo Frames will not help you, however if you are working on a storage network and even with some file sharing you will get a bonus. Also in order for Jumbo Frames to work, both sides of the communication must support it or it will not use the higher MTU, as well as all devices along the way.

In the comments of my article “Solaris 11: Network Configuration Basics” you will notice “Kristen” mentioned that the ipadm command has changed in newer builds of Solaris 11. At the time she was using a newer build than I had available to me, so I could not verify her claim, however now I have verified this change against the Solaris 11 Early Adopter release snv_173. So be prepared to make the following changes.

12 thoughts on “Solaris 11: Network Configuration Advanced”

these two are really great! They are so good actually that I, being a political scientist, am able to configure a trunk on my HP Microserver with Solaris Express 11 and an extra Intel Dual Nic Card. Many thanks!

What you could elaborate for the less clever ones like me – which mode for the trunk should be used on the switch (active or passive) and on SE11. Could not find an answer yet. I am using a Procurve 1800-24 which allows active and passive mode.

I am by no means an expert. I work on the server side and as such a lot of the networking side is just “magic” that said I will explain what I know.

First definitions.

Trunk – A trunk is a logical configuration which allows for multiple VLANs to be sent over the same physical port.
Aggregration Group (LAG) – Also called Ether-channel in the Cisco world, and also called a trunk in a lot of circles, although trunk is not really a good way to refer to it. This technology is when you combine the bandwidth of multiple interfaces into a logical interface which can be used to increase bandwidth and provide redundancy.

That said… Keep in mind that when you create a LAG there has to be two parts to the LAG, the switch side and the NIC side.

Active means that it will use all LAG members when sending traffic. Passive means that it will return traffic on the same interface it was received on, which as long as traffic is being sorted on the other end of the switch the LAG will function. Now how you configure it really depends on your environment. I personally like to keep my servers as vanilla as possible and allow the network configuration on the network side (frankly there is more than enough work on the server side to be adding more). Also in a lot of cases it is just flat out easier to do it on the switch. That said I would try and set up the configuration on the switch with the SE11 box using “auto”. Once you have a working configuration generate some traffic and unplug some LAG members and see what happens, you will also want to see what kind of speed you are getting from you LAG (an FTP test to a well connected host [read: better than your LAG] on your LAN works well for this – If you are trying to test a 4Gbps LAG on the switch you can hard code the ports for 100Mbps giving you a 400Mbps LAG for the purpose of testing speed to a 1Gbps FTP Server).

Also I forgot to mention. In some cases you will have to fiddle with the settings to get them just right, so don’t be afraid to tinker. Once you have a working config then it is time to test and validate.

have quite some time to test it today. The four HDDs in RaidZ result in around 200 mb/s read. Happy to get that on the LAN 😉
Will also have a look at the Jumbo Frames. But as I read so far on Solaris Express it does not make a big difference.

so I started and most went ok:
1) Went through your Basics
2) Created DHCP for Onboard Broadcomchip which I renamed to broadcomdhcp1
3) Made an aggregation of e1000g0 and e1000g1 (that is an extra NIC Card)
4) Assigned static IP v4 for aggregation (called trunk1)

However, now I am stuck:
1) For funny reasons I get a trunk1/-a AND a trunk1/v4
2) I can see the IP numbers in my net using a portscanner but can not ping on them
3) No access to internet.
4) No access to the SMB shares provided by this Solarisbox (this box works as a NAS in my private net)

Also keep in mind that generally numbering begins with “0” as opposed to “1” so it might make more sense to have your interfaces named trunk0. Though I generally stay with the default names unless I am doing VLANs then I will name them with accordance with the purpose of the VLAN (user0, server0, printer0, dmz0, etc).

My guess is that this should sort you out. However if it doesn’t I emailed you separately so you can respond back with more details.

Just in case:
root@snarf:~# ping 172.16.1.2
ping: sendto Network is unreachable
root@snarf:~# ipfstat -io
block out log all
pass out quick on lo0 all
pass out quick proto udp from any to any port = bootps
block in log all
pass in quick on lo0 all
pass in quick proto udp from any to any port = bootpc
root@snarf:~# svcadm disable svc:/network/ipfilter
root@snarf:~# ping 172.16.1.2
172.16.1.2 is alive
root@snarf:~# ipfstat -io
empty list for ipfilter(out)
empty list for ipfilter(in)