Solaris Link Aggregation using dladm

A Solaris "Link Aggregation" is the act of bonding several interfaces together to create a single logical interface.


A link aggregation consists of several interfaces on a system that are configured together as a single, logical unit. Link aggregation, also referred to as Trunking, is defined in the IEEE 802.3ad Link Aggregation Standard.

The IEEE 802.3ad Link Aggregation Standard provides a method to combine the capacity of multiple full-duplex Ethernet links into a single logical link. This link aggregation group is then treated as though it were, in fact, a single link.

The following are features of link aggregations:

1. Increased bandwidth – The capacity of multiple links is combined into one logical link.

2. Automatic failover/failback – Traffic from a failed link is failed over to working links in the aggregation.

3. Load balancing – Both inbound and outbound traffic is distributed according to user selected load balancing policies, such as source and destination MAC or IP addresses.

4. Support for redundancy – Two systems can be configured with parallel aggregations.

5. Improved administration – All interfaces are administered as a single unit.

6. Less drain on the network address pool – The entire aggregation is assigned one IP address.

Before going toward HOW-TO, Your aggregation configuration is bound by the following requirements:

1. You must use the dladm command to configure aggregations.

2. An interface that has been plumbed cannot become a member of an aggregation.

3. Interfaces must be of the GLDv3 type: bge, e1000g, xge, nge, rge, ixgb.

4. All interfaces in the aggregation must run at the same speed and in full duplex mode.

5. “Legacy” data link provider interfaces (DLPI ), such as the ce interface do not support Solaris link aggregations. Instead, you must configure aggregations for legacy devices by using Sun Trunking. You cannot configure aggregations for legacy devices by using the dladm command.

How aggregation works?

The MAC layer, which is part of GLDv3, is the central point of access to Network Interface Cards (NICs) in the kernel. At the top, it provides a client interface that allows a client to send and receive packets to and from NICs, as well as configure, stop and start NICs. A the bottom, the MAC layer provides a provider interface which is used by NIC drivers to interface with the network stack. In the figure above, the client is the Data-Link Service (DLS) which provides SAP demultiplexing and VLAN support for the rest of the stack. The Data-Link Driver (DLD) provides a STREAMS interface between Nemo and DLPI consumers.


The core of the link aggregation feature is provided by the "aggr" kernel pseudo driver. This driver acts as both a MAC client and a MAC provider. The aggr driver implements a MAC provider interface so that it looks like any other MAC device, which allows us to manage aggregation devices as if they were a regular NIC from the rest of Solaris.

HOW-TO?
 
Make sure your eeprom’s local-mac-address? variable is set to true.


# eeprom local-mac-address?

local-mac-address=false

# eeprom local-mac-address? = true
# eeprom local-mac-address?

local-mac-address?=true

PS: Above step is not applicable for x86.

1. Unplumb the interfaces to be aggregated:


# ifconfig bge0 down unplumb
# ifconfig bge1 down unplumb


2. Create a link-aggregation group with key 1. key Is the number that identifies the aggregation. The lowest key number is 1. Zeroes are not allowed as keys.

passive mode by default:

# dladm create-aggr -d bge0 -d bge1 1



# mv /etc/hostname.bge0 /etc/hostname.aggr1

3. Perform reboot. Good to go for reconfigure reboot but not required.

# reboot -- -rv

After reboot –

You can check the status of the aggregation with the dladm command...


# dladm show-aggr
key: 1 (0x0001) policy: L4 address: 0:14:4f:2b:be:18 (auto)
device address speed duplex link state
bge0 0:14:4f:2b:be:18 1000 Mbps full up attached
bge2 0:14:4f:2b:be:1a 1000 Mbps full up attached

Hope that helps.