Find it

Tuesday, August 25, 2009

Sun Solaris MPxIO

In general, multipathing is a method for redundancy and automatic fail-over that provides at least two physical paths to a target resource. Multipathing allows for re-routing in the event of component failure, enabling higher availability for storage resources. Multipathing also allows for the parallel routing of data, which can result in faster throughput and increased scalability.

The Solaris I/O multipathing feature is a multipathing solution for storage devices that is part of the Solaris operating environment. This feature was formerly known as Sun StorEdge Traffic Manager (STMS) or MPxIO.

Solaris Fibre Channel and Storage Multipathing software enables FC connectivity for the Solaris hosts. The software resides on the server and identifies the storage and switch devices on your SAN. It allows you to attach either loop or fabric SAN storage devices while providing a standard interface with which to manage them.

Multipathing is disabled by default for FC devices on SPARC based systems, but is enabled by default on x86 based systems.

Note - The multipathing feature is not available for parallel SCSI devices but is available for FC disk devices. Multipathing is not supported on tape drives or libraries or on IP over FC.

Example device name with multipath disabled:
/dev/dsk/c1t1d0s0

Example device name with multipath enabled:

/dev/dsk/c3t2000002037CD9F72d0s0

Well, We have learn about theory enough, now we will see how to enable it?

Enabling MPxIO -

MPxIO has a configuration file located @ /kernel/drv/fp.conf. /kernel/drv/fp.conf file is used to enable MPxIO and if needed exclude the interal disks from MPxIO

stmsboot command is also used to enables/disables/updates MPxIO configuration.

Enable MPxIO in /kernel/drv/fp.conf

1. Edit /kernel/drv/fp.conf file and have below entry uncommented.

mpxio-disable="no";

2. After editing fp.conf and having above entry in it active execute below command.

#stmsboot -u <<<<< Caution: It ask for reboot so for MPxIO enablement server needs downtime.

OK. So now after reboot you will have to verify if MPxIO is running or not.

I follow very immature way to check this out. I just do the format and see if I can see the external disks or not.

# format
Searching for disks...done

c2t60050768018A8023B80000000000013Ad0: configured with capacity of 12.00GB
c2t60050768018A8023B80000000000013Bd0: configured with capacity of 12.00GB
c2t60050768018A8023B80000000000013Cd0: configured with capacity of 12.00GB
c2t60050768018A8023B80000000000013Dd0: configured with capacity of 16.00GB
c2t60050768018A8023B80000000000013Ed0: configured with capacity of 16.00GB
c2t60050768018A8023B80000000000013Fd0: configured with capacity of 16.00GB

There are various commands by which you can manage your storage disks few of them are as listed -

1. To Display Paths

# mpathadm list lu
/dev/rdsk/c2t60050768018A8023B80000000000013Fd0s2
Total Path Count: 8
Operational Path Count: 8

2. Show detailed information about a disk/LUN

#mpathadm show lu /dev/rdsk/c2t60050768018A8023B80000000000013Fd0s2

<<<<<<< You can find more details of specific LUN >>>>>>>

3. Display world wide port names/Fiber card Firmware level -

# fcinfo hba-port
HBA Port WWN: 10000000c9446e11 <----------- WWPN
OS Device Name: /dev/cfg/c4
Manufacturer: Emulex
Model: LP9002L
Firmware Version: 3.90a7 (C2D3.90A7)
FCode/BIOS Version: Boot:3.20 Fcode:1.40a0
Serial Number: BG50103047
Driver Name: emlxs
Driver Version: 2.31p (2008.12.11.10.30)
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 1Gb
Node WWN: 20000000c9446e11

#fcinfo hba-port -l <<<<< Good Coomand for debugging >>>>>>

#fcinfo remote-port -sl -p 10000000c9446e11

lists all remote ports as well as the link statistics and scsi-target information. This is one of the excellent command I found for troubleshooting/debugging.

4. Display all LUNs visable to MPxIO

# luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Ad0s2
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Bd0s2
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Cd0s2
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Dd0s2
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Ed0s2
Node WWN:5005076801000477 Device Type:Disk device
Logical Path:/dev/rdsk/c2t60050768018A8023B80000000000013Fd0s2

# luxadm -e port
/devices/pci@1e,600000/lpfc@2/fp@0,0:devctl CONNECTED

>>>>> shows all hba ports and which ones are connected. >>>>>>>

# cfgadm -al -o show_FCP_dev

Enter the cfgadm command to verify the paths to the LUNs.

There are several other utilities by which you can manage/view your storage disk, above are few well known methods. I hope someone will get benefited by this article.

=================================================================================

Find HBA’s WWN

#prtpicl -v -c scsi-fcp | grep wwn (On all versions)
# prtconf -vp | grep -i ww

One small shell script I found handy -

#!/bin/sh
for i in `cfgadm | grep fc-fabric | awk ‘{print $1}’`; do
dev=”`cfgadm -lv $i | grep devices | awk ‘{print $NF}’`”
wwn=”`luxadm -e dump_map $dev | grep ‘Host Bus’ | awk ‘{print $4}’`”
echo “$i: $wwn”
done

9 comments:

  1. Have you ever used Solaris Volume Manager on top of Mpxio?

    ReplyDelete
  2. Yes, we are using MPxIO along with SVM disks. Anything specific that you want to share here?

    ReplyDelete
  3. Thanks Nilesh..i am new to Solaris track.. i will try these commands out and will reply if need some specific trouble shooting Steps....

    ReplyDelete
  4. How to disbale mpxio on solaris 10 x86. I tried to change mpxio_disable=yes; and manual entry in vfstab..stsmboot -d..all these are not working..pls help me in this issue

    ReplyDelete
  5. Hi Unknown,

    Any specific disk vendor that you're trying to enable multi-pathing?

    For e.g.

    Syntax to add Third Party Storage is mentioned in scsi_vhci.conf

    Note : Note Vendor ID ( EMC in my case) should be 8 Chars if it’s not 8 char’s you need to pad with spaces. BTW you can get Vendor ID ( EMC ) and Product ID (SYMMETRIX) from format inquiry on disk.


    device-type-scsi-options-list =
    "EMC SYMMETRIX", "symmetric-option";
    symmetric-option = 0×1000000;
    device-type-mpxio-options-list=
    "device-type=EMC SYMMETRIX", "load-balance-options=logical-block-options";
    logical-block-options="load-balance=logical-block", "region-size=18";

    After that enable the mpxio for fiber channel using below command. It would ask you to reboot the box and also updates the vfstab automatically for you.

    #stmsboot -D fp -e

    Once system is back. you can see the common device created by combining both paths. stmsboot -L for displaying that info.

    # stmsboot -L

    Thanks,
    Nilesh

    ReplyDelete
  6. Few more tips -

    How to enable MPxIO on Solaris boxes.

    # cd /kernel/drv

    # vi fp.conf >>>> Put below entries. Here we are enabling MPxIO on specific port basis!

    mpxio-disable="yes";
    name="fp" parent="/pci@0/pci@0/pci@8/pci@0/pci@1/SUNW,emlxs@0" port=0 mpxio-disable="no";
    name="fp" parent="/pci@0/pci@0/pci@8/pci@0/pci@9/SUNW,emlxs@0" port=0 mpxio-disable="no";

    # vi scsi_vhci.conf >>>> 3rd part related Storage array entry!

    device-type-scsi-options-list =
    "EMC SYMMETRIX", "symmetric-option";
    symmetric-option = 0x1000000;

    HTH,
    Nilesh Joshi

    ReplyDelete
  7. Hi Nilesh,

    When MPxIO is configured, Is there anyway to know how much traffic is flowing in each path.

    Somthing in similar to vmdmpadm iostat show

    regards
    Susanta

    ReplyDelete
  8. This blog contain excellent stuff useful to Learners.Thank you for sharing this. We are providing Best Sun Solaris Online Training. for more Visit SUN SOLARIS Online Training

    ReplyDelete