Dynamically Adding a RAW device to a Solaris zone.

Dynamically Adding a RAW device to a Solaris zone.

By default it is not possible to add raw device to zone without taking a reboot on zone however there is a famous saying => "There is always a way out for those clever enough to find it". So I'm, just kidding, don't take it seriously. :)

I found a little hack to accomplish the objective of adding raw device to zone without rebooting it. Here is a way out -

1) Add the device to the zonecfg

#zonecfg -z barvozone1
zonecfg:barvozone1> add device
zonecfg:barvozone1:device> set match=/dev/rdsk/c3t60050768018A8023B8000000000000F0d0s0
zonecfg:barvozone1:device> end
zonecfg:barvozone1>exit

2) use the mknod command to create the device in the zones dev folder

#ls -l /dev/rdsk/c3t60050768018A8023B8000000000000F0d0s0
lrwxrwxrwx   1 root     root          67 Feb 18 15:34 /dev/rdsk/c3t60050768018A8023B8000000000000F0d0s0 -> ../../devices/scsi_vhci/ssd@g60050768018a8023b8000000000000f0:a,raw

#ls -l /devices/scsi_vhci/ssd@g60050768018a8023b8000000000000f0:a,raw
crw-r-----   1 root     sys      118, 128 Mar  5 23:55 /devices/scsi_vhci/ssd@g60050768018a8023b8000000000000f0:a,raw

# cd /barvozone1/zonepath/dev

# mknod c3t60050768018A8023B8000000000000F0d0s0 c 118 128

That's it. The raw device is now visible within zone and now you can start with your stuffs without any downtime. Isn't it cool?

Identify ASM Disks from OS interface

There are several ways to identify ASM disks from the database interface however sometimes or in fact most of the times it might be the case that the System Engineer/Administrator doesn't have access to Oracle (ASM instance) or he may not aware of database operations so in that case how to recognize that a particular disk is being used and under control of ASM.

There are several occasions we need to order SAN disks/LUNs to either expand volume group, zpool or metadevice to increase a filesystem within it. Now consider, there are already few devices/disks available on system and looks free but if you are unsure whether a specific device/disk/LUN is being used by ASM, you can look at the header of the device. The od (octal dump) command can be used. E.g. od --read-bytes=128 --format=c displays the 1st 128 bytes of a device in character format.

For Solaris distributions –

root:aczfild3s:/var/tmp # od -c -N 128 /dev/rdsk/c5t6006016051402800F07C0E23B0CFDF11d0s0
0000000 \0 202 001 001 \0 \0 \0 \0 200 \0 \0 017 A 200 327 272
0000020 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000040 O R C L D I S K \0 \0 \0 \0 \0 \0 \0 \0
0000060 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000100 \n 020 \0 \0 \0 017 001 003 V 4 S _ P E R _
0000120 D A T A 0 1 _ 0 0 1 5 \0 \0 \0 \0 \0
0000140 \0 \0 \0 \0 \0 \0 \0 \0 V 4 S _ P E R   _
0000160 D A T A 0 1 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000200

This is a ASM disk as the ORCLDISK string exists in the header. The ASM disk name is V4S_PER_DATA01_0015.

For Linux distributions –


# od --read-bytes=128 --format=c /dev/dm-39
0000000 001 202 001 001 \0 \0 \0 \0 \0 \0 \0 200 w 374 211 343
0000020 244 } 030 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000040 O R C L D I S K \0 \0 \0 \0 \0 \0 \0 \0
0000060 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000100 \0 \0 020 \n \0 \0 001 003 N C A S _ D I S
0000120 K 1 _ 0 0 0 0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000140 \0 \0 \0 \0 \0 \0 \0 \0 N C A S _ D I S
0000160 K 1 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000200

The ASM disk name is NCAS_DISK1_0000.
 
For AIX distributions –


AIX is one of the “SENSIBLE” distributions I must say. Why? Check this out –

In Linux or Oracle-Sun Solaris the native volume manager or utilities around it does not identify if the disk is being used by ASM and it simply allow to format/erase the disk and allow to add the device to inline filesystem or volume group however AIX native volume manager is a step ahead & smart - disks being used by Oracle ASM cannot be also used with AIX LVM. The main reason for this is because the Oracle ASM software puts information on the raw disk to identify it as an Oracle disk, which wipes out any existing PVID or VGDA information.

If an attempt is made to use LVM on a disk already in an Oracle ASM disk group the following errors will be seen:

root> extendvg -f apps hdiskpower10
0516-1339 extendvg: Physical volume contains some 3rd party volume group.
0516-1397 extendvg: The physical volume hdiskpower10, will not be added to the volume group.
0516-792 extendvg: Unable to extend volume group.

root> mkvg -y lparvg hdiskpower128
0516-1339 mkvg: Physical volume contains some 3rd party volume group.
0516-1397 mkvg: The physical volume hdiskpower128, will not be added to the volume group.
0516-862 mkvg: Unable to create volume group.

LVM knows to search for the Oracle identifier on these disks, and will fail the operation. You can manually check for the identifier by running the lquerypv command.

# lquerypv -h /dev/hdiskpower7
00000000 00820101 00000000 80000000 F7BE76D7 ..............v.00000010 00000000 00000000 00000000 00000000
................00000020 4F52434C 4449534B 00000000 00000000 ORCLDISK........00000030 00000000 00000000 00000000 00000000 ................00000040 0A100000 00000103 44415441 5F303030 ........DATA_00000000050 30000000 00000000 00000000 00000000 0...............00000060 00000000 00000000 44415441 00000000 ........DATA....00000070 00000000 00000000 00000000 00000000 ................00000080 00000000 00000000 44415441 5F303030 ........DATA_00000000090 30000000 00000000 00000000 00000000 0...............000000A0 00000000 00000000 00000000 00000000 ................000000B0 00000000 00000000 00000000 00000000 ................000000C0 00000000 00000000 01F64610 3B306C00 ..........F.;0l. 000000D0 01F64610 3B9C7C00 02001000 00100000 ..F.;. .........000000E0 0001BC80 0000C800 00000002 00000001 ................000000F0 00000002 00000002 00000000 00000000 ................
 
Hope that helps.