Find it

Monday, October 22, 2012

Weird issue with VERITAS after replacing the disk

When performing a disk replacement in VxVM 4.1 and VxVM 5.0, the disk being replaced does not show up in the output of a "vxdisk list" command. Instead a "NONAMEs2" entry is seen.

root@XXXXX# vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME
VxVM vxdisk ERROR V-5-1-2272 NONAMEs2 not a valid dmpnode disk access name.
NONAMEs2 auto - - online
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
c1t0d0s2 auto rootdisk rootdg online c1t0d0s2
c1t1d0s2 auto - - online c1t1d0s2
c1t2d0s2 auto data01 data online c1t2d0s2
c1t5d0s2 auto data02 data online c1t5d0s2
c1t8d0s2 auto data03 data online c1t8d0s2
c1t9d0s2 auto data04 data online c1t9d0s2
c1t10d0s2 auto - - online c1t10d0s2
c1t11d0s2 auto - - online c1t11d0s2
c1t12d0s2 auto - - online c1t12d0s2
c1t13d0s2 auto - - online c1t13d0s2
- - rootmirror rootdg removed was:c1t3d0s2

To get rid of this, solution is to re-create device mapping, restart vxconfigd, initialize the device, add the disk back to the diskgroup.

# vxdisk rm NONAMEs2 (Remove the device entry in Volume Manager)
# cd /dev/vx/rdmp (Delete all files pertaining to NONAME*)
# cd /dev/vx/dmp (Delete all files pertaining to NONAME*)
# mv /etc/vx/disk.info /etc/vx/disk.info.old
# mv /etc/vx/array.info /etc/vx/array.info.old
# devfsadm -Cv
# vxddladm stop eventsource
# vxconfigd -k (This will recreate the device mapping and the disk.info file)

If the above solution fails, Rerun all commands up to and including: "mv /etc/vx/disk.info /etc/vx/disk.info.old" then run a reconfigure reboot.

It worked!!!! disk c1t3d0s2 is back....

root@XXXXX# vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME
c1t0d0s2 auto rootdisk rootdg online c1t0d0s2
c1t1d0s2 auto - - online c1t1d0s2
c1t2d0s2 auto data01 data online c1t2d0s2
c1t3d0s2 auto - - online c1t3d0s2
c1t5d0s2 auto data02 data online c1t5d0s2
c1t8d0s2 auto data03 data online c1t8d0s2
c1t9d0s2 auto data04 data online c1t9d0s2
c1t10d0s2 auto - - online c1t10d0s2
c1t11d0s2 auto - - online c1t11d0s2
c1t12d0s2 auto - - online c1t12d0s2
c1t13d0s2 auto - - online c1t13d0s2
- - rootmirror rootdg removed was:c1t3d0s2

Now as the disk is back in proper format perform action of  -

  • Initialize the device,
  • Add the disk to the disk group
  • Recover plex.

root@XXXXX# vxdisksetup -i c1t3d0 format=sliced
root@XXXXX# vxdg -g rootdg -k adddisk rootmirror=c1t3d0

root@XXXXX# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
c1t0d0s2 auto:sliced rootdisk rootdg online
c1t1d0s2 auto:none - - online invalid
c1t2d0s2 auto:sliced data01 data online
c1t3d0s2 auto:sliced rootmirror rootdg online
c1t5d0s2 auto:sliced data02 data online
c1t8d0s2 auto:sliced data03 data online failing
c1t9d0s2 auto:sliced data04 data online
c1t10d0s2 auto:none - - online invalid
c1t11d0s2 auto:none - - online invalid
c1t12d0s2 auto:none - - online invalid
c1t13d0s2 auto:none - - online invalid

root@XXXXX# vxrecover -bg rootdg

Check if recovery is in progress (vxtask list)

root@XXXXX# vxtask -l list
Task: 436 RUNNING
Type: PARENT
Operation: VXRECOVER
Started: Fri Oct 19 18:38:17 2012
Progress: 20.00% (1 of 5 jobs, 1 active)
Task: 441 RUNNING (Parent: 436)
Type: ATCOPY
Operation: PLXATT Vol opt Plex opt-02 Dg rootdg
Started: Fri Oct 19 18:40:57 2012
Throttle: 0
Progress: 30.16% 6326272 of 20972736 Blocks
Work time: 1 minute, 54 seconds (04:23 remaining)

root@XXXXX# vxprint -htg rootdg
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO

dg rootdg default default 0 1027335905.1025.XXXXX
dm rootdisk c1t0d0s2 auto 10175 143339136 -
dm rootmirror c1t3d0s2 auto 9919 143328960 -

v home - ENABLED ACTIVE 8405376 ROUND - gen
pl home-01 home ENABLED ACTIVE 8405376 CONCAT - RW
sd rootdisk-03 home-01 rootdisk 75566975 8405376 0 c1t0d0 ENA
pl home-02 home ENABLED ACTIVE 8405376 CONCAT - RW
sd rootmirror-05 home-02 rootmirror 75516096 8405376 0 c1t3d0 ENA

v opt - ENABLED ACTIVE 20972736 ROUND - gen
pl opt-01 opt ENABLED ACTIVE 20972736 CONCAT - RW
sd rootdisk-04 opt-01 rootdisk 54594239 20972736 0 c1t0d0 ENA
pl opt-02 opt ENABLED ACTIVE 20972736 CONCAT - RW
sd rootmirror-04 opt-02 rootmirror 54543360 20972736 0 c1t3d0 ENA

v rootvol - ENABLED ACTIVE 4202688 ROUND - root
pl rootvol-01 rootvol ENABLED ACTIVE 4202688 CONCAT - RW
sd rootdisk-02 rootvol-01 rootdisk 20351 4202688 0 c1t0d0 ENA
pl rootvol-02 rootvol ENABLED ACTIVE 4202688 CONCAT - RW
sd rootmirror-01 rootvol-02 rootmirror 0 4202688 0 c1t3d0 ENA

v swapvol - ENABLED ACTIVE 33560448 ROUND - swap
pl swapvol-01 swapvol ENABLED ACTIVE 33560448 CONCAT - RW
sd rootdisk-01 swapvol-01 rootdisk 4233215 33560448 0 c1t0d0 ENA
pl swapvol-02 swapvol ENABLED ACTIVE 33560448 CONCAT - RW
sd rootmirror-02 swapvol-02 rootmirror 4202688 33560448 0 c1t3d0 ENA

v var - ENABLED ACTIVE 16780224 ROUND - gen
pl var-01 var ENABLED ACTIVE 16780224 CONCAT - RW
sd rootdisk-05 var-01 rootdisk 37803839 16780224 0 c1t0d0 ENA
pl var-02 var ENABLED ACTIVE 16780224 CONCAT - RW
sd rootmirror-03 var-02 rootmirror 37763136 16780224 0 c1t3d0 ENA

Hope this helps someone who facing this issue..... Have a good night.