Friday, January 1, 2010

Veritas Volume Manager – vxassist

Veritas Volume Manager – 2 covers vxassist command for volume
management .
vxassist utility in Veritas Volume Manager is used to create volumes, add mirrors and logs to existing volumes, extend and shrink existing volumes, provides for the migration of data from a specified set of disks, and provides facilities for the on-line backup of existing volumes.
The default behavior of vxassist is to create volumes in the rootdg diskgroup if diskgroup is not specified . The default length is taken as number of blocks but this can be specified in Kilobytes or Megabytes or Gigabytes.

About Vxassist 1. Creating a Volume     1.1 Creating a Concatenated Volume     1.2 Creating a Striped Volume     1.3 Creating a RAID-5 Volume 2. Extending a Volume     2.1 Extending a volume up to certain length,     2.2 Extending by a Given Length 3. Shrinking a Volume     3.1 Shrinking to a Given Length     3.2 Shrinking by a Given Length 4. Removing a Volume 5. Mirroring a Volume     5.1 Creating & Mirroring a New Volume     5.2 Mirroring an Existing Volume     5.3 Mirroring All Volumes 6. Removing a Mirror 7. Adding a RAID-5 Log 8. Adding a DRL Log 9.  Removing a RAID-5 Log 10.Preparing a Volume for Online Backup 11.Displaying Volume Configuration Information 12.Next Steps

About vxassist

vxassist command syntax :

vxassist volume_name [attributes]

Commonly used options are given below (See man vxassist for complete list of supported options)
-g for specifying diskgroups
-b for background operation
-d file containing defaults for vxassist if not specified /etc/default/vxassist is used

Keywords used are make , mirror , move , growto ,growby ,shrintto ,shirnkby ,snapstart , snapshot ,snapwait

Attributes specify volumes layout disks controllar to include exclude etc

Device Nodes
Default rootdg disk group.
Block Device Node /dev/vx/dsk/volume_name
Raw Device Node /dev/vx/rdsk/volume_name
Other DiskGroups
Block Device Node /dev/vx/dsk/diskgroup_name/volume_name
Raw Device Node /dev/vx/rdsk/diskgroup_name/volume_name

1. Creating a Volume
1.1 Creating a Concatenated Volume

By default, vxassist creates a concatenated volume using the space available on a disk or on the number of disks in a diskgroup if the volume size specified is more then the one available on a single disk.

Disks can be specified from a diskgroup for a volume group but if not mentioned available disks areselected by the volume manager.

Command syntax :

· #vxassist make volume_name volume_length

To create a new volume appvol of 100 MB in the default disk group rootdg with available disks:
· #vxassist make appvol 100m

To create the volume appvol of 100MB on disk03
· #vxassist make appvol 100m disk03

1.2 Creating a Striped Volume

A striped volume contains at least one plex that consists of two or more subdisks located on two or more physical disks.

Command Syntax

· vxassist make volume_name length layout=stripe

To create a striped volume appvol2 with the default stripe unit size on the default number of disks
· #vxassist make appvol2 100m layout=stripe

To create a striped volume appvol2 100MB striped volume on three specific disks.
· #vxassist make appvol2 100m layout=stripe disk04 disk05 disk06

1.3 Creating a RAID-5 Volume
A RAID-5 volume contains a RAID-5 plex that consists of two or more subdisks located on two or more physical disks. Only one RAID-5 plex can exist per volume. A RAID-5 volume may also contain one or more RAID-5 log plexes, which are used to log information about data and parity being written to the volume.

Command Syntax :

· vxassist make volume_name length layout=raid5

To create the RAID-5 volume appvol4 with the default stripe unit size on the default number of disks with RAID-5 log,
· #vxassist make appvol4 100m layout=raid5.

2. Extending a Volume

Caution : If you plan to use volume size change (grow or shrink ) on a volume with data make sure you have the good backup of the volume available before carrying out such operation .

2.1 Extending a volume up to certain length,

Command syntax

· vxassist growto volume_name length

To extend vol3 upto 8000 sectors, type:
· #vxassist growto vol3 8000

2.2 Extending by a Given Length

Command Syntax
· vxassist growby volume_name length

To extend volapp by 1000 sectors, type:
· #vxassist growby volapp 1000

3. Shrinking a Volume
Caution – Do not shrink a volume below the size of the file system. If you have a VxFS file system, you can shrink the file system and then shrink the volume. If you do not shrink the file system first, you risk unrecoverable data loss.

Always make sure you have a good backup of the data volume to be shirnked.

3.1 Shrinking to a Given Length

Shrink a volume to a specific length as follows:

· vxassist shrinkto volume_name length

Make sure you do not shrink the volume below the current size of the file system or database using the volume. This command can be safely used on empty volumes.

To shrink volcat to 1300 sectors, type:

· #vxassist shrinkto volcat 1300

3.2 Shrinking by a Given Length

Shrink a volume by a specific length as follows:

· vxassist shrinkby volume_name length

To shrink volcat by 8000 sectors, type:

· #vxassist shrinkby volapp2 8000

4. Removing a Volume

Removing a volume requires removing all references to the volumes to be removed like unmounting the volume if mounted and removing its reference from /etc/vfstab .

An active volume has to be stopped first to stop all the activities to the volume only then it can be removed

Stopping Volume

Command Syntax

· vxvol stop volume_name

To stop appvol1

· #vxvol stop appvol1

Removing Volume
Command Syntax

· vxedit -rf rm volume_name

To remove the volume appvol1

· #vxedit -rf rm appvol11

5. Mirroring a Volume

Mirroring allow data to be written simultaneously to two disks so that in case of failure of one disk data is available from the other mirrored disk. Boot disks can be mirriored to provide alternatebooting disk .

5.1 Creating & Mirroring a New Volume

Command Syntax :

· vxassist make volume_name length layout=mirror

To create the mirrored volume, appvol22 of 100mb
· #vxassist make appvol22 100m layout=mirror

To create a volume with Dirty Region Logging enabled
· #vxassist make volume_name length layout=mirror,log

5.2 Mirroring an Existing Volume

Command syntax

· vxassist mirror volume_name

Creating a mirror of volume vol44:
· #vxassist mirror vol44

5.3 Mirroring All Volumes

To mirror all existing volumes to available disk space

· /etc/vx/bin/vxmirror -a

6. Removing a Mirror

Removing a mirror involves first dissociating the plex from its volume and then removing the plex and any associated subdisks completely.The last valid mirror plex associated with a volume cannot be removed.

Dissociate and remove the plex from its volume as follows:

· vxplex -o rm dis plex_name

vxprint -h shows the volume and associated plexes :

Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 v volm fsgen ENABLED 10240 – ACTIVE – - pl volm-01 volm ENABLED 10240 – ACTIVE – - sd disk01-02 volm-01 – 10240 0 – - – pl volm-02 volm ENABLED 10240 – ACTIVE – - sd disk02-03 volm-02 – 10240 0 – - -

To disassociate and remove volmir-02 from volmir, type:

· #vxplex -o rm dis volm-02

The output of vxprint -h for volm now shows volume volm with only one plex::

Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 v volm fsgen ENABLED 10240 – ACTIVE – - pl volm-01 volmir ENABLED 10240 – ACTIVE – - sd disk01-02 volmir-01 – 10240 0 – - -

7. Adding a RAID-5 Log

A RAID-5 log maintains a copy of the data and parity being written to the volume at any given time. If a system failure occurs, VxVM can replay the RAID-5 log to resynchronize the volume. This copies the data and parity that was being written at the time of failure from the log to the appropriate areas of the RAID-5 volume.

RAID-5 log plexes are used to log information about data and parity being written to the raid volume. A log plex is created for the volume by default when a raid volume is created but additional ones can be added by following command .

· vxassist addlog name_of_raid5_vol

8. Adding a DRL Log

Dirty region logging (DRL) is used with mirrored volume layouts. DRL keeps track of the regions that have changed due to I/O writes to a mirrored volume. Prior to every write, a bitmap is written to a log to record the area of the disk that is being changed. In case of system failure, DRL uses this information to recover only the portions of the volume that need to be recovered.

To put Dirty Region Logging into effect for a volume, a log subdisk must be added to that volume and the volume must be mirrored. Only one log subdisk can exist per plex.

The following example creates a log for the mirrored volume vol03:

· #vxassist addlog vol03

When vxassist is used to add a log subdisk to a volume, a log plex is also created to contain the log subdisk, by default.

Once created, the plex containing a log subdisk can be treated as a regular plex. Data subdisks can be added to the log plex. The log plex and log subdisk can be removed using the same procedures used to remove ordinary plexes and subdisks.


9. Removing a RAID-5 Log

Removing a RAID-5 log involves first dissociating the log from its volume and then removing the log and any associated subdisks completely.

Dissociate the log from its volume as follows:

· #vxplex -o rm dis plex_name

To identify the log plex, use the command:

· vxprint -ht raid5_volume_name

To disassociate the log plex volrd-02 from volrd, type:

· #vxplex -o rm dis volrd-02

The output of vxprint -h for volrd now shows:

Disk group: rootdg TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 v volrd raid5 ENABLED 32 – ACTIVE – - pl volrd-01 volrd ENABLED 32 – ACTIVE – - sd disk01-02 volrd-01 – 32 0 – - – sd disk02-03 volrd-01 – 32 0 – - -


10. Preparing a Volume for Online Backup

Volume manager provides an easy way to take backup of the online data by creating an offline snapshot mirror of the volume .

The vxassist snapstart operation creates a write-only backup mirror, which is attached to and synchronized with the volume to be backed up and later detached and made offline by vxassist snapshot command . The snapshot volume can be used by backup utilities while the original volume continues to be available for applications and users.

The volume backup method described here does not apply to RAID-5 volumes.

Backing up a volume with vxassist involves the following procedure

Create a snapshot mirror

snapstart starts creating a online snapshot mirror of the volume using the available disk space . The snapshot is completed with vxassist snapshot command when offline snapshot volume is created with a userdefined name .

Command Syntax

* vxassist snapstart volume_name

To create a snapshot mirror of a volume called vol8, type

* #vxassist snapstart vol8

Create a snapshot volume

Command Syntax

vxassist snapshot volume_name new_volume_name

To create a snapshot volume of vol8, type:
#vxassist snapshot vol8 snapvol8

You can now back up the snapshot volume by whatever means you prefer. To avoid wasting space, you can then remove the snapshot volume, which occupies as much space as the original volume

11 Displaying Volume Configuration Information

The vxprint command can be used to display information about how a volume is configured.

Display the volume, mirror, and subdisk record information for all volumes as follows:

#vxprint -ht

Disk group: rootdg

  Disk group: rootdg      DG  NAME        NCONFIG    NLOG      MINORS     GROUP-ID   DM  NAME        DEVICE     TYPE      PRIVLEN    PUBLEN     STATE   V   NAME        USETYPE    KSTATE    STATE      LENGTH     READPOL      PREFPLEX   PL  NAME        VOLUME     KSTATE    STATE      LENGTH     LAYOUT       NCOL/WID MODE   SD  NAME        PLEX       DISK      DISKOFFS LENGTH       [COL/]OFF    DEVICE     MODE      dg  rootdg      759096729.1025.tweety      dm  disk10      c1t0d0s2 sliced      559        1044400     -   dm  disk20      c2t0d0s2 sliced      559        1044400     -      v   pubs        fsgen      ENABLED ACTIVE       2288       SELECT       -   pl  pubs-01     pubs       ENABLED ACTIVE       2288       CONCAT       -          RW   sd  disk10-01 pubs-01      disk10    0          2288       0            c0t0d0     ENA      v   vol8      fsgen      ENABLED ACTIVE       20480      SELECT       -   pl  vol8-01   vol8       ENABLED ACTIVE       20480      CONCAT       -          RW   sd  disk10-02 vol8-0 disk10        2288       20480      0            c0t1d0     ENA 


where dg is a disk group, dm is a disk,
v is a volume,
pl is a plex,
sd is a subdisk.
The top few lines indicate the headers that match each type of output line that follows.
Each volume is listed along with its associated plex(es) and subdisk(s).
Display volume-related information for a specific volume as follows:

#vxprint -t volume_name
To display information about vol8, type:

#vxprint -t vol8


Veritas Volume Manager Cheat Sheet

The Setup: an Oracle database supporting an application called ABC, and given the following:

Assuming that we need to create filesystems intended for Oracle and:

we are using a SID of 'ABC'

we have nine disks, c0t0d1 through c0t0d9

we will be creating six volumes: /u01/oradata/ABC ( 4gb ), /u02/oradata/ABC (4gb ), /u03/oradata/ABC (8gb ), /u04/oradata/ABC ( 4gb ), /u01/app/oracle/product/8.0.5_ABC ( 4gb ), and /u01/admin/ABC ( 4gb ).

We added these disks to Volume Manager Control during installation with vxinstall, or later with vxdiskadd(1m). Both vxinstall and vxdiskadd can add either initialize a previously unused disk, or 'encapsulate' a disk already containing data, but not under Volume Manager control.

we could take the following steps to set up our filesystems.

Adding an EMC disk to Veritas

Use the inq utility to see if you have SCSI visibility to the disk

Use vxdisk -o alldgs list to see if Veritas has seen the disk

if not, try the following, and then repeat the previous step:

devfsadm

vxdctl initdmp

vxdctl enable

Creating a Disk Group in Volume Manager:

Create a vxvm disk group for the Oracle SID using vxdg(1m). It's important that all of the disks/filesystems necessary for the ABC database to run are included in the disk group, otherwise the database could not be moved to another system.

The easiest way to do this is to use vxdiskadd
vxdiskadd c0t0d0 c0t0d1 c0t0d2 . . .

Doing it the hard way:
vxdg init ABCdg ABCdg01=c0t0d1

This would create a disk group called ABCdg containing one physical disk, c0t0d1, which will be referred to by it's name within vxvm, ABCdg01. You cannot initiate a disk group without specifying at least one disk as a member of the group. You should not include any disks destined to be part of an Oracle Database in the default vxvm group, rootdg. This allows you to use the vxdg 'deport' and 'import' commands to migrate an entire vxvm disk group to another host.

Now that we've create the disk group for our database, we can add the rest of our disks to it:

vxdg -g ABCdg adddisk ABCdg02=c0t0d2

vxdg -g ABCdg adddisk ABCdg09=c0t0d9

Creating Volumes With vxassist

vxassist(1m) is a vxvm command that acts as a front-end to other vxvm commands, much as newfs(1m) acts as a simpler-to-use front-end to mkfs(1m). Using vxassist with it's -v flag will display the actual vxvm commands used during volume construction. vxassist also front-ends vxfs commands and normal UNIX utilities during volume construction. An example of this is using a singlevxassist command to set up a volume. Using vxassist simplifies many vxvm tasks, but you should pay careful attention to the defaults you will inherit when setting up volumes using this tool. Understanding the contents of the /etc/default/vxassist file is important, especially when striping disks.

To make our 8gb /u03/oradata/ABC volume, we could use the following command line:

vxassist -g ABCdg make ABC01 8g

Breaking it down into it's component parts:

vxassist

-g ABCdg # this volume will belong to the ABCdg disk group

make # tells vxassist that we are creating a new volume

ABC01 # name the volume

8g # specify the size of the volume

Here we create the other five volumes:

vxassist -g ABCdg make ABC02 8g

vxassist -g ABCdg make ABC03 8g

vxassist -g ABCdg make ABC04 8g

vxassist -g ABCdg make ABC05 8g

vxassist -g ABCdg make ABC06 8g

Creating a File System:

We now lay filesystems down on our newly created volumes. We assume here that you've agreed with the Oracle DBA's to use a block size of 8192 for our filesystems. Not specifying a block size allow Veritas to dynamically assign one for you based upon the size of the volume. Our volumes would default to 1024 for a block size on our 4 GB volumes, and 2048 for our 8GB volume. We will specify the largefiles option, because the mkfs_vxfs(1m) command uses nolargefiles as a default, unlike mkfs_ufs(1m) on Solaris 2.6 or greater, which uses largefiles as a default. If we forgot to do so, we could use fsadm ( fsadm_vxfs(1m) ) to set the largefiles bit on. fsadm can also query a mounted filesystem for the current setting of this flag. Be certain that whatever flags you set during file system creation or modification are reflect in vfstab, or the resulting file system may be un-mount-able.

mkfs -F vxfs -o bsize=8192,largefiles /dev/vx/rdsk/ABCdg/ABC01

Mounting a File System:

Creating and mounting filesystems go hand in hand. You must be certain that any special flags set at creation time are reflected in /etc/vfstab, or on the command line if there is no entry in vfstab for your filesystem.

mount -F vxfs -o largefiles /dev/vx/dsk/ABCdg/ABC01 /u03/oradata/ABC

Resizing a mounted volume and file system:

/etc/vx/bin/vxresize -g rootdg -F vxfs archive 16g

Recovering disks under VM control:

(if you want to live dangerously, try looking at dfwset01:/root/progs/hose_vx_dg.pl)

Pre-reqs

Procedure
1) comment the volume(s) you're destroying out of vfstab

2) umount the volume

3) stop it:
vxvol stop volume01

4) remove it:
vxedit -rf rm volume01

5) remove the disk(s) the volume was on from the disk group ( up to the last disk )
vxdg -g mydg rmdisk dmdiskname01

6) Remove the the disk(s) from VXVM control
vxdisk rm c1t1d51s2

7) If you wish do get rid of the disk group ( in order to recover the last disk ), you must destroy the disk group:
vxdg destroy mydg

8) remove the last disk
vxdisk rm c2t7d9s2

9) If you're removing the disks from the EMC port, be sure to clean up:
drvconfig
disks
vxdctl enable
vxdctl initdmp

10) Freak out and realize you whacked the wrong thing, use /root/eotw to recover.

Renaming a VM Disk:

Since names like "ABCdg10" are not as descriptive as they could be, you can rename it to something more useful.

vxedit rename ABCdg10 ABCdg_hot_spare

Using long disk names can be more descriptive, but will make using vxva trickier, as it will truncate your disk name somewhat if it is too long.

Moving a Disk Group to Another System:

1) umount the disks:

umount /u01/app/oracle/product/8.0.5_ABC

umount /u01/admin/ABC

umount /u01/oradata/ABC

umount /u02/oradata/ABC

umount /u03/oradata/ABC

umount /u04/oradata/ABC

2) Stop the volumes:

vxvol -g ABCdg stopall

3) deport the disk group from one system:

vxdg deport ABCdg

4) import them on another system:

vxdg import ABCdg

5) Start all of the volumes on the new system and resync mirrors in the background:

vxrecover -g ABCdg -sb

6) mount the filesystems

mount /u01/app/oracle/product/8.0.5_ABC

mount /u01/admin/ABC

mount /u01/oradata/ABC

mount /u02/oradata/ABC

mount /u03/oradata/ABC

mount /u04/oradata/ABC

This procedure assumes that the second system can see the disks, and also that you have commented the file systems out of vfstab on the old system.

Rename a Disk Group

/root
dfwns19 # df -k | grep reports
/dev/vx/dsk/reportsdg/ncr01 /u01/app/oracle/admin/REPORTS
/dev/vx/dsk/reportsdg/ncr02 /u01/app/oracle/product/8.0.5REPORTS
/dev/vx/dsk/reportsdg/ncr03 /u01/oradata/REPORTS
/dev/vx/dsk/reportsdg/ncr04 /u02/oradata/REPORTS
/dev/vx/dsk/reportsdg/ncr05 /u03/oradata/REPORTS
/dev/vx/dsk/reportsdg/ncr06 /u04/oradata/REPORTS
/dev/vx/dsk/reportsdg/ncr07 /u11/oraarch/REPORTS

dfwns19 # umount /u01/app/oracle/admin/REPORTS
dfwns19 # umount /u01/app/oracle/product/8.0.5REPORTS
dfwns19 # umount /u01/oradata/REPORTS
dfwns19 # umount /u02/oradata/REPORTS
dfwns19 # umount /u03/oradata/REPORTS
dfwns19 # umount /u04/oradata/REPORTS
dfwns19 # umount /u11/oraarch/REPORTS

dfwns19 # vxvol -g reportsdg stopall

dfwns19 # vxdg deport reportsdg

dfwns19 # vxdg -n nclproddg import reportsdg

dfwns19 # vxdg list

Mirroring and then breaking mirrors

# make an sd from your dm ( use the -g, or it won't work. The book is wrong ) #                     sdname       dmname,start_offset,sd_length vxmake -g dwproddg sd dwprodd15-01 dwprodd15,0,8389440  # create a plex from your sd vxmake -g dwproddg plex dw01-02 sd=dwprodd15-01   # attach the plex to your volume vxplex -g dwproddg att dw01 dw01-02  # watch it silver ( check out the pause and slow options. badass ) /tmp dfwora03 # vxtask -l list Task:      162 RUNNING Type:      ATCOPY Operation: PLXATT Vol dw01 0.0 Started:   Tue Dec 05 03:47:29 2000 Throttle:  0 Progress:  18.33% 1537352 of 8388608 Blocks  Work time: 1 minute, 26 seconds (06:23 remaining)  # more than one sd per plex for plexen > one dm vxmake -g dwproddg sd dwprodd17-01 dwprodd17,0,8833920 vxmake -g dwproddg sd dwprodd18-01 dwprodd18,0,7944000 vxmake -g dwproddg plex dw03-02 sd=dwprodd17-01,dwprodd18-01 vxplex -g dwproddg att dw03 dw03-02  # now for the scary part. dis-associating the plexes and sd's vxplex -o rm dis dw01-01 # zap the dm from the dg vxdg -g dwproddg rmdisk dwprodd01 # nuke the da vxdisk rm c2t2d13s2    

NAME STATE ID
nclproddg enabled 949356971.2501.dfwns19


Renaming a disk group and volume

bansecdbdg -> secproddg used as an example

# Unmount all volumes in the bansecdbdg disk group. # Stop all volumes vxvol -g bansecdbdg stopall  # Deport bansecdbdg vxdg deport bansecdbdg  # Import bansecdbdg using the -n option vxdg -n secproddg import bansecdbdg  # Recover the disk group by the new dg name vxrecover -g secproddg -sb  # Rename each volume vxedit -g secproddg rename bansecdb01 sec01 vxedit -g secproddg rename bansecdb02 sec02 . . . vxedit -g secproddg rename bansecdb08 sec08  # Update vfstab, your mount points and remount the volumes 

Resizing a disk, while specifying the disk

# Resize volume u11, and only use disk  dev20dg23 # x requires the volume to grow (i.e. does not let you specify a  # smaller than existing volume size), and b backgrounds the  # task (according to the tech) ./vxresize -F vxfs -bx -g dev20dg u11 16g dev20dg23 

Moving a volume off of a particluar disk

# Move u11 off of dm dev20dg20 vxassist -g dev20dg move u11 !dev20dg20

Manage your logical volumes with Veritas Volume Manager (VxVM) V5 for AIX 5.3 and 6.1

Introduction

In the world of UNIX® storage management, there are two primary leaders: IBM and Veritas (now Symantec). Both companies offer products that help UNIX system administrators manage storage with very flexible methods. Veritas offers the Veritas Volume Manager (VxVM), which is either packaged as a standalone add-on, or part of a larger package, such as the Veritas On-Line Storage Manager. VxVM is a storage management subsystem that lets you manage physical disks as logical devices. (Logical devices do not have the limitations of physical disks.)

Veritas Volume Manager

VxVM is a storage management subsystem that lets you manage physical disks as logical devices called volumes. A VxVM volume appears to applications and the operating system as a physical disk on which file systems, databases, and other managed data objects can be configured. VxVM provides easy-to-use online disk storage management for computing environments and Storage Area Network (SAN) environments. By supporting the Redundant Array of Independent Disks (RAID) model, VxVM can be configured to protect against disk and hardware failure, and to increase I/O throughput.

VxVM provides features that enhance fault tolerance and fast recovery from disk failure. VxVM overcomes physical restrictions imposed by hardware disk devices by providing a logical volume management layer that lets volumes span multiple disks. VxVM also provides the tools to improve performance, and ensure data availability and integrity. You can use VxVM to dynamically configure disk storage while the system is active.

VxVM and the operating system

VxVM operates as a subsystem between operating system and data management systems, similar to file systems and database management systems. VxVM is tightly coupled with the operating system. Before a disk can be brought under VxVM control, the disk must be accessible through the operating system device interface. VxVM is layered on top of the operating system interface services, and is dependent on how the operating system accesses physical disks.

VxVM is dependent upon the operating system for:

  • Operating system (disk) devices
  • Device handles
  • VxVM dynamic multipathing (DMP) metadevice

VxVM and data storage

VxVM uses two types of objects to handle storage management:

Physical objects
Physical disks, or other hardware, with block and raw operating system device interfaces that are used to store data.
Virtual objects
When one or more physical disks are brought under the control of VxVM, it creates virtual objects called volumes on those physical disks. Each volume records and retrieves data from one or more physical disks. Volumes are accessed by file systems, databases, or other applications the same way that physical disks are accessed.

Volumes are also composed of other virtual objects (plexes and subdisks) that are used in changing the volume configuration. Volumes and their virtual components are called virtual objects, or VxVM objects.

Physical disks

A physical disk is the basic storage device, or media, where the data is ultimately stored. You can access the data on a physical disk by using a device name to locate the disk. The physical disk device name varies with the computer system you use. Not all parameters are used on all systems. Typical device names are of the form hdisk(number) , where (number) specifies a unique disk number starting at 0.


Figure 1. Displaying a physical disk
Displaying a physical disk

Disk arrays

Performing I/O to disks is relatively slow because disks are physical devices that require time to move the heads to the correct position before reading or writing. If all of the read or write operations are done to individual disks, one at a time, the read-write time can become unmanageable. Performing these operations on multiple disks can help reduce the problem.

A disk array is a collection of physical disks that VxVM can represent to the operating system as one or more virtual disks or volumes. To the operating system, the volumes created by VxVM look and act like physical disks. Applications that interact with volumes should work the same way they do with physical disks. Figure 2 shows how VxVM presents the disks as several volumes, in a disk array, to the operating system.


Figure 2. Displaying disks in a disk array
Displaying disks in a disk array

Data can be spread across several disks within an array to distribute or balance I/O operations across the disks. Using parallel I/O across multiple disks improves I/O performance by increasing data transfer speed and overall throughput for the array.

Multipathed disk arrays

Some disk arrays provide multiple ports to access their disk devices. These ports, coupled with the host bus adaptor (HBA) controller and any data bus or I/ O processor local to the array, provide multiple hardware paths to access the disk devices. Such disk arrays, called multipathed, can be connected to host systems in many different configurations, such as multiple ports connected to different controllers on a single host, chaining of the ports through a single controller on a host, or ports connected to different hosts simultaneously.

Device discovery

Device discovery is the process of discovering the disks that are attached to a host. This feature is important for dynamic multipathing (DMP) because it needs to support a growing number of disk arrays from several vendors. In conjunction with the ability to discover devices attached to a host, the device discovery service lets you add support dynamically for new disk arrays. The operation, which uses a facility called the device discovery layer (DDL), does not require a reboot.

Virtual objects

Virtual objects in VxVM include:

  • Disk groups
  • Volume manager (VM) disks
  • Subdisks
  • Plexes
  • Volumes

The connection between physical objects and VxVM objects is made when you place a physical disk under VxVM control. After installing VxVM on a host system, you must bring the contents of physical disks under VxVM control by collecting the VM disks into disk groups and allocating the disk group space to create logical volumes.

To bring the physical disk under VxVM control, the disk must not be under LVM control. VxVM must take control of the physical disks; the disk cannot be under the control of another storage manager (such as LVM). Listing 1 shows an example of using thevxdisk command to list the disk status.


Listing 1. Using vxdisk to list disk status
                                 # vxdisk -e list DEVICE       TYPE      DISK         GROUP        STATUS       OS_NATIVE_NAME Disk_0       auto      -             -            online       hdisk51 Disk_1       auto      -             -            online       hdisk48 Disk_2       auto      -             -            online       hdisk47 Disk_3       auto      -             -            online       hdisk50 Disk_4       auto      -             -            online       hdisk49 Disk_5       auto      -             -            online       hdisk46 Disk_6       auto      -             -            online       hdisk44 Disk_7       auto      -             -            online       hdisk42 Disk_8       auto      -             -            online       hdisk45 Disk_9       auto      -             -            offline      hdisk43 Disk_10      auto      -             -            online       hdisk41 Disk_11      auto      -             -            online       hdisk40 Disk_12      auto      -             -            online       hdisk38 Disk_13      auto      -             -            online       hdisk37 Disk_14      auto      -             -            online       hdisk39 Disk_15      auto      -             -            online       hdisk36 Disk_16      auto      -             -            online       hdisk33 Disk_17      auto      -             -            online       hdisk34 Disk_18      auto      -             -            online       hdisk31 Disk_19      auto      -             -            online       hdisk35 Disk_20      auto      -            -     online      hdisk32 Disk_21   auto      -          -            LVM        hdisk27              

VxVM creates virtual objects and makes logical connections between the objects. The virtual objects are then used by VxVM to do storage management tasks.

Disk groups

A disk group is a collection of disks that shares a common configuration, and is managed by VxVM. A disk group configuration is a set of records with detailed information about related VxVM objects, their attributes, and their connections. A disk group name can be up to 31 characters long.

You can create additional disk groups when you need them. Disk groups allow you to group disks into logical collections. A disk group and its components can be moved as a unit from one host machine to another.

VM disks

When you place a physical disk under VxVM control, a VM disk is assigned to the physical disk. A VM disk is under VxVM control and is usually in a disk group. Each VM disk corresponds to one physical disk. VxVM allocates storage from a contiguous area of VxVM disk space. A VM disk typically includes a public region (allocated storage) and a small private region where VxVM internal configuration information is stored. Each VM disk has a unique disk media name, called a virtual disk name. You can either define a disk name of up to 31 characters, or allow VxVM to assign a default name of diskgroup(number), where diskgroup is the name of the disk group to which the disk belongs. Figure 3 shows an example.


Figure 3. VM Disk
VM Disk

Subdisks

A subdisk is a set of contiguous disk blocks. A block is a unit of space on the disk. VxVM allocates disk space using subdisks. A VM disk can be divided into one or more subdisks. Each subdisk represents a specific portion of a VM disk, which is mapped to a specific region of a physical disk. The default name for a VM disk is diskgroup(number). The default name for a subdisk is diskgroup(number-number), as shown in Figure 4, where diskgroup is the name of the disk group to which the disk belongs.


Figure 4. Subdisk
Subdisk

A VM disk can contain multiple subdisks, but subdisks cannot overlap or share the same portions of a VM disk. Figure 5 shows a VM disk with three subdisks. The VM disk is assigned to one physical disk.


Figure 5. Three subdisks assigned to one VM Disk
Three subdisks assigned to one VM Disk

Plexes

VxVM uses subdisks to build virtual objects called plexes. A plex consists of one or more subdisks located on one or more physical disks. For example, the plex vol01-01 is shown in Figure 6.


Figure 6. A plex with two subdisks
A plex with two subdisks

Volumes

A volume is a virtual disk device that appears to applications, databases, and file systems like a physical disk device, but does not have the physical limitations of a physical disk device. A volume consists of one or more plexes, each holding a copy of the selected data in the volume. Due to its virtual nature, a volume is not restricted to a particular disk or a specific area of a disk.

You can change the configuration of a volume by using VxVM user interfaces. Making configuration changes will not cause disruption to applications or file systems that are using the volume. For example, a volume can be mirrored on separate disks or moved to use different disk storage.

VxVM uses the default naming conventions of vol(number) for volumes and vol(number-number) for plexes in a volume. For ease of administration, you can choose to select more meaningful names for the volumes that you create.

VxVM 5.0 installation

This section introduces the VxVM installation and configuration commands you can use to do tasks associated with VxVM objects on AIX versions 5.3 and 6.10. VxVM relies on the following constantly running daemons and kernel threads for its operation:

vxconfigd
The VxVM configuration daemon maintains disk and group configurations, communicates configuration changes to the kernel, and modifies configuration information stored on disks.
vxiod
VxVM I/O kernel threads provide extended I/O operations without blocking calling processes. By default, 16 I/O threads are started at boot time, and at least one I/O thread must continue to run at all times.
vxrelocd
The hot-relocation daemon monitors VxVM for events that affect redundancy, and performs hot-relocation to restore redundancy.

Before you start installation, make sure that the PATH variable is set properly because most of the commands used in the installation are in the /sbin or /usr/sbin directory. Add the directories to your PATH environment variable, as shown in Listing 2.


Listing 2. PATH information
                 For a KSH Shell (or sh), use the command:   # PATH=/sbin:/usr/sbin:/etc/vx/bin:/opt/VRTS/bin:$PATH# export PATH  For a C Shell (csh or tcsh), use the command:   % setenv PATH /sbin:/usr/sbin:/etc/vx/bin:/opt/VRTS/bin:${PATH} 

Verifying the OS environment

Veritas Storage Foundation 5.0 from Symantec can only be installed on a system running AIX 5.3 or AIX 6. They must be at the proper maintenance level, with additional AIX patches installed, before installing Veritas Storage Foundation. Check the level of the OS by using the oslevel command on an AIX machine, as shown in Listing 3.


Listing 3. Check OS level
                 # oslevel 6.1.0.0  # oslevel -rq Known Recommended Maintenance Levels ------------------------------------ 6110-00  # oslevel 5.3.0.0 # oslevel -rq Known Recommended Maintenance Levels ------------------------------------ 5300-07 5300-06 5300-05 5300-04 5300-03 5300-02 5300-01 5380-00 

Prerequisites

The following Veritas Storage Foundation packages are required or optional (as noted) for installation.

VRTSperl PERL languageRequired
VRTSspt VERITAS support toolsRequired
VRTSvlic VERITAS licensing utilitiesRequired
VRTSvmdoc VERITAS Volume Manager online guidesOptional
VRTSvmman VERITAS Volume Manager manual pagesOptional
VRTSvmpro VERITAS Volume Manager ProviderRequired for VEA and ISP
VRTSvxvm VERITAS Volume ManagerRequired
VRTSveki VERITAS Kernel Interface componentRequired
VRTSvxfs VERITAS File SystemRequired
VRTSddlpr VERITAS Device Discovery Layer ProviderRequired
VRTSfspro VERITAS File System ProviderRequired for VEA, even if you are not installing the VERITAS File System software
VRTSob VERITAS Enterprise Administrator ServiceRequired for VEA and ISP
VRTSobgui VERITAS Enterprise AdministratorRequired for VEA client
VRTSFman VERITAS File System manual pagesOptional
VRTSFsdoc VERITAS File System online guidesOptional

Install Veritas Storage Foundation

To install Storage Foundation, download file sets from the Symantec Web site. It is best practice to check for the data integrity of the file sets by running the cksum command and compare to the source location.

  1. Log in as superuser. Use the installp command to install the required packages, as shown in Listing 4.

    Listing 4. installp
                              Note:- Check the size of the file by using cksum command as below,  # cksum VRTSvxvm.bff 1823433071 126310400 VRTSvxvm.bff  installp command is used as below,        installp -acXd VRTSvmdoc.bff VRTSvmdoc        installp -acXd VRTSvmman.bff VRTSvmman        installp -acXd VRTSveki.bff VRTSveki        installp -acXd VRTSspt.bff VRTSspt        installp -acXd VRTSvxvm.bff VRTSvxvm        installp -acXd VRTSvlic.bff VRTSvlic         installp -acXd VRTSvxfs.bff VRTSvxfs        installp -acXd VRTSperl.bff VRTSperl  Note:- All these commands can be put inside a ksh script and  installed by invoking that script. 

    Use the lslpp command to verify the complete installation of Veritas VxVM, as shown in Listing 5.



    Listing 5. Verify installation
                                                     # lslpp -l |grep -i vrts    VRTSspt                    5.0.1.0  COMMITTED  Veritas Support Tools    VRTSveki                   5.0.1.0  COMMITTED  Veritas Kernel Interface    VRTSvlic                   5.0.1.0  COMMITTED  VRTSvlic Symantec License   VRTSvxfs                   5.0.1.0  COMMITTED  Veritas File System    VRTSvxvm                   5.0.1.0  COMMITTED  Veritas Volume Manager   VRTSveki                   5.0.1.0  COMMITTED  Veritas Kernel Interface   VRTSvmman                  5.0.1.0  COMMITTED  Veritas Manual Pages   VRTSvdoc                   5.0.1.0  COMMITTED  Veritas Documentation  

  2. To install from a CD-ROM, mount a CD-ROM and begin installation as follows.
    1. Log in as superuser.
    2. Place the Veritas software disc in a CD-ROM drive connected to your system.
    3. Mount the CD manually:
      # mkdir -p /mnt/cdrom # mount -V cdrfs -o ro /dev/cdN /mnt/cdrom

      where N in /dev/cdN is a number for the CD device configured on the target system. Usually the value for N is 0.

      Check the CD device nodes on your system in the /dev directory, or use the method below and follow Step 4 onward. To invoke the common installer, run the install command on the CD as shown below.

      # /mnt/cdrom/CD_NAME/installer 

    4. If the VRTSvlic licensing package is not installed, a message asks if you want to install VRTSvlic. Follow the instructions to install the package.
    5. If the VRTSvlic licensing package is installed, the Product Status page displays:
      • Products available for installation.
      • Products currently installed.
      • Products that are licensed.
      • Options for operations you can initiate.

      Use the /opt/VRTS/bin/vxlicrep command to view a report of the license type for each product

    6. At the prompt on the Product Status page, enter L to add a license key and click Return to begin.
    7. At the next prompt, enter the license key and click Return.
    8. At the Product Status page, enter I for the product installer and click Return. The product installer is displayed.
    9. At the Veritas product installer, enter the number of the product you want to install and click Return. The product installation begins automatically.

      At the end of the process, the Veritas product installer is displayed again.

    10. You are prompted to enter the systems' names on which the software is to be installed. When prompted, clickEnter to continue.

Veritas licensing

Install the license key using the vxlicinst command, as shown in Listing 6. (The license key below is an example only. For actual installation, the key is obtained from Symantec.)


Listing 6. vxlicinst command output
                 Install License:-  # /sbin/vxlicinst  Symantec License Manager vxlicinst utility version 3.02.16.0 Copyright (C) 1996-2006 Symantec Corporation. All rights reserved.  Enter your license key : RGCU-CB9F-YCBX-BFYX-OTGH-GN4O-7KYP-PPC  Number of days left for Demo = 150  License key successfully installed for VERITAS Storage Foundation for DB2 License key successfully installed for VERITAS File System License key successfully installed for VERITAS Volume Manager License key successfully installed for VERITAS Mapping Services 

Other licensing commands

The Veritas licensing commands are in the VRTSvlic package. You must install VRTSvlic for the licensing process to work. There are two commands:

  • vxlicrep lets you view currently installed licenses.
  • vxlictest retrieves features encoded in a license key and describes them.

Enable the vxconfigd daemon

Execute the commands in Listing 7 to enable the vxconfigd daemon.


Listing 7. Enable the daemon
                 # vxconfigd -km disable   # vxdctl init # vxdctl enable 

There is more information about the vxconfigd daemon and vxdctl later in this section.

To check the Storage Foundation processes and confirm successful Veritas File System installation, use the following command:

# lsvfs vxfs vxfs    32      /sbin/helpers/vxfs      /sbin/helpers/vxfs

To confirm that the Volume Manager processes (vxconfigd, vxnotify, and vxrelocd) are running, use:

# ps -e |grep vx

Create VxVM disk groups

Listing 8 shows the output of the lspv command without VxVM configuration.


Listing 8. lspv output
                 # lspv hdisk0          00b28305ad573b54                    rootvg          active hdisk1          none                         None hdisk2          none                               None hdisk3          00b28305eea1b645                   None hdisk4          00b28305eb18f002                   None hdisk5          none                                          None hdisk6          none                                          None hdisk7          none                                None hdisk8          none                                 None hdisk9          none                                 None hdisk10         none                                         None hdisk11         none                                         None hdisk12         none                                 None hdisk13         none                                 None hdisk14         00b2830584f972f3                    None hdisk15         00b2830584f9751a                    None 

Now the disks have to be configured as a Veritas Volume disk group (vxvmdg) by initializing and adding disks. There are two levels of initialization for disks in the Volume Manager:

  • Formatting of the disk media itself. This must be done outside of the Volume Manager.
  • Storing identification and configuration information on the disk for use by the Volume Manager.
Volume Manager interfaces are provided to step you through this level of disk initialization.

A fully initialized disk can be added to a disk group and used to replace a previously failed disk, or to create a new disk group.


Listing 9. vxdg command
                 #cd /etc/vx/bin --this is done because the command vxdg is in this path.  #  vxdg -g vxvmdg adddisk vxvmdg1=Disk_10 # vxdg -g vxvmdg adddisk vxvmdg2=Disk_9 # vxdg -g vxvmdg adddisk vxvmdg3=Disk_7 # vxdg -g vxvmdg adddisk vxvmdg4=Disk_6 # vxdg -g vxvmdg adddisk vxvmdg5=Disk_5 # vxdg -g vxvmdg adddisk vxvmdg6=Disk_4 # vxdg -g vxvmdg adddisk vxvmdg7=Disk_3 #  vxdg -g vxvmdg adddisk vxvmdg8=Disk_2  

Display configured disks

Now you use the vxdisk list command to display various configured disks, as shown in Listing 10.


Listing 10. vxdisk command output
                  # vxdisk list DEVICE       TYPE            DISK         GROUP        STATUS Disk_0       auto:none       -            -      online invalid Disk_1       auto:none       -            -      online invalid Disk_2       auto:cdsdisk    vxvmdg8      vxvmdg       online Disk_3       auto:cdsdisk    vxvmdg7      vxvmdg       online Disk_4       auto:cdsdisk    vxvmdg6      vxvmdg       online Disk_5       auto:cdsdisk    vxvmdg5      vxvmdg       online Disk_6       auto:cdsdisk    vxvmdg4      vxvmdg       online Disk_7       auto:cdsdisk    vxvmdg3      vxvmdg       online Disk_8       auto:aixdisk    -            -            online Disk_9       auto:cdsdisk    vxvmdg2      vxvmdg       online Disk_10      auto:cdsdisk    vxvmdg1      vxvmdg       online Disk_11      auto:none       -            -            online invalid Disk_12      auto:LVM        -            -            LVM Disk_13      auto:cdsdisk    -            -            online aliased Disk_14      auto:LVM        -            -            LVM Disk_15      auto:cdsdisk    -            -            online aliased 

The disk listing sometimes displays in the format hdisk(number). Use the format accordingly while configuring with the vxdgcommand.

Troubleshooting and administration

This section includes some troubleshooting tips and common commands used to manage VxVM.

Disks go into Error state

Sometimes, when the disks go into error state, Veritas VxVM is unable to claim those disks as vxvmdgs. To fix this type of problem, run the following set of commands, in the following order.

To see corresponding mapping between hdisk and Disk lists, run the command in Listing 11.


Listing 11. Solution for disks going to error state
                  # vxdisk -e list DEVICE       TYPE      DISK         GROUP        STATUS       OS_NATIVE_NAME Disk_0       auto      -             -            online       hdisk51 Disk_1       auto      -             -            online       hdisk48 Disk_2       auto      -             -            online       hdisk47 Disk_3       auto      -             -            online       hdisk50 Disk_4       auto      -             -            online       hdisk49 Disk_5       auto      -             -            online       hdisk46 Disk_6       auto      -             -            online       hdisk44 Disk_7       auto      -             -            online       hdisk42 Disk_8       auto      -             -            online       hdisk45 Disk_9       auto      -             -            offline      hdisk43 Disk_10      auto      -             -            online       hdisk41 Disk_11      auto      -             -            online       hdisk40 Disk_12      auto      -             -            online       hdisk38 Disk_13      auto      -             -            online       hdisk37 Disk_14      auto      -             -            online       hdisk39 Disk_15      auto      -             -            online       hdisk36 Disk_16      auto      -             -            online       hdisk33 Disk_17      auto      -             -            online       hdisk34 Disk_18      auto      -             -            online       hdisk31 Disk_19      auto      -             -            online       hdisk35 Disk_20      auto      -             -            error        hdisk32 Disk_21      auto      -             -            LVM          hdisk27 Disk_22      auto      -             -            online       hdisk30 Disk_23      auto      -             -            offline      hdisk29 Disk_24      auto      -             -            online       hdisk28 Disk_25      auto      -             -            LVM          hdisk26   

In the above listing, Disk_20 is in error state and needs to be fixed to continue VxVM operation.

To see the list of hdisks with an error state:


Listing 12. Get list
                 # vxdisk -o alldgs list DEVICE       TYPE            DISK         GROUP        STATUS Disk_0       auto:none       -            -            online invalid Disk_1       auto:none       -            -            online invalid Disk_2       auto:none       -            -            online invalid Disk_3       auto:none       -            -            online invalid Disk_4       auto:none       -            -            online invalid Disk_5       auto:none       -            -            online invalid Disk_6       auto:aixdisk    -            -            online Disk_7       auto:none       -            -            online invalid Disk_8       auto:none       -            -            online invalid Disk_9       auto            -            -            offline Disk_10      auto:none       -            -            online invalid Disk_11      auto:none       -            -            online invalid Disk_12      auto:none       -            -            online invalid Disk_13      auto:aixdisk    -            -            online Disk_14      auto:aixdisk    -            -            online Disk_15      auto:none       -            -            online invalid Disk_16      auto:none       -            -            online invalid Disk_17      auto:none       -            -            online invalid Disk_18      auto:none       -            -            online invalid Disk_19      auto:none       -            -            online invalid Disk_20      auto:cdsdisk    -            -            error Disk_21      auto:LVM        -            -            LVM Disk_22      auto:none       -            -            online invalid Disk_23      auto            -            -            offline Disk_24      auto:none       -            -            online invalid Disk_25      auto:LVM        -            -            LVM #cd /etc/vx/bin 

Try to use the vxdisksetup and vxdiskunsetup commands to format the hdisk, as shown in Listing 13.


Listing 13. vxdisksetup and vxdiskunsetup
                 # /etc/vx/bin/vxdisksetup -if Disk_20 format=aixdisk VxVM vxdisk ERROR V-5-1-5433 Device Disk_20: init failed:         Disk sector size is not supported   # /etc/vx/bin/vxdiskunsetup  -C Disk_20 format=aixdisk VxVM vxdiskunsetup ERROR V-5-2-2208 format=aixdisk: Device address must  be of the form enclr_x where      x = one or more decimal digit 

Since both the commands are failing to clear the error state, see the detailed list using the vxdisk command for error one and proper one, as shown in Listing 14.


Listing 14. vxdisk
                 a) Disk with error state  # vxdisk list Disk_20 Device:    Disk_20 devicetag: Disk_20 type:      auto info:      format=cdsdisk,privoffset=256 flags:     online error private autoconfig pubpaths:  block=/dev/vx/dmp/Disk_20 char=/dev/vx/rdmp/Disk_20 guid:      {5d5aa3b8-1dd2-11b2-aeae-2a5848ed1fb8} udid:      AIX%5FVDASD%5FDISKS%5F600507630EFFFD670000000000000115 site:      - errno:     Disk is not useable, bad format Multipathing information: numpaths:   1 hdisk32 state=enabled  b) Proper one  # vxdisk list Disk_7 Device:    Disk_7 devicetag: Disk_7 type:      auto info:      format=none flags:     online ready private autoconfig invalid pubpaths:  block=/dev/vx/dmp/Disk_7 char=/dev/vx/rdmp/Disk_7 guid:      - udid:      AIX%5FVDASD%5FDISKS%5F600507630EFFFD670000000000000122 site:      - Multipathing information: numpaths:   1 hdisk42 state=enabled 

Now try with the chpv command:


Listing 15. chpv
                 # chpv -C hdisk32  # vxdisk list DEVICE       TYPE            DISK         GROUP        STATUS Disk_0       auto:none       -            -            online invalid Disk_1       auto:none       -            -            online invalid Disk_2       auto:none       -            -            online invalid Disk_3       auto:none       -            -            online invalid Disk_4       auto:none       -            -            online invalid Disk_5       auto:none       -            -            online invalid Disk_6       auto:aixdisk    -            -            online Disk_7       auto:none       -            -            online invalid Disk_8       auto:none       -            -            online invalid Disk_9       auto            -            -            offline Disk_10      auto:none       -            -            online invalid Disk_11      auto:none       -            -            online invalid Disk_12      auto:none       -            -            online invalid Disk_13      auto:aixdisk    -            -            online Disk_14      auto:aixdisk    -            -            online Disk_15      auto:none       -            -            online invalid Disk_16      auto:none       -            -            online invalid Disk_17      auto:none       -            -            online invalid Disk_18      auto:none       -            -            online invalid Disk_19      auto:none       -            -            online invalid Disk_20      auto:cdsdisk    -            -            error Disk_21      auto:LVM        -            -            LVM Disk_22      auto:none       -            -            online invalid Disk_23      auto            -            -            offline Disk_24      auto:none       -            -            online invalid Disk_25      auto:LVM        -            -            LVM   # lspv hdisk26         00cc17cec566971d                    rootvg          active hdisk27         00cc17ceebb54c8c                    rootvg          active hdisk28         none                                None hdisk29         none                                VeritasVolumes hdisk30         none                                None hdisk31         none                                None hdisk32         none                                VeritasVolumes hdisk33         none                                None hdisk34         none                                None hdisk35         00cc17ce59d7d248                    None hdisk36         none                                None hdisk37         none                                VeritasVolumes hdisk38         none                                None hdisk39         none                                VeritasVolumes hdisk40         none                                None hdisk41         none                                None hdisk42         none                                None hdisk43         none                                VeritasVolumes hdisk44         none                                VeritasVolumes hdisk45         none                                None hdisk46         none                                None hdisk47         00cc17cee3638f87                    None hdisk48         00cc17cee2ea1ed0                    None hdisk49         00cc17cee2ea1f4d                    None hdisk50         00cc17cee2ea1fe1                    None hdisk51         00cc17cec8c84486                    None 

Even after executing the chpv command, the Veritas volumes tag is not removed. Try enabling and disabling with the vxconfigddaemon.


Listing 16. vxconfigd
                 # vxconfigd -k  # vxdisk scandisks  # lspv hdisk26         00cc17cec566971d                    rootvg          active hdisk27         00cc17ceebb54c8c                    rootvg          active hdisk28         none                                None hdisk29         none                                VeritasVolumes hdisk30         none                                None hdisk31         none                                None hdisk32         none                                VeritasVolumes hdisk33         none                                None hdisk34         none                                None hdisk35         00cc17ce59d7d248                    None hdisk36         none                                None hdisk37         none                                VeritasVolumes hdisk38         none                                None hdisk39         none                                VeritasVolumes hdisk40         none                                None hdisk41         none                                None hdisk42         none                                None hdisk43         none                                VeritasVolumes hdisk44         none                                VeritasVolumes hdisk45         none                                None hdisk46         none                                None hdisk47         00cc17cee3638f87                    None hdisk48         00cc17cee2ea1ed0                    None hdisk49         00cc17cee2ea1f4d                    None hdisk50         00cc17cee2ea1fe1                    None hdisk51         00cc17cec8c84486                    None 

The tag is still not removed, and the disk is still in the error state.

Run the set of commands below for up to seek=3 or 4, and then run the other set of commands. The dd command clears the bad blocks, if any, and the error state is removed.


Listing 17. dd
                 # dd if=/dev/zero of=/dev/hdisk37 bs=512 seek=0 count=10 10+0 records in 10+0 records out  # dd if=/dev/zero of=/dev/hdisk37 bs=512 seek=1 count=10 10+0 records in 10+0 records out   # dd if=/dev/zero of=/dev/hdisk32 bs=512 seek=0 count=10 10+0 records in 10+0 records out 

Now run the vxdisksetup command, and it will clear the error state of the disk.


Listing 18. vxdisksetup
                 # vxdisksetup -if Disk_20 format=aixdisk  Now the Disk_20 is online:.  Disk_20      auto      -             -            online       hdisk32 

Adding hdisks into the Veritas Volume data group

Using a command line shell script, hdisks can be added into the Veritas Volume datagroup(vxvmdg), as shown in Listing 19.


Listing 19. Adding hdisks into Veritas Volume data group(vxvmdg)
                  # for i in 7 8 9 10 11 12 13 14; do > vxdg -g vxvmdg adddisk vxvmdg$j=hdisk$i > (( j += 1 )) > done  Note:- Here numbers 7,8,9,.... etc indicates hdisk numbers.  Now vxdisk command output displays the added disks as vxvmdg disks.  # vxdisk list DEVICE       TYPE            DISK         GROUP        STATUS hdisk0       auto:LVM        -            -            LVM hdisk1       auto:cdsdisk    -            -            online hdisk2       auto:cdsdisk    -            -            online hdisk3       auto:none       -            -            online invalid hdisk4       auto:cdsdisk    -            -            online hdisk5       auto:none       -            -            online invalid hdisk6       auto:aixdisk    -            -            online hdisk7       auto:cdsdisk    vxvmdg1      vxvmdg       online hdisk8       auto:cdsdisk    vxvmdg2      vxvmdg       online hdisk9       auto:cdsdisk    vxvmdg3      vxvmdg       online hdisk10      auto:cdsdisk    vxvmdg4      vxvmdg       online hdisk11      auto:cdsdisk    vxvmdg5      vxvmdg       online hdisk12      auto:aixdisk    vxvmdg6      vxvmdg       online hdisk13      auto:aixdisk    vxvmdg7      vxvmdg       online hdisk14      auto:cdsdisk    vxvmdg8      vxvmdg       online  

Other administrative commands

You can also use the following commands to handle Veritas VxVM:

vxdiskadm
The Volume Manager Support Operations menu interface, which provides a menu of disk operations. Each entry in the main menu leads you through a particular operation by providing information and asking questions. Default answers are provided for many questions, so common answers can be selected quickly.
vxdiskadd
Use this utility to add standard disks to the Volume Manager. vxdiskadd leads you through initializing a new disk by displaying information and asking questions.
vxdisk
A command-line utility for administering disk devices. You can use vxdisk to define special disk devices, to initialize information stored on disks that the Volume Manager uses to identify and manage disks, and to perform additional special operations.
vxdg
The command-line utility for operating on disk groups. You can use it to create new disk groups, to add and remove disks from disk groups, and to enable (import) or disable (deport) access to disk groups.
vxdisk list
Lists disk information and displays spare disks with a spare flag.
vxprint
Lists disk and other information, and displays spare disks with a SPARE flag.

Listing 20 shows an example of using the vxdiskadm command to control various Volume Manager operations. Any of these options can be selected for the required operation to be performed.


Listing 20. vxdiskadm usage
                 # vxdiskadm  Volume Manager Support Operations Menu: VolumeManager/Disk   1      Add or initialize one or more disks  2      Remove a disk  3      Remove a disk for replacement  4      Replace a failed or removed disk  5      Mirror volumes on a disk  6      Move volumes from a disk  7      Enable access to (import) a disk group  8      Remove access to (deport) a disk group  9      Enable (online) a disk device  10     Disable (offline) a disk device  11     Mark a disk as a spare for a disk group  12     Turn off the spare flag on a disk  13     Unrelocate subdisks back to a disk  14     Exclude a disk from hot-relocation use  15     Make a disk available for hot-relocation use  16     Prevent multipathing/Suppress devices from VxVM's view  17     Allow multipathing/Unsuppress devices from VxVM's view  18     List currently suppressed/non-multipathed devices  19     Change/Display the default disk layouts  20     Mark a disk as allocator-reserved for a disk group  21     Turn off the allocator-reserved flag on a disk  22     Change the disk naming scheme  list   List disk information    ?      Display help about menu  ??     Display help about the menuing system  q      Exit from menus  Select an operation to perform:  Below is the example for creating volume group vxvmdg and adding a disk to it.   Select an operation to perform: 1  Add or initialize disks Menu: VolumeManager/Disk/AddDisks    Use this operation to add one or more disks to a disk group.  You can   add the selected disks to an existing disk group or to a new disk group   that will be created as a part of the operation. The selected disks may   also be added to a disk group as spares. Or they may be added as   nohotuses to be excluded from hot-relocation use. The selected   disks may also be initialized without adding them to a disk group   leaving the disks available for use as replacement disks.    More than one disk or pattern may be entered at the prompt.  Here are   some disk selection examples:    all:          all disks   hdisk8:       a single disk (in the hdisk# naming scheme)   scsi3 :       all disks on scsi controller 3   fscsi6 :      all disks on fiber controller 6   xyz_0 :       a single disk (in the enclosure based naming scheme)   xyz_ :        all disks on the enclosure whose name is xyz  Select disk devices to add: [(pattern-list),all,list,q,?] hdisk2    Here is the disk selected.  Output format: [Device_Name]    hdisk2  Continue operation? [y,n,q,?] (default: y) y    You can choose to add this disk to an existing disk group, a   new disk group, or leave the disk available for use by future   add or replacement operations.  To create a new disk group,   select a disk group name that does not yet exist.  To leave   the disk available for future use, specify a disk group name   of "none".  Which disk group [,none,list,q,?] (default: none) vxvmdg  Create a new group named vxvmdg? [y,n,q,?] (default: y) y  Create the disk group as a CDS disk group? [y,n,q,?] (default: y) y  Use a default disk name for the disk? [y,n,q,?] (default: y) y  Add disk as a spare disk for vxvmdg? [y,n,q,?] (default: n) y  Add site tag to disk? [y,n,q,?] (default: n) y    A new disk group will be created named vxvmdg and the selected   disks will be added as spares to the disk group with default   disk names.    hdisk2 

Now you can use lspv or vxdisk to see that hdisk1 has been added to the vxvmdg group.

Listing 21 shows how to use vxdg to destroy any unwanted disk group entries.


Listing 21. Using vxdg to destroy unwanted disk group entry
                  #cd /etc/vx/bin  # vxdisk -e list DEVICE       TYPE      DISK         GROUP        STATUS       OS_NATIVE_NAME Disk_0       auto       -              -            online       hdisk15 Disk_1       auto       -              -            online       hdisk14 Disk_2       auto       -              -            online       hdisk13 Disk_3       auto       -              -            online       hdisk12 Disk_4       auto      TCd7-90536    TCg3-90536        online       hdisk10 Disk_5       auto      TCd6-90536    TCg3-90536        online       hdisk9 Disk_6       auto       -              -            online       hdisk11 Disk_7       auto      TCd5-90536    TCg2-90536        online       hdisk8 Disk_8       auto      TCd4-90536    TCg2-90536        online       hdisk7 Disk_9       auto      TCd3-90536    TCg1-90536        online       hdisk6 Disk_10      auto       -              -            online       hdisk5 Disk_11      auto       -              -            online       hdisk4 Disk_12      auto       -              -            LVM          hdisk0 Disk_13      auto       -              -            online       hdisk1 Disk_14      auto       -              -           LVM          hdisk3 Disk_15      auto       -              -            online       hdisk2  # for i in  6 7 8 9 10 11;do > vxdg destroy hdisk$i > done    # vxdisk -e list DEVICE       TYPE      DISK         GROUP        STATUS       OS_NATIVE_NAME Disk_0       auto      -             -            online       hdisk15 Disk_1       auto      -             -            online       hdisk14 Disk_2       auto      -             -            online       hdisk13 Disk_3       auto      -             -            online       hdisk12 Disk_4       auto      -             -            online       hdisk10 Disk_5       auto      -             -            online       hdisk9 Disk_6       auto      -             -            online       hdisk11 Disk_7       auto      -             -            online       hdisk8 Disk_8       auto      -             -            online       hdisk7 Disk_9       auto      -             -            online       hdisk6 Disk_10      auto      -             -            online       hdisk5 Disk_11      auto      -             -            online       hdisk4 Disk_12      auto      -             -            LVM          hdisk0 Disk_13      auto      -             -            online       hdisk1 Disk_14      auto      -             -            LVM          hdisk3 Disk_15      auto      -             -            online       hdisk2  

Changing the disk group format type

Listing 22 shows how to change the disk group format type, and how to fix resulting errors.


Listing 22.Changing disk group type
                   #cd /etc/vx/bin # for i in 7 6 5 4 3 2;do > vxdisksetup  -if Disk_$i format=aixdisk > done  # for i in 7 6 5 4 3 2;do > vxdisksetup  -if Disk_$i format=cdsdisk > done  # vxdg -g vxvmdg adddisk vxvmdg2=hdisk5 VxVM vxdg ERROR V-5-1-6478 Device hdisk5 cannot be added to a CDS disk group  [root@ortega01] /ezcfg/dat # vxdisk list DEVICE       TYPE            DISK         GROUP        STATUS hdisk0       auto:LVM        -            -            LVM hdisk1       auto:cdsdisk    -            -            error hdisk2       auto:cdsdisk    -            -            error hdisk3       auto:none       -            -            online invalid hdisk4       auto:cdsdisk    vxvmdg1      vxvmdg       online hdisk5       auto:aixdisk    -            -            online  If you get the above type of error, fix it using:  #/etc/vx/bin/vxdisksetup -if hdisk5 format=cdsdisk  Now run:   # vxdg -g vxvmdg adddisk vxvmdg2=hdisk5  # vxdisk list DEVICE       TYPE            DISK         GROUP        STATUS hdisk0       auto:LVM        -            -             LVM hdisk1       auto:cdsdisk    -            -             error hdisk2       auto:cdsdisk    -            -             error hdisk3       auto:none       -            -             online invalid hdisk4       auto:cdsdisk    vxvmdg1      vxvmdg      online hdisk5       auto:cdsdisk    vxvmdg2      vxvmdg      online