Linux Logical Volume Manager http://www.redhat.com/magazine/009jul05/features/lvm2/
LVM Administrator Guide https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/index.html
cheatsheet http://www.datadisk.co.uk/html_docs/redhat/rh_lvm.htm
Disk stacks:
Hard Drive => Kernel => Partitions => Physical Volumes (LVM2) => Volume Groups (LVM2)
=> Logical Volumes (LVM2) => Filesystems
extending physical volume, logical volume, filesystem:
pvresize, vgextend/vgreduce, lvresize, resize2fs
pv, vg, lv can be modified without umount, filesystem require unmount when modified.
commands:
pvdisplay; vgdisplay; lvdisplay, lvs; df -T (find filesystem type)
lvmdiskscan; pvscan; vgscan; lvscan: check/list all pv created for vg
pvcreate; vgcreate; lvcreate; lvextend
vgchange -a n vg_name / vgchange -a y vg_name: activate/deactivate vg
moving LVM disks between computers
The LVM structures are saved both on the disks and in the /etc/lvmconf directory so the only thing that has to be done to move a disk or a set of disks that contain a Volume Group is to make sure the machine that the VG belonged to will not miss it. That is accomplished with the
On machine #1:
Create New LVM filesystem:
Change LVM size:
Disk stacks:
Hard Drive => Kernel => Partitions => Physical Volumes (LVM2) => Volume Groups (LVM2)
=> Logical Volumes (LVM2) => Filesystems
extending physical volume, logical volume, filesystem:
pvresize, vgextend/vgreduce, lvresize, resize2fs
pv, vg, lv can be modified without umount, filesystem require unmount when modified.
commands:
pvdisplay; vgdisplay; lvdisplay, lvs; df -T (find filesystem type)
lvmdiskscan; pvscan; vgscan; lvscan: check/list all pv created for vg
pvcreate; vgcreate; lvcreate; lvextend
vgchange -a n vg_name / vgchange -a y vg_name: activate/deactivate vg
moving LVM disks between computers
The LVM structures are saved both on the disks and in the /etc/lvmconf directory so the only thing that has to be done to move a disk or a set of disks that contain a Volume Group is to make sure the machine that the VG belonged to will not miss it. That is accomplished with the
vgexport command. vgexport simply removes the structures for the VG from /etc/lvmconf, but does not change anything on the disks. Once the disks are in the new machine (they don't have to have the same ID's) the only thing that has to be done is to update /etc/lvmconf. Thats done with vgimport.On machine #1:
On machine #2:
vgchange -a n vg01 vgexport vg01
Notice that you don't have to use the same name for the Volume Group. If the vgimport command did not save a configuration backup use
vgimport vg01 /dev/sda1 /dev/sdb1 vgchange -a y vg01
vgcfgbackup to do it.Create New LVM filesystem:
- create lvm partition, type 8e: # fdisk /dev/sdb
- create pv: # pvcreate /dev/sdb1
- create vg: # vgcreate -s 16MB vg_test /dev/sdb1
( physical extent (PE) size of 16MB) - create lv: # lvcreate -l +100%FREE -n lv_test vg_test
( or lvcreate -L 200M -n lv_test vg_test
will create softlink /dev/vg_test/lv_test -> /dev/mapper/vg_test-lv_test) - create fs: # mkfs -t ext4 -m 1 -O ^has_journal -v /dev/vg_test/lv_test
Change LVM size:
- shutdown the vm
- add extra partition to hd with fdisk or gdisk, make sure it's Linux LVM (8e) type
- create physical volume: # pvcreate /dev/xvda3
- add pv to volume group: # vgextend VolGroup00 /dev/xvda3
- extend lv: # lvextend -l +100%FREE /dev/mapper/VolGroup00-LogVol00
- resize file system: # resize2fs /dev/mapper/VolGroup00-LogVol00
For root partition:
- shutdown the vm. Go the xenserver host, set to boot to cdrom
- xe vm-param-set uuid=
HVM-boot-policy=BIOS\ order - xe vm-param-set uuid=
HVM-boot-params:order=dc - xe vm-start name-label='<vm-name>'
- (after all done, reset back) xe vm-param-set uuid=
HVM-boot-policy= - reboot to rescue mode from CD for the server to see the new partition, choose "skip" to not scan system and mount /mnt/sysimage
- lvs # if not display, run lvmdiskscan, then pvscan first
- lvm vgchange -a y # make the logical volume known to the kernel
- e2fsck -f /dev/VolGroup00/LogVol00
- resize2fs -f /dev/VolGroup00/LogVol00 500G
- lvm lvreduce -L500G /dev/VolGroup00/LogVol00(last two steps can be combine as:lvreduce --resizefs --size -50G /dev/VolGroup00/LogVol00 )
- reboot to rescue mode with /mnt/sysimage, rebuild boot sector
(check /boot/grub/grub.conf /boot/grub/device.map. bikid to display partition labels, e2label, tune2fs to change partition labels;)
(Apparently, in XenServer 6, the device name change. When boot from CDROM, device is /dev/hda. When boot into linux VM, the device become /dev/xvda. To get around, make sure /boot/grub/device.map use /dev/xvda. Then boot to the harddrive, select "c" at the boot stage to get into grub> mode.
grub> root (hd0,0)
grub> setup (hd0)
grub> reboot
)
removing a logical volume
# lvremove /dev/myvg/my_volume
|
removing a volume group
Deactivate the volume group:
# vgchange -a n my_volume_group
|
# vgremove my_volume_group
|
removing a physical volume
# pvdisplay /dev/hda1
# vgreduce my_volume_group /dev/hda1
# pvscan
# pvremove /dev/hda1
|
http://www.turnkeylinux.org/blog/extending-lvmLogical Volume Management (AKA LVM) is a powerful, robust mechanism for managing storage space.
In TurnKey 11, instead of installing the root filesystem directly to a fixed size partition, we setup LVM by default, and install the root filesystem to a Logical Volume, which may later be expanded, even across multiple physical devices.
Unfortunately, as with anything powerful, to get the most out of LVM you first have to negotiate a learning curve. From the feedback we've been getting it seems that confusion regarding LVM is common with new users, so here's a quick "crash course"...
How LVM works
In LVM, there are several layers, each builds on top of the other:
PV[s] (Physical Volumes) -> VG[s] (Volume Groups) -> LV[s] (Logical Volumes) -> Filesystems.
Logical Volumes are allocated/extended within the boundaries of their underlying storage pool which is called a Volume Group in LVM terminology.
For example, in TurnKey the filesystem is installed by default to the /dev/turnkey/root Logical Volume, which is allocated within the turnkey Volume Group:
--- Logical volume --- LV Name /dev/turnkey/root VG Name turnkey LV UUID OzX3fe-aRQa-81XM-0vCV-8aJo-eUL4-6J90XJ LV Write Access read/write LV Status available # open 1 LV Size 17.0 GiB Current LE 4502 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 251:0
Out of the box the turnkey Volume Group doesn't have too much free space:
# vgdisplay --- Volume group --- VG Name turnkey System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 2 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 18.14 GiB PE Size 4.00 MiB Total PE 4645 Alloc PE / Size 4480 / 17.50 GiB Free PE / Size 165 / 660.00 MiB VG UUID IwaFL0-QCi8-iIUE-TWjQ-R906-PYpH-gMPaH9
We can only extend a Logical Volume within the free space of the underlying Volume Group. How much free space we currently have within the Volume Group can be seen in this part of the output:
Free PE / Size 165 / 660.00 MiB
In the above example we only have 660 MB to allocate to LVMs within the turnkey Volume Group. So if we want to extend the root LV we'll have to first extend the VG backs it up.
Volume Groups group together Physical Volumes. That's why they're called Volume Groups. This command will show us which Physical Volumes have been registered into LVM, and to which volume groups they have been assigned:
# pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name turnkey PV Size 18.15 GiB / not usable 4.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 4645 Free PE 165 Allocated PE 4480 PV UUID H1Prpv-0VXR-7moE-zsbt-eyVt-m0fQ-GkAT6w
In this example we only have one Physical Volume (the /dev/sda2 partition) in the turnkey Volume Group.
Extending a Logical Volume
Bottom line: if the underlying Volume Group doesn't have enough free space, to extend the Logical Volumeyou'll first have to extend the underlying Volume Group by adding another Physical Volume to it.
In VMWare you could either create a new virtual hard disk device to add to the volume group, or extend an existing virtual hard disk device, create a new partition with cfdisk, and add the new partition to theVolume Group:
# example #1: you've added to VMWare a new virtual hard disk called /dev/sdb pvcreate /dev/sdb vgextend turnkey /dev/sdb # example #2: you've expanded the existing sda hard disk cfdisk /dev/sda # creating /dev/sda3 (you may need to reboot before you can see this) pvcreate /dev/sda3 vgextend turnkey /dev/sda3
After you've extended the Volume Group, you are free to extend the underlying Logical Volume:
# lvextend -L+10G /dev/turnkey/root Extending logical volume root to 27.0 GiB Logical volume root successfully resized
Finally, you'll have to resize the filesystem within /dev/turnkey/root so it can see that the underlying block device just got 10G bigger:
# resize2fs /dev/turnkey/root resize2fs 1.41.11 (14-Mar-2010) Filesystem at /dev/turnkey/root is mounted on /; on-line resizing required old desc_blocks = 2, new_desc_blocks = 2 Performing an on-line resize of /dev/turnkey/root to 7077888 (4k) blocks. The filesystem on /dev/turnkey/root is now 7077888 blocks long.
----
Shrink LVM without dataloss
In this example we shrink a partition from 10G to 9G (xfs does not support shrink) :
First, we unmount.
# umount /dev/vg_blah/lv_blah
Check the file system.
# e2fsck -f /dev/vg_blah/lv_blah
Resize the file system.
# resize2fs -p /dev/vg_blah/lv_blah 9G
Reduce the size of the logical volume.
# lvreduce -L -1G /dev/vg_blah/lv_blah
Shrink the volume group if desired.
# vgreduce vg_blah /dev/sdxyLet's assume you've an LVM volume group named
vol01 that has a physical volume named /dev/sda2. You still have free space on the/dev/sda disk and you'd like to increase the size of the sda2 partition and extend the associated physical volume ... thus increasing the free space in your volume group.Here're the commands to achieve that pv expansion without a reboot:
# First unmount everything that uses the given volume group
# List all the logical volumes that are in vol01
$ lvdisplay /dev/vol01
--- Logical volume ---
LV Name /dev/vol01/lv01
VG Name vol01
LV UUID q3eQP3-4V9E-7yvo-LUZR-zbXz-F2Mh-Pn5fDt
LV Write Access read/write
LV Status available
# open 1
LV Size 139.70 GB
Current LE 35762
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:0
--- Logical volume ---
LV Name /dev/vol01/lv02
VG Name vol01
LV UUID b53h7W-VO2U-3Ok5-WvW3-GbDp-Tbvb-6bbdkw
LV Write Access read/write
LV Status available
# open 1
LV Size 29.31 GB
Current LE 7504
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:1
# Now unmount all these volumes (if they are mounted)
$ umount /dev/vol01/lv01
$ umount /dev/vol01/lv02
# In case you get a "device is busy" error, check out what processes use them (lsof /dev/vol01/lv01) and stop/kill them.
( you might have to turn swap off (using swapoff) if you happen to have the swap as a LV: cat /proc/swaps
shows the device(file)name(s) used for swap and
swapoff /dev/path/to/device
umounts the specific one if possible)
# Now deactive the volume group.
$ vgchange -a n vol01
# Increase the size of the sda2 partition to the desired value.
# You do this by deleting the partition and recreating it with a higher end cylinder.
# I assume that pvresize can handle partition expansions only if the partition start remains the same and the partition end cylinder is moved to a higher value.
$ fdisk /dev/sda
The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p
Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000e4bad
Device Boot Start End Blocks Id System
/dev/sda1 1 486 3903763+ 83 Linux
/dev/sda2 487 18722 146480670 8e Linux LVM
Command (m for help): d
Partition number (1-4): 2
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (487-60801, default 487):
Using default value 487
Last cylinder or +size or +sizeM or +sizeK (487-60801, default 60801):
Using default value 60801
Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)
Command (m for help): p
Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000e4bad
Device Boot Start End Blocks Id System
/dev/sda1 1 486 3903763+ 83 Linux
/dev/sda2 487 60801 484480237+ 8e Linux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
# Now comes the interesting part: we force the system with partprobe to re-read the partition table.
$ pvdisplay /dev/sda2
--- Physical volume ---
PV Name /dev/sda2
VG Name vol01
PV Size 139.69 GB / not usable 3.53 MB
Allocatable yes (but full)
PE Size (KByte) 4096
Total PE 35761
Free PE 0
Allocated PE 35761
PV UUID S0cDcl-7mr8-2AAb-172u-HClq-J2aQ-DfC2V5
$ partprobe (or kpartx -s /dev/sda
or # echo 1 > /sys/block/sda/device/rescanif not working, reboot, or inactivating the volume (bug) )
$ pvresize /dev/sda2
$ pvdisplay /dev/sda2
--- Physical volume ---
PV Name /dev/sda2
VG Name vol01
PV Size 462.04 GB / not usable 1.04 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 118281
Free PE 82520
Allocated PE 35761
PV UUID S0cDcl-7mr8-2AAb-172u-HClq-J2aQ-DfC2V5
# Voala! :-) The physical volume got increased.
# Time to reactivate the volume group.
$ vgchange -a y vol01
# Now check the volume group details.
$ vgdisplay vol01
--- Volume group ---
VG Name vol01
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 18
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 2
Act PV 2
VG Size 927.79 GB
PE Size 4.00 MB
Total PE 237515
Alloc PE / Size 68866 / 269.01 GB
Free PE / Size 168649 / 658.79 GB
VG UUID 0eY2tg-AOny-XsDL-wfLR-d290-2jxO-TDTNnQ
# You can now use lvextend to increase logical volumes in the volume group.
$ lvextend -L +10G --name lv01 /dev/vol01
# And resize2fs to extend the Ext2/3 filesystem on the logical volume.
$ resize2fs /dev/vol01/lv01
( For reiserfs, after extending the LV I had to extend reiserfs file system which requires this command:
resize_reiserfs -f /dev/vol01/lv01)
# And finally mount the filesystems.
$ mount /dev/vol01/lv01
$ mount /dev/vol01/lv02
# You're now ready.
http://forums.citrix.com/message.jspa?messageID=1377324
This set of steps enables you to resize an SR that is mounted on a shared SAN. This applies to XenServer 5 in a pool configurat
RESIZING LUNs for XENSERVER SRs
Perform steps 2-7 on the Pool Master:
1. Extend the volume/LUN
2. xe sr-list name-label
Note the uuid of the SR.
3. pvscan | grep “<the uuid you noted in the previous step>“
Note the device name (eg: PV /dev/sdj )
4. echo 1 > /sys/block
5. pvresize <device
6. xe sr-scan <the uuid you noted in the previous step>
7. Verify that the XE host sees the larger physical disk: pvscan | grep <the uuid you noted in step 2>
The thing of note is that in Linux is there are two components
My test host is connected to an HP EVA 8100. Once you know the device you want to work with (in my case it'll be /dev/dm-45
First we need to find the multipath ID
host# multipath -ll
... cut cut ...
3600508b40
size=25Gfeatures=1 queue_if_n
\_ round-robi
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robi
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready
We need to make note of the rather large number above (starting with 3600 in my case) for later. Resize the LUN on the EVA, and once it's complete then we need to get Linux to rescan the LUN's from the slave device level.
host# for i in `ls /sys/block
blockdev --rereadpt
done
Now if you do a dmesg you'll see that the devices see the disk size change.
host# dmesg
... cut cut ...
sd 5:0:2:6: sdac 62914560 512-byte hardware sectors: (32.2 GB/30.0
GiB)
sd 5:0:2:6: sdac Write Protect is off
sd 5:0:2:6: sdac Mode Sense: 97 00 10 08
sd 5:0:2:6: sdac Write cache: disabled, read cache: enabled, supports
DPO and FUA
sdac: detected capacity change from 2684354560
sdac: unknown partition table
... cut cut ...
However if you check with multipath,
host# multipath -ll 3600508b40
3600508b40
size=25Gfeatures=1 queue_if_n
\_ round-robi
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robi
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready
So we need to tell multipath that there has been a resize. (note we run multipathd
host# multipathd
multipathd
ok
multipathd
... cut cut ...
3600508b40
size=30G features=1 queue_if_n
\_ round-robi
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robi
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready
Now you can continue on with the pvresize.
http://blogs.citrix.com/2011/03/07/live-lun-resize-on-xenserver/
