Friday, December 21, 2012

Guideline for XenServer Switch Ports


Guidelines

  1. If using Spanning Tree Protocol (STP), use either Rapid Spanning Tree Protocol (RSTP) or enable PortFast on XenServer connected ports, depending on switch hardware support. PortFast allows a switch port running Spanning Tree Protocol (STP) to go directly from blocking to forwarding mode by skipping the learning and listening modes. PortFast should only be enabled on ports connected to a single host.
    WARNING: Use caution when enabling PortFast, and do so only on ports that do not connect to multi-homed devices such as hubs or switches.

  2. Disable Port Security or equivalent on XenServer connected ports.
    Port security prevents multiple MAC addresses from being presented to the same port. In a virtual environment, you see multiple MAC addresses presented from Virtual Machines to the same port. If you have enabled Port Security, it shuts down the port.

Troubleshooting

  1. Disable BPDU guard on XenServer connected ports.
    BPDU is a protection setting part of the STP that prevents you from attaching a network device to a switch port. When you attach a network device, the port shuts down and has to be enabled by an administrator. A PortFast port should never receive configuration BPDUs.

    Note: When BPDUs are received by a PortFast port, it indicates another bridge is connected to the port, and that there is a possibility of a bridging loop formation during the Listening and Learning phases. In a valid PortFast configuration, configuration BPDUs should never be received. Some Cisco switches support a feature called PortFast BPDU Guard, which is a feature that shuts down a PortFast-enabled port in the event a BPDU is received. This feature ensures that a bridging loop is not formed, because the port is shut down by the switch and removes the possibility of a loop forming.

  2. If you are using a 10/100 switch, change the port speed settings to Static
    If you are connecting to a 100 MBP/s port, set the PIF speeds to 100 MBPs Static with full duplex.

    Note: It is not necessary to change the speed or duplex settings when connecting to 1GB switches.

Tuesday, November 6, 2012

Free HA XenServer

http://www.infoworld.com/print/143867

Free high availability: Create a XenServer virtualization cluster

Wednesday, October 24, 2012

LVM admin

Linux Logical Volume Manager http://www.redhat.com/magazine/009jul05/features/lvm2/

LVM Administrator Guide https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/index.html

cheatsheet http://www.datadisk.co.uk/html_docs/redhat/rh_lvm.htm
Disk stacks:
     Hard Drive => Kernel => Partitions => Physical Volumes (LVM2) => Volume Groups (LVM2)
          => Logical Volumes (LVM2) => Filesystems



extending physical volume, logical volume, filesystem:
pvresize, vgextend/vgreducelvresize, resize2fs
pv, vg, lv can be modified without umount, filesystem require unmount when modified.

commands:
pvdisplay; vgdisplay; lvdisplay, lvs; df -T (find filesystem type)
lvmdiskscan; pvscan; vgscan; lvscan: check/list all pv created for vg
pvcreate; vgcreate; lvcreate; lvextend
vgchange -a n vg_name / vgchange -a y vg_name: activate/deactivate vg

moving LVM disks between computers
The LVM structures are saved both on the disks and in the /etc/lvmconf directory so the only thing that has to be done to move a disk or a set of disks that contain a Volume Group is to make sure the machine that the VG belonged to will not miss it. That is accomplished with the vgexport command. vgexport simply removes the structures for the VG from /etc/lvmconf, but does not change anything on the disks. Once the disks are in the new machine (they don't have to have the same ID's) the only thing that has to be done is to update /etc/lvmconf. Thats done with vgimport.
On machine #1:

vgchange -a n vg01
vgexport vg01
On machine #2:

vgimport vg01 /dev/sda1 /dev/sdb1
vgchange -a y vg01
Notice that you don't have to use the same name for the Volume Group. If the vgimport command did not save a configuration backup use vgcfgbackup to do it.

Create New LVM filesystem:

  1. create lvm partition, type 8e: # fdisk /dev/sdb
  2. create pv: # pvcreate /dev/sdb1
  3. create vg: # vgcreate -s 16MB vg_test /dev/sdb1
    ( physical extent (PE) size of 16MB)
  4. create lv: # lvcreate -l +100%FREE -n lv_test vg_test
    ( or lvcreate -L 200M -n lv_test vg_test
    will create softlink /dev/vg_test/lv_test -> /dev/mapper/vg_test-lv_test)
  5. create fs: # mkfs -t ext4 -m 1 -O ^has_journal -v /dev/vg_test/lv_test

Change LVM size:
  1. shutdown the vm
  2. add extra partition to hd with fdisk or gdisk, make sure it's Linux LVM (8e) type
  3. create physical volume: # pvcreate /dev/xvda3
  4. add pv to volume group: # vgextend VolGroup00 /dev/xvda3
  5. extend lv: # lvextend -l +100%FREE /dev/mapper/VolGroup00-LogVol00
  6. resize file system: # resize2fs /dev/mapper/VolGroup00-LogVol00
For root partition:
  1. shutdown the vm. Go the xenserver host, set to boot to cdrom
    • xe vm-param-set uuid= HVM-boot-policy=BIOS\ order
    • xe vm-param-set uuid= HVM-boot-params:order=dc
    • xe vm-start name-label='<vm-name>'
    • (after all done, reset back) xe vm-param-set uuid= HVM-boot-policy=
  2. reboot to rescue mode from CD for the server to see the new partition, choose "skip" to not scan system and mount /mnt/sysimage
  3. lvs # if not display, run lvmdiskscan, then pvscan first
  4. lvm vgchange -a y  # make the logical volume known to the kernel
  5. e2fsck -f /dev/VolGroup00/LogVol00
  6. resize2fs -f /dev/VolGroup00/LogVol00 500G
  7. lvm lvreduce -L500G /dev/VolGroup00/LogVol00(last two steps can be combine as:lvreduce --resizefs --size -50G /dev/VolGroup00/LogVol00 )
  8. reboot to rescue mode with /mnt/sysimage, rebuild boot sector
    (check /boot/grub/grub.conf /boot/grub/device.map. bikid to display partition labels, e2label, tune2fs to change partition labels;)
    (Apparently, in XenServer 6, the device name change. When boot from CDROM, device is /dev/hda. When boot into linux VM, the device become /dev/xvda. To get around, make sure /boot/grub/device.map use /dev/xvda. Then boot to the harddrive, select "c" at the boot stage to get into grub> mode.
    grub> root (hd0,0)
    grub> setup (hd0)
    grub> reboot
    )

removing a logical volume


# lvremove /dev/myvg/my_volume

removing a volume group
Deactivate the volume group:


# vgchange -a n my_volume_group
Remove the volume group:


# vgremove my_volume_group

removing a physical volume




# pvdisplay /dev/hda1
# vgreduce my_volume_group /dev/hda1


# pvscan
# pvremove /dev/hda1

http://www.turnkeylinux.org/blog/extending-lvm
Logical Volume Management (AKA LVM) is a powerful, robust mechanism for managing storage space.
In TurnKey 11,  instead of installing the root filesystem directly to a fixed size partition, we setup LVM by default, and install the root filesystem to a Logical Volume, which may later be expanded, even across multiple physical devices.
Unfortunately, as with anything powerful, to get the most out of LVM you first have to negotiate a learning curve. From the feedback we've been getting it seems that confusion regarding LVM is  common with new users, so here's a quick "crash course"...

How LVM works

In LVM, there are several layers, each builds on top of the other:
PV[s] (Physical Volumes) -> VG[s] (Volume Groups) -> LV[s] (Logical Volumes) -> Filesystems.
Logical Volumes are allocated/extended within the boundaries of their underlying storage pool which is called a Volume Group in LVM terminology.
For example, in TurnKey the filesystem is installed by default to the /dev/turnkey/root Logical Volume, which is allocated within the turnkey Volume Group:
--- Logical volume ---
  LV Name                /dev/turnkey/root
  VG Name                turnkey
  LV UUID                OzX3fe-aRQa-81XM-0vCV-8aJo-eUL4-6J90XJ
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                17.0 GiB
  Current LE             4502
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:0
Out of the box the turnkey Volume Group doesn't have too much free space:
# vgdisplay
  --- Volume group ---
  VG Name               turnkey
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               18.14 GiB
  PE Size               4.00 MiB
  Total PE              4645
  Alloc PE / Size       4480 / 17.50 GiB
  Free  PE / Size       165 / 660.00 MiB
  VG UUID               IwaFL0-QCi8-iIUE-TWjQ-R906-PYpH-gMPaH9
We can only extend a Logical Volume within the free space of the underlying Volume Group. How much free space we currently have within the Volume Group can be seen in this part of the output:
Free  PE / Size       165 / 660.00 MiB
In the above example we only have 660 MB to allocate to LVMs within the turnkey Volume Group. So if we want to extend the root LV we'll have to first extend the VG backs it up.
Volume Groups group together Physical Volumes. That's why they're called Volume Groups. This command will show us which Physical Volumes have been registered into LVM, and to which volume groups they have been assigned:
# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               turnkey
  PV Size               18.15 GiB / not usable 4.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              4645
  Free PE               165
  Allocated PE          4480
  PV UUID               H1Prpv-0VXR-7moE-zsbt-eyVt-m0fQ-GkAT6w
In this example we only have one Physical Volume (the /dev/sda2 partition) in the turnkey Volume Group.

Extending a Logical Volume

Bottom line: if the underlying Volume Group doesn't have enough free space, to extend the Logical Volumeyou'll first have to extend the underlying Volume Group by adding another Physical Volume to it.
In VMWare you could either create a new virtual hard disk device to add to the volume group, or extend an existing virtual hard disk device, create a new partition with cfdisk, and add the new partition to theVolume Group:
# example #1: you've added to VMWare a new virtual hard disk called /dev/sdb
pvcreate /dev/sdb
vgextend turnkey /dev/sdb

# example #2: you've expanded the existing sda hard disk
cfdisk /dev/sda  # creating /dev/sda3 (you may need to reboot before you can see this)
pvcreate /dev/sda3
vgextend turnkey /dev/sda3
After you've extended the Volume Group, you are free to extend the underlying Logical Volume:
# lvextend -L+10G /dev/turnkey/root
Extending logical volume root to 27.0 GiB
Logical volume root successfully resized
Finally, you'll have to resize the filesystem within /dev/turnkey/root so it can see that the underlying block device just got 10G bigger:
# resize2fs /dev/turnkey/root
resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/turnkey/root is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 2
Performing an on-line resize of /dev/turnkey/root to  7077888 (4k) blocks.
The filesystem on /dev/turnkey/root is now 7077888 blocks long.

----

Shrink LVM without dataloss

In this example we shrink a partition from 10G to 9G (xfs does not support shrink) :
First, we unmount.
# umount /dev/vg_blah/lv_blah
Check the file system.
# e2fsck -f /dev/vg_blah/lv_blah
Resize the file system.
# resize2fs -p /dev/vg_blah/lv_blah 9G
Reduce the size of the logical volume.
# lvreduce -L -1G /dev/vg_blah/lv_blah
Shrink the volume group if desired.
# vgreduce vg_blah /dev/sdxy
-----
Let's assume you've an LVM volume group named vol01 that has a physical volume named /dev/sda2. You still have free space on the/dev/sda disk and you'd like to increase the size of the sda2 partition and extend the associated physical volume ... thus increasing the free space in your volume group.

Here're the commands to achieve that pv expansion without a reboot:
# First unmount everything that uses the given volume group
# List all the logical volumes that are in vol01
$
lvdisplay /dev/vol01
  --- Logical volume ---
  LV Name                /dev/vol01/lv01
  VG Name                vol01
  LV UUID                q3eQP3-4V9E-7yvo-LUZR-zbXz-F2Mh-Pn5fDt
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                139.70 GB
  Current LE             35762
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:0
 
  --- Logical volume ---
  LV Name                /dev/vol01/lv02
  VG Name                vol01
  LV UUID                b53h7W-VO2U-3Ok5-WvW3-GbDp-Tbvb-6bbdkw
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                29.31 GB
  Current LE             7504
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:1

# Now unmount all these volumes (if they are mounted)
$
umount /dev/vol01/lv01
$ umount /dev/vol01/lv02
# In case you get a "device is busy" error, check out what processes use them (lsof /dev/vol01/lv01) and stop/kill them.
you might have to turn swap off (using swapoff) if you happen to have the swap as a LV:          cat /proc/swaps
          shows the device(file)name(s) used for swap and
          swapoff /dev/path/to/device
          umounts the specific one if possible)
# Now deactive the volume group.
$
vgchange -a n vol01
# Increase the size of the sda2 partition to the desired value.
# You do this by deleting the partition and recreating it with a higher end cylinder.
# I assume that pvresize can handle partition expansions only if the partition start remains the same and the partition end cylinder is moved to a higher value.
$
fdisk /dev/sda

The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000e4bad

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         486     3903763+  83  Linux
/dev/sda2             487       18722   146480670   8e  Linux LVM

Command (m for help): d
Partition number (1-4): 2

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (487-60801, default 487):
Using default value 487
Last cylinder or +size or +sizeM or +sizeK (487-60801, default 60801):
Using default value 60801

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000e4bad

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         486     3903763+  83  Linux
/dev/sda2             487       60801   484480237+  8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

# Now comes the interesting part: we force the system with partprobe to re-read the partition table.
$ pvdisplay /dev/sda2
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vol01
  PV Size               139.69 GB / not usable 3.53 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              35761
  Free PE               0
  Allocated PE          35761
  PV UUID               S0cDcl-7mr8-2AAb-172u-HClq-J2aQ-DfC2V5
$
partprobe 

(or kpartx -s /dev/sda
 or # echo 1 > /sys/block/sda/device/rescan

if not working, reboot, or inactivating the volume (bug) )
$
pvresize /dev/sda2
$ pvdisplay /dev/sda2
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vol01
  PV Size               462.04 GB / not usable 1.04 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              118281
  Free PE               82520
  Allocated PE          35761
  PV UUID               S0cDcl-7mr8-2AAb-172u-HClq-J2aQ-DfC2V5
# Voala! :-) The physical volume got increased.
# Time to reactivate the volume group.
$ vgchange -a y vol01
# Now check the volume group details.
$ vgdisplay vol01
  --- Volume group ---
  VG Name               vol01
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  18
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               3
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               927.79 GB
  PE Size               4.00 MB
  Total PE              237515
  Alloc PE / Size       68866 / 269.01 GB
  Free  PE / Size       168649 / 658.79 GB
  VG UUID               0eY2tg-AOny-XsDL-wfLR-d290-2jxO-TDTNnQ
# You can now use lvextend to increase logical volumes in the volume group.
$ lvextend -L +10G --name lv01 /dev/vol01
# And resize2fs to extend the Ext2/3 filesystem on the logical volume.
$ resize2fs /dev/vol01/lv01
( For reiserfs, after extending the LV I had to extend reiserfs file system which requires this command:
resize_reiserfs -f /dev/vol01/lv01)
# And finally mount the filesystems.
$ mount /dev/vol01/lv01
$ mount /dev/vol01/lv02
# You're now ready.

----
http://forums.citrix.com/message.jspa?messageID=1377324
This set of steps enables you to resize an SR that is mounted on a shared SAN. This applies to XenServer 5 in a pool configuration but will also work in stand-alone setups (no pool).

RESIZING LUNs for XENSERVER SRs

Perform steps 2-7 on the Pool Master:

1. Extend the volume/LUN from the SAN management console
2. xe sr-list name-label=<your SR name you want to resize>
Note the uuid of the SR.
3. pvscan | grep “<the uuid you noted in the previous step>“
Note the device name (eg: PV /dev/sdj )
4. echo 1 > /sys/block/device/device/rescan (e.g. echo 1 > /sys/block/sdj/device/rescan)
5. pvresize <device name> (eg: pvresize /dev/sdj )
6. xe sr-scan <the uuid you noted in the previous step>
7. Verify that the XE host sees the larger physical disk: pvscan | grep <the uuid you noted in step 2>

The thing of note is that in Linux is there are two components to a multipathed device. The raw devices (paths) that you see as /dev/sd{a,b,c,d} etc and the multipath device. Because of this there are two locations you need to advise of a disk size change.

My test host is connected to an HP EVA 8100. Once you know the device you want to work with (in my case it'll be /dev/dm-45) then you can proceed.

First we need to find the multipath ID
host# multipath -ll
... cut cut ...
3600508b40010517900008000cd6b0000 dm-45 HP,HSV210
size=25Gfeatures=1 queue_if_no_pathhwhandler=0rw
\_ round-robin 0 prio=100active
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robin 0 prio=20enabled
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready

We need to make note of the rather large number above (starting with 3600 in my case) for later. Resize the LUN on the EVA, and once it's complete then we need to get Linux to rescan the LUN's from the slave device level.

host# for i in `ls /sys/block/dm-45/slaves/`; do
blockdev --rereadpt /dev/$i
done

Now if you do a dmesg you'll see that the devices see the disk size change.

host# dmesg
... cut cut ...
sd 5:0:2:6: sdac 62914560 512-byte hardware sectors: (32.2 GB/30.0
GiB)
sd 5:0:2:6: sdac Write Protect is off
sd 5:0:2:6: sdac Mode Sense: 97 00 10 08
sd 5:0:2:6: sdac Write cache: disabled, read cache: enabled, supports
DPO and FUA
sdac: detected capacity change from 26843545600 to 32212254720
sdac: unknown partition table
... cut cut ...

However if you check with multipath, it still sees the old size

host# multipath -ll 3600508b40010517900008000cd6b0000
3600508b40010517900008000cd6b0000 dm-45 HP,HSV210
size=25Gfeatures=1 queue_if_no_pathhwhandler=0rw
\_ round-robin 0 prio=100active
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robin 0 prio=20enabled
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready

So we need to tell multipath that there has been a resize. (note we run multipathd, not multipath)

host# multipathd -k
multipathd> resize multipath 3600508b40010517900008000cd6b0000
ok
multipathd> show topology
... cut cut ...
3600508b40010517900008000cd6b0000 dm-45 HP,HSV210
size=30G features=1 queue_if_no_pathhwhandler=0 rw
\_ round-robin 0 prio=100active
\_ 5:0:2:6 sdac 65:192 activeready
\_ 5:0:3:6 sdad 65:208 activeready
\_ round-robin 0 prio=20enabled
\_ 5:0:0:6 sdaa 65:160 activeready
\_ 5:0:1:6 sdab 65:176 activeready

Now you can continue on with the pvresize.
http://blogs.citrix.com/2011/03/07/live-lun-resize-on-xenserver/

Tuesday, September 11, 2012

Xen command

harddrives in XenServer 6.0:
  rhel 4: /dev/xvda, cdrom: /dev/xvdd
  rhel 5: /dev/hda, cdrom: /dev/hdd
  rhel 6: /dev/xvda, cdrom: /dev/xvdd

/etc/xensource/pool.conf
Emergency Network reset, will remove config for all host PIFs, Bonds, VLANs and tunnels, all VMs will be shutdown forcefully: xe-reset-networking
# xe pool-join master-address=linhb425 master-username=root master-password=xxxxx
. build status report when xapi is down
xen-bugtool --yestoall
# xe-toolstack-restart
. backup pool metadata for all VMs
# xe-backup-metadata -d -u [sr uuid]
. top Xen process
# xentop
. processor stat in Dom0
# mpstat 5
. virtual memory in Dom0
# vmstat 2
. storage traffic stats
# iostat -d 2 6
. lists VMs that are running
# list_domains

XAPI configuation in sqlite DB
 /var/xapi/state.db
 information for:
 - Host
 - VM
 - XAPI
# /opt/xensource/bin/xapi
# xenstore-ls
/opt/xensource/sm/XE_SR_ERRORCODES.xml
 /opt/xensource/sm/sm.py
 /opt/xensource/sm/<Type>SR.py
. xapi messages
# tail -f /var/log/xensource.log
.XenSource Inverntory info
# cat /etc/xensource-inventory
Platform Configuration
. HBA # /usr/local/bin/scli
. Software RAID # mdadm
. open-iscsi; discover iSCSI targets available to this server
  # iscsiadm -m discovery -t st -p <ip>
  # iscsiadm -m discovery -type sendtargets -port 192.168.250.14
. open iscsi sessions
  # iscsiadm -m session
. Network
 /etc/sysconfig/network-scripts/ifcfg-*
 /etc/sysconfig/iptables
  # tcpdump -i eth0 -vvv
  # netstat -pl
  # ifconfig
  # route
  # dhclient eth0
. kernel and modules
  # lsmod
  # insmod/modprobe
  # rmmod
. LVM Volume Group name: VG_XenStorage_<uuid>
  # fdisk -l
  # vgs
  # lvs
  # pvs
  # lvm
. incorporating a host disk in dom0
  # mount /dev/VG_XenStorage_<uuid>/<VM_uuid>.<disk> /path/to/mount/point
# pvcreate /dev/cciss/c0d1p1
. find vhd usage
# vhd-util scan -f -m “VHD-*” -l VG_XenStorage-<UUID_of_StorageRepository> -p
# lvscan

Object Commands:
 xe <class>-<TAB>
Object Parameter Command
 xe <class>-param-<TAB>
. Filter
 xe vm-list power-state=halted
. Parameter
 xe vm-list params=name-label
. Minimum
 xe vm-list --minimal

xe commands:
. list snapshot
# xe vdi-list is-a-snapshot=true
.dump crash kernel
# xe host-crashdump-list
.Xen Hypervisor Boot message
# xe host-dmesg
# xe event-wait class=vm name-label=<name> power-state=running
# xe host-cpu-list
# xe log-get-keys
# xe host-bugreport-upload
# xe host-backup
# xe log-set-output level=debug key=storage output=file:/tmp/storage-debug
# xe log-set-output level=debug key=storage output=nil
# xe sm-list
# xe sr-create type=lvm content-type=user device-config:device=/dev/cciss/c0d1p1 name-label="Local RAID5"

Friday, July 27, 2012

rsync Command Examples

http://www.thegeekstuff.com/2010/09/rsync-command-examples/


rsync -avzr --delete --stats --progress -H --numeric-ids --include 'P*' --exclude '*' /path/to/source/ /path/to/dest


rsync is used to perform the backup operation in UNIX / Linux.
rsync utility is used to synchronize the files and directories from one location to another in an effective way. Backup location could be on local server or on remote server.

Important features of rsync

  • Speed: First time, rsync replicates the whole content between the source and destination directories. Next time, rsync transfers only the changed blocks or bytes to the destination location, which makes the transfer really fast.
  • Security: rsync allows encryption of data using ssh protocol during transfer.
  • Less Bandwidth: rsync uses compression and decompression of data block by block at the sending and receiving end respectively. So the bandwidth used by rsync will be always less compared to other file transfer protocols.
  • Privileges: No special privileges are required to install and execute rsync

Syntax

$ rsync options source destination
Source and destination could be either local or remote. In case of remote, specify the login name, remote server name and location.

Example 1. Synchronize Two Directories in a Local Server

To sync two directories in a local computer, use the following rsync -zvr command.
$ rsync -zvr /var/opt/installation/inventory/ /root/temp
building file list ... done
sva.xml
svB.xml
.
sent 26385 bytes  received 1098 bytes  54966.00 bytes/sec
total size is 44867  speedup is 1.63
$
In the above rsync example:
  • -z is to enable compression
  • -v verbose
  • -r indicates recursive
Now let us see the timestamp on one of the files that was copied from source to destination. As you see below, rsync didn’t preserve timestamps during sync.
$ ls -l /var/opt/installation/inventory/sva.xml /root/temp/sva.xml
-r--r--r-- 1 bin  bin  949 Jun 18  2009 /var/opt/installation/inventory/sva.xml
-r--r--r-- 1 root bin  949 Sep  2  2009 /root/temp/sva.xml

Example 2. Preserve timestamps during Sync using rsync -a

rsync option -a indicates archive mode. -a option does the following,
  • Recursive mode
  • Preserves symbolic links
  • Preserves permissions
  • Preserves timestamp
  • Preserves owner and group
Now, executing the same command provided in example 1 (But with the rsync option -a) as shown below:
$ rsync -azv /var/opt/installation/inventory/ /root/temp/
building file list ... done
./
sva.xml
svB.xml
.
sent 26499 bytes  received 1104 bytes  55206.00 bytes/sec
total size is 44867  speedup is 1.63
$
As you see below, rsync preserved timestamps during sync.
$ ls -l /var/opt/installation/inventory/sva.xml /root/temp/sva.xml
-r--r--r-- 1 root  bin  949 Jun 18  2009 /var/opt/installation/inventory/sva.xml
-r--r--r-- 1 root  bin  949 Jun 18  2009 /root/temp/sva.xml

Example 3. Synchronize Only One File

To copy only one file, specify the file name to rsync command, as shown below.
$ rsync -v /var/lib/rpm/Pubkeys /root/temp/
Pubkeys

sent 42 bytes  received 12380 bytes  3549.14 bytes/sec
total size is 12288  speedup is 0.99

Example 4. Synchronize Files From Local to Remote

rsync allows you to synchronize files/directories between the local and remote system.
$ rsync -avz /root/temp/ thegeekstuff@192.168.200.10:/home/thegeekstuff/temp/
Password:
building file list ... done
./
rpm/
rpm/Basenames
rpm/Conflictname

sent 15810261 bytes  received 412 bytes  2432411.23 bytes/sec
total size is 45305958  speedup is 2.87
While doing synchronization with the remote server, you need to specify username and ip-address of the remote server. You should also specify the destination directory on the remote server. The format is username@machinename:path
As you see above, it asks for password while doing rsync from local to remote server.
Sometimes you don’t want to enter the password while backing up files from local to remote server. For example, If you have a backup shell script, that copies files from local to remote server using rsync, you need the ability to rsync without having to enter the password.
To do that, setup ssh password less login as we explained earlier.

Example 5. Synchronize Files From Remote to Local

When you want to synchronize files from remote to local, specify remote path in source and local path in target as shown below.
$ rsync -avz thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/
rpm/Basenames
.
sent 406 bytes  received 15810230 bytes  2432405.54 bytes/sec
total size is 45305958  speedup is 2.87

Example 6. Remote shell for Synchronization

rsync allows you to specify the remote shell which you want to use. You can use rsync ssh to enable the secured remote connection.
Use rsync -e ssh to specify which remote shell to use. In this case, rsync will use ssh.
$ rsync -avz -e ssh thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/
rpm/Basenames

sent 406 bytes  received 15810230 bytes  2432405.54 bytes/sec
total size is 45305958  speedup is 2.87

Example 7. Do Not Overwrite the Modified Files at the Destination

In a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source.
Use rsync -u option to do exactly that. (i.e do not overwrite a file at the destination, if it is modified). In the following example, the file called Basenames is already modified at the destination. So, it will not be overwritten with rsync -u.
$ ls -l /root/temp/Basenames
total 39088
-rwxr-xr-x 1 root root        4096 Sep  2 11:35 Basenames

$ rsync -avzu thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/

sent 122 bytes  received 505 bytes  114.00 bytes/sec
total size is 45305958  speedup is 72258.31

$ ls -lrt
total 39088
-rwxr-xr-x 1 root root        4096 Sep  2 11:35 Basenames

Example 8. Synchronize only the Directory Tree Structure (not the files)

Use rsync -d option to synchronize only directory tree from source to the destination. The below example, synchronize only directory tree in recursive manner, not the files in the directories.
$ rsync -v -d thegeekstuff@192.168.200.10:/var/lib/ .
Password:
receiving file list ... done
logrotate.status
CAM/
YaST2/
acpi/

sent 240 bytes  received 1830 bytes  318.46 bytes/sec
total size is 956  speedup is 0.46

Example 9. View the rsync Progress during Transfer

When you use rsync for backup, you might want to know the progress of the backup. i.e how many files are copies, at what rate it is copying the file, etc.
rsync –progress option displays detailed progress of rsync execution as shown below.
$ rsync -avz --progress thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ...
19 files to consider
./
Basenames
     5357568 100%   14.98MB/s    0:00:00 (xfer#1, to-check=17/19)
Conflictname
       12288 100%   35.09kB/s    0:00:00 (xfer#2, to-check=16/19)
.
.
.
sent 406 bytes  received 15810211 bytes  2108082.27 bytes/sec
total size is 45305958  speedup is 2.87
You can also use rsnapshot utility (that uses rsync) to backup local linux server, or backup remote linux server.

Example 10. Delete the Files Created at the Target

If a file is not present at the source, but present at the target, you might want to delete the file at the target during rsync.
In that case, use –delete option as shown below. rsync delete option deletes files that are not there in source directory.
# Source and target are in sync. Now creating new file at the target.
$ > new-file.txt

$ rsync -avz --delete thegeekstuff@192.168.200.10:/var/lib/rpm/ .
Password:
receiving file list ... done
deleting new-file.txt
./

sent 26 bytes  received 390 bytes  48.94 bytes/sec
total size is 45305958  speedup is 108908.55
Target has the new file called new-file.txt, when synchronize with the source with –delete option, it removed the file new-file.txt

Example 11. Do not Create New File at the Target

If you like, you can update (Sync) only the existing files at the target. In case source has new files, which is not there at the target, you can avoid creating these new files at the target. If you want this feature, use –existing option with rsync command.
First, add a new-file.txt at the source.
[/var/lib/rpm ]$ > new-file.txt
Next, execute the rsync from the target.
$ rsync -avz --existing root@192.168.1.2:/var/lib/rpm/ .
root@192.168.1.2's password:
receiving file list ... done
./

sent 26 bytes  received 419 bytes  46.84 bytes/sec
total size is 88551424  speedup is 198991.96
If you see the above output, it didn’t receive the new file new-file.txt

Example 12. View the Changes Between Source and Destination

This option is useful to view the difference in the files or directories between source and destination.
At the source:
$ ls -l /var/lib/rpm
-rw-r--r-- 1 root root  5357568 2010-06-24 08:57 Basenames
-rw-r--r-- 1 root root    12288 2008-05-28 22:03 Conflictname
-rw-r--r-- 1 root root  1179648 2010-06-24 08:57 Dirnames
At the destination:
$ ls -l /root/temp
-rw-r--r-- 1 root root    12288 May 28  2008 Conflictname
-rw-r--r-- 1 bin  bin   1179648 Jun 24 05:27 Dirnames
-rw-r--r-- 1 root root        0 Sep  3 06:39 Basenames
In the above example, between the source and destination, there are two differences. First, owner and group of the file Dirname differs. Next, size differs for the file Basenames.
Now let us see how rsync displays this difference. -i option displays the item changes.
$ rsync -avzi thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
>f.st.... Basenames
.f....og. Dirnames

sent 48 bytes  received 2182544 bytes  291012.27 bytes/sec
total size is 45305958  speedup is 20.76
In the output it displays some 9 letters in front of the file name or directory name indicating the changes.
In our example, the letters in front of the Basenames (and Dirnames) says the following:
> specifies that a file is being transferred to the local host.
f represents that it is a file.
s represents size changes are there.
t represents timestamp changes are there.
o owner changed
g group changed.

Example 13. Include and Exclude Pattern during File Transfer

rsync allows you to give the pattern you want to include and exclude files or directories while doing synchronization.
$ rsync -avz --include 'P*' --exclude '*' thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
./
Packages
Providename
Provideversion
Pubkeys

sent 129 bytes  received 10286798 bytes  2285983.78 bytes/sec
total size is 32768000  speedup is 3.19
In the above example, it includes only the files or directories starting with ‘P’ (using rsync include) and excludes all other files. (using rsync exclude ‘*’ )

Example 14. Do Not Transfer Large Files

You can tell rsync not to transfer files that are greater than a specific size using rsync –max-size option.
$ rsync -avz --max-size='100K' thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
./
Conflictname
Group
Installtid
Name
Sha1header
Sigmd5
Triggername

sent 252 bytes  received 123081 bytes  18974.31 bytes/sec
total size is 45305958  speedup is 367.35
max-size=100K makes rsync to transfer only the files that are less than or equal to 100K. You can indicate M for megabytes and G for gigabytes.

Example 15. Transfer the Whole File

One of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.
If network bandwidth is not an issue for you (but CPU is), you can transfer the whole file, using rsync -W option. This will speed-up the rsync process, as it doesn’t have to perform the checksum at the source and destination.
#  rsync -avzW  thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp
Password:
receiving file list ... done
./
Basenames
Conflictname
Dirnames
Filemd5s
Group
Installtid
Name

sent 406 bytes  received 15810211 bytes  2874657.64 bytes/sec
total size is 45305958  speedup is 2.87

I'm using rsnapshot filesystem snapshot utility to make incremental snapshots of local and remote filesystems for 10 production servers running on RHEL 5.x system. The rsnapshot commands makes extensive use of hard links,
so if the file doesn't change, the next snapshot is simply a hard link to the exact same file. How do I use the rsync command to copy my the entire snapshot directory /raid6/rsnapshot (around 4TB) to a remote server?

The rsync command can preserve hard links and make the exact copy of /raid6/rsnapshot directory to a remote server using the following syntax:
rsync -az -H --delete --numeric-ids /path/to/source server2:/path/to/dest
Where,
  1. -a : Archive mode (i.e. recurse into directories, and preserve symlinks, file permissions, file modification times, file group, file owner, device files & special files)
  2. -z : Compress file data during the transfer
  3. -H : Preserve hard links (i.e. copy hard links as hard links)
  4. --delete : Delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized i.e. keep exact replica of your /raid6/rsnapshot directory.
  5. --numeric-ids : Transfer numeric group and user IDs rather than using user and group names and mapping them at both ends.
In short type the following command as root user:
# rsync -az -H --delete --numeric-ids /raid6/rsanpshot backupserver2:/backups
Smaller size directories can be dumped to usb 2.0/3.0 or eSata external hard drives using the same syntax. First, mount usb hard drive:
# mount /dev/sdXY /mnt/usbdisk
Use the rsync as follows:
# rsync -az -H --delete --numeric-ids /raid6/rsanpshot /mnt/usbdisk


Setup Rsync with SSH on UNIX / Linux (rsync without password)

Question: When I perform rsync, it asks for my password on the remote server before starting the transfer. I would like to avoid this, and perform rsync without password. Can you explain with an example on how to setup rsync over ssh without password on Linux?
Answer: The following steps explains how to setup rsync over ssh that doesn’t ask for a password. This is helpful when you are scheduling a cron job for automatic backup using rsync.

1. Test rsync over ssh (with password):

Do a rsync to make sure it asks for the password for your account on the remote server, and successfully copies the files to the remote server.
The following example will synchronize the local folder /home/ramesh to the remote folder /backup/ramesh (on 192.168.200.10 server).
We discussed in detail about rsync in our previous 15 rsync examples articles.
This should ask you for the password of your account on the remote server.
rsync -avz -e ssh /home/ramesh/ ramesh@192.168.200.10:/backup/ramesh/

2. ssh-keygen generates keys.

Now setup ssh so that it doesn’t ask for password when you perform ssh. Use ssh-keygen on local server to generate public and private keys.
$ ssh-keygen
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Note: When it asks you to enter the passphrase just press enter key, and do not give any password here.

3. ssh-copy-id copies public key to remote host

Use ssh-copy-id, to copy the public key to the remote host.
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.200.10
Note: The above will ask the password for your account on the remote host, and copy the public key automatically to the appropriate location. If ssh-copy-id doesn’t work for you, use the method we discussed earlier to setup ssh password less login.

4. Perform rsync over ssh without password

Now, you should be able to ssh to remote host without entering the password.
ssh 192.168.200.10
Perform the rsync again, it should not ask you to enter any password this time.
rsync -avz -e ssh /home/ramesh/ ramesh@192.168.200.10:/backup/ramesh/
If you want to schedule this rsync backup job automatically, use cron to set it up.

Compare directories difference
$ rsync -rvnc --delete source/ dest/

  • -n: dry run
  • -c: compute and compare checksum