day-to-day issues

08.20.2013
- upgrade from XenServer 6.1 to XenServer 6.2, redhat vm stop working. Could not find kernel-xen packages update as noted in XenServer 6.2 released notes, known issues.
Booted to recover, changing referred partition from /dev/sda# to /dev/xvda# in fstab solved the problem.

10.04.2013
- same as 08.20.2013 issue. A RHEL6.4_6x from XS6.1 would not run XS6.2.  Run "grub-install /dev/xvda" got error:
expr: non-numberic argument
expr: non-numberic argument
The file /boot/grub/stage1 not read correctly.

• Have tried to change /boot from ext4 to ext3. not working
• Change kernel version, not working

Finally, these made it work:

# grub
grub> device (hd0) /dev/xvda
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

04.23.2014
- after increase memory of VM, the VM could not start. log shows "The given VMs failed to release memory when instructed to do so"
The host is running out of memory. Try to shutdown some VM or reduce the memory of VMs resolve the problem.

04.23.2014
- when copy a big (~4G) file, the VM went into a quick reboot (about 14 seconds).

/var/log/messages does not show any errors, except reboot:

Apr 23 11:31:21 dev2 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: No such file or directory
Apr 23 11:35:02 dev2 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Apr 23 11:35:02 dev2 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1169" x-info="http://www.rsyslog.com"] start
Apr 23 11:35:02 dev2 kernel: Initializing cgroup subsys cpuset
Apr 23 11:35:02 dev2 kernel: Initializing cgroup subsys cpu
.....
Apr 23 11:35:03 dev2 ntpd[1382]: frequency initialized -5.758 PPM from /var/lib/ntp/drift
Apr 23 11:35:04 dev2 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: No such file or directory

05.13.2014
linux switch to read only file system
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=51306

As a workaround, remount the Linux file system using this command to return it to the proper state:
# mount -o remount /

possible solution: disable journaling
# Create ext4 fs on /dev/sda10 disk
mkfs.ext4 /dev/sda10

# Enable writeback mode. This mode will typically provide the best ext4 performance.
tune2fs -o journal_data_writeback /dev/sda10

# Delete has_journal option, This step disables journal
tune2fs -O ^has_journal /dev/sda10
# Or when creating
mkfs.ext4 -O ^has_journal /dev/sda10

# Required fsck
e2fsck -f /dev/sda10

# Check fs options, if has_journal option exist - you have journal
dumpe2fs /dev/sda10 | grep 'Filesystem features'

For more performance add fstab opions: data=writeback,noatime, 
i.e: /dev/sda10 /opt ext4 defaults,data=writeback,noatime 0 0

"discard" will enable trimming, discard options is default in ext4
Adding discard is a terrible choice if your goal is performance. It is also meaningless for non-flash drives. If you have a flash drive, and performance is the goal, schedule a nightly fstrim cron job on the relevant partitions.
Note: Using the discard flag for an ext3 root partition will result in it being mounted read-only.
Warning: Users need to be certain that their SSD supports TRIM before attempting to mount a partition with the discard flag. Data loss can occur otherwise!

07.22.2014
VM in XenServer 6.2 cannot reboot. Showing error "The tapdisk failed".
tried these, did not work.

1. From XenServer, "pvscan" to find out VG_XenStorage- ,
2. # vgchange -ay
3. to see if your Logical Volume Manage group has busted VHDs, making sure no errors appear or reports of bad headers/footers
   # vhd-util scan -m "VHD*" -f -c -l VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b -p -v
4. scan your storage repository and see if any errors appear
   # xe sr-scan uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b
5. The last item you can do - on a live system - but this is the big one and a reference article can be found here (http://blogs.citrix..../xs-mgt-volume/).
   # lvrename /dev/VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b/MGT /dev/VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b/MGTbak
   # lvchange -ay
   From XenCenter, select your storage repository and under Storage, select SCAN SR.
6. Un-plug and plug storage. All VMs off
   # xe pbd-list sr-uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b
   # xe pbd-unplug uuid=b8ba43c6-ded0-4f60-aa97-d27fd7daf8cf
   Wait a moment and the run the following to re-attach your storage:
   # xe pbd-plug uuid=b8ba43c6-ded0-4f60-aa97-d27fd7daf8cf
   Now, we complete with an sr scan:
   # xe sr-scan uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b

possible cause: too many vm on the host that it cannot allocate memory for the vm.

07.24.2014
VM console not responding. reset VNC console
live migrate to another host in the pool then migrate back, or

1. identify <dom-id> for the frozen VM
2. stop the current running console:
   # kill `ps aux|grep domain/<dom-id>|grep -v grep|awk '{print $2}'`
3. restart VNC console:
   # /usr/lib/xen/bin/vncterm -v 127.0.0.1:1 -x /local/domain/<dom-id>/serial/0 &