Wednesday, November 9, 2011

Install KVM Virtualization on RedHat

http://www.cyberciti.biz/faq/centos-rhel-linux-kvm-virtulization-tutorial/

KVM (Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions Intel VT or AMD-V. How do I install KVM under CentOS or Red Hat Enterprise Linux version 5.5?

The Linux kernel 2.6.20 and above included KVM. RHEL 5.5 (and upcoming RHEL 6) supports KVM out of box and it has also been ported to FreeBSD as a loadable kernel module. However, this tutorial is tested on both CentOS and RHEL 5.5 only running 64 bit Intel Xeon CPU (with Intel VT) and 64 bit kernels with SELinux running in enforcing mode.

Why KVM?

XEN allows several guest operating systems to execute on the same computer hardware and it is also included with RHEL 5.5. But, why use KVM over XEN? KVM is part of the official Linux kernel and fully supported by both Novell and Redhat. Xen boots from GRUB and loads a modified host operating system such as RHEL into the dom0 (host domain). KVM do not have concept of dom0 and domU. It uses /dev/kvm interface to setup the guest operating systems and provides required drivers. See the official wiki for more information.

Required Packages

You must install the following packages:
  1. kmod-kvm : kvm kernel module(s)
  2. kvm : Kernel-based Virtual Machine
  3. kvm-qemu-img : Qemu disk image utility
  4. kvm-tools : KVM debugging and diagnostics tools
  5. python-virtinst : Python modules and utilities for installing virtual machines
  6. virt-manager : Virtual Machine Manager (GUI app, to install and configure VMs)
  7. virt-viewer: Virtual Machine Viewer (another lightweight app to view VM console and/or install VMs)
  8. bridge-utils : Utilities for configuring the Linux Ethernet bridge (this is recommended for KVM networking)

KVM Package Group

RHEL comes with KVM software group which includes full virtualization support with KVM. You can list all packages in the group as follows:
# yum groupinfo KVM
Sample outputs:
Loaded plugins: rhnplugin, security
Setting up Group Process
Group: KVM
 Description: Virtualization Support with KVM
 Mandatory Packages:
   celt051
   etherboot-zroms
   etherboot-zroms-kvm
   kmod-kvm
   kvm
   kvm-qemu-img
   qcairo
   qffmpeg-libs
   qpixman
   qspice-libs
 Default Packages:
   Virtualization-en-US
   libvirt
   virt-manager
   virt-viewer
 Optional Packages:
   celt051-devel
   etherboot-pxes
   etherboot-roms
   etherboot-roms-kvm
   gpxe-roms-qemu
   iasl
   kvm-tools
   libcmpiutil
   libvirt-cim
   qcairo-devel
   qffmpeg-devel
   qpixman-devel
   qspice
   qspice-libs-devel

A Note About libvirt

libvirt is an open source API and management tool for managing platform virtualization. It is used to manage Linux KVM and Xen virtual machines through graphical interfaces such as Virtual Machine Manager and higher level tools such as oVirt. See the official website for more information.

A Note About QEMU

QEMU is a processor emulator that relies on dynamic binary translation to achieve a reasonable speed while being easy to port on new host CPU architectures. When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, and S390 guests. See the official website for more information.

A Note About Virtio Drivers

Virtio is paravirtualized drivers for kvm/Linux. With this you can can run multiple virtual machines running unmodified Linux or Windows VMs. Each virtual machine has private virtualized hardware a network card, disk, graphics adapter, etc. According to Redhat:
Para-virtualized drivers enhance the performance of fully virtualized guests. With the para-virtualized drivers guest I/O latency decreases and throughput increases to near bare-metal levels. It is recommended to use the para-virtualized drivers for fully virtualized guests running I/O heavy tasks and applications.

Host Operating System

Your main operating system such as CentOS or RHEL is known as host operating system. KVM is a Linux kernel module that enables a modified QEMU program to use hardware virtualization. You only need to install KVM under host operating systems.

KVM Domains

It is nothing but a guest operating system running under host operating system. Each kvm domain must have a unique name and ID (assigned by system).

Guest Operating Systems

KVM supports various guest operating systems such as
  1. MS-Windows 2008 / 2000 / 2003 Server
  2. MS-Windows 7 / Vista / XP
  3. FreeBSD
  4. OpenBSD
  5. Sun Solaris
  6. Various Linux distributions.
  7. NetBSD
  8. MINIX
  9. QNX
  10. MS DOS
  11. FreeDOS
  12. Haiku
  13. Amiga Research OS

Install KVM

Type the following command to install KVM under RHEL or CentOS:
# yum install kvm virt-viewer virt-manager libvirt libvirt-python python-virtinst
OR
# yum groupinstall KVM

Important Configuration And Log Files (Directories) Location

The following files are required to manage and debug KVM problems:
  1. /etc/libvirt/ - Main configuration directory.
  2. /etc/libvirt/qemu/ - Virtual machine configuration directory. All xml files regarding VMs are stored here. You can edit them manually or via virt-manager.
  3. /etc/libvirt/qemu/networks/ - Networking for your KVM including default NAT. NAT is only recommended for small setup or desktops. I strongly suggest you use bridged based networking for performance.
  4. /etc/libvirt/qemu/networks/default.xml - The default NAT configuration used by NAT device virbr0.
  5. /var/log/libvirt/ - The default log file directory. All VM specific logs files are stored here.
  6. /etc/libvirt/libvirtd.conf - Master libvirtd configuration file.
  7. /etc/libvirt/qemu.conf - Master configuration file for the QEMU driver.

TCP/UDP Ports

By default libvirt does not opens any TCP or UDP ports. However, you can configure the same by editing the /etc/libvirt/libvirtd.conf file. Also, VNC is configured to listen on 127.0.0.1 by default. To make it listen on all public interfaces, edit /etc/libvirt/qemu.conf file.

Our Sample Setup

+-------------> vm#1 ( 10.10.21.71 / 123.1.2.4, CentOS MySQL Server)
                                                                             |
                                                                             +-------------> vm#2 ( 10.10.21.72 / 123.1.2.5, FreeBSD 7 Web Server)
LAN --> Switch    -->  eth0 --> -+   10.10.21.70                             |
                                 | ---> br0 -+      +----------------+       +-------------> vm#3 ( 10.10.21.73 / 123.1.2.6, OpenBSD 4.x Firewall )
                                 |           | ===> |   RHEL Server  |  -----+
                                 |           |      |      KVM       |       +-------------> vm#4 ( 10.10.21.74 / 123.1.2.7, Solaris 10 Testing Server )
                                 | ---> br1 -+      +----------------+       |
Wan --> ISP Router --> eth1 --> -+   123.1.2.3                               +-------------> vm#5 ( 10.10.21.71 / 123.1.2.8, Windows Server Testing Server )
                                                                             |
                                                                             +-------------> vm#6 ( 10.10.21.71 / 123.1.2.9, RHEL Mail Server )
(Fig.01: Our sample server setup - you need to scroll to see complete diagram)
Where,

Host Configuration

  • OS - RHEL / CentOS v5.5 is our host operating system.
  • Host has two interface eth0 and eth1
  • LAN - eth0 with private ip
  • Internet - eth1 with public IPv4/IPv6 address.
  • Disk - 73x4 - 15k SAS disk in RAID 10 hardware mode. All VMs are stored on same server (later I will cover SAN/NFS/NAS configuration with live migration).
  • RAM - 16GB ECC
  • CPU - Dual core dual Intel Xeon CPU L5320 @ 1.86GHz with VT enabled in BIOS.

Virtual Machine Configuration

  • Bridged mode networking (eth0 == br0 and eth1 == br1) with full access to both LAN and Internet.
  • Accelerator virtio drivers used for networking (model=virtio)
  • Accelerator virtio drivers for disk (if=virtio) and disk will show up as /dev/vd[a-z][1-9] in VM.
  • Various virtual machines running different guest operating systems as per requirements.
With bridged networking you can share actual network device with KVM machines. This is required for servers with multiple network cards and gives you good performance. You can choose to put multiple segments into one bridged network or to divide it into different networks interconnected by routers.

Our Sample Setup

The following describes the networking used by our setup:
+------+       +-----+
LAN      --> | eth0 |  <==> | br0 |       -> KVM VMs connected to LAN for SAN/NFS
  +------+       +-----+
                               10.10.x.y/24
  +------+       +-----+
Internet --> | eth1 |  <==> | br1 |       -> KVM VMs connected to the Inernet
  +------+       +-----+
                               123.1.2.0/28
Where,
  • All other clients can reached to all VMs via br1 which is connected to public interface. br1 is our default gateway.
  • br0 is connected to private LAN to access other servers, services and storage devices such as SAN/NAS or NFS servers. br0 route is configured via route-br0 static networking configuration file.

Turn Off NetworkManager

The NetworkManager (GUI) tool can create problems with bridged based networking so disable it as follows, enter:
# chkconfig NetworkManager off
# chkconfig network on
# service NetworkManager stop
Edit /etc/sysconfig/network, enter:
# vi /etc/sysconfig/networkUpdate file as follows:
 
NETWORKING=yes
HOSTNAME=kvm42.nixcraft.net
GATEWAY=br1
 
Save and close the file.

br0: Configure Bridging for eth0

Edit /etc/sysconfig/network-scripts/ifcfg-eth0, enter:
# vi /etc/sysconfig/network-scripts/ifcfg-eth0Update it as follows:
 
 
DEVICE=eth0
ONBOOT=yes
ETHTOOL_OPTS="autoneg off speed 100 duplex full"
BRIDGE=br0
HWADDR=b8:ac:6f:65:31:e5
 
Save and close the file. Create /etc/sysconfig/network-scripts/ifcfg-br0, enter:
# vi /etc/sysconfig/network-scripts/ifcfg-br0Update it as follows (note options are case sensitive i.e. Bridge and bridge are two different options)
 
DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=10.10.21.70
NETMASK=255.255.255.192
DELAY=0
 
Save and close the file. Finally, create static routing file /etc/sysconfig/network-scripts/route-br0, enter :
# vi /etc/sysconfig/network-scripts/route-br0Edit it as follows:
10.0.0.0/8 via 10.10.21.122 dev br0
Save and close the file. Delete old /etc/sysconfig/network-scripts/route-eth0 (if exists):
# rm /etc/sysconfig/network-scripts/route-eth0

br1: Configure Bridging for eth1

Edit /etc/sysconfig/network-scripts/ifcfg-eth1, enter:
# vi /etc/sysconfig/network-scripts/ifcfg-eth1Update it as follows:
 
 
DEVICE=eth1
ONBOOT=yes
ETHTOOL_OPTS="autoneg off speed 100 duplex full"
BRIDGE=br1
HWADDR=00:30:49:8c:48:ad
 
Please note that test server is set to 100Mbps full duplex. Save and close the file. Edit /etc/sysconfig/network-scripts/ifcfg-br1, enter:
# vi /etc/sysconfig/network-scripts/ifcfg-br1Define public IP address including gateway as follows:
 
DEVICE=br1
BOOTPROTO=static
ONBOOT=yes
IPADDR=123.1.2.3
NETMASK=255.255.255.248
GATEWAY=123.1.2.200
TYPE=Bridge
DELAY=0
 
Save and close the file.

Restart Network Service

Type the following command:
# service network restartMake sure everything is working fine:
# brctl showSample outputs:
bridge name bridge id  STP enabled interfaces
br0  8000.0030488e31ac no  eth0
br1  8000.0030488e31ad no  eth1
virbr0  8000.000000000000 yes
Verify IPs and routing tables:
# ip addr show br0
# ip addr show br1
# ip route
# ping cyberciti.biz
# host google.com


Now, you have installed the KVM software and required drivers on the host operating systems. You also configured bridged based networking which will provide direct access to the Internet. It is time to create your first guest operating systems. For demonstration purpose you will install CentOS Linux v5.5 using the Internet using virt-manager GUI tool.

The virt-manager is the easiest way to install guest operating system using CDROM or Internet. It is a a desktop tool for managing virtual machines. It provides the ability to control the lifecycle of existing machines (bootup/shutdown,pause/resume,suspend/restore), provision new virtual machines, manage virtual networks, access the graphical console of virtual machines, and view performance statistics. You need to use this tool locally or remotely over the ssh session.

Step # 1: Download CentOS Linux Network Installation CD

Visit the official website and grab CentOS network installation disk and store it in /opt or /tmp directory. The wget command can be used to download an ISO file quickly:
# cd /tmp
# wget http://mirrors.hns.net.in/centos/5.5/isos/x86_64/CentOS-5.5-x86_64-netinstall.iso

Step #2: Creating CentOS Linux Guests With virt-manager

Type the following command at local server
# virt-manager
OR run virt-manager remotely over the ssh session, enter:
# ssh -X -C root@kvmserver42.nixcraft.in
# virt-manager

Sample outputs:

Fig.01: Creating A Virtualized Guest With virt-manager GUI Tool
Fig.01: Creating A Virtualized Guest With virt-manager GUI Tool

Next, click the New button to create a new guest and just follow on screen instructions. The following is the wizard based installation procedure:

Fig.02: New guest wizard to install CentOS Linux
Fig.02: New guest wizard to install CentOS Linux
Fig.03: Name the KVM virtual machine
Fig.03: Name the KVM virtual machine
Fig.04: Define virtualization method
Fig.04: Define virtualization method (full virtualization due to Intel VT)
Fig.05: Define installation method (CDROM/Network/HTTP/NFS etc)
Fig.05: Define installation method (CDROM/Network/HTTP/NFS etc)
Fig.06: Local media installation via CDROM or ISO image
Fig.06: Local media installation via CDROM or ISO image
Fig.07: KVM Storage setup (network [must be mounted] or local or block disk)
Fig.07: KVM Storage setup (network iscsi/nfs must be mounted or local or block disk)
Fig.08: KVM Set Virtual network (NAT) or Shared physical device (bridge br0 or br1)
Fig.08: KVM Set Virtual network (NAT) or Shared physical device (bridge br0 or br1)
Fig.09: KVM Guest VM Memory and CPU Allocation Setup
Fig.09: KVM Guest VM Memory and CPU Allocation Setup
Fig.10: KVM VM setup done and you can start guest installation
Fig.10: KVM VM setup done and you can start guest installation
Within minutes you will see a vnc window and with guest operating system installation process as follows:
Fig.11: KVM - guest operating system installation procedures started using vnc
Fig.11: KVM - guest operating system installation procedures started using vnc
Now, just follow on screen installation instructions and install CentOS as per your requirements. The above procedure can be repeated for MS-Windows, OpenBSD, FreeBSD and all other supported guest operating systems.

virt-manager requires GUI locally and it is slow. On other hand virt-install is a command line tool for provisioning new virtual machines using the "libvirt" hypervisor management library. The tool supports both text based & graphical installations, using serial console, SDL graphics or a VNC client/server pair. The guest can be configured to use one or more virtual disks, network interfaces, audio devices, and physical host devices (USB, PCI). You can use the virt-install to create virtualized guests as follows to install FreeBSD or CentOS.

Install FreeBSD As Guest OS

Again, use the wget command to grab FreeBSD ISO image:
# cd /tmp
# wget ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64/7.3/FreeBSD-7.3-RELEASE-amd64-disc1.iso

Type the following command to install FreeBSD 7.3 64 bit with 512 MB RAM, 1 vcore cpu, 4GB disk space, br0 and br1 network interface:
# virt-install \
-n freebsd \
-r 512 \
--vcpus=1 \
--os-variant=freebsd7 \
--accelerate \
-v \
-c /nfsclient/iso/FreeBSD-7.3-RELEASE-amd64-disc1.iso \
-w bridge:br0 \
-w bridge:br1 \
--vnc \
--disk path=/raid10/kvm/freebsd73.img,size=4

Sample outputs:

Starting install...
Creating storage file...                                                                     | 4.0 GB     00:00
Creating domain...                                                                           |    0 B     00:01
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
Use your local vnc viewer or type the following at server itself:
# vncviewer
OR over ssh session, enter:
# ssh -X -C root@kvmserver42.nixcraft.in
# virt-viewer freebsd

Sample outputs:

Fig.01: Creating guest with virt-install and installing guests with vnc
Fig.01: Creating guest with virt-install and installing guests with vnc

Once again, just follow on screen installation instructions and install FreeBSD as per your requirements. The above procedure can be repeated for MS-Windows, OpenBSD, FreeBSD and all other supported guest operating systems.

Install CentOS As Guest OS

Type the following command to install CentOS using http mirror method with 512MB RAM, 4GB disk space stored on nas server mounted at /nas, br0 and br1 network interface:
# virt-install \
-n centos \
-r 512 \
--vcpus=1 \
--os-variant=rhel5.4 \
--accelerate \
-v \
-l http://mirrors.kernel.org/centos/5.5/os/x86_64/ \
-w bridge:br0 \
-w bridge:br1 \
--vnc \
--disk path=/nas/kvm/centos.img,size=4

You can grab installer screen using vnc as follows (or use your local vnc viewer):
# ssh -X -C user@server.example.com
# virt-viewer centos

Understanding virt-install Command Line Options

  1. -n centos - Name of the new guest virtual machine instance. This must be unique amongst all guests known to the hypervisor on this machine, including those not currently active.
  2. -r 512 - VM memory allocation.
  3. --vcpus=1 - VM cpu allocation.
  4. --os-variant=rhel5.4 - Optimize the guest configuration for a type of operating system called rhel5.4.
  5. --accelerate - When installing a QEMU guest, make use of the KVM or KQEMU kernel acceleration capabilities if available. Use of this option is recommended unless a guest OS is known to be incompatible with the accelerators. The KVM accelerator is preferred over KQEMU if both are available.
  6. -v - This guest should be a fully virtualized guest
  7. -l http://mirrors.kernel.org/centos/5.5/os/x86_64/ - Install using http mirror.
  8. -w bridge:br0 - Connect the guest to the host network. In this example connect to a bridge device in the host called "br0". Use this option if the host has static networking config & the guest requires full outbound and inbound connectivity to/from the LAN. Also use this if live migration will be used with this guest.
  9. -w bridge:br1 - Same as above but using br1 so that guest can have full outbound and inbound connectivity to/from the Internet.
  10. --vnc - Setup a virtual console in the guest and export it as a VNC server in the host.
  11. --disk path=/nas/kvm/centos.img,size=4 - Path to the file, disk partition, or logical volume to use as the backing store for the guest's virtual disk.
Kickstart is a network installation system for RHEL, Fedora and CentOS Linux distributions. Another good option is Cobbler which is a Linux provisioning server that centralizes and simplifies control of services including DHCP, TFTP, and DNS for the purpose of performing network-based operating systems installs. In this tutorial, I'm going to show you how to use kickstart file to install CentOS.

Create Kickstart file

An automated installation method to install CentOS / Fedora or RHEL is recommend to automate procedure. Using kickstart, a system administrator can create a single file containing the answers to all the questions that would normally be asked during a typical RHEL Linux installation. Use kickstart GUI tool called "Kickstart Configurator" (run system-config-kickstart command to start the tool) to create a file called ks.cfg as follows:
auth  --useshadow  --enablemd5
bootloader --location=mbr
zerombr
clearpart --all --initlabel
text
firewall --enabled --port=22:tcp
firstboot --disable
keyboard us
network --device eth0 --bootproto static --ip 10.10.21.76 --netmask 255.255.255.240 --gateway 10.10.21.100 --nameserver 10.10.21.1,10.10.21.2 --noipv6
network --device eth1 --bootproto static --ip 123.1.2.6 --netmask 255.255.255.240 --gateway 123.1.2.100 --nameserver 10.10.21.1,10.10.21.2 --hostname centos.nixcraft.in --noipv6
lang en_US
logging --level=info
url --url=http://mirrors.nixcraft.in/centos/5.5/os/x86_64/
reboot
rootpw --iscrypted $1$somepassword
selinux --enforcing
skipx
timezone  America/New_York
install
part / --bytes-per-inode=4096 --fstype="ext3" --grow --size=1
part swap --recommended
%packages
@core
--nobase
%post
(
echo '10.0.0.0/8 via 10.10.21.100' > /etc/sysconfig/network-scripts/route-eth0
sed -i 's/LABEL=\//& console=ttyS0/' /etc/grub.conf
echo 'S0:12345:respawn:/sbin/agetty ttyS0 115200' >> /etc/inittab
echo "ttyS0" >> /etc/securetty
echo 'IPV6INIT=no' >> /etc/sysconfig/network
echo 'install ipv6 /bin/true' >> /etc/modprobe.conf
) 1>/root/post_install.log 2>&1
Upload this file to a web server as ks.cfg. You can use nfs server too.

virt-install: Install CentOS using Kickstart

Type the following command:
# virt-install \
-n centos \
-r 2048 \
--vcpus=1 \
--os-variant=rhel5.4 \
--accelerate \
-v \
-w bridge:br0 \
-w bridge:br1 \
--disk path=/emc/kvm/centos.img,size=100 \
-l http://mirrors.nixcraft.in/centos/5.5/os/x86_64/ \
-nographics \
-x "ks=http://10.10.21.3/static/ks.cfg ksdevice=eth0 ip=10.10.21.76 netmask=255.255.255.240 dns=10.10.21.1 gateway=10.10.21.100"

The -x option is used to pass additional kernel command line to the installer when performing a guest install. The ks option sets ks file location and rest are networking options so that installer can fetch ks.cfg and do automated installation for you.


There are various log files stored on the host system to assist with debugging KVM related problems. In this tutorial, I will cover log file locations and tools used to track down KVM problems.

=> $HOME/.virtinst/virt-install.log - virt-install tool log file.
=> $HOME/.virt-manager/virt-manager.log - virt-manager tool log file.
=> /var/log/libvirt/qemu/ - Log files for each running virtual machine. If centos is virtual machine name, than log file is /var/log/libvirt/qemu/centos.log.
You can use the grep and other Linux tools to view this files:
# tail -f /var/log/libvirt/qemu/freebsd.log
# grep something $HOME/.virtinst/virt-install.log

Connecting To Console

Use the virsh command to connect to guest serial console as follows :
# virsh list
# virsh console freebsd

This is useful to troubleshoot problem such as networking and much more from host itself. However, you need to configure guest operating system for a serial console. See how to setup a serial console for
FreeBSD and CentOS virtual machines.

KVM Configuration Files

You can edit configuration file to define hardware properties for VMs or other setup. They are located in /etc/libvirt/qemu/ directory. Here is sample FreeBSD guest configuration file.
less /etc/libvirt/qemu/freebsd.xml
OR
virsh dumpxml freebsd
Sample outputs:

<domain type='kvm'>
  <name>freebsd</name>
  <uuid>6b7f44df-b67a-b1e1-0f9a-40c9ad760b0a</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel5.4.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
<pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' cache='none'/>
      <source file='/nfs/freebsd73.img'/>
      <target dev='hda' bus='ide'/>
    </disk>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <interface type='bridge'>
      <mac address='54:53:01:12:4c:0a'/>
      <source bridge='br0'/>
      <model type='ne2k_pci'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:53:02:1e:9c:0b'/>
      <source bridge='br1'/>
      <model type='ne2k_pci'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
<input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>
 
When you create guests with the virt-manager or virt-install, the guests configuration files are created automatically in the /etc/libvirt/qemu/ directory. You can use this file error checking. You can edit this file using a text editor or virsh command itself:
virsh edit freebsd
virsh edit centos

KVM tools

Type the following command to install kvm-tools package which contains some diagnostics and debugging tools for KVM, such as kvmtrace and kvm_stat, enter:
# yum -y install kvm_tools
You need to mount debugfs, enter:
# mount -t debugfs debugfs /sys/kernel/debug
# kvm_stat

Sample outputs:

Fig.01: Getting stats about KVM with kvm_stat
Fig.01: Getting stats about KVM with kvm_stat

You can use normal tools to troubleshoot and get other information using the following tools:

  • ps, pstree, and top
  • vmstat, iostat, and lsof
  • tcpdump, brctl, ip, and ifconfig
I'm using KVM to run multiple virtual machines under Redhat Enterprise Linux server 5.5. I've installed FreeBSD 7.x 64 bit as guest operating systems. How do I redirect the FreeBSD version 6, 7 or 8 virtual machine console to a serial port using virsh console command for management purpose from host itself?

FreeBSD does support a dumb terminal on a serial port as a console. This is useful for quick login or debug guest system problem without using ssh. First, login as root using ssh to your guest operating systems:
$ ssh vivek@freebsd.nixcraft.in
su -

Edit /boot/loader.conf, enter:
# vi /boot/loader.conf
Append the following entry:
console="comconsole"
Save and close the file. Edit /etc/ttys, enter:
# vi /etc/ttys
Find the line that read as follows:

ttyd0  "/usr/libexec/getty std.9600"   dialup  off secure
Update it as follows:
ttyd0   "/usr/libexec/getty std.9600"   vt100   on secure
Save and close the file. Reboot the guest, enter:
# reboot
After reboot, you can connect to FreeBSD guest as follows from host (first guest the list of running guest operating systems):
# virsh list
Sample outputs:

Id Name                 State
----------------------------------
  3 ographics            running
  4 freebsd              running
Now, connect to Freebsd guest, enter:
virsh console 4
OR
virsh console freebsd
Sample outputs:

Fig.01: Virsh console connected to FreeBSD guest
Fig.01: Virsh console connected to FreeBSD guest

References:

The virsh command can be used to mange local or remote guest operating systems. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains.

List Running VMS

Type the following command:
# virsh list
Sample outputs:

Id Name                 State
----------------------------------
  1 centos.nixcraft.in   running
  2 freebsd              running

Shut Down A Guest

# virsh list
# virsh shutdown dominName
# virsh shutdown freebsd
# virsh shutdown 3

Rebooting A Guest

# virsh list
# virsh reboot domaiName
# virsh reboot 3
# virsh reboot win2008biz

Forcefully Stop A Guest

Force a guest to stop with the virsh command if it is not responding or crashed
# virsh list
# virsh destroy domainName
# virsh destroy openbsd.nixcraft.in

Get Information About Guest

# virsh list
# virsh dominfo dominName
# virsh dominfo 2
# virsh dominfo freebsd

Sample outputs:

Id:             2
Name:           freebsd
UUID:           6b7f44df-b67a-b1e1-0f9a-40c9ad760b0a
OS Type:        hvm
State:          running
CPU(s):         1
CPU time:       26.3s
Max memory:     524288 kB
Used memory:    524288 kB
Autostart:      disable

Get Information About Node

# virsh nodeinfo
Sample Outputs:

CPU model:           x86_64
CPU(s):              4
CPU frequency:       2394 MHz
CPU socket(s):       1
Core(s) per socket:  4
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:         8181332 kB

Conclusion

The first part in this series covered KVM installation, KVM bridged based networking, KVM guest setup using virt-install, virt-manager, kickstart based guest installation and troubleshooting KVM with log files. Stay tunned for next part of series which will cover other advanced topics such as:
  • VM storage configuration using NFS and iSCSI
  • VM live migration
  • VM security and firewall
  • VM load balancing
  • VM instillation using Cobbler
The virtual network (virbr0) used for Network address translation (NAT) which allows guests to access to network services. However, NAT slows down things and only recommended for desktop installations. To disable Network address translation (NAT) forwarding type the following commands:

Display Current Setup

Type the following command:
# ifconfig
Sample outputs:

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:7921 (7.7 KiB)
Or use the following command:
# virsh net-list
Sample outputs:

Name                 State      Autostart
-----------------------------------------
default              active     yes
To disable virbr0, enter:
# virsh net-destroy default
# virsh net-undefine default
# service libvirtd restart
# ifconfig


I've setup FreeBSD and OpenBSD under KVM. Everything works great except ftp protocol. I've no firewall installed inside FreeBSD and OpenBSD. How do I configure Linux host KVM firewall to allow ftp traffic?

You need to configure iptables to allow all traffic to be forwarded across the bridge from KMV guest. There are two options here:

Option #1: Configure Iptables

Type the following command as root user on KVM host (RHEL / CentOS host system):
/sbin/iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
/sbin/service iptables save
/sbin/service iptables restart

Option #2: Disable Iptables On Bridges

Edit /etc/sysctl.conf, enter:
# vi /etc/sysctl.conf
Append the following config directives:

# Disable iptables on bridges so that ftp will work
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
Save and close the file. Reload the changes:
# sysctl -p

There are various ways to start virtual machines at at boot time. This means you don't have to type virsh start vmName command. This can be done by marking a vm as autostart. To configure a domain to be automatically started at boot. It will create an softlink at /etc/libvirt/qemu/autostart/. So if your VM name is debianlenny1, your config file name should be /etc/libvirt/qemu/debianlenny1.xml and the softlink should be created at /etc/libvirt/qemu/autostart//debianlenny1.xml

KVM: Autostart a Domain / VM Command

Type the following command:
virsh autostart vmName
virsh autostart debianlenny1
Sample outputs:
Domain debianlenny1 marked as autostarted
Also, make sure /etc/init.d/libvirtd service is stared on boot:
chkconfig libvirtd on

OpenBSD is a UNIX like system based on Berkeley Software Distribution (BSD). It was forked form NetBSD and created by Theo de Raadt in 1995. OpenBSD is famous for quality code and documentation without compromising position on software licensing. OpenBSD is also well know for focus on security features such as Memory protection, cryptography, randomization and much more in default base installation.

OpenBSD Installation

You need to first download OpenBSD iso image by visiting this page. You can use the wget command to grab the same as follows. The following command will download OpenBSD v4.7 64 bit (AMD64) iso file:
cd /tmp
wget http://filedump.se.rit.edu/pub/OpenBSD/4.7/amd64/install47.iso

Install OpeBSD 4.7 AMD64

Type the following command to install OpenBSD v4.7 64 bit with 512 MB RAM, 1 vcore cpu, 5GB disk space, br0 and br1 network interface:
 
virt-install \
-n openbsd1 \
-r 512 \
--vcpus=1 \
--os-variant=openbsd4 \
--accelerate \
-v -c /tmp/install47.iso \
-w bridge:br0 -w bridge:br1 \
--vnc \
--disk path=/raid10/kvm/openbsd1.img,size=5
 
Sample outputs:
Starting install...
Creating domain...                                                                                                                                                                                                    |    0 B     00:01
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
Use your local vnc viewer or type the following at server itself to view installer screen:
# vncviewer
OR run it over the ssh session, enter:
# ssh -X -C root@kvmserver42.nixcraft.in
# virt-viewer openbsd1

Sample outputs:

Fig.01: Creating OpenBSD guest with virt-install and installing the same with vnc
Fig.01: Creating OpenBSD guest with virt-install and installing the same with vnc

Now, just follow on screen installation instructions and install OpenBSD as per your requirements.

After Reboot My OpenBSD Guest Hangs At The Boot Screen...

Optional: Configure Serial Console Login and Redirection

OpenBSD version 4.5 and above made changes in kernel which prevents booting OpenBSD using KVM technology. OpenBSD 4.4 runs with mbbios enabled. However, due to some sort of bug in KVM it does not boot and hangs while displaying a message which read as "Starting tty flags". The workaround is to disable mpbios. This may or may not happen with your installation, but if it hanged, use the following procedure.

Step # 1: Boot OpenBSD With -c Option

At boot> prompt type the following command:
bsd -c
disable mpbios
quit
Sample session:
Fig.01: OpenBSD KVM Hangs at setting tty flags workaround
Fig.01: OpenBSD KVM Hangs at setting tty flags workaround

Now OpenBSD will boot without any problem. Login as root and type the following command to disable mpbios permanently:
# config -ef /bsd
disable mpbios
quit
Sample session:
Fig.02: Disabling mpbios for OpenBSD running under Linux KVM
Fig.02: Disabling mpbios for OpenBSD running under Linux KVM

Now you can use OpenBSD without any problem. Please note that this problem only occured under RHEL 5.5 64bit (KVM83) running OpenBSD 4.7 64bit guest. The config command can be used to build kernel compilation directories or modify a kernel. In this case you've modified the kernel to boot without mpbios support. The mpbios driver extracts configuration information from the BIOS on multiprocessor systems that conform to the Intel MultiProcessor Specification. See config and mpbios man page for further information.

I'm running a KVM virtual machine (FreeBSD 7 and Fedora Linux 13 desktop) under Redhat Enterprise Linux 5.5. However, I'm not getting the remote access via VNC (Virtual Network Computing) to access my Fedora Linux graphical desktop or FreeBSD console. How do I turn on VNC support without reinstalling both operating systems?

You need to attach your vncserver with the guest operating system console. This can be done using any one of the following method:

Method # 1: Command Line Option

Normally, QEMU (/usr/libexec/qemu-kvm) uses SDL to display the VGA output. With the -vnc option option, you can have QEMU listen on VNC display display and redirect the VGA display over the VNC session. When using the VNC display, you must use the -k parameter to set the keyboard layout if you are not using en-us. Valid syntax for the display is as follows:
-vnc :0
-vnc 192.168.1.5:0
-vnc 0.0.0.0:5
-vnc 0.0.0.0:1 -k en-us
####  Require that password based authentication is used for client connections ####
-vnc 0.0.0.0:1,password -k en-us
 
In the following example start centos1 guest vm using vnc
/usr/libexec/qemu-kvm -S -M rhel5.4.0 -m 1024 -smp 1 -vnc 0.0.0.0:1 -k en-us -name centos1 -monitor pty -boot c -drive file=/var/lib/libvirt/images/centos1.img

Method # 2: qemu-kvm VM Config File (Recommended)

You need to edit your VM config file which is in XML format. The config file is located at /etc/libvirt/qemu directory. In this example, edit centos1.xml as follows:
# vi /etc/libvirt/qemu/centos1.xml
Append the following line before final </devices>:

 
 <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
 
Here is my sample config file:
 
<domain type='kvm'>
  <name>centos1</name>
  <uuid>88d067cf-e5f7-7229-f35f-472a9c884864</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel5.4.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
<pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' cache='none'/>
      <source file='/emc/vms/images/host302.f02.dc05.corplan.nixcraft.net.in/c/centos1.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='xx:yy:zz:ee:f4:63'/>
      <source bridge='br0'/>
      <model type='virtio'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:52:xx:yy:zz:ee'/>
      <source bridge='br1'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>
 
Finally, restart your libvirtd:
# /etc/init.d/libvirtd restart
# virsh shutdown centos1
# virsh start centos1

How Do I Use VNC Client?

Type your VNC (KVM) server ip and port:
Fig.01: Connecting to KVM Guest Using VNC Client
Fig.01: Connecting to KVM Guest Using VNC Client

Once connected you can view your desktop:
Fig.02: RHEL 6 Guest Beta Desktop Over VNC Session Running Under KVM
Fig.02: RHEL 6 Guest Beta Desktop Over VNC Session Running Under KVM

Make sure you browse the desktop using ssh tunnel or over some sort of VPN session.

How Do I Find Out Current VNC Setting For Any Given Domain / VM?

Type the following command:
# virsh vncdisplay domainName
# virsh vncdisplay 3
# virsh vncdisplay centos1

Sample outputs:

:2

How Do I Password Protect My VNC Session?

The passwd attribute provides a VNC password in clear text (so make sure your xml config file is only readable by root user). Edit centos1.xml file as follows:
 
 <graphics type='vnc' port='-1' autoport='yes' passwd='YOUR-PASSWORD-HERE' keymap='en-us'/>
 
OR
 
 <graphics type='vnc' port='-1' autoport='yes' listen='192.168.1.5' passwd='YOUR-PASSWORD-HERE' keymap='en-us'/>
 
Where,
  • type='vnc': The graphics element has a mandatory type attribute which takes the value "sdl", "vnc", "rdp" or "desktop". In this case it is set to VNC for remote access.
  • autoport='yes': The autoport attribute is the new preferred syntax for indicating autoallocation of the TCP port to use.
  • passwd='YOUR-PASSWORD-HERE': The passwd attribute provides a VNC password in clear text.
  • keymap='en-us': The keymap attribute specifies the keymap to use.
  • listen='192.168.1.5': The listen attribute is an IP address for the server to listen on.
Save and close the file. Restart services as follows:
# /etc/init.d/libvirtd restart
# virsh start centos1

No comments:

Post a Comment