Friday, December 6, 2013

openssl

SSL Certificates HOWTO http://www.tldp.org/HOWTO/SSL-Certificates-HOWTO/index.html
http://www.freebsdmadeeasy.com/tutorials/freebsd/create-a-ca-with-openssl.php
http://wiki.samat.org/CheatSheet/OpenSSL
http://pages.cs.wisc.edu/~zmiller/ca-howto/

Files:
  • Certificate Signing Request (.csr)
  • Base64-encoded X.509 Certificate (.cer or .crt)
  • DER-encoded binary X.509 Certificate (.cer, .der or .crt)
  • Cryptographic Message Syntax Standard (PKCS#7) Certificate (.p7b, .p7r or .spc)
  • Personal Information Exchange Format (PKCS#12 Certificate (.pfx or .p12)
    PKCS#12 is the only format what can be used to export a certificate and its private key
  • Certificate Revocation List (.crl)
  • Microsoft serialized certificate store (.sst)
  • Certificate Trust List (.stl)
  • Privacy-enhanced Electronic Mail (.pem)
    a refinement of base64 encoding
  • Private Key (.key)

Generating Client/Server certificates with a local CA
*make sure openssl points to the correct instillation (%which openssl).  Mine is aliased to /usr/local/openssl/bin/openssl
Generate a CA
1)    openssl req -out ca.pem -new -x509
        -generates CA file "ca.pem" and CA key "privkey.pem"
Generate server certificate/key pair
        - no password required.
2)    openssl genrsa -out server.key 1024
3)    openssl req -key server.key -new -out server.req
4)    openssl x509 -req -in server.req -CA CA.pem -CAkey privkey.pem -CAserial file.srl -out server.pem
        -contents of "file.srl" is a two digit number.  eg. "00"
Generate client certificate/key pair
5)    Either choose to encrypt the key(a) or not(b)
        a. Encrypt the client key with a passphrase
            openssl genrsa -des3 -out client.key 1024
        b. Don't encrypt the client key
            openssl genrsa -out client.key 1024
6)    openssl req -key client.key -new -out client.req
7)    openssl x509 -req -in client.req -CA CA.pem -CAkey privkey.pem -CAserial file.srl -out client.pem
        -contents of "file.srl" is a two digit number.  eg. "00"

Check certificate content:
# openssl x509 -text -in /var/lib/puppet/ssl/certs/puppet.pem
# openssl crl -text -in /var/lib/puppet/ssl/ca/ca_crl.pem
Simulate SSL connection:
# openssl s_client -host puppet -port 8140 -cert /path/to/ssl/certs/node.domain.com.pem -key /path/to/ssl/private_keys/node.domain.com.pem -CAfile /path/to/ssl/certs/ca.pem
-
# ssldump
-

General OpenSSL Commands


These commands allow you to generate CSRs, Certificates, Private Keys and do other miscellaneous tasks.

  • Generate a new private key and Certificate Signing Request
    openssl req -out CSR.csr -new -newkey rsa:2048 -nodes -keyout privateKey.key
  • Generate a self-signed certificate (see How to Create and Install an Apache Self Signed Certificate for more info)
    openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt
  • Generate a certificate signing request (CSR) for an existing private key
    openssl req -out CSR.csr -key privateKey.key -new
  • Generate a certificate signing request based on an existing certificate
    openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey privateKey.key
  • Remove a passphrase from a private key
    openssl rsa -in privateKey.pem -out newPrivateKey.pem

Checking Using OpenSSL


If you need to check the information within a Certificate, CSR or Private Key, use these commands. You can also check CSRs and check certificates using our online tools.

  • Check a Certificate Signing Request (CSR)
    openssl req -text -noout -verify -in CSR.csr
  • Check a private key
    openssl rsa -in privateKey.key -check
  • Check a certificate
    openssl x509 -in certificate.crt -text -noout
  • Check a PKCS#12 file (.pfx or .p12)
    openssl pkcs12 -info -in keyStore.p12

Debugging Using OpenSSL


If you are receiving an error that the private doesn't match the certificate or that a certificate that you installed to a site is not trusted, try one of these commands. If you are trying to verify that an SSL certificate is installed correctly, be sure to check out the SSL Checker.

  • Check an MD5 hash of the public key to ensure that it matches with what is in a CSR or private key
    openssl x509 -noout -modulus -in certificate.crt | openssl md5
    openssl rsa -noout -modulus -in privateKey.key | openssl md5
    openssl req -noout -modulus -in CSR.csr | openssl md5
  • Check an SSL connection. All the certificates (including Intermediates) should be displayed
    openssl s_client -connect www.paypal.com:443

Converting Using OpenSSL


These commands allow you to convert certificates and keys to different formats to make them compatible with specific types of servers or software. For example, you can convert a normal PEM file that would work with Apache to a PFX (PKCS#12) file and use it with Tomcat or IIS. Use our SSL Converter to convert certificates without messing with OpenSSL.

  • Convert a DER file (.crt .cer .der) to PEM
    openssl x509 -inform der -in certificate.cer -out certificate.pem
  • Convert a PEM file to DER
    openssl x509 -outform der -in certificate.pem -out certificate.der
  • Convert a PKCS#12 file (.pfx .p12) containing a private key and certificates to PEM
    openssl pkcs12 -in keyStore.pfx -out keyStore.pem -nodes
    You can add -nocerts to only output the private key or add -nokeys to only output the certificates.
  • Convert a PEM certificate file and a private key to PKCS#12 (.pfx .p12)
    openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile CACert.crt

puppet

http://puppetlabs.com/puppet/puppet-enterprise
Puppet SSL Explained http://www.masterzen.fr/2010/11/14/puppet-ssl-explained/
replace puppet ca cert http://blkperl.github.io/replace-puppet-ca.html

- clean certificate for client, regenerate cert
. on puppet master  # puppet cert --clean {node certname} 
. on client # rm -r etc/puppet/ssl; rm -r /var/lib/puppet/ssl
. regenerate on client by restart puppet on clean
. on master, check there is new cert wait to be signed:  # puppetca --list 
. on master, sign the requested cert:  # puppetca --sign <node-name> 
- Re-generate the puppet master’s certificate
. clean the old cert
  # puppetca --clean "$(puppet master --configprint certname)"
  or
  # find $(puppet master --configprint ssldir) -name "$(puppet master --configprint certname).pem" -delete
. Start a non-daemonized WEBrick puppet master instance, and wait for it to generate and sign a new certificate:
  # puppet master --no-daemonize --verbose
or  # puppet master --verbose --no-daemonize --cert_name "Puppet CA: `hostname -f`"
  stop the temporary puppet master with ctrl-C after see the “notice: Starting Puppet master version 2.6.9” message.
. restart puppet master
simplest way is just remove /var/lib/puppet/ssl directory and restart puppetmaster )

- manually verifying the certificate with openssl
# openssl verify -CAfile /etc/puppet/ssl/certs/ca.pem /etc/puppet/ssl/certs/myhostname.domain.com.pem
- look at the certificate content
# openssl x509 -text -noout -in /var/lib/puppet/ssl/certs/puppet.pem
- to see the puppet master’s certified hostnames, ssldir
# puppet master --configprint certname
# puppet master --configprint ssldir
# puppet master --configprint manifest
# puppet master --configprint modulepath
- fingerprinting
. on the node
puppet agent --test --fingerprint
. on master
puppetca --list node.domain.com --fingerprint
-

Friday, November 1, 2013

Expand a Xen virtual disk using LVM

http://www.craigdunn.org/2010/06/howto-expand-a-xen-virtual-disk-using-lvm/

HOWTO: Expand a Xen virtual disk using LVM

So you have a Citrix Xen VM with a Linux filesystem partitioned using LVM and need to expand the size of the root partition? – This is a quick how to that is fairly well tested.
Firstly, I recommend that you backup your data!
In this example, we’ll be adding another 5G to a current 6G logical partition, giving us 11G in total. In this example, our set ups include an extra 2G logical partition so the disk size in total will be 13G.


1 ) Shut down the VM


2 ) Identify the VM and the disk VDI


xenhost# xe vm-disk-list vm="vmhost"
Disk 0 VBD:
uuid ( RO)             : 941baebf-ca8d-0626-f088-30d314433287
    vm-name-label ( RO): vmhost
       userdevice ( RW): 0
 
Disk 0 VDI:
uuid ( RO)             : 03218b6d-dbb1-4181-aa67-5273938af532
       name-label ( RW): 0
    sr-name-label ( RO): TEMPORARY
     virtual-size ( RO): 8589934592
Now resize the VDI from the Xen Center command line:
xenhost# xe vdi-resize uuid=03218b6d-dbb1-4181-aa67-5273938af532 disk-size=13GiB

3 ) Start the VM, login and become root


4 ) Run fdisk and re-create the partition


You will need to identify the root LVM partition in fdisk, remove it, and create it using the largest block size available.
In this example, our LVM partition is the second one (xvda2) – so we select d to delete, 2 to specify the partition, then n to create a new second partition using up to the last cylinder.
vmhost# fdisk /dev/xvda
 
The number of cylinders for this disk is set to 1697.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
 
Command (m for help): p
 
Disk /dev/xvda: 13.9 GB, 13958643712 bytes
255 heads, 63 sectors/track, 1697 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           1          13      104391   83  Linux
/dev/xvda2              14        1044     8281507+  8e  Linux LVM
 
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (14-1697, default 14):
Using default value 14
Last cylinder or +size or +sizeM or +sizeK (14-1697, default 1697):
Using default value 1697
 
Command (m for help): wq
The partition table has been altered!
 
Calling ioctl() to re-read partition table.
 
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
vmhost#
This WARNING error can safely be ignored.

5 ) Reboot the VM


6 ) Resize the LVM physical volume


You can use pvdisplay to see the current size of the volume:
vmhost# pvdisplay
  --- Physical volume ---
  PV Name               /dev/xvda2
  VG Name               VolGroup00
  PV Size               7.90 GB / not usable 23.41 MB
...snip...
Now run pvresize to grow the physical volume to cover the whole partition…
vmhost# pvresize /dev/xvda2
Now re-run pvdisplay and you should see a bigger volume
vmhost# pvdisplay
  --- Physical volume ---
  PV Name               /dev/xvda2
  VG Name               VolGroup00
  PV Size               12.90 GB / not usable 25.51 MB
...snip...

7 ) Resize the LVM logical volume


vmhost# lvresize /dev/VolGroup00/LogVol00 -L +5G
  Extending logical volume LogVol00 to 10.94 GB
  Logical volume LogVol00 successfully resized

8 ) Grow the file system


This example assumes you are running an ext3 filesystem, other filesystem types may vary – run resize2fs over the volume.
vmhost# resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 2867200 (4k) blocks.
The filesystem on /dev/VolGroup00/LogVol00 is now 2867200 blocks long.

Tada!

And that’s it, you should now see an extra 5G available in your root partition
vmhost# df -h /
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       <strong>11G</strong>  3.8G  6.3G  38% /

Wednesday, October 30, 2013

cloud renting

Google App Engine https://developers.google.com/appengine/docs/whatisgoogleappengine

http://www.rackspace.com/cloud/servers/
http://4sysops.com/archives/amazon-ec2-pricing-for-dummies-part-1-only-pay-for-what-you-use/
"Basically, there are four major factors to remember when it comes to EC2 price calculation: S3 storage, EBS storage, EC2 bandwidth, and EC2 instance type. In my case, the storage costs are negligible ($2), the bandwidth costs are significant ($15) and the EC2 instance costs are decisive ($82). "

cloud rate calculator
  1. In general, one database = one instance. You spin up instances, and do what you like with them. Definitely possible to have more connections to it.
  2. Hours per month is just that. How many hours per month you have the instance active. If you plan to have the instance active 24/7, you may find more cost effective alternatives with other cloud providers. If you run it less often than that, you save money when it's not active. It's billed hourly to your account at the rate specified.
  3. Upload data is counted at the standard transfer rates. A few GBs doesn't cost much, but you will be paying for the service starting the moment you spin up the instance.
"simple" monthly calculator http://calculator.s3.amazonaws.com/calc5.html
Database Service Pricing http://aws.amazon.com/rds/pricing/
http://bitnami.com/cloud-cost-calculator
There are a lot of nice UI calculators (newvem, trusted advisor, ylastic, etc) that will do it for you. Personally I developed one for myself (it is free for everybody) and use it to analyze AWS accounts of my clients to generate a cost report for various AWS services (EC2, RDS, EBS, ELB, S3, etc...).

http://stackoverflow.com/questions/158765/amazon-ec2-cost
Rackspace delivers a better service that Amazon AWS at a fraction of the cost. A basic Rackspace Cloud Server (dedicated only to you) costs around $11/mo and their customer service is astoundingly good. (For example, you can actually TALK to someone via phone or live chat, instead of having to post in community support forums. With amazon you have to subscribe to an annual service contract in order to talk to anyone, which costs around $250/year)
As of 2012, this answer is mostly incorrect. Both Rackspace and Amazon EC2 give you a virtual machine; so on both services you are sharing the physical hardware. For small deployments, Amazon is much cheaper than Rackspace because Amazon is FREE for the first year using micro instances. Even after that, for hosting a web server Amazon is still cheaper when you avail yourself of reserved instances/spot instances. Rackspace does have a more compelling support offering,
Although AWS seems to be inexpensive at the begining, but in reality it is very expensive.
After using AWS for some time, I discovered it is actually very expensive, you better rent one or more dedicated servers. which will give you more processing power and will cost much less.AWS costs will shoot in the sky if your website gets many visitors, serve many files and requests... these small tiny fractions that they charge sum up. Examples: last month I used AWS, I paid 10$ just for requests into S3, which means they count how many requests you made and sum them up and charge you for this... but you still have to pay for the actual bandwidth of these requests! which sounds crazy for me.. why do I have to pay for the requests if I already pay for bandwidth used & for storage costs?you also have to pay for IO requests! which means everytime you read from or write to the hard drive you they are adding up small fractions!these small fractions may look very small, but if your website is very busy, you will be surprised at the end of the monthCosts: instance + traffic /from/to instance + hard drive + requests to hard drive + S3 storage + S3 bandwidth + S3 requeststhe best thing to do is to use S3 as a storage, and server your files from your own servers.. this way you have to pay for storage and your requests + bandwith usage will be minimal....
If you are running a server 24x7 then buy reserved instances. For example a reserved instance of Linux small instance running 24x7 for 3 years will cost a total of $572.36. $257 for purchased of reserved instance then $8.64 per month until the 3 years runs out. An on-demand instance would cost you $43 per month, so reserved is more cost effective for 24x7 machines. (remember, this is a machine you have full control - run many websites - over like a personal box)
If you are doing dev tests, then use a Spot Instance or on-demand. An amazon Linux spot instance is 0.007 per hour right now. An on-demand instance is 0.06 per hour. A spot instance dies when it is turned off, an on-demand can be stopped and started. Spot instances are used by people who need to do short term work like web crawling or census data junk that lasts for a short time.
It costs almost nothing to play around with AWS so you get an idea of the costs. It costs me less than a dollar to play around for 3 days during work hours. I created on-demand machines, I created spot instances and ran them all night, I spent about 12 hours total. Its cheap to learn. You will learn more experimenting than you will asking questions in forums.

Friday, October 4, 2013

change the clock source in the system

  • Red Hat Enterprise Linux 4:
    • Checking the current clock source:
      • One can easily perform this task through searching in /var/log/dmesg file for "time":
        [root@localhost ~]# grep "time" /var/log/dmesg
        ...
        Using HPET for base-timer
        Using HPET for gettimeofday
        Using hpet for high-res timesource
        ...
        
    • Verifying documentation on time source kernel parameters:
      • In order to check what time source may be used with RHEL4 kernel, one should read the kernel documentation file /usr/share/doc/kernel-doc-2.6.9/Documentation/kernel-parameters.txt:
                clock=          [BUGS=IA-32, HW] gettimeofday timesource override.
                                Forces specified timesource (if available) to be used
                                when calculating gettimeofday(). If specified timesource
                                is not available, it defaults to PIT.
                                Format: { pit | tsc | cyclone | pmtmr }
        
                hpet=           [IA-32,HPET] option to disable HPET and use PIT.
                                Format: disable
        
                notsc           [BUGS=IA-32] Disable Time Stamp Counter
        
    • Overriding the clock source configuration:
      • adding clock= to the kernel stanza:
        title Red Hat Enterprise Linux AS (2.6.9-89.ELsmp)
                root (hd0,0)
                kernel /vmlinuz-2.6.9-89.ELsmp ro root=LABEL=/ rhgb quiet clock=pmtmr
                initrd /initrd-2.6.9-89.ELsmp.img
        
  • Red Hat Enterprise Linux 5:
    • Checking the current clock source:
      • Unfortunately, as the original snapshot of RHEL5 kernel was done at a time in which the actual linux-2.6 clocksource code was just being introduced for i386 and x86_64, it is not accurate to list the content of files into /sys/devices/system/clocksource/* directories in order to determine the current and the available system clock sources. Instead, to determine the clock source accurately, one should proceed the same as in RHEL4 way: searching in /var/log/dmesg file:
        [root@localhost ~]# grep "time" /var/log/dmesg
        ...
        time.c: Using 3.579545 MHz WALL PM GTOD PIT/TSC timer.
        time.c: Detected 3200.238 MHz processor.
        ...
        
    • Verifying clock source available kernel parameters:
      • In order to check what time source may be used with RHEL5 kernel, one should read the kernel documentation file /usr/share/doc/kernel-doc-2.6.18/Documentation/kernel-parameters.txt:
                clock=          [BUGS=IA-32, HW] gettimeofday clocksource override.
                                [Deprecated]
                                Forces specified clocksource (if available) to be used
                                when calculating gettimeofday(). If specified
                                clocksource is not available, it defaults to PIT.
                                Format: { pit | tsc | cyclone | acpi_pm | tsccount | notsccount }
        
                clocksource=    [GENERIC_TIME] Override the default clocksource
                                Override the default clocksource and use the clocksource
                                with the name specified.
        
                hpet=           [IA-32,HPET] option to disable HPET and use PIT.
                                Format: disable
        
                notsc           [BUGS=IA-32] Disable Time Stamp Counter
        
    • Overriding the clock source configuration:
      • adding clocksource= to the kernel stanza:
        title Red Hat Enterprise Linux (2.6.18-128.el5)
                root (hd0,0)
                kernel /vmlinuz-2.6.18-128.el5 ro root=LABEL=/ rhgb quiet clocksource=acpi_pm
                initrd /initrd-2.6.18-128.el5.img
        
  • Red Hat Enterprise Linux 6:
    • Checking the current clock source:
      • listing the /sys/devices/system/clocksource/*/current_clocksource file content:
        [root@localhost ~]# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
        hpet
        
    • Verifying available clock sources:
      • listing the /sys/devices/system/clocksource/*/available_clocksource file content:
        [root@localhost ~]# cat /sys/devices/system/clocksource/clocksource0/available_clocksource
        hpet acpi_pm
        
      • browsing the documentation file /usr/share/doc/kernel-doc-2.6.32/Documentation/kernel-parameters.txt:
                clocksource=    [GENERIC_TIME] Override the default clocksource
                                Format:                         Override the default clocksource and use the clocksource
                                with the name specified.
                                Some clocksource names to choose from, depending on
                                the platform:
                                [all] jiffies (this is the base, fallback clocksource)
                                [ACPI] acpi_pm
                                [ARM] imx_timer1,OSTS,netx_timer,mpu_timer2,
                                        pxa_timer,timer3,32k_counter,timer0_1
                                [AVR32] avr32
                                [X86-32] pit,hpet,tsc,vmi-timer;
                                        scx200_hrt on Geode; cyclone on IBM x440
                                [MIPS] MIPS
                                [PARISC] cr16
                                [S390] tod
                                [SH] SuperH
                                [SPARC64] tick
                                [X86-64] hpet,tsc
        
                hpet=           [X86-32,HPET] option to control HPET usage
                                Format: { enable (default) | disable | force |
                                        verbose }
                                disable: disable HPET and use PIT instead
                                force: allow force enabled of undocumented chips (ICH4,
                                        VIA, nVidia)
                                verbose: show contents of HPET registers during setup
        
                notsc           [BUGS=X86-32] Disable Time Stamp Counter
        
    • Overriding the clock source configuration:
      • Dynamically and temporarily (not persistent to reboots):
        [root@localhost ~]# echo "acpi_pm" > /sys/devices/system/clocksource/clocksource0/current_clocksource
        
      • Permanently, adding clocksource= to the kernel stanza:
        title Red Hat Enterprise Linux Server (2.6.32-71.18.2.el6.x86_64)
                root (hd0,0)
                kernel /vmlinuz-2.6.32-71.18.2.el6.x86_64 ro root=LABEL=/ crashkernel=auto clocksource=acpi_pm
                initrd /initramfs-2.6.32-71.18.2.el6.x86_64.img
        
  • An overview on hardware clock and system timer circuits:
    When it comes to talk about a system's clock, the hardware sits at the very bottom. Every typical system has several devices, usually implemented by clock chips, that provide timing features and can serve as clocks. So, which hardware is available depends on the particular architecture. The clock circuits are used both to keep track of the current time of the day and to make precise time measurements. The timer circuits are programmed by the kernel,so they issue interrupts at a fixed, and predefined, frequency. For instance, IA-32 and AMD64 systems have at least one programmable interrupt timer (PIT) as a classical timer circuit, which is usually implemented by an 8254 CMOS chip. Let's briefly describe the clock and timer circuits that are usually found with any nearly modern system of those architectures:
    • Real Time Clock (RTC): The RTC is independent of the system's CPU and any other chips. As it is energized by a small battery, it continues to tick even when the system is switched off. The RTC is capable of issuing interrupts at frequencies ranging between 2Hz and 8,192Hz. Linux uses the RTC only to derive the time and date at boot time.
    • Programmable Interrupt Timer (PIT): The PIT is a time-measuring device that can be compared to the alarm clock of a microwave oven: it makes the user aware that the cooking time interval has elapsed. Instead of ringing a bell, the PIT issues a special interrupt called timer interrupt, which notifies the kernel that one more time interval has elapsed. As the time goes by, the PIT goes on issuing interrupts forever at some fixed (architecture-specific) frequency established by the kernel.
    • Time Stamp Counter (TSC): All 80x86 microprocessors include a CLK input pin, which receives the clock signal of an external oscillator. Starting with the Pentium, 80x86 microprocessors sport a counter that is increased at each clock signal, and is accessible through the TSC register which can be read by means of the rdtsc assembly instruction. When using this register the kernel has to take into consideration the frequency of the clock signal: if, for instance, the clock ticks at 1GHz, the TSC is increased once every nanosecond. Linux may take advantage of this register to get much more accurate time measurements.
    • CPU Local Timer: The Local APIC (Advanced Programmable Interrupt Controller) present in recent 80x86 microprocessors provide yet another time measuring device, and it is a device, similar to the PIT, which can issue one-shot or periodic interrupts. There are, however, a few differences:
      • The APIC's timer counter is 32bit long, while the PIT's timer counter is 16bits long;
      • The local APIC timer sends interrupts only to its processor, while the PIT raises a global interrupt, which may be handled by any CPU in the system;
      • The APIC's timer is based on the bus clock signal, and it can be programmed in such way to decrease the timer counter every 1, 2, 4, 8, 16, 32, 64, or 128 bus clock signals. Conversely, the PIT, which makes use of its own clock signals, can be programmed in a more flexible way.
    • High Precision Event Timer (HPET): The HPET is a timer chip that in some future time is expected to completely replace the PIT. It provides a number of hardware timers that can be exploited by the kernel. Basically the chip includes up to eight 32bit or 64bit independent counters. Each counter is driven by its own clock signal, whose frequency must be at least 10MHz; therefore the counter is increased at least once in 100 nanoseconds. Any counter is associated with at most 32 timers, each of which composed by a comparator and a match register. The HPET registers allow the kernel to read and write the values of the counters and of the match registers, to program one-shot interrupts, and to enable or disable periodic interrupts on the timers that support them.
    • ACPI Power Management Timer (ACPI PMT): The ACPI PMT is another clock device included in almost all ACPI-based motherboards. Its clock signal has a fixed frequency of roughly 3.58MHz. The device is a simple counter increased at each clock tick. However the ACPI PMT is preferable to the TSC if the operating system or the BIOS may dynamically lower the CPU's frequency or voltage. When this happens, TSC's frequency changes causing time warps and others side-effects, while the frequency of ACPI PMT does not.

  • Xen Problem

    XenServer is headache http://sysconfig.org.uk/2011/01/so-long-xenserver/

    - clocksource problem https://wiki.debian.org/Xen/Clocksource

    When running no-citrix kernel, sometime running to clocksource problem.
    The clocksource problem appears after you suspend a Linux host and the kernel in the virtual machine starts spewing this:
    Mar  5 09:24:17 co kernel: [461562.007153] clocksource/0: Time went backwards: ret=f03d318c7db9 delta=-200458290723043 shadow=f03d1d566f4a offset=14
    It turns out that the Citrix kernel uses jiffies as the clocksource per default, while Debian uses the xen clocksource.
    So by just running this on the domU fixes the problem:
    echo "jiffies"> /sys/devices/system/clocksource/clocksource0/current_clocksource
    
    There is no need to decouple the clock from the host. To make this change permanent, you need to addclocksource=jiffies to the bootparameters of your domU kernel.

    For fully virtualized (HVM) instances
    To find your current clocksource
    cat /sys/devices/system/clocksource/clocksource0/current_clocksource
    To see available clocksources
    cat /sys/devices/system/clocksource/clocksource0/available_clocksource

    For para-virtualized (PV) instances,
    echo 1 > /proc/sys/xen/independent_wallclock
    In order to ensure the setting sticks on boot, you should also issue a:
    echo "xen.independent_wallclock = 1" >> /etc/sysctl.conf

    Friday, September 20, 2013

    Script XenServer Backups

    http://technocrat.kagan.cc/2013/08/script-xenserver-backups.html

    After upgrading to Citrix XenServer 6.1 Advanced Edition a while ago, I have been enjoying great features, such as "High-Availability" and "VM Protection Policies". I especially liked VM protection with automated snapshots and exports that took practically no time to set up and worked fairly well for a number of small to midsize environments we have partitioned our virtual infrastructure into. Well, my enjoyment proved to be short-lived, XenServer 6.2 came along delivering scalability improvements and also introducing many changes, click here for more information.

    I found many of those changes (like XenServer being fully open source; inclusion of software maintenance; simple, single edition packaging; socket based perpetual and annual licensing; etc.) interesting and welcomed them with open arms. All but one:
    This spelled the end of my automated protection days (once again, read more about it here). Luckily, I was still able to draw on my by-now-virtually-nonexistent Linux skills (yes, with a lot of help from plentiful online resources) to replace automation magic with scripting and scheduling.

    In case someone else finds this helpful and to give credit where it's due, here's a collection of resources:


    • Great post that provides step-by-step instructions and script samples for pretty much everything you are going to need to get started - http://www.charleslabri.com/back-up-xenserver-6-to-network-share-with-fancy-scripting-and-lots-of-fun-and-no-downtime/
    • Email notifications required a quick modification of the "/etc/ssmtp/revaliases" file (a line similar to root:username@xyz.com:smtp.xyz.com); more details here
    • To get things scheduled right refer to one of many resources that describe cron, for example this one
    • To delete old backup files a quick refresher on the find command might be needed, check out the following reference or this two part article - part 1, part 2
    • Finally, if you run Windows based VMs and want to use VSS (seems to work well under W2K8 not so much under W2012), you may want to look into running "C:\Program Files\Citrix\XenTools\install-XenProvider.cmd" to install VSS provider
    That's it. I have used information above to schedule jobs that take VM snapshots, export them to NFS-based shares, and keep a set number of them. So far, everything works pretty well.

    * http://nchrissos.wordpress.com/2013/09/05/free-backup-for-citrix-xenserver-live-virtual-machines/
    http://www.charleslabri.com/back-up-xenserver-6-to-network-share-with-fancy-scripting-and-lots-of-fun-and-no-downtime/
    http://www.howtogeek.com/131181/how-to-backup-citrix-xen-vms-for-free-with-xen-pocalypse-bash/

    Wednesday, August 21, 2013

    password recovery

    Follow the below steps to reset the StoreVault root password:

    1. Connect the serial cable between StoreVault and PC
    2. Reboot the system
    3. Access StoreVault boot menu via serial port using Hyper Terminal application with default setting(BPS-9600_DB-8_P-None-SB-1_FC-None)
    3. The StoreVault begins the boot process.
    4. While booting press Ctrl-C to display the boot menu.
    Boot menu lists following option:
         1) Normal Boot
         2) Boot without /etc/rc
         3) Change Password
         4) Initialize all disks
         4a) Same as option 4, but create a flexible root volume.
         5) Maintenance mode boot
    5. Select option 3 to change the password and reboot the system.

    Note: Do not try with other boot menu options, it may erase all your data.

    Tuesday, August 20, 2013

    day-to-day issues

    08.20.2013
    - upgrade from XenServer 6.1 to XenServer 6.2, redhat vm stop working. Could not find kernel-xen packages update as noted in XenServer 6.2 released notes, known issues.
    Booted to recover, changing referred partition from /dev/sda# to /dev/xvda# in fstab solved the problem.

    10.04.2013
    - same as 08.20.2013 issue. A RHEL6.4_6x from XS6.1 would not run XS6.2.  Run "grub-install /dev/xvda" got error:
    expr: non-numberic argument
    expr: non-numberic argument
    The file /boot/grub/stage1 not read correctly.
    • Have tried to change /boot from ext4 to ext3. not working
    • Change kernel version, not working
    Finally, these made it work:

    # grub
    grub> device (hd0) /dev/xvda
    grub> root (hd0,0)
    grub> setup (hd0)
    grub> quit

    04.23.2014
    - after increase memory of VM, the VM could not start. log shows "The given VMs failed to release memory when instructed to do so"
    The host is running out of memory. Try to shutdown some VM or reduce the memory of VMs resolve the problem.

    04.23.2014
    - when copy a big (~4G) file, the VM went into a quick reboot (about 14 seconds).

    /var/log/messages does not show any errors, except reboot:
    Apr 23 11:31:21 dev2 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: No such file or directory
    Apr 23 11:35:02 dev2 kernel: imklog 5.8.10, log source = /proc/kmsg started.
    Apr 23 11:35:02 dev2 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1169" x-info="http://www.rsyslog.com"] start
    Apr 23 11:35:02 dev2 kernel: Initializing cgroup subsys cpuset
    Apr 23 11:35:02 dev2 kernel: Initializing cgroup subsys cpu
    .....
    Apr 23 11:35:03 dev2 ntpd[1382]: frequency initialized -5.758 PPM from /var/lib/ntp/drift
    Apr 23 11:35:04 dev2 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: No such file or directory
    


    05.13.2014
    linux switch to read only file system
    http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=51306

    As a workaround, remount the Linux file system using this command to return it to the proper state:
    # mount -o remount /

    possible solution: disable journaling
    # Create ext4 fs on /dev/sda10 disk
    mkfs.ext4 /dev/sda10

    # Enable writeback mode. This mode will typically provide the best ext4 performance.
    tune2fs -o journal_data_writeback /dev/sda10

    # Delete has_journal option, This step disables journal
    tune2fs -O ^has_journal /dev/sda10
    # Or when creating
    mkfs.ext4 -O ^has_journal /dev/sda10

    # Required fsck
    e2fsck -f /dev/sda10

    # Check fs options, if has_journal option exist - you have journal
    dumpe2fs /dev/sda10 | grep 'Filesystem features'

    For more performance add fstab opions: data=writeback,noatime, 
    i.e: /dev/sda10 /opt ext4 defaults,data=writeback,noatime 0 0

    "discard" will enable trimming, discard options is default in ext4
    Adding discard is a terrible choice if your goal is performance. It is also meaningless for non-flash drives. If you have a flash drive, and performance is the goal, schedule a nightly fstrim cron job on the relevant partitions.
    Note: Using the discard flag for an ext3 root partition will result in it being mounted read-only.
    Warning: Users need to be certain that their SSD supports TRIM before attempting to mount a partition with the discard flag. Data loss can occur otherwise!

    07.22.2014
    VM in XenServer 6.2 cannot reboot. Showing error "The tapdisk failed".
    tried these, did not work.

    1. From XenServer, "pvscan" to find out VG_XenStorage- ,
    2. # vgchange -ay
    3. to see if your Logical Volume Manage group has busted VHDs, making sure no errors appear or reports of bad headers/footers
      # vhd-util scan -m "VHD*" -f -c -l VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b -p -v
    4. scan your storage repository and see if any errors appear
      # xe sr-scan uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b
    5. The last item you can do - on a live system - but this is the big one and a reference article can be found here (http://blogs.citrix..../xs-mgt-volume/).
      # lvrename /dev/VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b/MGT /dev/VG_XenStorage-5f5684ac-8561-e047-1ada-ee9bdba25a2b/MGTbak
      # lvchange -ay
      From XenCenter, select your storage repository and under Storage, select SCAN SR.
    6. Un-plug and plug storage. All VMs off
      # xe pbd-list sr-uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b
      xe pbd-unplug uuid=b8ba43c6-ded0-4f60-aa97-d27fd7daf8cf
      Wait a moment and the run the following to re-attach your storage:
      # xe pbd-plug uuid=b8ba43c6-ded0-4f60-aa97-d27fd7daf8cf
      Now, we complete with an sr scan:
      # xe sr-scan uuid=5f5684ac-8561-e047-1ada-ee9bdba25a2b
    possible cause: too many vm on the host that it cannot allocate memory for the vm.

    07.24.2014
    VM console not responding. reset VNC console
    live migrate to another host in the pool then migrate back, or
    1. identify <dom-id> for the frozen VM
    2. stop the current running console:
      # kill `ps aux|grep domain/<dom-id>|grep -v grep|awk '{print $2}'`
    3. restart VNC console:
      # /usr/lib/xen/bin/vncterm -v 127.0.0.1:1 -x /local/domain/<dom-id>/serial/0 &