Tag Archives: cloud

Storage performance difference between KVM and Xen

Posted on by | 1 Reply

Several blogs and manuals with examples on kvm or xen setups use NFS as storage backend. Mostly they state that for production use iSCSI is recommended. However there are examples where NFS is part of the architecture, eg. OpenNebula. I tried to find specific statistics on the performance differences between NFS, iSCSI and local storage. During this search I encountered some pointers that NFS and Xen is not a good combination, but never a straight comparison.

I decided to invest some time and setup a small test environment and run some bonnie++ statistics. This is not a scientific designed experiment, but a test to show the differences between the platforms. Two test platforms are setup, 1 with a Xen server (DL360G6) (xen1) and a 12 disk SATA storage server (storage1), and another with a KVM server (DL360G5) (kvm1) and a 2 disk SATA storage server (storage2) . Both servers are connected with a gigabit network. I’ve also run a test with a 100mb/s network between the kvm1 and storage2 server. For reference I’ve also done tests with the images on localdisk.

I realize that LVM and iSCSI storage is most efficient, but storage with image files is very convenient and in case of cloud setups sometimes the only option.

Seq output Seq input Random
Per Chr Block Rewrite Per Chr Block Seeks
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Xen-guest-via-nfs-tapaio 1G 3570 5 2436 0 1366 0 26474 41 24831 0 6719.0 1
xen-guest-via-iscsi 1G 25242 40 12071 1 15175 0 32071 42 47742 0 7331.3 1
kvm-guest-nfs-1gb-net 1G 8140 16 17308 3 11864 2 40861 81 71711 3 2126.6 54
kvm-guest-nfs-qcow-100mb 1G 1922 3 9874 1 3994 0 10720 22 10441 0 595.4 33
kvm-guest-nfs-qcow-100mb-2nd 1G 9735 21 2039 0 3197 0 10729 22 10463 0 685.3 38
kvm-guest-nfs-qcow-100mb-3rd 1G 5327 10 7378 1 4421 0 10655 18 10512 0 706.3 39
xenserver-nfsmount 1G 41507 60 60921 7 29687 1 33427 48 64147 0 4674.4 11
kvmserver-nfs-1G 20G 31158 52 32044 17 10749 2 19152 28 18987 1 90.3 1
localdisk-on-nfs-server-cloudtest3 4G 41926 65 43805 7 18928 3 52943 72 56616 3 222.6 0

The  conclusion of the tests is that local storage is fastest. NFS storage with Xen is not a good combination. Xen runs best with iSCSI backed storage. KVM with NFS runs significantly better. It is safe to say that if you want to use NFS use it with KVM, not with Xen. In any case iSCSI is always the best option for Xen. I have not yet tested KVM with iSCSI but I expect this to perform better than NFS.

OpenNebula EC2 error

Posted on by | Leave a reply

While experimenting with OpenNebula and trying to build a public cloud with the EC2 interface to OpenNebula I encountered the following problem in the code:

[rogierm@cloudtest3 one]$ econe-upload /home/rogierm/test.img
/usr/lib/ruby/1.8/rdoc/ri/ri_options.rb:53: uninitialized constant RI::Paths (NameError)
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/lib/ruby/1.8/rdoc/usage.rb:72
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/local/one/bin/econe-upload:61

I fixed this problem by adding the following line (above the other require statements) in econe-upload, or any other command giving the same error:

require 'rdoc/ri/ri_paths'