Xen vs. KVM: Ending the Debate

Based on the reader comments, I apparently pushed some hot buttons in my last post.  Basically, I stated that by encouraging Linux developers to focus on KVM, the Linux Foundation is taking a somewhat small-minded view of virtualization.  Of course, I expected there to be some reaction from the Linux community to a statement like that.  But needling the Linux fan base was not my intent, yet it was evident from some of the comments that my post was unintentionally interpreted as an attack on KVM and Linux.  Ironically, it was the whole “versus” game between Xen and KVM that I was criticizing to begin with!  In this post, I’ll expound on my position and hopefully clarify the key takeaway point.  But first, I must address a few common technical inaccuracies that crept into the reader comments:

1. ESX employs full virtualization, which fully emulates the hardware.  Xen and Hyper-V employ paravirtualization and do not fully emulate hardware.  KVM employs limited support for paravirtualization through special network and memory drivers. 
2. Xen, Hyper-V and ESX are type-1 virtual machine monitors (VMMs)/hypervisors.  The hypervisor runs in Ring 0, while the parent partition (dom0) and the child partitions (domU, guest VMs) run in Ring 1.  KVM, Virtual Server, VirtualBox, Parallels, VMware Workstation, etc., are type-2 VMMs.  The VMM runs in Ring 3 as an application.  However, through it’s limited paravirtualization, KVM is able to achieve performance closer to that of a type-1 hypervisor when compared to typical type-2 VMMs.
3. Hyper-V does not require installing an entire instance of Windows Server 2008.  Selecting the Hyper-V role essentially installs a stripped-down version of Windows Server or Server Core in the parent partition (dom0).

Why do these architectural differences matter?  For starters, type-1 VMMs generally perform much better than their type-2 counterparts.  One reader who commented on my last post said that in his own tests, KVM performed better than Xen and ESX.  He then admits in the next sentence that he doesn’t like Xen much anyway.  So, obviously, I’m not buying it.  Even if his testing was valid, I’m sure the better performance was only with Linux guest VMs.  And how about stability?  With a type-1 VMM, if one guest VM crashes, it should not affect the other VMs.  But with a type-2 VMM in which the host potentially is running other application workloads alongside the VMM, the failure of one application could impact the VMM.  Now, you can say that the Linux KVM host can be stripped down to basically only run virtualization workloads.  But, if you’re going to strip down your Linux host to make it function more like a type-1 VMM, why not just run a type-1 VMM?

But, let’s assume that the technology improves and that KVM performance and stability become near that of a type-1 VMM.  The issue is bigger than performance, though.  It’s more about the mentality.  The commoditization of hypervisors mean that they should be treated almost like hardware.  When you buy servers, you buy them with the expectation that you can run a desired range of operating systems (or hypervisors) on them.  It doesn’t matter whether they’re running Intel or AMD, whether they’re from Dell or HP, etc.  Same goes with hypervisors (though the CPU does matter to an extent when migrating VMs between hosts). 

One reader suggested that more and more functionality will be added to hypervisors so that they become more like full-fledged operating systems.  But the goal is not to move hypervisors up the stack by making them more like full operating systems.  If the hypervisor natively can run applications, then is it really still virtualizing?  It will just be an interface between the app and the hardware, which is what an operating system is.  Rather, hypervisors are moving down the stack and becoming closer to the hardware, as evident by the work Intel and Phoenix are doing to add virtualization instructions to chipsets and BIOS respectively.

The point is, enterprises run various operating system and OS workloads, and like the hardware, the hypervisor should support those workloads equally and indifferently.  If my goal is to virtualize all those OS workloads, why would I want to install an operating system on bare-metal hardware just so it can host those OS workloads on top of it?  Why use an operating system to host operating system workloads when I can use a type-1 VMM?  And shouldn’t the VMM sit at a lower layer than the operating system?

Of course, part of the Linux Foundation’s job is to promote Linux, and I have no problem with that.  As reader Bill Murray pointed out, there definitely are use cases for KVM, but by and large, those use cases are going to be predominately around hosting Linux workloads.  Do you really see shops hosting Windows workloads on KVM?  So for the Linux Foundation to blanketly encourage developers to focus on KVM just keeps the Xen-vs.-KVM debate going, which you think the Foundation would avoid considering the whole Xen and kernel-inclusion debacle.  Furthermore, it just seems out of touch.  Enterprises already are standardizing around Xen, ESX and Hyper-V.  Sure, the Linux Foundation can promote KVM where it’s appropriate, but shouldn’t it also be encouraging developers to improve Linux’s performance and support on those platforms?  Novell understands this, and partnered with Microsoft to support SUSE Linux workloads on Hyper-V.  Conversely, Microsoft is working to improve support of Windows workloads on Novell’s Xen-based virtualization platform. 

The bottom line is that none of the Big 3 hypervisors are going away anytime soon.  Developers can accept this and support these platforms and the enterprise customers that have adopted them.  Or, they can live in the niche.

Follow Me on Twitter