Johan De Gelas

 

When I first heard about Intel’s VT-x and AMD’s SVM technology I expected to see performance improvements over the software based solutions such as Binary Translation and Paravirtualization. Both AMD and Intel gave the impression that they were about to “enhance” and “accelerate” the current purely software based solutions.

What AMD and Intel did was extend x86 to make “Classic Virtualization” possible, very similar to the old IBM mainframe virtualization. In other words, hardware virtualization support does not really “enhance” Binary Translation or Paravirtualization; it is a completely different approach. First generation hardware virtualization was even a step back from a performance view, but one that enabled many steps forward. The first generation of virtualization has been improved, and is now adopted by VMware to support 64-bit guest OSes and by Xen to run unmodified OSes (such as Windows). So right now, hardware virtualization still has a long way to go while software virtualization is mature and represents the current standard.

Second generation virtualization (VT-x+EPT and AMD-V+NPT) is more promising, but while it can improve performance significantly it is not guaranteed that it will improve performance across all applications due to the heavy TLB miss cost. On the flip side of the coin: software virtualization is very mature, but there is very little headroom left to improve. The smartest way is to use a hybrid approach, and that is exactly what VMware, Xen, and Microsoft have been doing. Read More »

Paravirtualization is not that different from Binary Translation. BT changes “critical” or “dangerous” code into harmless code on the fly; paravirtualization does the same thing, but in the source code. Of course, changing the source code allows a bit more flexibility than changing everything on the fly, which has to happen quickly. One advantage is that paravirtualization eliminates a lot of unnecessary “traps” by the VMM (or Hypervisor), even more than BT.

The hypervisor provides hypercall interfaces for critical kernel operations such as memory management, interrupt handling, and time keeping. These hypercalls will only happen when it is necessary. For example, most of the memory management is done by the different guest OSes. The Hypervisor will only be “called” for things like page table updates and DMA accesses.

The best feature of the Xen implementation of virtualization is the way I/O is handled. I/O devices in the VM are just simplified interfaces that link to real native drivers in a privileged VM (called Domain 0 in Xen). This means there is no emulation involved, and the overhead is significantly reduced. That this is more than Xen propaganda is illustrated by VMware ESX: while VMs running on early ESX versions had rather low network performance, VMs running on ESX 3.x have very acceptable network performance thanks to a cleverly implemented paravirtualized vmxnet network driver. Read More »

 

Virtualization Challenges

The grandfathers of virtualization, such as the IBM S/370, used a very robust system to allow the hypervisor to control the virtual machines. Every privileged instruction by a virtual machine caused a “trap”, an error, as it was trying to execute a “resource management” instruction while running in a less privileged ring. The VMM intercepts all those traps and emulates the instruction, without jeopardizing the integrity of the other guests. In order to improve performance, the developers of the guest OS and VMM (both at IBM) tried to minimize the number of traps and reduce the time required to take care of the various traps.

This kind of virtualization was not possible on x86 as the 32/64-bit Intel ISA does not trap every incident that should lead to VMM intervention. One example is the POPF instruction that disables and enables interrupts. The problem is that if this instruction is executed by a guest OS in ring 1, an x86 CPU does not make a fuss about it, but simply ignores it. The result is that if a guest OS is trying to disable an interrupt, the interrupt is not disabled at all, and the VMM has no way of knowing that this is happening. As always, the good old x86 ISA is a bit chaotic: it has 17 of these “non-interceptable, cloaked for the VMM” instructions ^[1]. The conclusion is that x86 cannot be virtualized the way that the old mainframes were virtualized. Incidentally, the PowerPC and Alpha ISA’s are clean enough to be virtualized in the classic manner.

The above is much more than a quick simplified history lesson. Keep this in mind when we discuss what Intel and AMD have been doing with VT-x and AMD-V. Read More »

First dual-core in 2005, then quad-core in 2007: the multi-core snowball is rolling. The desktop market is still trying to find out how to wield all this power; meanwhile, the server market is eagerly awaiting the octal-cores in 2009. The difference is that the server market has a real killer application, hungry for all that CPU power: virtualization.

While a lot has been written about the opportunities that virtualization brings (consolidation, hosting legacy applications, resource balancing, faster provisioning…), most publications about virtualization are rather vague about the “nuts and bolts”.

Performance? Isn’t that a non-issue? Modern virtualization solutions surely do not lose more than a few percent in performance, right? The answer is quite a bit different from what some of the sponsored white papers want you to believe. Read More »