One of the themes that I touched on in my recent webcast is this idea that virtualization represents a tradeoff of simplicity for efficiency. Specifically, I mentioned the complexity added to the software stack when virtualization is used, and how that added complexity pays for itself over the life of the system by enabling a number of dynamic optimizations of physical, software, and human assets (i.e., dynamic load-balancing to optimize bandwidth, dynamic application serving to eliminate overprovisioning of software licenses, reallocation of tech support time from commuting to actual maintenance via remote management technologies).

In this short post, I want to take a closer look at the hardware side of virtualization’s tradeoffs, as an avenue to a better understanding of how it actually generates value by enabling system architects to trade statically (over)provisioned hardware for dynamically provisioned hardware.

Bandwidth and transistors

Processor and system architects often speak of moving functionality into or out of software, as if software were some ephemeral reality with no physical costs. A better way to talk about this move would be to speak of moving functionality from fixed-function, statically-allocated hardware into general-purpose, dynamically-allocated hardware. 

To take a concrete example, consider the case of moving encryption functionality from a dedicated coprocessor into “software.” By replacing a coprocessor with an encryption program running on general purpose hardware, you move the encryption function from a fixed pool of transistors (i.e., the encryption IC) into a dynamically allocated pool of transistors that includes RAM, cache, and the various on-die components of a processor (both core and un-core). Sure, the software-run encryption may not be as fast, but if the coprocessor was infrequently used enough then it’s definitely more efficient from a total system cost and power draw perspective.

Virtualization entails a similar type of tradeoff, in that it lets you trade infrequently used but statically provisioned hardware for more intensively used, dynamically provisioned hardware.

Virtualization and dynamic provisioning of resources

Because virtualization increases the depth of the software stack, it ups the aggregate amount of code and data moving through all of the links and caches in a system. This has the net effect of increasing the memory and bandwidth footprint of each virtualized application. 

Starting from the processor and moving outwards, this increase in code and data boosts the following: instruction execution activity in the core’s back end, instruction decode activity in the core’s front end, cache utilization, frontside bus traffic, memory subsystem traffic, and memory utilization. At the level of the network (in the case of app streaming and utility computing), virtualization increases overall network traffic and back-end storage needs.

You’ll notice, however, that all of the increased elements that I just mentioned have one thing in common: they’re all typically overprovisioned to prevent bottlenecks. When you multiply the overprovisioning of all of these system- and network-specific elements across a datacenter’s worth of computers, then you have a lot of extra cache, memory space, link bandwidth, and instruction decode and execution bandwidth, all scattered around the datacenter in tiny, isolated pockets. These pockets of overprovisioned resources are individually too small to be used, but collectively they represent a huge waste in a large datacenter.

However, virtualization lets you pool compute activity together so that it can be tightly fitted to a smaller number of more highly utilized machines. In this way, the machines that host virtualized software see increases in all the elements listed above (instruction decoded and execution activity, link bandwidth, cache and memory space), but the rest of the machines see decreased utilization to the point where they can be brought offline.