The motivations are strong and compelling when making the choice to move towards virtualization as the solution for your data center.

Some of the factors include:

• Lower energy costs
• Greatly reduced data center physical footprint.
• More efficient use of hardware resources
• High availability
• Business continuity / easier disaster recovery
• Lower admin to server ratio

Before the migration is to move forward a detailed audit of current equipment, platforms, applications, etc must take place. All computing resources must be properly documented and enumerated. All backups need to be made current. If hardware is going to be re-tasked and used in the new virtualization schema, thorough testing needs to be done to assure full functionality and reliability.

Thorough preparation in the form diagramming, establishing documented process flows, and devising effective storage strategies is directly relational to the effectiveness and efficiency of your virtual migration and the resultant infrastructure. The planning stage is a good time to implement new ideas that will assist in ease of administration and further data center expansion. As with any IT infrastructure the complexity scales with data center size. Depending on the size of the data center, the virtual migration can be rolled out in phases to ease the transition and deal with issues on a smaller scale than would be necessary if the entire data center was migrated at once.

When the time comes to convert or migrate from purpose-built hardware servers to virtual machines, there are several options to consider. If the data center is small or your company can afford a small window of downtime, you can take servers out of production and then migrate to virtual machines. Although this procedure is more hands-on, it allows you to deal with issues out of band without affecting data integrity. For the larger data centers or data centers that cannot afford a maintenance window for migration (which is most likely the scenario), live migrations can be performed using one of many vendor-supplied tools that are readily available to automate the migration process, live, from bare metal server to virtual machine.

Once the decision has been made to take advantage of virtualization in your data center, the key to successfully accomplishing your objectives is research, and planning. The outcome and success of your virtualized environments will likely dictated by the level of preparation and detail you have provided in your project manifest.

This has certainly been the talk of the industry for a while, focusing on management tools above and beyond the VMM itself.  To a large part, this a response to open source VMMs (like Xen or KVM) and free VMMs (like Hyper-V). If the VMM is free, then vendors must somehow make money elsewhere in the
stack – for Microsoft and Linux vendors, it’s obvious how this works, but VMWare is clearly driving to differentiate in other ways.  Products from VMware are looking at high-availability, disaster recovery and backup as ways to add value by bringing formerly high-end server features to standard x86.

To some extent this creates a bit of tension though.  One of the reasons why virtualization is necessary is that modern operating systems have grown incredibly large and perform a lot of complex and non-essential work inside the kernel.  This additional complexity can result in instability and security vulnerabilities.  For VMMs to really add value, they need to stay lean (like VMware ESXi – which can be as small as 32MB) and avoid the temptation to throw in every feature and the kitchen sink.  That just will result in another bloated software layer – and then someone will come along and try to put yet another layer of indirection beneath the VMM, or perhaps remove the VMM entirely!

This obviously creates an interesting paradox as virtualization vendors will have to tread a fine line between adding features and compactness and efficiency.  Perhaps one area that has yet to see a lot of overlap with
virtualization, but is a natural candidate is security.

Security and virtualization are fundamentally very similar.  Virtualization focuses on ensuring that multiple operating systems ‘play nice together’ and share the same hardware safely.  To some extent, security is one aspect of ’working well together’.  It’s easy to imagine how virtualization could improves security – high risk activities, such as browsing the web, file sharing, etc. could all be done in one VM (with limited access to data), while more important and secure activities (taxes, peresonal documents) could reside in another VM. Of course, this assumes that the VMMs don’t inherit some of the existing security vulnerabilities – which would be yet another reason to keep a VMM slim and compact.

These are just some thoughts on how virtualization might evolve over the coming years, but here’s an opportunity for you to chime in.  How would you like to see virtualization evolve?

I recently had the opportunity to participate in a live Webcast focusing on The ROI of Virtualization, along with representatives from Intel and IBM. We had a lively discussion on the subject. The webcast is still available as an archive, if you’d like to check it out.

As usual, my favorite part of the event was at the end, where my co-presenters and I finish with our prepared remarks and take questions from the audience. I also had a few questions of my own. One of the most interesting points was regarding new innovations in hardware and software design, and where that will be taking the next generation of virtualization technology. I asked my copresenters – Bob Zuber, Worldwide Product Marketing Manager from IBM, and RK Hiremane, Senior Product Marketing Engineer from Intel “to outline some of those future directions that they see their respective companies pursuing. Here is a short excerpt from that Q&A:

Steve: What do you see in the future, what types of innovations can we expect, to help us achieve even greater efficiencies through virtualization?

RK: I think today, “virtualization 2.0 is becoming very important, based on the principles of consolidation and so forth. As we go forward, as many of the analysts have predicted, this is going to lead to further automation, and we believe it will lead to more service-oriented, policy-driven automation of the data center, and better efficiencies of the data center.

Bob: From a policy point of view, there will be certain things we look at from a power point of view. You may have workloads that are more performance-oriented or more energy-conservative, so you may have policies that say, “I want to direct that workload to the most appropriate platform to get the job done, in the most efficient manner. Today, when you look at power supplies, memory DIMMs, processors — you may come back and say, for this type of workload, I’m not going to have the ultimate performance from a processor point of view, but have better wattage from that point of view. You may forego and sacrifice certain attributes, and being able to have that policy director understand the underlying hardware attributes to direct the workload, that’s the promising piece that we see and the most exciting. It’s all about that automation and how we automatically move things around, but more importantly, how does a hardware provider build the most efficient piece of hardware that addresses a certain type of workload, but then likewise, can be adaptable to change, as the customer demands change, but from a workload perspective.

This is just one example of what you might call “virtualization 3.0, or the next generation of a technology that is already gaining a major foothold in corporate data centers. Since much of the appeal today is driven by cost and energy efficiencies, the policy-driven automation built into the coming generation of virtualization solutions will address those issues directly, and compound the savings.

From where I sit, that looks pretty promising. Is there a downside? I suppose extra processing power to automatically determine where to direct each task, bouncing it between machines that are more processor-intensive or more energy efficient, could potentially add to the overhead in the hypervisor, and adversely effect performance. As with anything, it’s all in the execution. If it’s implemented skillfully, there should be no discernible performance hit.

What do you think about new virtualized servers that think for themselves, and route workloads according to their own interpretations of the policies we set. Is it IT utopia at last?

Virtualization technology has made great strides in allowing the creation of the next generation of efficient, easily manageable, highly available, and dynamic data centers. Future developments in virtualization as a technology, the processes involved and hardware technology employed will be impressive and ground-breaking.

Desktop virtualization is a very hot topic these days as companies try to get even more efficient and consolidate resources, make better use of IT administrative hours and deploy desktops based on the needs of individual department tasks and computing requirements. Virtual desktops can be maintained centrally and are versatile when it comes to the end user access. The same virtual desktop used within the building can be accessed remotely.

Virtualization will be explored as an option in areas previously not considered feasible. Cell phone companies are already investigating its applicability in providing multiple phone environments on the same hardware. In the future you could choose your handset and then specify whether you wanted blackberry OS, Windows Mobile, or both.

A different mindset will have to be embraced to fully make use of the benefits inherent in virtualization. Steps normally applied to the flow of a project will either be shortened or removed all together. The use of virtualization will continue to spur increased ingenuity as the path from concept to testing, to production, is greatly shortened when virtual environments can be rolled out within minutes. When you remove the step of having to acquire physical computing assets before you can test your ideas, you take a great chunk of time and budgetary needs out of the equation towards the quest for a refined business solution.

Hardware technology will better enable the use of virtualization as components and will be designed to better facilitate the virtualization process, hypervisor, and interaction of data I/O between the software and physical layers. Advancements are already being made in these areas that will revolutionize the way data is accessed and manipulated. In the future I can see servers being purchased with the hypervisor already included in firmware, which will then be easily updated and managed with an even smaller foot print.

The future of virtualization means a refinement in the way people interact with technology and above all, better efficiency on all levels.

Virtualization is certainly one of the trendiest technologies and ripe for measurement.  The latest announcements of CPUs from Intel and AMD have all explicitly mentioned and showcased improvements in virtualization performance, along with a flurry of feature names like VPID, Pacifica, Nested Page Tables and Extended Page Tables.  Most of these I’ve described in prior article, but to summarize, a lot of these features are about shifting the burden of virtualization from software (that is, the VMM) onto the hardware by making some hardware operations more complete.  Take for example, THE VPID in Nehalem (incidentally AMD has had an equivalent feature for a while), which reduces transition times between VMs by about a third (compared to the prior generation Penryn).  While it’s great that VM transitions are faster, it’s really hard for an average user to understand what that really means in terms of virtualization performance.

Unfortunately, virtualization benchmark is a very complex and difficult subject.  To start with – it’s rather hard to define.  Virtualization by itself is not useful and isn’t important to measure.  Server users ultimately care about the performance of their virtualized workloads – which is pretty well understood with benchmarks like sAP sales & distribution (SAP SD).  In theory, a user could run SAP SD on raw hardware and then on the same hardware with virtualization and measure the performance changes. That’s by far the simplest and easiest test – but it also doesn’t really capture the complexity of virtualization.  

The aggregate performance of a server across a set of virtualized workloads is much more interesting.  It reflects reality, how virtualization is used in the data center, much better and also adds in the complexity of multiple VMs contending for CPU, memory and I/O.  The latter should help to distinguish between the mediocre and excellent platforms.  Next, there is the question of what workloads to put in the VMs – should they be homogeneous (all print servers on a single piece of hardware) or heterogeneous (mixing DBMS, file servers, etc. and if so, what should the mix be)?  Those questions are all rather difficult and depend on what IT staff usually do in the real world.

Once these questions have been answered, it will give users a clearer picture of what features actually provide real value for virtualization. Right now, there are no standard benchmarks unfortunately, although SPEC has a working committee that has focused on this problem since 2006.  

Of course, the work won’t stop there as there are other interesting questions – how to assess the power efficiency of a virtualized solution, or compare different architectures, but some standardized benchmarks would go a long way towards helping virtualization become more buyer friendly.

I recently came across a thought-provoking blog post by StorageIO analyst and author Greg Schultz. Ruminating on themes in his new book “The Green and Virtual Data Center,” Schultz laments that the natural link between the concepts of virtualization and data center consolidation has led many to mistakenly conclude that these terms are synonymous. They aren’t. And even though consolidation is one of the leading reasons to virtualize today, it’s not the only good reason to consider the technology.

Furthermore, while the inherent cost and energy savings in consolidation are fueling much of the growth and interest in virtualization today, that won’t always be the case. Does that mean that when the economy rebounds, virtualization will become yesterday’s news? Far from it, argues Schultz. Despite “the perception and industry messaging that incorrectly pigeon holes virtualization to mean consolidation, and consolidation to mean virtualization,” Schultz warns us not to be sucked into the notion that virtualization without consolidation doesn’t have a bright future. “To the contrary, the reality is that there is life beyond consolidation (and) there are even more scenarios and far greater market opportunity for non-consolidation virtualization deployments over time, than what has already been seen for first wave of consolidation-centric virtualization scenarios.”

What are some of the reasons besides consolidation for virtualizing your servers, storage or other IT resources? How about simplified maintenance and technology updates, not to mention load-balancing, fault tolerance, and supporting business continuance (BC) and disaster recovery (DR)? According to Schultz, “We are just seeing the tip of the virtualization iceberg (or mountain).”

That’s good news if you’re in the virtualization business, since perhaps as few as 15% of all servers and storage systems are actually suitable to be consolidated – yet nearly all are suitable to be virtualized. What non-consolidation reasons can you think of to virtualize your server, storage, or even desktop systems? Or do you think virtualization is simply a consolidation-crazed “flash in the pan,” being driven by the IT community’s collective kneejerk reaction to an abysmal economy? I’d like to hear your views.

To maintain optimal uptime, high availability, and business continuity a comprehensive and up to date disaster recovery plan must be in place and meticulously documented. Disasters can be caused by power outages, human error, and certainly natural disasters. Server or data center downtime can cost the company dearly  whether it be monetary or a negative impact on reputation. Disaster recovery plans must also be tested thoroughly from time to time to make sure they are effective and meet recovery time constraints. In a data center that does not use virtualization this becomes a costly investment. A testing environment must be set up utilizing similar hardware to emulate what is already in place for an accurate test and analysis.

One of the many benefits of virtualization is the ease in which disaster recovery plans can be created and tested. When testing disaster recovery for virtual machines you eliminate the necessity of similar hardware as the VM’s are hardware independent. You can test your recovery plan using machines that have been taken out of production as long as they have the same hypervisor or virtual machine monitor on them.

Furthermore, the need to install the OS and all of the related applications to bring a machine back up is eliminated as virtual machines are just a collection of files that can easily be backed up in their entirety by copying them off (or backing them up) to another location. To restart the backed up virtual machine just copy it over to an operational server and start it up. This greatly reduces the time required for recovery.

If your entire data center has been brought down due to any nature of disaster the high portability of virtual machines allows you to transfer them to an alternate data center that is out of the affected area and restart. Your entire data center can be restarted at an alternate location using virtualization.

To further enhance and shorten the disaster recovery timeframe there are tools available that can facilitate the process by automatically transferring and restarting your virtual infrastructure at a predefined location. These tools are capable of advanced features such as the ability to start virtual machines in a pre-defined order so that any dependencies are resolved before the virtual machines that rely on them are started. A good example would be a virtual machine that ran an application which required a database to be running before it could start.        

Utilizing virtualization in your data center or server room assists you in overcoming some of the most trying and difficult tasks you may ever have to face. Due to the simple concept of virtualization separating hardware and the guest OS or virtual machine many tedious processes can become much simpler through automation and proper planning.

In keeping with the purpose of using virtualization technology the data storage solution must also be resilient and dynamic. Options in tasking and deploying storage for a virtual machine must follow suit with the core technology. Your choice for data storage needs to have high availability and fault tolerance for optimal uptime. Access to the data needs to be fast so as not to affect the performance of the virtual machine. Optimally, the data storage should be easily expandable without creating additional volumes.

The storage technology used and its method of access typically determines if all or parts of the objectives are met and if so, to what degree.

The practice of data storage on the same hardware as the virtual machine is strictly taboo as it defeats the purpose of deploying virtualization in the first place. Virtualization data storage is usually accomplished with Network Attached Storage (NAS) or a Storage Area Network (SAN). Both forms of storage have their benefits and drawbacks as will be explained later in this article.

A NAS – or Network Attached Storage comes in the form of a hardware device that usually contains one or more arrays of hard disks handled by one or more controllers. The advantage of having multiple controllers is that in the case one fails the other may take over the failed controllers RAID array, often in a somewhat performance deprecated state but none the less still operational. NAS devices come with an operating system that is meant to control the storage functions needed and provide device setup and management access in the form of secure shell access or VIA a web browser.  NAS uses common file based protocols to allow access to storage with NFS being popular for UNIX systems and CIFS for windows systems.

A SAN – or Storage Area Network is a high speed network of interconnected data storage devices. The storage devices connected can be of different brands. SAN’s are commonly used in somewhat larger storage implementations than NAS due to their cost and complexity, although they can encompass NAS devices. The drawbacks of using a SAN as compared to NAS are cost, and more complex administration.

Another difference between the two storage options is that with NAS you use file level operations and SAN’s uses Block level I/O.

Using network storage for your virtual machines there is always the concern of latency. This problem is being overcome with the use of TOE (TCP Offload Engine) enabled network cards and faster fiber channel and copper interconnects.

Your choice of virtual machine data storage ultimately should depend on the size and demands of your virtualization deployment scenario and of course budget.

An entire class of new tools is emerging to help developers and system administrators generate JeOS images. Such tools can generate bootable system images in several formats with the components selected by the user. Some of the more sophisticated tools even have automatic dependency resolution features that can guess what packages will be needed by analyzing a target application. Many of these tools are designed to build images that are based on specific Linux distributions.

One of the most powerful tools in this field is rPath’s Builder platform, which can create virtual images based on several several distributions, including CentOS and Ubuntu. It offers support for numerous output formats including those compatible with VMware, Microsoft Virtual PC, Virtual Iron, Parallels Workstation, and Xen. It also recently gained support for the  Amazon Machine Image format, which is used by Amazon’s Elastic Compute Cloud service. The rBuilder platform leverages the advanced Conary package management system to perform automatic dependency resolution.

The Ubuntu Linux distribution also provides its own unique solution for JeOS construction. The vmbuilder tool, which is designed to be used at the command line, is a quick and easy way to generate custom Ubuntu images. It is generally used with KVM and it’s particularly useful for software developers who want to create self-contained Ubuntu server testing environments.

There are a lot of emerging JeOS tools that are still under development. Novell’s SUSE Studio, which is still in closed alpha testing, provides an extremely rich graphical user interface for building SUSE appliance images with custom packages, special configurations, and custom branding. SUSE Studio has several default profiles that can be used to quickly create images with complete GNOME and KDE environments or a lightweight text-only environment.

With some of the latest tools, the process of building virtual appliances is just as easy as selecting a set of packages from a web-based package management interface. As JeOS tools become more sophisticated, the challenges associated with generating custom platform images will swiftly decline.

In our last post, we discussed some of the ways that virtualization is used both by individual consumers, and by enterprises. Recapping, the most prominent use is server consolidation, but there are plenty of other ways that virtualization can prove beneficial.

One particularly interesting topic is how the various uses of virtualization have influenced and impacted the design of modern servers. For the most part, the goal of virtualization and server consolidation is to minimize the power, heat and space foot print consumed by servers. As it turns out, this generally means that workloads are consolidated onto larger four socket servers which tend to maximize the capacity for guest operating systems, rather than the smaller single or dual socket servers.

Memory Capacity

Virtualization tends to heavily emphasize memory capacity, which is often the limiting factor in a given environment. As we discussed earlier, a regular operating system usually keeps the kernel in main memory to avoid frequent page faults. In a similar fashion, the entire VMM is usually kept in main memory, and is rarely paged to disk. In a virtualized environment, when a guest OS becomes active, large portions will be moved back to memory. This isn’t a problem when a server has one or two guest OSes, but when there are a dozen or so virtual machines on a server all contending for memory, the pressure becomes quite substantial. Each OS might eat up 0.5-1GB, plus another 0.5GB for the VMM. This can be a serious issue if several of the VMs are likely to be utilized at the same time – since contention for memory will result in a swap storm, with important data constantly being shuttled back and forth between memory and disk, and extremely poor responsiveness. Ultimately, memory capacity is often the gating factor in how many VMs can be hosted on a given server – which encourages server vendors to differentiate their offerings by providing more memory capacity. For instance, IBM’s current X4 chipset supports up to 1TB of memory in a single system – four times the capacity of most other x86 servers.

Reliability

Virtualization also substantially changes the equation for reliability and uptime. Isolation is the key to virtualization’s success – without it, IT departments would likely not be willing to deal with the problems associated with running dozens of applications together on the same server. Virtualization improves reliability and availability by insulating each guest OS from one another (and from any applications running above a guest OS). However, every guest OS is equally dependent on the VMM and hardware operating correctly; a fatal problem in either will cause the guest OS to become unavailable. With a dozen VMs on a single server, this means that the negative consequences of a hardware failure are a dozen times worse than before. The hardware (and the VMM) must become correspondingly more reliable in order to keep the same uptimes. This is one area that has rarely been mentioned and deserves extra emphasis – while virtualization does a lot to improve reliability, the hardware must be increasingly reliable as well to maintain the same level of service.

This is actually one way that IBM has historically differentiated their high-end server offerings. IBM actually uses a custom chipset for their 4-16 socket servers, with extra reliability and availability features designed into the silicon. Several years ago, I wrote an article about the IBM X3 chipset (http://www.realworldtech.com/page.cfm?ArticleID=RWT042405213553), which unlike most other servers at the time had memory scrubbing, hot-swap and bit-steering. IBM has since updated their line with the X4 chipset, which is discussed at a high level in this white paper (http://virtualizationconversation.com/wp-content/uploads/2008/12/ibm_x-architecture_blueprint.pdf).

I/O System

The last real impact of virtualization is on the I/O subsystem. As we discussed in our second post on virtualization (insert link), I/O is not really encompassed in current virtualization technologies – although it will be in the future. The only major changes required in the I/O system itself are the IOMMU, which is an element of the chipset. Currently, none of the standard chipsets incorporate this feature, although IBM has included an IOMMU in their server chipsets since the X3.

When it comes to the disk system, virtualization generally tends to increase the number of random seek requests initiated by the disk, simply because rather than sending a single stream of requests, the system is sending perhaps a dozen streams all intermingled. This can present a bit of a problem, in that random reads and writes are by far the most difficult disk pattern to deal with. This in turn tends could necessitate massive disk arrays or other exotic solutions. Given current trends though, it is more likely that solid-state drives will be adopted in situations with especially demanding I/O patterns, since they can service many times more random I/O requests.

The only other ingredient is the network interfaces, which are far simpler in many regards. The network I/O simply needs to have enough aggregate bandwidth to satisfy the various VMs residing on a given server. The only foreseeable impact here is that some enterprises may choose to more rapidly adopt 10 gigabit Ethernet, if current gigabit Ethernet is a bottleneck.