Virtualization: Power, Service-Interface Compatibility, and Design
Summary: Virtualization may be loosely defined as presenting an apparent architecture for downstream interaction that abstracts and encapsulates the actual implementation. (5 printed pages)
Virtualization is a powerful technique for extending the value of your hardware and software investments. Virtualization may be used to maximize utilization of expensive resources, enable the dynamic allocation of resources to high-priority applications, extend the effective lifespan of technology investments, improve the productivity of developers and quality assurance engineers, improve opportunities for collaboration, and simplify the management of diverse systems.
Virtualization is frequently used to provide service-interface compatibility with an existing traditions (that is, non-service–oriented) architecture. Virtual file systems, for example, may implement the interfaces designed for access to a single disk, but use them to access data spread across in-memory, disk-based, and tape-based storage. This compatibility permits applications that are written to interact with the traditional implementation to successfully interact with the virtualized implementation.
Virtualization may also be by design. Interfaces that abstract implementations may be designed to deliver specific business value, anticipating that multiple implementations will deliver that value in unique ways. This commonality lets service providers compete in the delivery of the business value. An abstracted printer interface, for example, may allow the user of a document-design application to choose from a variety of printer options, each offering specific features and each implemented to interact with a specific real device. Microsoft's Common Language Runtime (CLR) is a more complex example of a specification that is virtual by design.
In both cases, a common set of interfaces provides access to diverse implementations of services that deliver appreciable business value.
Three major patterns dominate the relationship between virtual interfaces and underlying implementations. We will term these virtual segments, virtual composites, and virtual appliances. Let us explore these patterns both in terms of how they are used today, and how we can apply them to the new challenges we face every day.
The virtual segment pattern allows a single piece of hardware to offer multiple apparent architectures or instances of an architecture.
The classic example of this pattern is the virtual machine, pioneered by IBM in the 1960s and still very much present today. A single physical computer may manage many virtual machines, each with its own instance of an execution environment. Or, "sandbox" applications may manage multiple instances of the same operating system (OS). This is often done to permit running privileged instructions in the virtual OS so that they cannot endanger the managing OS.
Different operating systems may be run in virtual machine instances to give users access to multiple application portfolios and development environments, such as running a Macintosh emulator on a Windows PC, or the other way around. Virtual versions of outdated operating systems can also permit the continued support of applications that are difficult to upgrade, or that must be run in a specific environment (for auditing purposes, for example), or that are still in use by customers and for which technical support must still be provided.
The real-world applications of virtual machines are manifold. Beyond those already mentioned, virtual machines can be used to:
- Provide computing resources to multiple end users or long-running applications with managed quality of service (QoS).
- Scale legacy solutions by moving them onto more modern, more powerful, less expensive, and more easily managed hardware.
- Develop and demonstrate network interactions across operating systems using a single computer, effectively emulating the deployment environment within the development environment.
- Emulate a device of limited capabilities—such as a Smartphone—so that developers can easily execute code on the target "device" from their development environment on, for example, a Windows PC.
- Provide quality assurance engineers with an array of environments (both in terms of OS version and in terms of configuration) in which applications may be tested.
Microsoft supports virtual machine technology in its VirtualPC, VirtualPC for Mac, and VirtualServer 2005 offerings.
Another important example of the virtual segment pattern is the virtual local area network (VLAN). VLANs partition data centers and local site networks into zones to manage secure access, localize broadcast traffic, and to guarantee available bandwidth for specific applications.
A classic example of the virtual segment pattern is disk segmentation. Disk segmentation establishes multiple file systems on a single device, primarily to sandbox storage. You might do this, for example, to protect the "system" from the users, or even some users from other users. Another reason for segmentation is to support a variety of file system formats, including "legacy" (FAT16, for example), "foreign" (Windows on Unix), and raw (such as a swap partition in BSD Unix).
The virtual composite pattern describes the composition of an array of devices into a single apparent device.
An early example of this pattern was the disk array, which offers a single apparent file system across multiple disks (typically all attached to a single controller). The disk array enables users to manipulate very large files, very large collections, and to achieve performance and reliability through "striping" (distributing data for a single file across multiple disks) and check bits (providing redundancy for recovering data when one of the disks failed).
Storage virtualization has evolved rapidly. Advances in operating systems made the creation of disk-spanning file systems less dependent on specialized hardware, and permitted dynamic expansion and contraction of file systems. Hierarchical file systems incorporated faster and slower storage technologies into a single file system, permitting "unlimited" storage that prioritized access for more-recently-used information. Storage-area networks and network-attached storage devices permit sharing vast file systems by all of the computers on a local network.
The virtual composite pattern applied to computing devices allowed Web hosting to scale to meet the explosive demand that began in the mid-1990s. Techniques such as virtual IP addressing allowed an array of machines to act as a single logical service provider. Similar strategies have been used to ensure availability of critical data using clustering techniques for database servers. "Blade" servers refined this pattern for super-scalar service delivery.
Utility computing is the current focus of the virtual composite pattern applied to computing devices. The utility computing pattern supports the dynamic provisioning of resources in the data center, such as the dynamic distribution of services to available compatible computers as demand requires. In a sophisticated utility computing environment, machines may be automatically unmapped from a lower priority use, reconfigured, and brought up for a new use to meet current demand. (The application of the virtual machine pattern is also very useful for effecting this transition.) The utility pattern may even be used to exploit idle desktop resources for "overnight" processing to solve very complex problems like the models used in aircraft engineering, risk projection, and logistical planning. (Some use the term "utility computing" synonymously with the term "grid computing," while others make a definite distinction between the two patterns.)
Microsoft offers broad support for the virtual composite pattern, including Windows Storage Server 2003 for storage virtualization, Windows clustering for server virtualization, and the Dynamic Systems Initiative support for the dynamic data center (Microsoft's implementation of the utility computing pattern).
The virtual appliance pattern describes the common practice of using a set of software interfaces to encapsulate and abstract access to a connected—including network-connected—resource.
An important sub-pattern of the virtual appliance pattern is the virtual device pattern, where the encapsulated resource is (or mimics) a physical device. As noted earlier, abstract device drivers revolutionized how we communicate with printers. Analogous abstractions are used to communicate with scanners, storage devices, and audio hardware.
It has been so long since applications have talked directly to disks and magnetic tape devices that many developers cannot conceive of programming without file and file system abstractions. "Files" and "directories" are appliances: they provide a common set of useful interfaces across a broad variety of physical devices, encapsulating both file layout strategies and I/O control primitives. Even when communicating with a custom locally-attached device, experienced developers will write a device driver rather than have the application communicate directly with the device.
A further refinement of the virtual appliance pattern involves a host machine offering access to one of its devices to other computers across the network. Network file systems and remote printers are examples of this pattern. Network appliances, such as print servers, specialize in delivering this pattern.
This evolution has led to a powerful version of the virtual appliance pattern: the practice of treating a general-purpose computer as a collection of network attached appliances. A single computer may provide file services, print services, fax services, e-mail services, Web information services, authentication services, and so on. Even management interfaces have become virtualized—allowing like services to be managed with like interfaces regardless of the computer or operating system delivering the service.
The service definitions provided in the Information Technology Infrastructure Library (ITIL) body of work codify and encourage this conception of computers as service delivery appliances. One may argue that the roots of service orientation lie in the virtual appliance pattern.
Microsoft operating systems rigorously apply the virtual appliance pattern and are continuing to refine it; the Longhorn release of Windows will significantly expand the set of managed interfaces to the capabilities of the operating system. Microsoft is pursuing network appliances with equal aggression, both through our focus on service orientation and through our implementation of the ITIL services in the Microsoft Operations Framework (MOF).
Virtualization allows solutions that are too big for a single device to use multiple devices as if they were one. Virtualization allows special-purpose and rarely-accessed solutions to be supported without dedicating systems to them. Virtualization allows computing resources and entire networks to be reconfigured to meet the demands of the moment.
Virtualization patterns permit information technology assets to be managed and evaluated based on the value they deliver to the organization. Effective virtualization strategies increase organizational flexibility by focusing on delivering capabilities, rather than on installing and managing hardware.