Introduction to Virtualization
Before virtualization became commonplace, organizations had to deploy a dedicated physical server for each application, such as print servers, email servers, and database servers.
This approach forced businesses to underutilize their hardware, resulting in increased costs and overcrowding in data centers.
To solve this problem, IBM invested in developing resource-sharing (virtualization) solutions for mainframes in the late 1960s and early 1970s. Then in 1999, VMware revolutionized the industry by bringing virtualization to the x86 architecture.
The key technology behind this transformation is called a hypervisor. It is a software layer that sits between the hardware and the operating systems and manages resources.
The fundamental architecture of a virtualized server is as follows.
- The server is the bottom layer, consisting of CPU, primary memory (RAM), and secondary memory (SSD or HDD).
- Then, instead of running a single operating system, a hypervisor, or a middle layer, virtualizes the hardware resources and allocates them to multiple virtual machines (VMs), which can be imagined as the top layer.
So, each VM gets its own virtual CPU, virtual RAM, virtual storage, and virtual network.
Virtualization, as we understand it today, has come a long way since the 60s.
Initially, virtualization did not address storage challenges, and each virtual machine's data was still tied to a single physical machine. This meant that whenever the hardware failed, applications and data were lost.
To overcome this, technologies like VMware's vMotion and Fault Tolerance were introduced. These technologies enable live migration of VMs between servers.
For VM mobility and failover, a shared storage network, SAN or NAS, is required. This is where storage virtualization becomes important.
So, while server virtualization abstracts computing resources, storage virtualization abstracts data storage and allows IT teams to build scalable, flexible, and fault-tolerant environments.
In the next sections, we will understand how virtualization works, its different types, and how server and storage virtualization complement each other. We will also understand the benefits and challenges of these approaches.
Operating Principles of Virtualization
Virtualization is made possible by the hypervisor. It is a lightweight software layer installed on a physical server, which abstracts the underlying hardware. It also enables the creation of multiple virtual machines.
Each VM operates as an independent system with its own operating system, applications, and virtualized hardware.
There are two types of hypervisors:
2. Type 2 Hypervisors: These are hosted and run on top of an existing operating system as software. They allow running multiple OSs on the same PC. Examples include Oracle VirtualBox, VMware Workstation, and VMware Fusion.
How Virtual Machines Are Created and Managed
Once the hypervisor is installed, an administrator can create virtual machines by allocating virtual resources from the physical host to them.
Each virtual machine is assigned a vCPU, or virtual CPU, which is a portion of the physical processor's computing power; a vRAM, or virtual RAM, which is a slice of the host's total memory; a virtual storage, which is a virtual hard disk drive (for example, VMDK for VMware and VHD for Hyper-V); and lastly, a vNIC, which is a virtual network interface card, a software-defined network adapter. Here is an example.
Let us assume we have a server with 24 CPU cores, 96 gigabytes of RAM, 1 terabyte of storage, and dual 10 GbE network adapters, and it has to host three virtual machines. This is how these virtual machines may be created:
| VM Type | vCPU | vRAM | vStorage |
| Web Server | 8 | 16 GB | 200 GB |
| Database Server | 8 | 32 GB | 400 GB |
| Application Server | 4 | 16 GB | 200 GB |
The hypervisor can dynamically schedule workloads across the physical hardware and ensure that resources are used efficiently while maintaining isolation between virtual machines.
Types of Server Virtualization
We understood how server virtualization allows multiple workloads to run on a single physical machine. This is done by abstracting hardware resources.
However, different virtualization methods exist because every method is a trade-off between performance, compatibility, and VM isolation.
Early virtualization efforts had to balance these three factors.
- Compatibility: Should any operating system be able to run without modification?
- Performance: Should we avoid virtualization overhead, i.e., overuse of CPU and memory resources?
- Scalability: Should virtualization be lightweight enough for cloud-native workloads?
Three major types of server virtualization evolved, each optimized for different scenarios. Let's understand these types:
Full Virtualization
- This approach provides complete hardware emulation and allows guest operating systems to run unmodified.
- The hypervisor manages all the interactions between the guest operating system and the physical hardware. This ensures strong isolation.
- While this approach supports any operating system that runs on the hardware, it also introduces performance overhead due to the need to emulate privileged instructions. Some examples are VMware ESXi, Microsoft Hyper-V, and KVM.
Para-Virtualization
- This approach optimizes performance by making the guest operating system aware that it is running in a virtualized environment. So instead of hardware emulation, the guest operating system communicates with the hypervisor using specialized hyper-calls.
- This reduces virtualization overhead but also requires operating system modifications or specialized drivers. Xen in para-virtualized mode and KVM with virtIO drivers use this method.
OS-Level Virtualization
- This is also called containerization, and it takes a different approach that eliminates the need for separate hypervisors. Instead, a single operating system kernel is used to run multiple isolated instances (each of which is known as a container).
- Because of this approach, IT admins can achieve efficient resource utilization and rapid scalability. However, this approach requires that all containers use the same OS kernel. Technologies like Docker, Kubernetes, and LXC make this method ideal for cloud-native applications and microservices.
Link Between Server and Storage Virtualization
Server virtualization has undoubtedly transformed IT infrastructure. When virtual machines run on shared physical resources, the IT environment’s efficiency and flexibility increase manifold.
However, virtualized workloads require storage. Without an optimized storage management strategy, the benefits of server virtualization can be very limited.
In a traditional IT environment, storage is tied to physical hardware. This makes it difficult to scale, allocate, and manage. Also, this creates challenges in virtualized environments where VMs need storage that is flexible, resilient, and independent of the underlying hardware. Simply provisioning storage for individual virtual machines can cause bottlenecks. It can also make management overheads very complicated.
This is where storage virtualization becomes important.
What Is Storage Virtualization?
Storage virtualization is the process of abstracting physical storage resources to present them as a single logical storage pool. With this technology, IT teams can centralize how they manage, automate, and utilize storage capacity. This reduces the complexity of traditional storage systems.
With storage virtualization, administrators do not have to deal with multiple physical disks. Instead, they interact with the virtual storage layer that can allocate space where needed.Virtualization software can intercept input/output requests from virtual machines and can also map them to the appropriate physical storage location.
One of the biggest advantages of storage virtualization is that it disguises the complexity of physical storage infrastructure. In environments like Storage Area Network (SAN) or Network Attached Storage (NAS), storage virtualization enables IT departments to manage storage in a way that is more efficient, helps them improve backup and data disaster response, and improves overall system performance.
It also eliminates rigid constraints of physical storage, as storage virtualization can allow a business to scale effortlessly and optimize the use of storage resources.
Types of Storage Virtualization
Storage virtualization came into being to address the challenges that IT administrators were facing in managing large-scale storage environments. Different workloads and applications require different ways to manage storage. Because of this, two main methods of storage virtualization have come into being.
1. Block-Level Storage Virtualization
- This method is commonly used in SAN (Storage Area Network) environments. In this method, raw storage blocks of a fixed size, which can function independently, are pooled together. A virtualization layer abstracts the logical storage (for example, drive partitions) from the physical storage (for example, SSDs or HDDs). This allows efficient allocation of storage across multiple drives.
- In this method, high-speed protocols like Fiber Channel (FC) or iSCSI are used to provide low-latency, high-performance data access. This is essential for databases, virtual machines, and enterprise applications. Since block storage operates at a lower level than file storage, it integrates well with operating systems and appears as a locally attached storage.
- Block storage virtualization also is easier to manage in large SAN environments because it allows IT administrators to enable centralized storage control. This in turn makes it easy for them to allocate, migrate, and scale storage resources dynamically.
2. File-Level Storage Virtualization
- This method is designed for NAS (Network Attached Storage) environments. In this environment, storage is accessed through file-level protocols like NFS for Linux and SMB for Windows. So instead of dealing with storage blocks, in the file-level virtualization approach, files and directories are abstracted from their physical storage locations.
- Because of this, applications can access data without knowing where that data physically resides. This method is ideal for file-sharing environments, collaboration tools, and content repositories. In these applications, multiple users need access to the same data.
- So when NAS storage is pooled into a unified namespace, file-level storage virtualization can then simplify file migration, management, and scalability without interrupting operations.
- Here is an overview of the key differences between block- and file-level storage virtualization
| Feature | Block-Level Storage Virtualization | File-Level Storage Virtualization |
| Storage Type | SAN (Storage Area Network) | NAS (Network Attached Storage) |
| Data Access | Blocks of raw storage | Files and directories |
| Structure | Lacks a predetermined structure | Files & folders arranged in a hierarchy |
| Performance | High-performance, low-latency | Moderate performance |
| Common Use Cases | Databases, Virtual Machines, Enterprise Apps | File sharing, collaboration, content storage |
| Protocols Used | Fibre Channel, iSCSI | SMB, NFS |
Benefits of Virtualization
Virtualizing servers and storage brings a lot of advantages to enterprises. Here is an overview.
1. Hardware Consolidation
Many virtual machines can share one physical server, which can raise the overall utilization and reduce the number of physical servers and drives needed. This can reduce hardware and energy costs significantly.
2. Flexibility
Storage virtualization allows dynamic allocation of capacity, as volumes can be expanded on the fly by pulling more storage devices. This allows administrators to manage a single large pool instead of many silos.
3. Better Utilization of Resources
Unused disk space across many devices can be aggregated and used efficiently. This provides for better space utilization and reduces the wastage of capacity. For example, in cases where thin provisioning is used in virtualized storage, multiple VMs can share a pool and consume physical space only as needed.
4. Simplified Management
A centralized interface can manage the entire virtual storage pool. This makes it easy to allocate, monitor, and back up data across the IT environment. Snapshots and cloning of virtual disks can be done centrally.
5. Reduced Vendor Lock-in
When storage is abstracted from the hardware, then an organization can mix different brands and models behind the virtualization layer. This heterogeneous integration means an array from vendor A and another from vendor B can contribute to one storage pool. This protects investments and makes it easy for IT admins to add new tech without being tied to one vendor.
Challenges of Virtualization
Now that we understand the benefits of server and storage virtualization, it is also important to understand the challenges they present.
1. Increased Complexity
A virtualization layer raises the complexity of the IT environment because there are more components to manage, such as hypervisors, virtual switches, and volume managers. All of these need to be configured and maintained. Managing various software and hardware elements in tandem can be difficult when so many moving parts are involved.
2. Performance Overhead
Virtualization introduces an indirection layer, which can cause input/output latency. There is extra processing, such as context switches and input/output through the hypervisor or network, which can reduce system performance as compared to bare metal. So, administrators have to ensure that the virtualization stack is optimized to avoid bottlenecks.
3. Backup and Disaster Recovery Complexity
While snapshot and replication features are available for administrators, they cannot solely rely on virtualization features. A virtualized setup still needs a solid backup strategy. Additionally, data recovery in a virtualized storage environment can be non-trivial because one has to consider multiple layers, such as virtual disk, host file system, and underlying storage during the recovery efforts.
Conclusion
We've seen how storage virtualization can be a game changer for organizations. It can improve flexibility, scalability, and efficiency in data management. While block-level storage virtualization promises high performance for databases and enterprise applications, file-level virtualization can simplify shared data access in collaborative environments.
IT professionals need to choose the right approach depending on the workload demands from the IT infrastructure.
Virtualization continues to evolve, and organizations need to integrate the right storage technologies to make sure that they get the best performance, lowest complexity, and robustness for their IT systems. IT professionals who can understand these different storage virtualization models will effectively build resilient and
About The Author
Data Recovery Expert & Content Strategist
