Windows 8 Platform Storage – Part 2

A prior blog entry began our whirlwind introductory tour of some of the more significant platform storage enhancements being introduced in Windows Server 8 – this follow-up entry concludes our introduction.

In Windows 8 Platform Storage – Part 1, we listed your storage related requirements. We also described our resultant set of assumptions guiding Windows 8 platform storage development. Finally, we had a brief introduction to some Windows 8 platform storage capabilities including:

• Improved NTFS metadata integrity on commodity storage
• Maximized NTFS availability through online file system scan and repair and
• Storage Spaces – platform based storage virtualization for business critical application deployment on cost-effective commodity storage hardware

The following additional Windows 8 platform storage enhancements support objectives listed in the prior blog:

• Data Deduplication – Windows Server 8 implements state of the art (sub-file) data chunking and compression to deliver optimal data deduplication, and thereby maximize storage capacity utilization for a variety of workloads. The implementation is transparent to the primary workload on the server – this is achieved by minimizing I/O impact through scheduled and selective optimization, and minimizing server impact through low resource (CPU/memory) usage. Further, the implementation strongly optimizes for high data integrity – this is achieved by performing data integrity validation on all data as well as metadata, and by ensuring redundancy for all metadata as well as “popular” data chunks. Windows 8 servers that enable deduplication as well as BranchCache apply the same algorithms both to on-disk and on-the-wire savings, thereby benefiting from faster download times as well as reduced bandwidth utilization over wide area networks. The below table lists a sampling of expected savings from deduplication for common workloads:

• Windows Storage Management API – a comprehensive and extensible WMI v2 based storage management API enables easier “single point” administration and management of heterogeneous storage infrastructure (including Storage Spaces). In-box PowerShell command-lets enable simplicity of end-to-end provisioning and ongoing administration, as well as inherent support for remote execution and scripting. Application developers including those delivering sophisticated storage management solutions (e.g. System Center or equivalent) can now rely on a single Windows API. Storage hardware vendors can enable easy administration of their gear either through the industry standard Storage Management Initiative Specification (SMI-S) or via a Windows host-based Storage Management Provider interface.

A comprehensive single-pane-of-glass storage management GUI utilizes this API to deliver task based, multi-machine heterogeneous storage management.

• Offloaded Data Transfer – joint innovation with major external storage array vendors has resulted in Windows Server 8 providing an incredibly efficient mechanism to perform cross-application and cross-machine data transfers.

In situations where data is simply moved from a source location to a target and wherein no computations are performed on the dataset (e.g. file copying or virtual machine provisioning from a “golden image”), Windows Server 8 works with advanced external storage arrays to directly transfer this dataset without requiring the data to be transferred to/from the Windows servers. Obviously, this minimizes impact on host network, host CPU, or host memory and enables increased efficiencies through better external storage array and storage network utilization. In the future, we expect additional enhancements to external storage arrays that will also enable offloaded data transfers cross-storage-arrays and also across geographically distributed datacenters.

On Windows 8 servers which store data on advanced storage arrays supporting this capability, administrators can (transparently) enjoy the benefits of very fast and efficient data transfers through the simple act of “dragging and dropping” files using Windows Explorer. Further, applications have access to simple yet powerful APIs so they too can directly invoke this functionality.

• Awareness and utilization of thinly provisioned disks – beginning with Windows Server 8, NTFS identifies thinly provisioned disks and enables transparent capacity reclamation (e.g. when files are deleted) for better capacity utilization. Further, sophisticated applications can also identify thinly provisioned disks through both WMI or via PowerShell, and can invoke commands to enable capacity reclamation when appropriate. Applications are also notified via standard events whenever capacity exhaustion thresholds are crossed on such thinly-provisioned storage, thereby enabling them to take appropriate action.

• Storage Optimizer – Periodically, without any user intervention, a storage optimizer task completes maintenance activities including intelligent compaction of file system allocation to enable capacity reclamation on thinly provisioned disks (including on thinly provisioned spaces).

Platform storage enhancements in Windows Server 8 enable a cost-effective, scalable, and continuously available dynamic IT environment supporting business agility and flexibility. There is much more left to share about these and additional platform storage capabilities in Windows Server 8 – so stay tuned!

Rajeev Nagar

Group Program Manager – Windows Storage & File Systems