Windows 8 Platform Storage – Part 1

Organizations of every size – small and medium businesses, enterprises, and commercial hosting providers – have an insatiable need for storage. In Windows Server 8, we have invested significantly to deliver the most cost-effective platform for scalable and continuously available data access. This blog provides an overview of the rationale for our platform storage investments. Also, we will begin a whirlwind introductory tour of some of the more significant platform storage enhancements being delivered in Windows Server 8 – a follow-up blog entry (Windows 8 Platform Storage – Part 2) will conclude this introduction.

Our customers have very concisely articulated their storage needs:

(a) Guarantee data integrity

(b) Ensure service availability

(c) Maximize value derived from capital investment in storage infrastructure

(d) Minimize storage management complexity and associated ongoing operational costs

The following assumptions underlie our feature enhancements within Windows Server 8 to address the abovementioned needs:

(a) By delivering a richer set of storage virtualization capabilities in the platform, a diverse subset of customer workloads can be successfully deployed on cost-effective industry standard (commodity) storage components.

(b) Storage vendors strive hard to make their components more robust. Nevertheless, given an increased reliance on commodity storage infrastructure, partial or complete failures of storage components are to be expected. Despite such failures, data integrity as well as service uptime must be maintained.

(c) As storage needs scale, more storage infrastructure is deployed. At large scale, storage component failures become “routine” and must be easily handled without commensurate increased complexity.

(d) Maximizing utilization of all deployed storage capacity enables containment of the rate of growth of deployed storage, and, therefore, decreases need for additional capital expenditures.

(e) Some customers incur significant capital expenditure to deploy very highly capable “external storage arrays” for mission critical workloads within their datacenters. Joint innovation with our partners, and deep integration of Windows with these sophisticated storage arrays results in significant additional value to such customers.

(f) A uniform and rich set of storage management capabilities within Windows that can enable multi-machine heterogeneous storage management will reduce storage management complexity and associated operational costs while increasing organizational agility and flexibility.

The following subset of Windows Server 8 platform storage enhancements targets objectives described earlier:

• Improved NTFS metadata integrity on commodity storage – Beginning with Windows 8, NTFS has been enhanced to solely rely on the “flush” command in lieu of “forced unit access” (aka write-through) for all operations that require write ordering to ensure file system metadata integrity. This enhancement decreases the possibility of metadata inconsistency due to unexpected power loss on commodity hard disk drives while enabling the disk to cache associated data as long as safely possible. Cost-effective industry standard SATA drives can, therefore, be more reliably utilized.

• Maximized NTFS availability through online file system scan and repair – very large volumes (e.g. 64TB) can now be confidently deployed without fear of service downtime associated with file system check/repair operations. Windows 8 performs online scanning for corruptions, online identification of error conditions for on-disk file system data structures, and online repair. Windows 8 now incurs file system downtime only when absolutely needed and such downtime is proportional only to the specific number of (online) detected corruptions. All of these enhancements together deliver significantly improved NTFS uptime. The below graphic contrasts downtime incurred to detect and correct a single NTFS corruption, between Windows Server 2008R2 and Windows Server 8:

Further, this capability is enhanced through Cluster Shared Volume integration in clustered deployments, and manageability is significantly improved through file system health reporting in Action Center, Server Manager, and via PowerShell.

• Storage Spaces – Windows Server 8 delivers powerful platform-based storage virtualization capabilities enabling utilization of cost-effective commodity storage hardware for business critical application deployments. Two new concepts are introduced:

(a) Storage Pools: these are units of capacity aggregation, administration, and workload isolation

(b) Spaces (virtual disks): functionally equivalent to physical disks from the perspective of all users and applications, spaces deliver additional sophisticated capabilities including just-in-time allocation as well as resiliency to physical disk failures

As briefly introduced in Bill Laing’s post, Storage Spaces enable a diverse set of customers ranging from enthusiasts (such as Bill) to enterprises and hosting entities, to easily deploy continuously available cost-effective storage using commodity components. Storage Spaces were designed to be highly scalable supporting deployments ranging from just a few terabytes to multiple petabytes. Supported disk connectivity options include SATA (Serial ATA) and SAS (Serial Attached SCSI) – the latter option is expected to be more widely utilized in business environments. Features include:

• Optimal and flexible storage utilization through thin-provisioning and trim support
• Easy (elastic) capacity expansion – simply add more drives to the pool
• Resiliency to physical disk failures – high-performance mirroring (two or three copies), or parity based redundancy (with integrated journaling), is combined with rapid recovery and integrated per-pool hot-spares. As a result, despite storage hardware failures, applications and associated services can continue operating. Proactive background scrubbing further assists in minimizing possibility of downtime despite media failures.
• Application driven seamless error correction – some enterprise-class applications have traditionally maintained data checksums (or equivalent) enabling them to auto-detect data corruptions. Such applications can now be enhanced to (also) intelligently auto-correct and, thereby, seamlessly recover from data corruptions. This is enabled by an API which supports rendering all mirrored copies consistent from an application-determined “correct copy”.
• Pools can be comprised of heterogeneous media types (SSDs, or various HDDs e.g. 15K RPM, 10K RPM, or 7200 RPM) and space allocation can be targeted to specific underlying media for optimal performance.
• Ease of management via integration with the new Windows Storage Management API (see Windows 8 Platform Storage – Part 2) and full PowerShell support. The ability to delegate administration per-pool and per-space enables multi-tenant deployments.
• Continuously available and scale-out deployments through integration with Failover Clustering as well as with Cluster Shared Volumes. Note that clustered deployments require fixed provisioning.

In a nutshell, Storage Spaces deliver scale, availability, data integrity, and high performance on commodity hardware without “breaking the bank”.

A follow-on blog entry (Windows 8 Platform Storage – Part 2) will complete this introduction to some of the noteworthy platform storage enhancements in Windows 8. Until then, do let us know what you think about the upcoming Windows 8 features mentioned here.

Rajeev Nagar

Group Program Manager – Windows Storage & File Systems