Microsoft Enterprise Platforms Support: Windows Server Core Team
EPS Team Blogs
Product Team Blogs
“Why can’t I have a drive bigger than 2 TB?”
This is a common question. And as storage has gotten bigger and cheaper, I see it more and more. So let’s take a few minutes to talk about the mysterious 2 TB limit.
There are actually three different 2 TB limitations that are commonly hit...
Partition Size Limitation
The partition size is pretty straight forward. On an MBR (Master Boot Record) disk, the locations where the partition sizes are stored are only 4 bytes long. Since this is in hexadecimal, the largest value we can stuff in there is all F’s. So the max value would 4,294,967,295 in decimal.
FF FF FF FFh = 4294967295d
This maximum partition size is not in bytes, it is in number of sectors. Since currently sectors are limited to 512 bytes, the maximum size ends up being 2 TB.
4,294,967,295 sectors * 512 bytes/sectors = 2,199,023,255,040 bytes or 2TB.
Number of Clusters
The second limitation is harder to spot. It is a limitation of NTFS. NTFS is limited to (2^32 -1) clusters….no matter what. The smallest cluster size possible is 512 bytes (1 sector). So again the math leaves us at 2,199,023,255,040 or 2TB.
(2^32)-1 = (4,294,967,296)-1 = 4,294,967,295 clusters
4,294,967,295 clusters * 512 bytes/cluster = = 2,199,023,255,040 bytes or 2TB
Here is a quick reference chart to help you see the maximum NTFS size for each cluster size.
NTFS Max Size
17,592,186,040,320 (16TB) Default cluster size
Cluster size really depends on your needs. While 512 is fine if you just have a bunch of tiny files, it isn’t as efficient for say a volume with just SQL DBs. Also, a tiny cluster size can adversely affect VSS. But that is a topic for another time.
This is by far the hardest to understand as it requires some basic SCSI knowledge. Microsoft Windows operating systems support two different SCSI standards when it comes to reads and writes. There is a third but it is very old and is mostly just used on tape devices. So let’s just forget about that one and stick to the two that are relevant.
These two standards are Read10/Write10 and Read16/Write16. This all has to do with the way the CDB (Command Descriptor Block) is structured.
Read10/Write10 – This standard reserves bytes 2-5 to define the LBA (Logical Block Address). Think of LBA as sector numbers….it makes it easier on your brain. So we have 4 bytes that can define the addressable sectors. Just like in the ‘partition size limitation’ we are back to dealing with a 4 byte number used to define all the addresses on the drive.
And just like before, the above is just the possible number of address (number of sectors). Multiply by the standard sector size of 512 bytes and we get…
What this all means is that when Windows is using the Read10/Write10 standard, then the biggest drive that will be supported is 2TB.
Read16/Write16 – Sometimes called LBA64 by some vendors, this standard reserves bytes 2-9 to define the LBAs. That would be 8 bytes, each byte being 8 bits in size. Now here is where we start getting into some really big numbers. The approximate size comes out to be around 8ZB (zettabytes). Here’s a quick chart to put it in some perspective.
So it is going to be a while before we have to worry about running into the limitation of Read16/Write16.
Exceeding the limitations
Each limitation has a way of getting around it. Otherwise we’d still be stuck at 2TB.
Partition size limitation – There are actually two ways around this. The first way is to convert your disks to dynamic disks and create volume sets. This functionality has been around since Windows 2000. This doesn’t really allow you to increase the partition size. What it does is give you the ability to chain your partitions together to form larger volumes. So I could use two drives of 2TB and create a volume of roughly 4TB in size.
The second method to bypass the partition size limitation is to use a GPT (Guid Partition Table) configuration. In Windows 2003 SP1 Microsoft introduced its implementation of the GPT. A disk configured to be GPT rather than the MBR style would have a 32 sector partition array instead of a tiny 64 byte partition table.
NOTE: 32 sectors is equal to 16,384 bytes
The partitions that can be defined on a GPT disk can be up to 16EB in size.
Number of clusters – There really isn’t a way around this limitation. NTFS is currently still limited in its number of clusters. However, you can get past 2TB by making sure your cluster size is larger than the minimum size of 512 bytes.
It is important to note that for the most part, if you create a FAT partition and then convert it to NTFS the cluster size will come out as 512 bytes. There are ways around this but most of them require that you know ahead of time that you are going to be converting your FAT partition to NTFS.
SCSI goo – There isn’t anything in Windows you can do to get around the limitation of Read10/Write10 as Windows is already able to use Read16/Write16. However, the hardware MUST support it as well. Windows will query the storage devices and negotiate if Read10/Write10 or Read16/Write16 is to be used. When in doubt, check with your storage vendor.
Robert Mitchell Senior Support Escalation Engineer Microsoft Enterprise Platforms Support
NOTE: This is a three year old blog post that I did before 4k drives really started to become popular. If you want to read about 4k drives, I published an article on the Windows IT Pro site about it.
Also, I didn't write about booting because that's outside the scope of my posting. This posting was simply about the various things in Windows storage that you bump into while trying to push past 2 TB.
Thanks Robert for giving me a clear understanding of the 2TB limit.
I have only a basic knowledge of hard Drives, so its good to know these things.