Thoughts from the EPS Windows Server Performance Team
Useful Microsoft Blogs
When addressing system performance issues, a key element that is often overlooked is Disk Fragmentation. Even on a brand new system with plenty of RAM and high-end processors, the performance of the hard disk may be a bottleneck causing system performance issues. It takes time to load large data files into memory - issues become particularly noticeable when dealing with movies, video clips, database files or .ISO image files which may easily be several gigabytes in size. On a freshly formatted disk, these files load fairly quickly. Over time, however you may start to notice performance degradation - caused by disk fragmentation.
We touched on disk fragmentation when we were discussing the Page File a couple of months ago, but we never really got into the nuts and bolts of it. To understand disk fragmentation though, you need to understand the basic structure of hard disks. When you format a hard disk, the formatting process divides the disk into sectors, each of which contains space for 512 bytes of data. The file system then combines groups of sectors into clusters. A cluster is the smallest unit of space available for holding a single file - or part of a file. On NTFS disks, the cluster sizes are determined based on the drive size as shown below (this information is also available in Microsoft KB 314878). When formatting disks it is possible to change the cluster size, however this may cause additional performance issues.
Using the information above, if you were to take a 100MB video file, the file would be divided into roughly 25,000 pieces. If you save this 100MB file onto a freshly formatted disk, the information would be written in contiguous clusters. Since all of the clusters holding the data for this file are physically adjacent to each other, the mechanical components of the hard disk work very efficiently, pulling the data in one operation. In addition, the hard disk's cache and the Windows disk cache can anticipate data requests and fetch data from nearby clusters. This data can then be retrieved by an application from cached memory which is faster than retrieving the information from the disk itself.
Seems pretty straightforward, right? The problem is that the hard disks don't stay neatly organized for very long. Whenever you add data to an existing file, the file system has to allocate more clusters for storage. Typically, these clusters wind up being in a different physical location on the disk. As you delete files, you create gaps in the arrangement of the contiguously stored files. As you save new files (and this is especially true for large files), the file system uses up all of these bits of free space - resulting in the new files being scattered all over the disk in noncontiguous pieces. And thus we end up with fragmented disks and system performance issues because the disk heads have to spend time moving from cluster to cluster before they can read or write the data.
Enter Disk Defragmenter. This utility physically rearranges the files so that they are stored (as much as possible) in physically contiguous clusters. In addition to the consolidation of files and folders, the Defragmenter utility also consolidates free space - meaning that it is less likely for new files to be fragmented when you save them. For operating systems prior to Windows Vista, you had to manually run the utility or schedule automatic defragmentation via a scheduled task. On Windows Vista, Disk Defragmenter runs as a low-priority background task that is automatically run on a weekly basis without requiring user intervention. On Windows Server 2008, which uses the same Disk Defragmenter, the automatic defragmentation is not enabled by default. Also, the color-coded display that was part of earlier versions of the utility has been retired (believe it or not, more than a few people have asked about that!). Aside from the GUI version of the tool, you can also use a command-line version that enables some more granular control over the process. The utility name is DEFRAG.EXE and does require administrative privileges to run. The basic operation of the utility involves passing it a driver letter, for example: defrag.exe c: would perform a defragmentation of the C: drive. You can also specify other options through the use of command-line switches:
So now that we've covered what disk fragmentation is and how to address it, there are some caveats. You must have at least 15 percent free space on the disk volume before Disk Defragmenter can completely defragment the volume. If you have less free space, then a partial defragmentation will occur (unless you force the defragmentation with the -f switch). Also, you cannot defragment a volume that has been marked by the OS as possibly containing errors. This is where you would need to use the CHKDSK.EXE utility to ensure that there are no underlying disk issues. Some other things to look out for:
And with that, it's time to wrap up this post. Until next time ...
- CC Hameed
This post is a good overview of how to use defrag and what to use it for, but I'm interested to know how fragmentation performance issues might come into play when using RAID5 or other disk striping or storage i.e. SAN technology is used.
Do the performance penalties come into play due solely to the seek times of getting data from disparate parts of the disk or are there file system table efficiencies to be gained by defragmentation?
How would an administrator identify that fragmentation is actually causing a slow down on a system?
Bit of deja vu here, been a while (or an OS or two) since I discussed defrag seriously. Nice article
What is the allocation unit size that MS could recommend for 30-50 GB partitions so that wasted space is minimum yet fragmentation is also minimum?
I can understand about the color coded display being removed, but some sort of progress indicator is a must. Please consider adding at least a basic progress indicator to Windows 7.
I'll assume the version is more specifically for Vista as XP doesn't have as many options. XP's Defrag is also dated from 2001.
When addressing system performance issues, a key element that is often overlooked is Disk Fragmentation
We all know disk fragmentation isn bad, but cluster sizes on Windows boxes make lots of little pieces out of one big piece. When those pieces get out of order, we see slowdowns. CC Hameed of the Microsoft Performance Team explains the process in this
Automatic defragmentation is indeed the best solution to fix this persistent problem. After I installed an auto defragger (third party) I disabled auto defrag for the backups drive, but left it on for the OS and program files partitions. It's awesome since it does a great job yet I dont need to defrag manually or schedule it for midnights. Saves me time and effort. If auto defrag can help even a home user with just 3 PCs (ok, so one's a laptop), I can understand its usefulness in the corporate sector where its money and labor saving potential is quite high.
Very nice article indeed!
Next step is how you put it to work in a large organization: how do you start and schedule the job without slowing the computer down and thus disturbing tjhe user at times when it is critical that the user is not disturbed.
Great article. This was a big help to us when we were troubleshooting some issues with our application and we suspected disk problems.
One quick suggestion: it doesn't appear that the /? shows all of the above flags and settings. Admittedly I only tried it on XP Pro and not Vista, but it sure would be nice to have all of the help options display.
Do you know what -g switch means? (It's used in ScheduledDefrag task)
Hi is there any recommendation of how much Free space (%) is required on C:\ DRIVE ?
Is there any correlation of free space on drive affecting the performance throughput (particularly C:\ Drive) ?
What about programs running in the background? I have problems with 'driver changed - restarting' appearing.
I'm an amature. How do I get defragmetter to work from the Start/Programs/accessories/System Tools/Disk defragmetter. I get an error message that says...Disk Defragmetter would not start. Is there a simple way to fis this or do I need to go the the DOS window as the above article explains?
can diskfragmenting be harmfull if done too much? what about disk fragmenting a few times in a row?