Blogs

The Case of the Failed File Copy

  • Comments 68
  • Likes

The other day a friend of mine called me to tell me that he was having a problem copying pictures to a USB flash drive. He’d been able to copy over two hundred files when he got this error dialog, after which he couldn’t copy any more without getting the same message:

Unfortunately, the message, “The directory or file cannot be created”, provides no clue as to the underlying cause and the dialog explains that the error is unexpected and does not suggest where you can find the “additional help” to which it refers.  My friend was sophisticated enough to make sure the drive had plenty of free space and he ran Chkdsk to check for corruption, but the scan didn’t find any problem and the error persisted on subsequent attempts to copy more files to the drive. At a loss, he turned to me.

I immediately asked him to capture a trace with Process Monitor, a real-time file system and registry monitoring tool, which would offer a look underneath the dialogs to reveal actual operating system errors returned by the file system.  He sent me the resulting Process Monitor PML file, which I opened on my own system. After setting a filter for the volume in question to narrow the output to just the operations related to the file copy, I went to the end of the trace to look back for errors. I didn’t have to look far, because the last line appeared to be the operation with the error causing the dialog:

To save screen space, Process Monitor strips the “STATUS” prefix from the errors it displays, so the actual operating system error is STATUS_CANNOT_MAKE.  I’d never seen or even heard of this error message. In fact, the version of Process Monitor at the time showed a raw error code, 0xc00002ea, instead of the error’s display name, and so I had to look in the Windows Device Driver Kit’s Ntstatus.h header file to find the display name and add it to the Process Monitor function that converts error codes to text.

At that point I could have cheated and searched the Windows source code for the error, but I decided to see how someone without source access would troubleshoot the problem. A Web search took me to this old thread in a newsgroup for Windows file system developers:

Sure enough, the volume was formatted with the FAT file system and the number of files on the drive, including those with long file names, could certainly have accounted for the use of all available 512 root-directory entries.

I had solved the mystery. I told my friend he had two options: he could create a subdirectory off the volume’s root and copy the remaining files into there, or he could reformat the volume with the FAT32 file system, which removes the limitation on entries in the root directory.

One question remained, however. Why was the volume formatted as FAT instead of FAT32? The answer lies with both the USB drive makers and Windows format dialog. I’m not sure what convention the makers follow, but my guess is that many format their drives with FAT simply because it’s the file system guaranteed to work on virtually any operating system, including those that don’t support FAT32, like DOS 6 and Windows 95.

As for Windows, I would have expected it to always default to FAT32, but a quick look at the Format dialog’s pick for one of my USB drives showed I was wrong:

I couldn’t find the guidelines used by the dialog anywhere on the Web, so I looked at the source and found that Windows defaults to FAT for non-CD-ROM removable volumes that are smaller than 4GB in size.

I’d consider this case closed, but I have two loose ends to follow up on: see if I can get the error message fixed so that it’s more descriptive, and lobby to get the default format changed to FAT32.  Wish me luck.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • One problem with a FAT32 default for non-CD-ROM removables may be other devices which handle USB storage devices. I for once stumbled upon many devices like car stereos and cameras, which cannot cope with FAT32 and wont read them (usually results in locking up or non-secriptive "cannot read" error messages). Might this be a reason why this guideline for the format dialog has been chosen?

  • Interestingly I ran into the same problem two days ago. It reminded me (it turns out now that I was correct) of DOS limitations and so I simply tried to copy the file to another directory and all was well. It good to know what exactly the problem is. And a lot many people may be facing the same problem and not realising what the solution is. Maybe MS can add this to one of their tips section or something.

  • Is there any camera/MP3 player in the market that supports FAT16 but not FAT32?

    Btw, I thought it did make sense to use FAT16 when those Compact-Flash cards first released - they were in 1-32MB range only.

  • It's been years since I've seen a problem with not having enough root directory entries available. I'll admit, I didn't guess the nature of the problem before you explained it, but I feel fairly confident that I'd have diagnosed it properly in the field.

    I used to see this fairly frequently when Iomega ZIP drives first became popular, because people were treating them like floppy diskettes, and because the new VFAT extensions in Windows 95 contributed to the use of root directory entries.

    Exhausting all of the root directory entries on a 3 1/2" floppy diskette (112-- see http://www.pcguide.com/ref/hdd/file/fatRoot-c.html) wasn't easy with MS-DOS. You'd need an average file size of about 12.71KB to do that, and although I'm sure a significant number of users saw root directory entry exhaustion on floppy diskettes, I'd imagine most floppies had full data areas before they had full root directories.

    With the advent of VFAT, and the larger storage capacity of ZIP disks, the conditions for root directory exhaustion became much more favorable. Users weren't familiar with making subdirectories on floppies, and they'd store tons of files in the root directory of their ZIP media. On a 100MB ZIP disk, piling on files of 200KB or less would exhaust the 512 available root directory entries. If you used long filenames, additional root directory entries would be populated with long filename data.

    MS-DOS and Windows 9X return a 'Disk Full' error when you run out of root directory entries on a FAT volume. I saw multiple cases where users had purchased additional ZIP media because they erroneously thought their other medias were "full", when in reality they just exhausted all the root directory entries. *smile*

    I notice that the NTFSD posting referred to plumbing this status code out in Windows XP. I find it quite humorous that Windows Vista, the present-day flagship version of the Windows operating system, returns a more arcane and less helpful message to an error condition than more than decade-old Microsoft operating systems.

  • The minute you wrote "cannot create" I thought "hmm, the root directory is full".

    But the drawbacks of FAT aside: The big cluster sizes usually enforced by FAT16 (due to its very nature of only having 2^16 clusters available to it), is very useful for those of us using digital cameras.

    I shoot in raw format, and my raw files are 16GB in size. Calculate the overhead of 32KB clusters vs 1KB clusters, and be amazed. The size of the cluster tables alone are staggering... (and as the camera, IMO, should buffer the cluster table, it will steal the camera's memory as well, and I suspect not always without side-effects given certain models with the not-so-latest firmware version)

    Instead of making FAT32 the default, please suggest that they (again) open up the possibility for 128KB big clusters. Even if that support is only extended to the format command line utility. Make the command switch "/yesIknowwhatIamdoingGoAway!" if need be.

    Currently, format hints at large cluster supports, but at the end of the formatting operation, it backs down and says "no soup for you" (paraphrased).

    For other devices where you might care for long file names, created date (in addition to modification dates), huge (2GB+) files and so on, FAT32 is nice. But for most purposes, this is not needed on memory cards. It only gets in the way!

    Actually, I'd go so far as saying that if you consider FAT32 instead of FAT16, you might as well go all the way and turn it into a NTFS partition!

    --

    Rune

  • I had a similar experience on a Windows XP machine last summer. My girlfriend’s sister had been taking a large amount of medium sized pictures that she wanted to copy to a 512 MB flash drive. The error message that XP showed was in no way better than the one on Vista (it was actually quite horrible). She and her father, both pretty inexperienced with computers, where both convinced that there had to be a failure with the USB stick and were going return it to the store where they've bought it for warranty repairs.

    As luck would have it I got hold of it first and started off with looking at it with the "Disk Management" tool in XP and saw that it was using FAT instead of FAT32. As soon as I formatted the stick to FAT32 everything worked ok. I then had a hard time to try and explain to them what the difference with FAT and FAT32 is (remember that they where computer illiterates). Now when I read you blog I start tp wonder how many USB sticks has been returned or thrown away because of similar problems elsewhere.

    Therefore, to help the poor users out there who do not understand these kinds of ancient limitations and file systems I hope that you can get them to change the default to FAT32 for the formatting dialog soon – Vista SP1 perhaps?? And if nothing else, have them make a proper error message for this so that mere mortals can understand what to do.

  • Obviously didn't happen this time but it can also fail if the absolute path gets too long (255? in NTFS).

  • u r unbeatable Mark, love you...

  • I concur with Dennis.

    In the Windows world, only ancient stuff like DOS/Win95 don't support FAT32, and those don't support USB anyways. But many non-PC devices read flash cards or USB disks, such as cameras, DVD players, direct-from-camera printers, etc. These devices started to support FAT32 only when flash sizes were approaching 2GB, which was not that long ago. So Windows tried to format devices to work with them.

  • Ha, I've seen that one coming after the first two paragraphs :-)

    The main problem with FAT32 is that unlike FAT16, Microsoft claims to have patents covering it and thus every manufacturer that wants to pre-format its devices with FAT32 or wants to build a car stereo/DVD player/etc. that supports FAT32 would have to pay royalties to MS. They all try to avoid that.

  • The drive in question being under 500 megabytes, FAT16 seems to be a /good/ default choice anyway. Of course the users should store their files under a subdirectory, as you said. Another advantage of FAT vs. FAT32 in this case is the users could d**space it if need be - assuming win 9x of course. Why MS chose not to include a doublespace file system in 2k and over is beyond me. (Yes, I know about EFS, but it's not an answer).

    Is there a "3rd" party d**space add-on for 2k+ ?

    Regards

    --

    Ninho

  • This is similar yet different to a problem I occasionally encounter. And please contact me if you think you can help, but I can't replicate the situation on demand at the moment. I'll have to wait until I encounter it at home...

    Anyway, Here is the gist of it: I run as a 'User' user, not an 'Administrator'. I download a zip file containing pictures. I use 7-zip to extract the zip to .\[zip name]. I browse the pictures. I then try to Cut the folder and Paste it into a directory that is symlinked from my user directory to a directory on my second hard drive.

    Sometimes, the Cut / Paste fails saying insufficient privileges, when I certainly DO have privileges for the entire source and destination (it comes from my desktop to something with explicitly defined permissions. I get the UAC dialogue to allow the move to which I supply my Admin password and then I am told that it cannot be done, Try Again?

    It's weird because it's random. Sometimes deleting the unzipped directory and then re-unzipping the folder will allow it. Sometimes copy and paste (rather than Cut and Paste) will allow it.

    It just seems like a mix-up to what I feel should be a straightforward task, haha.

  • You were the best before you started to work at Microsoft, and you are still the best.  I've never seen anyone go so deep into the Windows system since Peter Norton.

    As a computer technician, I find myself digging deeper in areas I know nothing about and actually learning something.  I disarmed a root kit and restored the CD-ROM's functionality by researching, testing, and documenting.

    Then comes an article like this one where it goes into deep detail about how Windows sees a USB drive, and I learn of another tool to help me troubleshoot issues with processes.

    I've place this blog as a feed and I plan on reading it every week.

    Thanks Mark for bringing your technical expertise and determination to make the Windows OS the best it can be.

  • So Mark will Windows default to exFAT after Vista SP1 ships? And is exFAT going to be available on XP?

  • I got my 2 GB Flash drive to use for Vista as Ready Boost but I didn't notice any speed diff so I stared dropping files and folder into it using it as a back up. Nice.

    Then I plugged it in and it said this is empty. I checked via properties and it showed the drive to correctly about 1/2 full.

    I never looked to see its original format. Based upon the above posts do you think I should format it FAT32?