The Storage Team Blog about file services and storage features in Windows and Windows Server.
Chat Topic: Join the NTFS file system team to ask questions about NTFS in Windows VistaDate: Wednesday, July 12, 2006Wendy [MSFT] (Moderator):Welcome to today’s chat with the NTFS Team. We will try to answer as many questions as we can today. Participants should type their questions, select the “Ask the experts” check box, and click “Send.” Those posts will go into a private queue, from which our experts will draft answers and repost questions in the upper window with their answers. (To confirm: if you selected the “Ask the experts” check box when you posted, you don’t need to resubmit. Wendy [MSFT] (Moderator):We’ll tell you all about what’s new (and what’s the same) and answer questions relating to Transactional File System (TxF), symbolic links, NTFS interop between Windows XP and Windows Vista, and much more!
Surendra[Msft] (Expert):Hi I'm Surendra Verma. I'm the development manager for the file-systems group.Senthil [MSFT] (Expert):I'm the PM for NTFS. I have been with the NTFS team for just about 6 months now and have been working in the file systems area for the last 4 years.Wendy [MSFT] (Moderator):Hi Everyone! My name is Wendy and I just recently joined the Longhorn/Windows Vista team. Previously I was part of the R2 and MSN 9.0 Beta Teams. My job is to see that you BTs get what you need to have the best beta experience ever! We’ve got lots of great things in the works for you BTs! Dana [MSFT] (Expert):Quick bio: I am the program manager for NTFS' transactional technologies in Vista.Malcolm [MSFT] (Expert):Hi, my name is Malcolm, and I'm working as a developer on NTFS.Jill [MSFT] (Expert):Hi, I'm Jill Zoeller and I'm the Community PM for the Core File Services team.Dan [MSFT] (Expert):Hi, my name is Dan. I am one of the NTFS developers.
Senthil [MSFT] (Expert):Q: If you guys own this feautre, any work being planned to clean up the NTFS permissions editing UI?A: What kind of problems are you seeing with this UI?Andy [msft] (Expert):Greetings. I'm Andy Herron and I'm a developer on the NTFS local file system. I work on both core NTFS and TXF. Have worked on NT for roughly 14 years.
Jill [MSFT] (Expert):Q: Will we get a more reliable defrag?A: Defrag is still a work in progress. We posted an FAQ recently on our blog at https://blogs.technet.com/filecab/articles/440717.aspx to answer some of the common questions.Malcolm [MSFT] (Expert):Q: Has there been a change in how ACLs are handled in Vista? (e.g., in XP - an admin could change permissions even if they didn't have the appropriate access privilages)A: No. The specific issue you're asking about is by design - owners have reserved rights on objects in addition to those provided explicitly in the ACL. That's a requirement of C2 certification, and has been in NT since the beginning.Malcolm [MSFT] (Expert):Q: With development of WinFS stopped, is it now up to the NTFS team to come up with solutions for all file requirements for the future?A: NTFS will continue to develop new storage solutions over time, such as the great work we've done with Vista. I don't think that means we'll be the only innovators in the storage space, however :)Malcolm [MSFT] (Expert):Q: Will Vista make any specific uses of NTFS file streams, file streams seem to be overlooked by most people and to the average user serve little purposeA: Although not directly related to NTFS - NTFS simply stores stream information as requested by applications - XPSP2 and newer have used streams to store metadata for untrusted downloads, and similar things. Explorer and IE in Vista continue to use streams for metadata of user files. WS03 introduced new APIs for enumerating streams.Wendy [MSFT] (Moderator):For those just joining us, today’s chat is about NTFS. To post a question, please type your question, select the “Ask the experts” check box, and click “Send.” That way, we can track which questions we still need to answer.
Dana [MSFT] (Expert):Q: What do each of you find are the most exciting and groundbreaking changes in NTFS Vista?A: Some highlights include self-healing, symbolic links and transactions. I am biased of course but IMHO NTFS supporting transactions is really the most groundbreaking change. This feature allows you to coordinate file system operations with each other, the registry (also transactional), SQL, MSMQ and any other transactional resource that works with DTC. Think what this does to your coding style! You no longer have to figure out what changes were made to a file and back them out if there was an error!Jill [MSFT] (Expert):Q: If a user has a laptop and has his My Docements redirected to a USERS share on the server and available offline, and sets encrypt the offline files will they also be encrypted on the share on the server.A: This is the NTFS team here today. The Offline Files team would love to see your question in the beta newsgroup though! Try asking in the microsoft.beta.longhorn.networking.filesystems newsgroup.
Jill [MSFT] (Expert):Q: running the defrag when the computer next switches on - is USELESS. what if this is a lap top and i am now on battery. the defrag should NEVER run when on battery. low priority I/O is useless if is still causes seeks to ruin high priorityI/OA: Thisi s all good feedback for the defrag team, but we're here with the NTFS team today. Please log your feature request for having defrag not run on battery. Malcolm [MSFT] (Expert):Q: Are there plans to have NTFS work on external USB devices, or is that the purpose of exFAT?A: NTFS is supported on removable media currently, and will be supported on media that can be surprise removed in Vista (eg., flash devices.) It may not be the wisest thing to do, since NTFS will rewrite the same sectors frequently, which is not optimal for flash devices. The primary intent for NTFS is USB disks.Surendra[Msft] (Expert):Q: If a USB external NTSF disk is used with a laptop and a folder is encrypted, can the same user open and read the contents of the folder if it is plugged into his desktop?A: No, not unless they've exported the encryption key (certificate) along with the data. Andy [msft] (Expert):Q: Is there any movement towards placing files more intelligently so they dont get fragged so much?A: Yes, we've got background defragmentation going on now as a service. We've also got a prefetcher that based on usage patterns will attempt to fill the cache with what it expects will be asked for next. From the NTFS side, this all goes on above us. The only thing we do is try to minimize the number of disk runs allocated to any given file up front at file extention time. That logic has been refined but not majorly expanded in Vista.Senthil [MSFT] (Expert):Q: Should I be able to access a symlinked folder via the network?A: Yes, remember that the client and the server needs to be Vista. By default only local to local links are turned on. You would need to enable the Local to Remote symlinks through the policy editor.Surendra[Msft] (Expert):Q: Will we get a more reliable defrag?A: Here's some info I got from the defrag team for this:
Defrag scheduled by default on all volumes – no need to worry about defrag anymore
New streamlined and simplified UI
Defrag engine improvements
More meaningful metrics for free space fragmentation – free scpace fragmentation was rarely > 0% before
By default only do partial defrag which consolidates files only if they are in extents smaller that 64 MB. This allows for defrag of large files even when there isn’t much free space available. It also prevents moving large extents around when it is not necessary.
Improved and more efficient free space consolidation pass
Improved cancellation support -- defrag can exit even if in the middle of defragmenting a really large file (not possible before – caused machines to fail to reboot for patches etc)
Using Low priority I/O and low priority CPU to minimize impact on other activity while running (I can defrag while watching video off of the same spindle)
Dan [MSFT] (Expert):Q: Are there any specific changes in NTFS that might affect things such as file recovery (using Winhex, Encase or the like?)A: NTFS in Vista has a self healing feature that may affect file recovery. Basically upon detection of certain types of corruption, NTFS will repair the corruption in place. The goal is to restore consistency of the metadata much like chkdsk will except it does not require a lock of the entire volume. If user does not want it enable by default, it can be disabled on a per volume basis using fsutil.exe. Dana [MSFT] (Expert):Q: will ntfs be slowly evolving towards an easy winfs implementation from now on are are the two mutually exclusive?A: WinFs was initially built on top of Transactional NTFS (TxF) for storing files. WinFs did a lot for you -- including providing a rich new programming environment. NTFS will always support existing users and today we are evolving our APIs not providing a whole new environment. As we go forward I expect that there will be enhancements but to what extent has not yet been planned. WinFs and NTFS/TxF were never mutually exclusive but complementary.Malcolm [MSFT] (Expert):Q: Why are Administrators not given permission to edit security on disk drives by default?A: This question is probably better to take up with the security folks. Basically the fear is that a large amount of the Windows userbase runs as Administrators, and providing too much access allows for viruses, worms, and badly behaved applications to cause problems. UAC is intended to overcome this and require elevation before writing to certain areas in the filesystem. The restriction on Administrators on the root of a volume comes from that. You can, however, take ownership of the root and manipulate ACLs afterwards.Senthil [MSFT] (Expert):Q: Are the deadly slow problems, accessing files and drive in Explorer, NTFS-related or Explorer-related?A: I think the performance related issues are being looked at from all angles. The NTFS team is aggresively working to identify and solve the performance related issues. Surendra[Msft] (Expert):Q: With NTFS supporting transactions, does that mean no more registry corruption? =)A: Registry doesn't use the NTFS transaction implementation. It does however expose transactional behavior like NTFS does.
More directly to the point, transactions can help with incosistent updates, but if you're talking about corruption due to bad hardware etc, transactions can't help with that.Jill [MSFT] (Expert):Q: My VISTA build 5456 has a default scheduled task defrag.exe -c that runs once a week. I have a C; D: and G: NTFS disk and a E: DVD-RW. Is that only defragging the c drive and if so do I need to change the line to defrag.exe -cdg or add three entries.A: Great question--I believe that defrag covers all volumes by default (not counting your DVD drive). This is a good question for the newsgroups since the NTFS team here today doesn't own defrag (it's a separate team). Surendra[Msft] (Expert):Q: Are there any changes to the NTFS Change Journal?A: The change journal is now on for all machines in Vista. It's turned on by the services (like search indexer).
Dan [MSFT] (Expert):Q: What does self-healing mean? Can you tell us (me) more about that? :)A: Self healing is a new feature in Vista. Basically upon detection of certain types of on disk corruption, NTFS will repair the corruption in place. The goal is to restore consistency of the metadata much like chkdsk will except it does not require a lock of the entire volume. If user does not want it enable by default, it can be disabled on a per volume basis using fsutil.exe.Surendra[Msft] (Expert):Q: Does the NTFS from Vista influenze an NTFS formatted disk example : "NTFS formatted by XP data drive on local machine", or will there be no change to the NTFS system be inflicted on that drive?A: NTFS format is unchanged from XP.Andy [msft] (Expert):Q: I'm really interested in how NTFS works. Is there any (technical) documentation available?A: "Inside Windows 2000" has a pretty good section on NTFS. Haven't checked Inside Windows Server 2003 but I believe it may provide an updated section. Rajeev Nagar, Windows NT File System Internals, OSR Press is also good, though a bit dated.Malcolm [MSFT] (Expert):Q: Question for the Queue: Will each FSO object still take up approximately 1K in MFT usage? Meaning will an NTFS volume with 1 million objects still be 1GB in size?A: Yes. The size per file in the MFT is 1k under most conditions, so each object will require a minimum of 1k of MFT space.Dana [MSFT] (Expert):Q: Is the transactional file system something that developers can utilize via an API, or is this an internal system service?A: Via APIs. There are enhancements to the filter manager for filters; new Win32 api's for applications. We are working on managed code api's as well. TxF (Transactional NTFS) is NTFS -- not a new service. It is built on top of new kernel technology (Kernel Transaction Manager) there are no services started for TxF. (TxF does its recovery at boot before any services even start.)Malcolm [MSFT] (Expert):Q: Is there a really easy way to make a symlink?A: Open CMD, and type 'mklink'. This tool will allow you to create junctions, hardlinks and symbolic links.Wendy [MSFT] (Moderator):For those just joining us, today’s chat is about NTFS. To post a question, please type your question, select the “Ask the experts” check box, and click “Send.” That way, we can track which questions we still need to answer.Jill [MSFT] (Expert):Q: Is there a help option with defrag for command line switched I have tried -? /? -help and /help and it appears to just close.A: Please log a bug on this. I think there should be command-line help for all command-line tools. Andy [msft] (Expert):Q: What is exFAT? What is DTC?A: exFAT is a new file system based on FAT that allows for larger volumes. If you search www.microsoft.com for exFAT there is an interesting slide deck by Vishal that covers it. The intention for it is for those applications where NTFS is not appropriate such as embedded devices exchanging data with PCs. DTC is Distributed Transaction Coordinator.Senthil [MSFT] (Expert):Q: My testing shows Vista Beta2 takes ~twice as long as XP to format an external USB drive (NTFS format). Will this be fixed in the final release?A: There was a bug in XP which was not going through and writing out 0's to the disk during formatting. Vista fixed this and that is the reason for the observed performance difference.Senthil [MSFT] (Expert):Q:  With hardlinks already supported in older versions of NTFS, will 'mklink' be available for down-level operating systems?A: No. Symbolic links will be supported for vista+Malcolm [MSFT] (Expert):Q: Will you actually utilize the LastAccessTimeStamp for core functionality in Vista/Longhord, as appears to not be the case in previous Windows Server Operating Systems?A: LastAccess has a very coarse update granularity (we don't update it more frequently than one hour.) The reason is for performance - we'd have to do this update very frequently otherwise. As a result, it's not generally something that is useful to applications (including other components in Windows) to use. It's primarily intended as a user feedback mechanism.Dan [MSFT] (Expert):Q: Can self healing be turned on for specific files or is it all files?A: Self healing is either on/off on a per volume basis. You can use fsutil.exe to do that.Andy [msft] (Expert):Q: Will TxF internals be documented to allow Secure Delete utilities to be written?A: For now the answer is "no".Malcolm [MSFT] (Expert):Q: can we use expanded drive letters such as cc:\ or something to break the current limit?A: Mountpoints are a better scheme to avoid this limit, and have existed since Windows 2000. In addition, UNC can be used to access network shares. DOS drive letters are a very historical remnant.Senthil [MSFT] (Expert):Q: This is more of a server oriented NTFS question but are volumes aligned on a 64th sector now by default?A: I think there was a change related to supporting 4K sector disks that warranted this, but i'm not entirely confident that this change has been made. Dana [MSFT] (Expert):Q: Does support for transactions in the file system include non-NTFS file system (I am thinking FAT here, but of course there are other FS types out there as well)??A: Not really. You cannot open a file on FAT with a transaction. But, saying that if you copy a file from a non-transacted read-only file systems (ex: CDFS) to Transactional NTFS, that transaction will work. Jill [MSFT] (Expert):Q: more of a bitlocker question really but worth asking - assuming I encrypt my ntfs file system with bitlocker and subsequently suffer a hardware failure that requires a new motherboard - is the ntfs file system on the encrypted drive lost ?A: We've got a question out to the Bitlocker team now...if we get an answer we'll post back. Also, Wendy just said to mention that the Bitlocker feature focus next week is cancelled.Malcolm [MSFT] (Expert):Q: In Vista, there is very little support for FAT-32. Is the general MS consensus that users don't use anything but NTFS these days? Is there a defined plan to phase out the older file systems?A: FAT has serious limitations, particularly in scalability cases. NT retains a modular design that is open to all filesystems, and we're even working on new drivers in the case of UDF or exFAT. So the plan isn't to phase out non-NTFS filesystems, but that FAT really isn't a filesystem designed for the future.Senthil [MSFT] (Expert):Q: File operations are horribly slow in our latest interim build, have there been changes checked in to resolve these issues and will we see those changes in the next interim? I'm specifically talking about file copy operations and archive expanding.A: We know of certain paths that are having perf problems and we are actively working on fixes..Andy [msft] (Expert):Q: Are you familiar with the error message "The data present in the reparse point buffer is invalid" and do you know if it involves some kind of bug?A: Not familiar with it and it sounds you should submit it as a bug.Wendy [MSFT] (Moderator):For those just joining us, today’s chat is about NTFS. To post a question, please type your question, select the “Ask the experts” check box, and click “Send.” That way, we can track which questions we still need to answer.Malcolm [MSFT] (Expert):Q: What features of NTFS in Vista will / won't be visible / usable to other OSs on a dual-boot system running XP or Windows Server 2003 as well as Vista?A: We've worked really hard to minimise changes to the on-disk structure. This means that dual booting XP/2003 will be supported while accessing the same volumes. However, since XP/2003 won't understand the new Vista features, those platforms won't be able to do things like participate in transactions, perform transactional recovery, resolve symbolic links, etc. So generally, the data is interchangable, but the features are specific to each platform.Surendra[Msft] (Expert):Q: Will there be any changes in how the old school file attributes (eg; read-only on a directory via the attrib cmd) be handled in NTFS on Vista?A: No...Senthil [MSFT] (Expert):Q: What are Junctions, symbolic links, hardlinks?A: Directory junctions:
Directory junctions allow a local directory or a remote share to be grafted into the local namespace. In other words a directory junction appears to the user as a normal directory, but it is actually pointing to another directory that may be local or remote (exposed as a share).
Senthil [MSFT] (Expert):Q: What are Junctions, symbolic links, hardlinks?A: Symbolic Links allow a file or directory to point to another file or directory, respectively, where the target file or directory can be local as well as remote.
Dana [MSFT] (Expert):Q: Does the self healing or other disk utilities and maintenance tools make use of the TxF, i.e is it possible that some files could be rolled back automatically whilst other dependant ones are left untouched and at an incompatible version causing instabilty?A: The second part of your question is "no". Out of the box the component servicing (think Windows Update) and System Restore use TxF. They coordinate their updates with transactional updates on the registry so that all of the changes make it or none of them. That is what you want! Consider what happens when someone does a force shutdown right in the middle of updating their system. Before, there was a good chance you corrupted your system. Now, TxF (and the registry) will roll back all those changes before the system even boots.Dan [MSFT] (Expert):Q: What is the trigger point for detecting that a file is corrupted therefore needs to be healed ?A: When a file is loaded from disk, NTFS performs basic metadata verification. If corruption is detected at that point, self healing (if enabled) will kick in to do the repair. There are also many other places where NTFS will perform consistency checks and may invoke self healing.Wendy [MSFT] (Moderator):If you need to leave the chat early, don’t forget to let us know how we’re doing! To the right of your emoticon pull-down box is a “Chat Feedback” option. If you don’t feel the chat was helpful, please ensure that you provide verbose comments on why you didn’t feel it was helpful and what we can do to improve the experience for you!! If you would like to provide more feedback than the 1000 characters will allow, feel free to eMail us at LHBeta@microsoft.com.
Malcolm [MSFT] (Expert):Q: It was mentioned that NTFS was not an optimal format for flash disks - "since NTFS will rewrite the same sectors frequently". But I have seen suggestions that ReadyBoost wants NTFS format on USB flash for performance. Can you elaborate?A: The ReadyBoost team are trying to minimise the number of writes done to the USB device, as this is not the optimal case for them. Instead, ReadyBoost uses USB for accelerating reads which would otherwise have had to go to disk. Clearly it's impractical to eliminate all writes, but the bias is in favor of reads. Perhaps this would be a good starter question for a ReadyBoost chat in the future ;)Surendra[Msft] (Expert):Q: If a user shows up with an installation DVD and a portable hard drive, they could just copy anything and everything from your system over in a few minutes, just by restarting the computer and booting off of the installation DVD. Is that true?A: Once you've lost physical access to your computer, unless the data is encrypted (look into bit-locker) you have no security. They can read your files, copy them out, modify them... that's always been the case. bit-locker is new in vistaSenthil [MSFT] (Expert):Q: Does Microsoft intend on any major changes to NTFS to work with more than 2TB with encryption and unlimited Cluster Numbers?A: Nothing has been planned on this front.Dana [MSFT] (Expert):Q: Database best practice is to put transaction log data on a volume other than the one being deliberately updated, then update the primary volume. Will Vista writes resemble databases in this respect?A: No. In fact, consider removable disks in your scenario -- that wouldn't work for the general case. Our "default" transaction log for each volume is on the volume. You can create a secondary log that is used by an application-owned directory tree. Since the application owns this space, it could put logs on another volume. Senthil [MSFT] (Expert):Q: Question for the Queue: What other behavior flags will we be able to manipulate with fsutil?A: There has been a whole buch of things added and fsutil /? will give the complete list.Malcolm [MSFT] (Expert):Q: Can't find the Transational NTFS Win32 APIs... can you give me one of them so I can find it?A: Start with http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ktm/fs/createtransaction.asp. Note that this is preliminary documentation subject to change. In this particular case, large changes are planned, and perhaps you should re-read this documentation after we have some more Vista releases.Senthil [MSFT] (Expert):Q: I'm not familiar with symlinks but have used linkd.exe resource kit util. Is it roughly the same thing?A: linkd is used for hardlinks, mklink can create both symbolic and hardlinks. Dana [MSFT] (Expert):Q: Which folders are the TxF entries stored?A: TxF works on the whole volume. The exception to that is if an application creates a secondary "store" (we call it a secondary resource manager (RM)) where all transactions are "managed" by that secondary rm. The metadata for TxF is stored in $EXTEND or (by default) in the root of the secondary RM.Malcolm [MSFT] (Expert):Q: is data forking still in the new ntfs?A: Depends what you mean by data forking. If you mean forking in the Macintosh sense of the term, NTFS implements streams for this purpose, which are far more flexible. NTFS in Vista will retain support for Streams that have shipped in previous releases.Andy [msft] (Expert):Q: Why is it exFAT and not exNTFS? Is NTFS not needed?A: NTFS is being enhanced on an ongoing basis so we didn't need an exNTFS. The need for exFAT comes from providing other companies (think embedded devices such as cameras) a file system spec that they can write to that is somewhat backward compatible logically with FAT but eliminates some of the design limitations. It is currently not our intetion to have embedded devices (not running Windows) try to write to NTFS.Surendra[Msft] (Expert):Q: Now that WinFS is scrapped, are there any plans to replace NTFS still?A: There were never any plans to replace NTFS. Scrapping of WinFS doesn't change that.Jill [MSFT] (Expert):Q: Any changes to how defragmentation works? Like automatic defragging of smaller files or changes to the defrag API?A: We're recently posted an FAQ at https://blogs.technet.com/filecab/articles/440717.aspx to cover some common questions. We are also planning a defrag chat on the 26th. Also, in the microsoft.beta.longhorn.filesystems.local beta newsgroup, search for posts from Georgi Matev. Surendra[Msft] (Expert):Q: Do you already have plans / ideas for the next version of NTFS?A: Yes we do! We're heads down with Vista right now.
Andy [msft] (Expert):Q: Does self healing have limitations in terms of what volumes would be healed? Like NTFS only / mapped drives/ mapped network shares ?A: Local volume only since exclusive access must be guarenteed. For first version, some repairs to ntfs internal files are limited. This will be expanded in the future.Surendra[Msft] (Expert):Q: We used to worry about allocation unit sizes in FAT/FTA32 because of wasted space. Is that a problem with the current NTFS?A: With NTFS you generally dont' need huge cluster size (allocation size) to scale to larger volumes, so in that case it helps with wasted space.Wendy [MSFT] (Moderator):For those just joining us, today’s chat is with the NTFS Team. We’ll tell you all about what’s new (and what’s the same) and answer questions relating to Transactional File System (TxF), symbolic links, NTFS interop between Windows XP and Windows Vista, and much more!
To post a question, please type your question, select the “Ask the experts” check box, and click “Send.”
Dana [MSFT] (Expert):Q: more of a political question .. You have already stated that the TxF internals wont initailly be published (obvious security reasons really) but is this likely to cause third party disk utility developers to sob to the EU demanding more access to the fs?A: Cannot comment on the EU. We do work closely with filter-driver develoers at the IFS-Plugfest. If a vendor needs to get into the bowels of TxF -- they can meet with me and other develoers there. There are various alternatives to work closely with MSFT but its handle 1 by 1.Malcolm [MSFT] (Expert):Q: What are junctions, hardlinks?A: A junction is a special file that tells the system to redirect to another file on disk. A hardlink represents another link to the same file. The implementations here are radically different, which in turn gives rise to different rules. For example, hardlinks are not supported on directories, and junctions can only point to directories. Hardlinks must refer to files on the same volume, whereas junctions do not. These are only really similar in a conceptual sense, in that they both allow one name to refer to a different object.Malcolm [MSFT] (Expert):Q: Any changes to alternate stream functionality?A: Not as such. This is a mature feature. There may be more users; DIR now supports displaying streams in a file, for example. But from NTFS, the feature is the same.Surendra[Msft] (Expert):Q: more of a political question .. You have already stated that the TxF internals wont initailly be published (obvious security reasons really) but is this likely to cause third party disk utility developers to sob to the EU demanding more access to the fs?A: I can't relate to legal questions. However, NTFS format is not officially documented.Surendra[Msft] (Expert):Q: With the arrival of more tools - and even viruses - that use ADS (Alternate Data Streams), will there be tools in Vista that support the enumeration and management of ADS from the GUI and/or command line?A: There're APIs (new) in Vista that let you enumerats alternate data streams.Jill [MSFT] (Expert):Q: All of this looks like something I should learn more about. Where's the best place to start?A: Can you be more specific about the type of information you're lookign for? IT Pro docs, developer docs...? We do have a blog at http://blogs.technet.com/filecab/ but we haven't blogged a lot about NTFS yet. Documentation for Vista is in progress so what's available publicly is limited. Dana [MSFT] (Expert):Q: It seems as though Vista NTFS will suffice just fine and the number of people who could have actually *truly* leveraged WinFS seems minimal (relatively speaking amongst home users). Was this found to be *A* case in the decision that came of WinFS?A: I don't think any one on this chat really has the answer to your question.Senthil [MSFT] (Expert):Q: Could we easily mount hard drives to folders so they appears to be on one drive?A: Mount points allow the root of a local filesystem to be mounted into any portion of the existing namespace. It is made visible to the user as a normal directory.
Andy [msft] (Expert):Q: With the arrival of more tools - and even viruses - that use ADS (Alternate Data Streams), will there be tools in Vista that support the enumeration and management of ADS from the GUI and/or command line?A: The shell hasn't quite caught up with the expanding usage of this feature. In Win2003 Server we added FindFirstStream/FindNextStream to quickly enumerate streams. Don't know of a shipping command line tool that provides this functionality. It should probably be added to fsutil (not a commitment to do so at this time). We are in discussions with the shell team about how to expose this.Jill [MSFT] (Expert):Q: Thanks JillA: Any time! You can always find me in the beta newsgroups if you have other doc-related questions. I can pass them along to our doc teams.Wendy [MSFT] (Moderator):We will need to wrap up this chat in about 20 minutes. Please post any other questions (select the “Ask the experts” check box) that you would like us to answer. Thanks!Surendra[Msft] (Expert):Q: What is the difference between a hard link and a sym link?A: Hard-link is build internally with NTFS and can only be put on files within a given volume. Sym-links (new in Vista) are build on top using reparse points allow you to link files and folders across volumes and machines. Deletion of all symlinks to a file doesn't delete the file - think of them as soft references. Also, a file can be deleted even if there's sym-links on it. Hard-links are basically alternate names for files, and if and only if all of the are delete the file is deleted. Dan [MSFT] (Expert):Q: Secure Delete utilities were incomplete on previous versions of NTFS because of undocumented access to the MFT required to wipe the "internal" extent. Will access to the MFT be documented to allow a true secure delete to occur?A: We do not have any APIs that will help wiping out an MFT entry. I would recommend user encrypt their files first. This will cause the data portion to become non-resident. Utilities can then go and wipe these encrypted data clusters. What's left in the MFT record will then be just the name of the file. In Vista, we added FSCTL_SET_ZERO_ON_DEALLOCATION to help zeroing out data clusters. Note, this is just zeroing it out. It does not meet the standard of secure delete.Andy [msft] (Expert):Q: What does the NTFS file attribute 'N' stand for, as displayed by explorer? (I've seen this on the \ProgramData folder (and child files/folders) as well as the junctions, not much else)A: I'm not able to find this right now. I believe it's "not content indexed"Wendy [MSFT] (Moderator):Q: Thanks all, you have been very helpful :)A: Before you leave the chat, don’t forget to let us know how we’re doing! To the right of your emoticon pull-down box is a “Chat Feedback” option. If you don’t feel the chat was helpful, please ensure that you provide verbose comments on why you didn’t feel it was helpful and what we can do to improve the experience for you!! If you would like to provide more feedback than the 1000 characters will allow, feel free to eMail us at LHBeta@microsoft.com.Jill [MSFT] (Expert):Q: Is Single Instance Storage (SIS) working in Vista?A: This is a Windows Server "Longhorn" feature only, so it's not in Vista.Surendra[Msft] (Expert):Q: do you know if NTFS format is unofficially documented?A: It's not officially documented.Senthil [MSFT] (Expert):Q: Are the default NTFS permissions for various folders -- root, Program Files, Users, etc. -- the same as in XP? Where is this documented?A: The root folder system acl is different and it disallows creation of files. I don't have the exact acl that is applied to the system folder.Senthil [MSFT] (Expert):Q: Does MS have any intentions of releasing kernel debugger extension DLLs to the public for debugging NTSF data structures ?A: No, the NTFS data structures are confidiential.Malcolm [MSFT] (Expert):Q: What is the difference between a junction point, and a directory symlink?A: Junctions and directory symlinks have subtly different semantics. For example, symbolic links are always evaluated on the client in a network scenario, whereas junctions are evaluated on the server. Also, ACLs are handled differently too. However, they are broadly similar.Malcolm [MSFT] (Expert):Q:  So there are APIs - there have always been APIs (although a little tricky to use) to enumerate and access Alternate Data Streams. Where are the tools for IT Pros who don't code?A: In CMD. Type CMD /R. I believe this is a new feature in Vista.Andy [msft] (Expert):Q: do you know if NTFS format is unofficially documented?A: Unknown as I haven't checked lately. If it exists it will need some updating as the TXF functionality in Vista has added quite a bit. The format itself is backward compatible but there are many additional metafiles.Surendra[Msft] (Expert):Q: Are there any changes to NTFS that everyday home users will notice, if so what are they?A: One thing I can think of is that people will be able to format NTFS on removable (USB) drives without needing to go through inscrutable UI to change device properties to turn on "optimize for performance"Wendy [MSFT] (Moderator):Q: Can I buy you all lunch sometime? lol. =) Nothing like schmoozing! Thanks for all your help so far.A: Absolutely!!! But be forewarned we have expensive tastes and we're always hungry!Surendra[Msft] (Expert):Q: NTFS has always been called transactional with logs that can be replayed. But an earlier answer stated "Consider what happens when someone does a force shutdown ... there was a good chance you corrupted your system". Please elaborate.A: I think that refers to issues with IDE and some SATA drives that don't honor write-through.Malcolm [MSFT] (Expert):Q: When trying to use FSUTIL, no matter the account used (even administrator), I get a message that says it requires an account with administrative privilege. Why?A: That was a bug. Fsutil always required users to be Administrators, but with the introduction of UAC, the exact meaning of Administrator became confused somewhat. This should be functional now.Jill [MSFT] (Expert):Q: For everyone interested in Defrag, we have a chat scheduled for July 26. Be there or be fragged!Andy [msft] (Expert):Q: Are there any enhancements for use in a clustered environment?A: From the file system perspective, only bugfixes and a more reliable experience in a clustered environment when using TXF as the node taking over a volume will rollback all transactions in flight but not committed from the failing node automatically on mount. The cluster team surely has more enhancements.Dan [MSFT] (Expert):Q: Is there a way to query the history of which files have been healed ?A: You can look into the system event log for NTFS healing events (currently event id 130-133). If there are too many healing activities within a short period of time, the detail will not be posted to the event log. Instead, you will get an event about this and when normal self healing event posting can be resumed, summary of how many healing event has occurred will be posted.Dana [MSFT] (Expert):Q: NTFS has always been called transactional with logs that can be replayed. But an earlier answer stated "Consider what happens when someone does a force shutdown ... there was a good chance you corrupted your system". Please elaborate.A: Before we protected the file system metadata not the user's data when an unexpected shutdown hit. What I was suggesting earlier is what you would see if you interrupted installation of multiple system files. What do you think happens if mid-way through updating your system, 1/2 of update is not completed? If you are lucky, the system boots to the point where the servicing infrastructure can back-out the changes. I have never been that lucky. :-) Today, TxF backs out those changes before booting the system.Malcolm [MSFT] (Expert):Q: So if I understeand correct, the NTFS is the same as XP, does that mean the 3ed party tools for partititon management working in XP should (drivers excepted perhaps) also work in Vista?A: That's a very large question. It's always been our goal to be as compatible as possible, although sometimes it's just not possible. PartitionMagic 8, for example, is known to have serious problems interacting with Vista. These kinds of tools interact with the system in a very low-level way, and Vista is a very large release. However, we've implemented volume shrink in Vista, and volume extend was in Windows Server 2003, and we'd strongly encourage you to try these features out.Senthil [MSFT] (Expert):Q: Does MS have any plans to document changes that have been made to NTFS driver in Vista for the benefit of File System Filter Driver Developers. I am talking about technical articles - not business level white papers.A: Any relevant changes will go into the IFS Kit which is now a part of the WDK.Dan [MSFT] (Expert):Q: Is there any API to find out if a particular file was healed and if yes then when etc ?A: No specific api. Just check the system event log. If there are too many within a short period of time, only summary will be posted.Surendra[Msft] (Expert):Q: do you plan to inheritate attributes? i checked the "index this folder..." checkbox on a folder, and it took ages to finish...A: There're no current plans to support inheritable attributes natively within NTFS.Malcolm [MSFT] (Expert):Q: FSUTIL issue is exhibiting on BETA 2 code.A: Are you running elevated?Wendy [MSFT] (Moderator):Before you leave the chat, don’t forget to let us know how we’re doing! To the right of your emoticon pull-down box is a “Chat Feedback” option. If you don’t feel the chat was helpful, please ensure that you provide verbose comments on why you didn’t feel it was helpful and what we can do to improve the experience for you!! If you would like to provide more feedback than the 1000 characters will allow, feel free to eMail us at LHBeta@microsoft.com.
Malcolm [MSFT] (Expert):Q: So then, what is the difference between a shortcut and the links?A: A shortcut is a file implemented by Shell32 via the IShellLink interface. These files tell shell to jump to a different location. A symbolic link is implemented under the filesystem API, so regular applications calling CreateFile get the benefit of links without needing to implement their own code, as they would with IShellLink.Dan [MSFT] (Expert):Q: Is there a way to query the history of which files have been healed ?A: Just check the system event log. It's under NTFS event (currently event id 130-133). If there are too many within a short period of time, only summary will be posted.Dana [MSFT] (Expert):Q: So then, what is the difference between a shortcut and the links?A: shortcuts are explorer icons that explorer uses; symlinks are file-system things. So, you cannot open up a shortcut in word but you can open a symlink to a word document.Senthil [MSFT] (Expert):Q: Why booting Windows Vista from external USB or firewire drive is not supported as on OSX ?A: This is a feature that is being worked on and might be considered for a future release. Surendra[Msft] (Expert):Q: todya - diskeep can put files unused for over a month adt the end of the volume. the trouble is this causes the bitmap cache to cache those fiel positions - and new file writes are written at teh end of the volumeninstead of the bgeinning ofthe volA: NTFS allocation algorithm uses a cache of free space and hint of where free space might lie. The cache is sorted by size and disk offset. NTFS use a best fit alogorithm to find a spot close to the previous allocation for the same file. There's no distinction between end of the volume or the beginning of the volume from a placement point of view. I'd almost want to ask why diskeep is using end of the volume for this if that's not optimal for perf.Malcolm [MSFT] (Expert):Q: Re: Q62 volume extend...Is that possible on the System drive? Does it need to be dynamic, or is basic okay?A: You'll need to have unpartitioned space immediately following the volume being extended. The system volume is extendable.Senthil [MSFT] (Expert):Q: One thing that was always annoying on XP and before was the long delays when a bad block is detected over and over, rather than telling you and trying to recover as much data as possible around that block. Is there anything to address this?A: Hmm..the bad block detected once will be stored away in the bad sector file and will not be used again. This has been the behaviour for ever..Surendra[Msft] (Expert):Q: With a symbolic link, what happens if the target file has been deleted, and you try to open it through its link? [With a hard-link, the file dies only when all links have been deleted, so this is a symlink-only problem]A: You get error indicating the file doesn't exist.Dan [MSFT] (Expert):Q: FSUTIL issue is exhibiting on BETA 2 code.A: Need more info. What is the problem?Wendy [MSFT] (Moderator):Before you leave the chat, don’t forget to let us know how we’re doing! To the right of your emoticon pull-down box is a “Chat Feedback” option. If you don’t feel the chat was helpful, please ensure that you provide verbose comments on why you didn’t feel it was helpful and what we can do to improve the experience for you!! If you would like to provide more feedback than the 1000 characters will allow, feel free to eMail us at LHBeta@microsoft.com.Malcolm [MSFT] (Expert):Q: Will there be greater UI exposure to streams and symbolic links?A: As mentioned previously, the DIR command will now display symbolic links, and can be used to enumerate streams.Senthil [MSFT] (Expert):Q: If you guys own this feautre, any work being planned to clean up the NTFS permissions editing UI?A: What exactly needs cleaning?Wendy [MSFT] (Moderator):Best Question goes to Bob R for asking "Database best practice is to put transaction log data on a volume other than the one being deliberately updated, then update the primary volume. Will Vista writes resemble databases in this respect?" Congrats! Please email the LHBeta alias with your full shipping address and shirt sizeSurendra[Msft] (Expert):Q: Will you do bugfixing from now on or are there any features which are not in place yet?A: For Vista we're in lockdown and stabilizing to get a quality product out. The featues are all in.Andy [msft] (Expert):Q: I wonder if when moving or copy files between two servers from a third PC, the flow will go through server-to-server and not via the PC ?A: From server A to server B through client C, no, data goes through C (lots of delegation issues there). For Vista client to either Vista or Win2k3 server, copying data from a share to a share (same or otherwise), the data will not go through the client. This is new functionality.Surendra[Msft] (Expert):Q: When a Volume Shadow Copy is made, is the MFT copied - and the "old" clusters marked in the MFT so they're not reused?A: No, it doesn't need to since VSS does a differential block copy to maintain previous versions of the volume.Senthil [MSFT] (Expert):Q: will the current partition magic work on NTFS?A: I don't think that the current version works, but they *might* be working on fixing this.Surendra[Msft] (Expert):Q: Can directories be moved while the filesystem is mounted to reduce fragmentation? If not, why not?A: Yes, directory blocks can be moved as well for defrag.Senthil [MSFT] (Expert):Q: Will we still be able to use disk management tools to "zero write" the NTFS partitions or is it going to needs new tools?A: are you taking about secure delete?. Format has an option that you can mention to write 0's n number of times.Dan [MSFT] (Expert):Q: Will compression ever improve in NTFS (newer better algorithms)?A: No change in Vista. Malcolm [MSFT] (Expert):Q: What is the TrustedInstaller account? I see he owns most files and has the most access rights to the file system.A: This account existed in XP as well, I believe. It's used so that files can be written to by installers, but not by regular administrators - at least not by default. Through this scheme, DLLs can be protected against silly installers that will downgrade versions etc, which results in system instability; the only mechanism to update these files can perform those checks. Of course, you can take ownership of these files, if there's a really pressing need.Senthil [MSFT] (Expert):Q: if booting from external drives is permitted can ntfs protect against unauthorised access - physical security is not always possible but card readers and finger print scanners help - if booting from a usb drive is allowed this could cause ussues...A: I think this problem is being dealt with by the group that is working on this feature..Malcolm [MSFT] (Expert):Q: Using the Shrink Partition feature, can I decide which side of the partition gets shrunk?A: Unfortunately, no. It moves clusters to the beginning of the volume, and performs the shrink from the end. There is no mechanism in Vista for moving partitions, either. In that sense, Vista is not a replacement for more sophisticated partitioning tools, it just provides the basic cases that is often enough for many people.Andy [msft] (Expert):Q: Media Center seems to cause much fragmentation - the constant creation and deletion of huge files. I've seen files with 100k+ fragments on my media center. Any special support for that in Vista?A: I believe they've done some work in allocating larger chunks at record time. It's stanard tradeoff between burning disk space and fragmenting. We have not done any specific work for recording, only some work for better response in playback.Andy [msft] (Expert):Q: any installations doesn't require a psw. Is not a good idea to implement that ?A: better directed to the security group for that oneMalcolm [MSFT] (Expert):Q: for resizing drives can data that is blocking the resize be moved within its partition to make the resize work?A: Isn't that not true by definition? One of the great things about shrinking in Vista is that it can be done on online volumes. One of the limitations of this is there are more clusters that can't be moved. There's no facility for moving these clusters, because it would cause problems since the volume is still online.Surendra[Msft] (Expert):Q: why are hardlinks used so much in vista (and presumably longhorn). they're a pain to manageA: Sym-links are used a lot more for some cases (My Document folder) etc for backward compat with XP.
Hardlinks are used in some more cases, for system executables (for beter system update) that you shouldn't care about much, right?Jill [MSFT] (Expert):Thanks to all the beta testers who joined us today! There were some great questions here.Senthil [MSFT] (Expert):Time to go,guys..thanks and hopefully we will be able to do this another time.Surendra[Msft] (Expert): Thanks guys. Geat questions.Dana [MSFT] (Expert):Thanx for all the great questions!
Malcolm [MSFT] (Expert):Thanks everyone for showing interest in the new functionality in Vista!Andy [msft] (Expert):Thanks for the good questions. Sorry couldn't get to them all.
Dan [MSFT] (Expert):Thanks for asking all these great questions!