Alternate Data Streams in NTFS

Alternate Data Streams in NTFS

  • Comments 14
  • Likes

This blog has been a long time coming. There is a bit of confusion about the subject of alternate data streams (ADS) and no small amount of suspicion. So I want to take a few minutes to set the record straight on ADS.

A couple years ago I wrote a blog on NTFS attributes.

http://blogs.technet.com/b/askcore/archive/2010/08/25/ntfs-file-attributes.aspx

You might want to review that blog before continuing. I’ll wait….

Welcome back.

One of the common questions I get is, “Robert. What is an alternate data stream?”

My reply is always the same, “It is a data stream that is alternate”.

I don’t mean to be smart aleck about it…but that’s what it is. We know from my older blog that a file is divided up into ‘attributes’ and one of these attributes is $DATA or simply called the data attribute. It is the part of the file we put data into. So if I have a text file that says, “This is my text”, then if I look at the data attribute, it will contain a stream of data that reads, “This is my text”. However, this is the normal data stream, sometimes called the primary data stream, but more accurately it is called the unnamed data stream. Why? Because it is a data stream that has no name. In the jolly land of programming it is referred to as $DATA:””

image

The name of the stream will appear between the quotes. Since this is an unnamed data stream, there isn’t anything there.

Now that we know what the unnamed data stream looks like, we can start thinking in terms of alternates. Knowing that the place we normally store data is the unnamed data stream, if a stream has a name, it is alternate. So if I had a file with an ADS named SecondStream, its full name would be, $DATA:”SecondStream”

image

This is all good and fine, but unlike the unnamed data stream, we can’t see the ADS. Or can we? The answer is, yes we can. But you have to use a method different than just opening the file in NotePad.

There are a number of tools out there that will allow you to view and manipulate ADS. One that Microsoft has provided for years is called STREAMS.EXE.

http://technet.microsoft.com/en-us/sysinternals/bb897440.aspx

STREAMS.EXE will display any ADS the file has.

clip_image005

The formatting is a little different.

image

STREAMS.EXE is fine, and I’ve used it for years, but with the release of Win8/Server 2012, I’ve discovered a new way of dealing with ADS….Windows PowerShell. Using the cmdlet, Get-Item, I can get more information than I did with STREAM.EXE.

clip_image008

The output shows not only the name of the ADS and its size, but also the unnamed data stream and its size is also listed (shown as :$DATA). And now that I know the name of the ADS, I can use the Get-Content cmdlet to query its contents.

clip_image009

STREAM.EXE can’t display what’s actually in an ADS. Here’s another trick that STREAM.EXE can’t do….create data streams. Using Set-Content, I’ll create a second ADS in the same file and add a line of text.

clip_image010

And again, we can query for the streams using Get-Item.

clip_image011

And finally, we can remove an ADS using the Remove-Item cmdlet.

clip_image012

Now we know what ADS is, how to query for ADS, how to create ADS, and how to delete ADS. So what is the big deal?

The big deal is that since ADS isn’t easily visible, it has become a cute way to hide data. Unfortunately it has also been used in the past to hide malicious code. This is how ADS got a bad name. In fact, a number of people that approach me about ADS already know that they have files with alternate data streams and they think they are infected with viruses.

Calm down. The mere presence of an ADS doesn’t mean that there is a problem. In fact, Microsoft uses ADS for a number of functions. I can almost guarantee that if you are reading this, you probably have some ADS on your computer. Let’s take a look at a couple examples.

Internet Explorer: Ever download an executable file from the Internet and then get warned about it when you ran it? How does that work?

When the file is downloaded, IE slaps an ADS on it. The stream will store a tag that tells Windows what zone the file was downloaded from.

clip_image013

Look Familiar?

So using what I’ve learned so far, I can look at one of the files I’ve downloaded from the internet and see if there is an ADS on it.

clip_image014

Yes, it is called ‘Zone.Identifier’. And then we can query the contents of the ‘Zone.Indentifier’ ADS.

clip_image016

Now we know that the file was downloaded from zone 3. Using the zone chart we can see it came from the Internet zone.

Value Setting
------------------------------
0     My Computer
1     Local Intranet Zone
2     Trusted sites Zone
3     Internet Zone
4     Restricted Sites Zone

Notice that my test download file is in a test directory. This means I moved the file here from my download directory. This is the cool thing about ADS, since it is part of the file, it moves with the file. Even if I copied it, the ADS would be on the new copy as well.

Other Internet browsers use ADS in a similar fashion.

File Classification Infrastructure: FCI is very dependent on ADS. The way that the classification works is that it puts tags on your files that allows you to keep track of what the file was classified as, no matter what happens with the file. It could be edited, copied, moved to another server, and its classification tags remain intact.

Others: Office files and Outlook Express file use ADS. And it isn’t limited to Microsoft programs. Numerous programs utilize the ADS functionality.

The point is that if you discover ADS on your system, it isn’t necessarily a bad thing. And just blindly stripping these data streams out of files can actually do a great deal of harm.

And now that you have some tools to use for querying alternate data streams, they won’t be so scary.

Thank you for your time and I hope this was educational.

Robert Mitchell
Senior Support Escalation Engineer
Microsoft Corp.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • How will streams be affected if the underlying filesystem is FAT32? If I save a file to a USB flash drive (which is normally formatted as FAT32), will I lose the stream content? Thanks (gkdemo@live.com)

  • @Gkhalsa

    The Stream only Works on NTFS, you will lose all the Stream Data if you copy it to The FAT32 FS.

    and even after you copy a file from NTFS --> FAT32 --> NTFS your Stream is Lost, as FAT32 doesnt

    understand ADS.

  • Correct. FAT file systems do not support more than one data stream.

  • Alternate data streams are fun, but aren't they going away?  I mean, ReFS does not support them, and the plan is to do away with a lot of non-mainstream (no pun intended :) features such as transactions and hard links, even though some are currently in use in the default OS installation, isn't it?

  • Great article. . .

    Just to add that there are also some other PowerShell v3, cmdlets for Alternate data streams:

    Test-AlternateDataStream & Unblock-File.

  • ReFS is a different animal.  It is meant to focus on reliability and as such only carries a subset of the functionality that NTFS provides.  As such, there will be some scenarios that NTFS is a better fit and some where ReFS is the logical choice.

  • ...and that is why virtually no one wants to use ReFS right now, at least until it plays catch-up with NTFS compatibility-/feature-wise (EFS, streams, compression, etc.) and especially performance-wise. When we first heard of a new filesystem for Windows and then read all about the reliability enhancements, it was quite disheartening to learn that performance actually *decreased* with the new filesystem, and it doesn't even seem to be designed to address NTFS's shortcomings in this area. We've already seen both are possible (e.g., ZFS).

    We're left with a Sophie's Choice for Windows in this era of giant data: do you want data integrity, or do you want performance and compatibility?

  • DIR of Win Vista/2008 or higher supports for a quick peak of ADS

    dir /r

  • I received a zip file that was supposed to contain files with ADS, but there were no ADSs. Is there a tool out there that archives files like WinZip that handles ADS? Preferably supported on Windows Server 2008?

  • Very well explained. Thank you, information is of great value for me.

  • How does SMB/CIFS handle ADS?

  • Does it ever occur to you arrogant nerds at Microsoft that NO ONE OUT THERE wants this crap? We're sick of wasting our precious time working around the bugs, quirks, and viruses in Windows and Office. Did anyone on the ADS team stop to think that WE CANNOT PERFORM A SIMPLE BACKUP -- just writing files to a CD -- without the hours-long process being repeatedly halted by your mindless "Confirm Stream Loss" dialogs? Maybe you're just too immature to

    grasp the sinister side of Bill Gates and crew. I recommend reading (if you can still read hard-copy printed material) the book by Tony Bove, Just Say NO to Microsoft. It will open your eyes to the malicious side your employer.

  • In windows 8, Ads file cannot be called by start command.. same command works in xp.. Why? have you tried to call hidden file using start??