Mark Russinovich’s technical blog covering topics such as Windows troubleshooting, technologies and security.
On November 3 2009, Sysinternals retired NewSID, a utility that changes a computers machine Security Identifier (machine SID). I wrote NewSID in 1997 (its original name was NTSID) because the only tool available at the time for changing machine SIDs was the Microsoft Sysprep tool, and Sysprep doesn’t support changing the SIDs of computers that have applications installed. A machine SID is a unique identifier generated by Windows Setup that Windows uses as the basis for the SIDs for administrator-defined local accounts and groups. After a user logs on to a system, they are represented by their account and group SIDs with respect to object authorization (permissions checks). If two machines have the same machine SID, then accounts or groups on those systems might have the same SID. It’s therefore obvious that having multiple computers with the same machine SID on a network poses a security risk, right? At least that’s been the conventional wisdom.
The reason that I began considering NewSID for retirement is that, although people generally reported success with it on Windows Vista, I hadn’t fully tested it myself and I got occasional reports that some Windows component would fail after NewSID was used. When I set out to look into the reports I took a step back to understand how duplicate SIDs could cause problems, a belief that I had taken on faith like everyone else. The more I thought about it, the more I became convinced that machine SID duplication – having multiple computers with the same machine SID – doesn’t pose any problem, security or otherwise. I took my conclusion to the Windows security and deployment teams and no one could come up with a scenario where two systems with the same machine SID, whether in a Workgroup or a Domain, would cause an issue. At that point the decision to retire NewSID became obvious.
I realize that the news that it’s okay to have duplicate machine SIDs comes as a surprise to many, especially since changing SIDs on imaged systems has been a fundamental principle of image deployment since Windows NT’s inception. This blog post debunks the myth with facts by first describing the machine SID, explaining how Windows uses SIDs, and then showing that - with one exception - Windows never exposes a machine SID outside its computer, proving that it’s okay to have systems with the same machine SID. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so MIcrosoft's support policy will still require cloned systems to be made unique with Sysprep.
Windows uses SIDs to represent not just machines, but all security principals. Security principals include machines, domain computer accounts, users and security groups. Names are simply user-friendly representations for SIDs, allowing you to rename an account and not have to update access control lists (ACLs) that reference the account to reflect the change. A SID is a variable-length numeric value that consists of a structure revision number, a 48-bit identifier authority value, and a variable number of 32-bit subauthority or relative identifier (RID) values. The authority value identifies the agent that issued the SID, and this agent is typically a Windows local system or a domain. Subauthority values identify trustees relative to the issuing authority, and RIDs are simply a way for Windows to create unique SIDs based on a common base SID.
You can use the Sysinternals PsGetSid tool to view a machine’s SID by running it with no command-line arguments:
Here, the revision number is 1, the authority is 5, and there are four subauthority values. At one point during the design of Windows NT, the machine SID might have been used for network identification, so in order to assure uniqueness, the SID that Setup generates has one fixed subauthority value (21) and three randomly-generated subauthority values (the numbers following “S-1-5-21” in the output).
Even before you create the first user account on a system, Windows defines several built-in users and groups, including the Administrator and Guest accounts. Instead of generating new random SIDs for these accounts, Windows ensures their uniqueness by simply appending a per-account unique number, called a Relative Identifier (RID), to the machine SID. The RIDs for these initial accounts are predefined, so the Administrator user always has a RID of 500:
After installation, Windows assigns new local user and group accounts with RIDs starting at 1000. You can use PsGetSid to view the name of the account for a specified SID, and here you can see that the local SID that has a RID of 1000 is for the Abby account, the name of the administrator account Windows prompted me to name during setup:
In addition to these dynamically created SIDs, Windows defines a number of accounts that always have predefined SIDs, not just RIDs. One example is the Everyone group, which has the SID S-1-1-0 on every Windows system:
Another example, is the Local System account (System), which is the account in which several system processes like Session Manager (Smss.exe), the Service Control Manager (Services.exe) and Winlogon (Winlogon.exe) run:
When an account logs on to a Windows system, the Local Security Authority Subsystem (LSASS -Lsass.exe) creates a logon session and a token for the session. A token is a data structure the Windows kernel defines to represent the account and it contains the account’s SID, the SIDs of the groups that the account belongs to at the time it authenticated, and the security privileges assigned to the account and the groups. When the last token that references a logon session is deleted, LSASS deletes the logon session and the user is considered logged off. Here you can see my interactive logon session, displayed with the Sysinternals LogonSessions utility:
And here you can see a token Lsass has created for the session in Process Explorer’s handle view. Note that number following the account name, 7fdee, matches the logon session ID shown by LogonSessions:
By default, processes inherit a copy of their parent process’s token. Every process running in my interactive session, for example, has a copy of the token that they inherited originally from the Userinit.exe process, the process Winlogon creates as the first of any interactive logon. You can view the contents of a process’s token by double-clicking on the process in Process Explorer and switching to the Security page of the process properties dialog:
When one of my processes opens an operating system object, like a file or registry key, the security subsystem executes a permission check that evaluates entries in the object’s access control list (ACL) that reference a SID included in the process’s token.
A similar check happens for remote logon sessions, which are the kind created by a “net use” of a remote computer’s share. To successfully connect to a share you must authenticate to the remote system with an account known to that system. If the computer is part of a Workgroup, then the credentials you specify must be for a local account on the remote system; for a Domain-joined system, the credentials can be for a remote system’s local account or a Domain account. When you access a file on the share, the file server driver on that system uses the token from the logon session for the permission check, leveraging a mechanism called impersonation.
The Microsoft-supported way to create a Windows installation that’s ready for deployment to a group of computers is to install Windows on a reference computer and prepare the system for cloning by running the Sysprep tool. This is called generalizing the image, because when you boot an image created using this process, Sysprep specializes the installation by generating a new machine SID, triggering plug-and-play hardware detection, resetting the product activation clock, and setting other configuration data like the new computer name.
However, some IT administrators install Windows on one of their systems, install and configure applications, then use deployment tools that don’t reset the SIDs of the copies of the Windows installations. The best practice up to now has been to run a SID-resetting utility like NewSID to change SIDs. These utilities generate a new machine SID, try to find all the locations on a system, including all the file system and registry ACLs, that contain copies of the machine SID, and update them to the new SID. The reason that Microsoft doesn’t support systems modified in this way is that, unlike Sysprep, these tools don’t necessarily know about all the places where Windows stashes away references to the machine SID. The reliability and security of a system that has a mix of the old and new machine SID can’t be guaranteed.
So is having multiple computers with the same machine SID a problem? The only way it would be is if Windows ever references the machine SIDs of other computers. For example, if when you connected to a remote system, the local machine SID was transmitted to the remote one and used in permissions checks, duplicate SIDs would pose a security problem because the remote system wouldn’t be able to distinguish the SID of the inbound remote account from a local account with the same SID (where the SIDs of both accounts have the same machine SID as their base and the same RID). However as we reviewed, Windows doesn’t allow you to authenticate to another computer using an account known only to the local computer. Instead, you have to specify credentials for either an account local to the remote system or to a Domain account for a Domain the remote computer trusts. The remote computer retrieves the SIDs for a local account from its own Security Accounts Database (SAM) and for a Domain account from the Active Directory database on a Domain Controller (DC). The remote computer never references the machine SID of the connecting computer.
In other words, it’s not the SID that ultimately gates access to a computer, but an account’s user name and password: simply knowing the SID of an account on a remote system doesn’t allow you access to the computer or any resources on it. As further evidence that a SID isn’t sufficient, remember that built-in accounts like the Local System account have the same SID on every computer, something that would be a major security hole if it was.
As I said earlier, there’s one exception to rule, and that’s DCs themselves. Every Domain has a unique Domain SID that’s the machine SID of the system that became the Domain’s first DC, and all machine SIDs for the Domain’s DCs match the Domain SID. So in some sense, that’s a case where machine SIDs do get referenced by other computers. That means that Domain member computers cannot have the same machine SID as that of the DCs and therefore Domain. However, like member computers, each DC also has a computer account in the Domain, and that’s the identity they have when they authenticate to remote systems.
Some articles on SID duplication, including this KB article, warn that if multiple computers have the same SID, that resources on removable media like an NTFS-formatted firewire disk can’t be secured to a local account. What they fail to mention is that permissions on removable media provide no security regardless, because a user can connect them to computers running operating systems that don’t honor NTFS permissions. Moreover, removable media tend to have default permissions that grant access to well-known SIDs, such as to the Administrators group, which are the same on all systems. That’s the fundamental rule of physical security and why Windows 7 introduced Bitlocker-to-Go, which enables you to encrypt removable storage.
The final case where SID duplication would be an issue is if a distributed application used machine SIDs to uniquely identify computers. No Microsoft software does so and using the machine SID in that way doesn’t work just for the fact that all DC’s have the same machine SID. Software that relies on unique computer identities either uses computer names or computer Domain SIDs (the SID of the computer accounts in the Domain).
It’s a little surprising that the SID duplication issue has gone unquestioned for so long, but everyone has assumed that someone else knew exactly why it was a problem. To my chagrin, NewSID has never really done anything useful and there’s no reason to miss it now that it’s retired. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so Microsoft’s support policy will still require cloned systems to be made unique with Sysprep
Though the responses have continued to grow, I do not see a response to my posting number 3291283. The scenario and steps used are not machine-state related, nor are they joined to a domain (so is not Domain SID related).
The case is solely machine SID related as the account for the local Administrator account (using a RID of 500) is the same for each machine since the machine SIDs themselves are the same.
Just looking for your comments on this and the test I proposed.
Though I didn't think it would be an issue, I did a quick experiment with vitual machines just to confirm.
After you change the administrator password on one of the machines, you'll not gain access to the other machine. Instead,you'll be prompted to provide the correct username & password. The test was done with Windows Server 2003.
Its one thing to write an academic article about whether something is a security risk or not. Its another thing to remove a utility that thousands of people use because you've decided for all people that they dont need to use it after 12 years. If you were wrong for 12 years its possible that you are wrong now. Write your article and let the people decide for themselves retiring a program based on your own thought experiment is incredibly arrogant. Thanks Jesus.
@Ed: You of course are free to continue to believe that SID duplication is an issue, but I wouldn't want my systems managed by an IT staff that makes decisions based on superstition. And it's surprising to me that you're so concerned over SID duplication while at the same time ignoring all the other duplicated state that is addressed by Sysprep.
About time someone debunked the newsid myth. I challenged it in 2005 or 2006 and could't find a reason for it but back then I was a nobody back then and I had not idea how to publish my results.
I had never used newsid nor recommended it. I did some serious testing when a customer told me they preferred it over Sysprep. My testing show newsid wasn't even needed.
1. Create an image without sysprep.
2. Depoy it to multiple machines.
3. Manually rename the machines (because you can't have computers with the same name on the same network.)
5. Join them to the domain.
Test everything and nothing ever failed.
Now, because Sysprep will fix the name and there are so many other features of sysprep, if anyone would rather use newsid over sysprep they aren't really thinking it through.
Great Stir-up Mark !!
Goes to show indeed that people don't realy know their systems...
I think general confusion occurs ragarding apps and the way they identify their clients (Nothing to do with the way Windows uses the SIDs).
Apps likes SMS, WSUS, etc use an ID to uniquely identify clients. It is these IDs that when cloned cause problems, nothing much to do with the SID... If the SID is used as source for these IDs (as a factor, or as a whole), then changing the SID could automatically lead to a ID regeneration, and therefore problem solved, even if it isn't a real SID problem.
When imaging systems, even with Sysprep, Unique identifiers still have to be thouroughly inspected. Each app needing its own regisrty tweaks or file deletions (LanDesk needing the deletion of its GUID from the registry, for example, same thing with the SMS 2.0 GUID file, etc). There is a greater chance that if it is a Microsoft app, sysprep will do it for you...
Many reactions to your great can of worms post are more deployment issues relating to duplicate IDs, not SIDs...
The real question, therefore, is how do I (product vendor) generate an ID for my client ? Well, the SID is supposed to be unique - huh, well I'll use that !! It's not an SID issue as such but when AppID=SID, then things go horribly wrong in my app... Only solution = change SID...
So the bottom line is most likely : to be safe, follow best practises & Sysprep. Also, know your apps !!!
Thanks for your neat post !
Many times I see the new-born generation of "sys-admins" go too fast, too quick, too much "googling" and showing they know it all well. And none of them really have an answer when it comes to really deep problems that are just slightly different from their "google-paper". Because they often still lack the real internal & technology knowledge and do not wish to spend time studying enough on that. Pity for those who are pointing with their comments at respected and longtime experienced guys, because they actually show their own lack of knowledge. As these longtime experienced guys spent long hours really studying & testing/verifying on the why's and how's from software/operating-software systems ever since it was released.
My advice is always: read and wonder why, then ask and/or read more and learn!
The further we come, the more we discover how little we really know...
@ Mark, again your work shows you belong among to the best & finest in this world of IT
"Microsoft’s official policy on SID duplication will also now change and look for Sysprep to be updated in the future to skip SID generation."
Would this update in sysprep to skip SID generation be an option? As you say, if you use a clone as the first DC in a domain you would want to create a new SID.
"For the DC you want to create from a clone, create a unique installation or run Sysprep."
If Microsoft completely remove SID generation from SYSPREP this could cause issues. Although I could use a scripted windows install for the first DC in a new domain.
A very interesting article, it created a good debate in a usually docile office. It has also caused a lot of debate here, if not slightly misunderstood by some.
After trying to take all this in and now having my head swimming all I have is ONE question. If you yourself were an IT admin and had to roll out new workstations which method would YOU use. The image then sysprep method or the install from scratch method ( either by hand or by scripting, including all the applications and drivers). I know Vista and Windows 7 have the new WIM format but they also can be imaged ( cloned ) first and THEN be used in the Desktop Deployment Toolkit so the question is still valid for them.
Interesting read for sure.
I've been using Ghost for cloning for the last 4-5 years, around 70 pc's in the org. back then.
When I was working out the procedures, I found Sysprep to be a major PITA to work with, and looked for other solutions. Then found wininternals.com and started using newsid.
my procedure in short:
1: install OS, update etc
2: join domain, install software, do customizing, leave domain. Unplug network
3: reboot as standalone
4: make image
5: load image to new pc, reboot, rename/newsid/run the F-secure ID regeneration util
6: reboot, attach network cable, join domain
Then figured out that everything ran just as good even if I forgot running newsid, so then i just skipped that step, 3 years ago.
The pc's won't connect to wsus until they're joined the domain after cloning. And they'll connect to the Antivirus policy server and create a new unique id.
I would never clone Domain controllers etc without running sysprep/newsid
So my findings correlate with Mark R's, and it also makes sense to me that running Newsid can solve other id-related issues.
But there's one thing that got me thinking: Our accounting system runs on MSSQL and depends heavily on DCOM.(And sloppy coding - users have to run as admin! to ensure consistent running...)
And we've had some strange trouble on some pcs while others been running fine. From some of the comments above, I think these issues may be SID related, intentional or not.
@Dean: If it were me (and it is as I am part of a large company that does this for many companies) I would use MDT and completely automate my deployment process. Doing this removes any possibility of duplicate anything except processes and you never have to even think of this battle.
When we clone virtual machines in the lab, we don't run any kind of NewSID process. Again, we let the VMware tool handle that for us, so a sysprepmini Setup process occurs. Since it handles the process via a MS supported mechanism (Sysprep) there is no issue.
All that being the case, I personally believe that as long as you leave the domain out of the equation until AFTER cloning, there should be no issue with a duplicate SIDs given the information in Mark's article and my own experience with OS's post Windows 2000.
As someone else stated I had problems when trying to setup an SQL enviroment and one would not be able to see admin account or similar on domain.
What I allways did were removing the network adaptor. Upon restart it was installed again and voila new SID, I think, and then join the domain again...
Think I left domain first, don't remember... but the reinstall of network adapter were the point here!
Earlier I posted about a computer not joining the domain with apparently a sid problem, but it turned out to be something else related to a network firewall issue, so I recant.
But, I still think the newsid tool should be left hanging around a bit longer, based on some of the arguments on the blog. Although, I understand the argument to not support it.
Leave a comment ? Why ? I haven't seen any answer from Mark to any of the comments already left here...
On the other hand, exchanging experiences is always a positive thing.
And the last paragraph on the article barely mentions WSUS, which others and I am interested in.
I'm not sure what you're looking for: WSUS has it's own per-machine ID that's independent of the SID and that is reset by Sysprep. The same goes for the other state that many have confused with the SID, including NLB GUIDs and SMS IDs.
Again, please do not comment unless you've read the entire post AND all the existing comments. I will not publish your comment if it has nothing new to add to the discussion.