Mark Russinovich’s technical blog covering topics such as Windows troubleshooting, technologies and security.
On November 3 2009, Sysinternals retired NewSID, a utility that changes a computers machine Security Identifier (machine SID). I wrote NewSID in 1997 (its original name was NTSID) because the only tool available at the time for changing machine SIDs was the Microsoft Sysprep tool, and Sysprep doesn’t support changing the SIDs of computers that have applications installed. A machine SID is a unique identifier generated by Windows Setup that Windows uses as the basis for the SIDs for administrator-defined local accounts and groups. After a user logs on to a system, they are represented by their account and group SIDs with respect to object authorization (permissions checks). If two machines have the same machine SID, then accounts or groups on those systems might have the same SID. It’s therefore obvious that having multiple computers with the same machine SID on a network poses a security risk, right? At least that’s been the conventional wisdom.
The reason that I began considering NewSID for retirement is that, although people generally reported success with it on Windows Vista, I hadn’t fully tested it myself and I got occasional reports that some Windows component would fail after NewSID was used. When I set out to look into the reports I took a step back to understand how duplicate SIDs could cause problems, a belief that I had taken on faith like everyone else. The more I thought about it, the more I became convinced that machine SID duplication – having multiple computers with the same machine SID – doesn’t pose any problem, security or otherwise. I took my conclusion to the Windows security and deployment teams and no one could come up with a scenario where two systems with the same machine SID, whether in a Workgroup or a Domain, would cause an issue. At that point the decision to retire NewSID became obvious.
I realize that the news that it’s okay to have duplicate machine SIDs comes as a surprise to many, especially since changing SIDs on imaged systems has been a fundamental principle of image deployment since Windows NT’s inception. This blog post debunks the myth with facts by first describing the machine SID, explaining how Windows uses SIDs, and then showing that - with one exception - Windows never exposes a machine SID outside its computer, proving that it’s okay to have systems with the same machine SID. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so MIcrosoft's support policy will still require cloned systems to be made unique with Sysprep.
Windows uses SIDs to represent not just machines, but all security principals. Security principals include machines, domain computer accounts, users and security groups. Names are simply user-friendly representations for SIDs, allowing you to rename an account and not have to update access control lists (ACLs) that reference the account to reflect the change. A SID is a variable-length numeric value that consists of a structure revision number, a 48-bit identifier authority value, and a variable number of 32-bit subauthority or relative identifier (RID) values. The authority value identifies the agent that issued the SID, and this agent is typically a Windows local system or a domain. Subauthority values identify trustees relative to the issuing authority, and RIDs are simply a way for Windows to create unique SIDs based on a common base SID.
You can use the Sysinternals PsGetSid tool to view a machine’s SID by running it with no command-line arguments:
Here, the revision number is 1, the authority is 5, and there are four subauthority values. At one point during the design of Windows NT, the machine SID might have been used for network identification, so in order to assure uniqueness, the SID that Setup generates has one fixed subauthority value (21) and three randomly-generated subauthority values (the numbers following “S-1-5-21” in the output).
Even before you create the first user account on a system, Windows defines several built-in users and groups, including the Administrator and Guest accounts. Instead of generating new random SIDs for these accounts, Windows ensures their uniqueness by simply appending a per-account unique number, called a Relative Identifier (RID), to the machine SID. The RIDs for these initial accounts are predefined, so the Administrator user always has a RID of 500:
After installation, Windows assigns new local user and group accounts with RIDs starting at 1000. You can use PsGetSid to view the name of the account for a specified SID, and here you can see that the local SID that has a RID of 1000 is for the Abby account, the name of the administrator account Windows prompted me to name during setup:
In addition to these dynamically created SIDs, Windows defines a number of accounts that always have predefined SIDs, not just RIDs. One example is the Everyone group, which has the SID S-1-1-0 on every Windows system:
Another example, is the Local System account (System), which is the account in which several system processes like Session Manager (Smss.exe), the Service Control Manager (Services.exe) and Winlogon (Winlogon.exe) run:
When an account logs on to a Windows system, the Local Security Authority Subsystem (LSASS -Lsass.exe) creates a logon session and a token for the session. A token is a data structure the Windows kernel defines to represent the account and it contains the account’s SID, the SIDs of the groups that the account belongs to at the time it authenticated, and the security privileges assigned to the account and the groups. When the last token that references a logon session is deleted, LSASS deletes the logon session and the user is considered logged off. Here you can see my interactive logon session, displayed with the Sysinternals LogonSessions utility:
And here you can see a token Lsass has created for the session in Process Explorer’s handle view. Note that number following the account name, 7fdee, matches the logon session ID shown by LogonSessions:
By default, processes inherit a copy of their parent process’s token. Every process running in my interactive session, for example, has a copy of the token that they inherited originally from the Userinit.exe process, the process Winlogon creates as the first of any interactive logon. You can view the contents of a process’s token by double-clicking on the process in Process Explorer and switching to the Security page of the process properties dialog:
When one of my processes opens an operating system object, like a file or registry key, the security subsystem executes a permission check that evaluates entries in the object’s access control list (ACL) that reference a SID included in the process’s token.
A similar check happens for remote logon sessions, which are the kind created by a “net use” of a remote computer’s share. To successfully connect to a share you must authenticate to the remote system with an account known to that system. If the computer is part of a Workgroup, then the credentials you specify must be for a local account on the remote system; for a Domain-joined system, the credentials can be for a remote system’s local account or a Domain account. When you access a file on the share, the file server driver on that system uses the token from the logon session for the permission check, leveraging a mechanism called impersonation.
The Microsoft-supported way to create a Windows installation that’s ready for deployment to a group of computers is to install Windows on a reference computer and prepare the system for cloning by running the Sysprep tool. This is called generalizing the image, because when you boot an image created using this process, Sysprep specializes the installation by generating a new machine SID, triggering plug-and-play hardware detection, resetting the product activation clock, and setting other configuration data like the new computer name.
However, some IT administrators install Windows on one of their systems, install and configure applications, then use deployment tools that don’t reset the SIDs of the copies of the Windows installations. The best practice up to now has been to run a SID-resetting utility like NewSID to change SIDs. These utilities generate a new machine SID, try to find all the locations on a system, including all the file system and registry ACLs, that contain copies of the machine SID, and update them to the new SID. The reason that Microsoft doesn’t support systems modified in this way is that, unlike Sysprep, these tools don’t necessarily know about all the places where Windows stashes away references to the machine SID. The reliability and security of a system that has a mix of the old and new machine SID can’t be guaranteed.
So is having multiple computers with the same machine SID a problem? The only way it would be is if Windows ever references the machine SIDs of other computers. For example, if when you connected to a remote system, the local machine SID was transmitted to the remote one and used in permissions checks, duplicate SIDs would pose a security problem because the remote system wouldn’t be able to distinguish the SID of the inbound remote account from a local account with the same SID (where the SIDs of both accounts have the same machine SID as their base and the same RID). However as we reviewed, Windows doesn’t allow you to authenticate to another computer using an account known only to the local computer. Instead, you have to specify credentials for either an account local to the remote system or to a Domain account for a Domain the remote computer trusts. The remote computer retrieves the SIDs for a local account from its own Security Accounts Database (SAM) and for a Domain account from the Active Directory database on a Domain Controller (DC). The remote computer never references the machine SID of the connecting computer.
In other words, it’s not the SID that ultimately gates access to a computer, but an account’s user name and password: simply knowing the SID of an account on a remote system doesn’t allow you access to the computer or any resources on it. As further evidence that a SID isn’t sufficient, remember that built-in accounts like the Local System account have the same SID on every computer, something that would be a major security hole if it was.
As I said earlier, there’s one exception to rule, and that’s DCs themselves. Every Domain has a unique Domain SID that’s the machine SID of the system that became the Domain’s first DC, and all machine SIDs for the Domain’s DCs match the Domain SID. So in some sense, that’s a case where machine SIDs do get referenced by other computers. That means that Domain member computers cannot have the same machine SID as that of the DCs and therefore Domain. However, like member computers, each DC also has a computer account in the Domain, and that’s the identity they have when they authenticate to remote systems.
Some articles on SID duplication, including this KB article, warn that if multiple computers have the same SID, that resources on removable media like an NTFS-formatted firewire disk can’t be secured to a local account. What they fail to mention is that permissions on removable media provide no security regardless, because a user can connect them to computers running operating systems that don’t honor NTFS permissions. Moreover, removable media tend to have default permissions that grant access to well-known SIDs, such as to the Administrators group, which are the same on all systems. That’s the fundamental rule of physical security and why Windows 7 introduced Bitlocker-to-Go, which enables you to encrypt removable storage.
The final case where SID duplication would be an issue is if a distributed application used machine SIDs to uniquely identify computers. No Microsoft software does so and using the machine SID in that way doesn’t work just for the fact that all DC’s have the same machine SID. Software that relies on unique computer identities either uses computer names or computer Domain SIDs (the SID of the computer accounts in the Domain).
It’s a little surprising that the SID duplication issue has gone unquestioned for so long, but everyone has assumed that someone else knew exactly why it was a problem. To my chagrin, NewSID has never really done anything useful and there’s no reason to miss it now that it’s retired. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so Microsoft’s support policy will still require cloned systems to be made unique with Sysprep
if I read you correctly (I tried), the real underlying problem is the encoding of a 'useless' (or more precisely a 'uselessly complex') value in the local user SIDs. Seems a fixed machine part like 'S-1-5-10' would fit anyway. If it doesn't need to be unique, why just not use a known value, like for the predefined accounts?
Then the system would check for local accounts by comparing the RID-less part to that special prefix instead of the machine SID. Changing the machine SID would then become a simple matter.
Hope I didn't overlooked deeper problems.
I took me just under 2 hours to read all the comments and to follow and read most of the embedded links. It has enlightened me as to the root cause of many of the issues encountered when ghosting domain members/controllers in our labs. This is just a rehash and possibly not worth the posting time, but these are the key points I found after digesting the information.
1) Always use sysprep before making an image for deployment to reset other identifiers and machine state.
2) Retrieving data from EFS with the original owner's assistance is pretty much the only valid use for newSid. This is why Bit-Locker should be used for encryption.
3) "SIDs must be unique within the authority in which they are used" - http://blogs.msdn.com/aaron_margosis/archive/2009/11/05/machine-sids-and-domain-sids.aspx
Third party apps that use the MachineSID to identify a machine within a larger grouping need their own ID; MachineSID is only guaranteed to be unique within their authority which is the local machine.
Of course, these generated id for a machine must be unique across the third-party's grouping of machines (WSUS IDs, SMS IDs, NLB GUIDs)... similar to an Active Directory DomainSID.
4) MSDTC also has an ID, the CID, which is not related to MachineSID and it is generated the first time MSDTC runs.
Best practice is to install MSDTC after cloning or use a startup script to execute well documented procedures to solve this issue.
5) A machine that has been added to a domain will have a DomainSID. The machine must not be imaged at this point for use in creating other domain member machines.
6) The Active Directory SID is the MachineSID of the very first DC used to create the Domain. You may not re-use this SID within the domain or for other domains that need to communicate with the first domain.
7) If you have a local machine user account with the same username and password (ie Administrator) you have a security hole that has nothing to do with SIDs.
Thanks for kicking off this awesome discussion.
I total;ly agree with you. I still run sysprep on all my images but as you stated, to clear out state specific entries for apps and not the machine SID. It's amazing how many people re-count tales of newsid saving their bacon without being able to give any technical detail about how! Its funny really as you wrote the darn thing yet they seem to forget that! For my part i have never seen an app that uses the machine sid. Mark, continue the great work. Excellent blog as always. You're a credit to the inducstry. ;)
Hey mark, just 1 question..
Was this just a design choice? Your point is that SIDs must be unique within the authority in which they are used. So obviosuly, domain sids need to be unique in the domain. Did the original designets just then extend that to the machine side to keep the logic the same and the indentifier format i.e. the SID s-x-xx-xxxx etc. the same?
funny in light of this blog:
Yes, the article is incorrect: Kevin would not be able to access John's share and the files on it unless he authenticated with John's user name and password, which he presumably doesn't know.
Thanks for the great article. One quick question - We're currently running a Windows 2000 domain, and about to upgrade to 2008 R2. We have several servers (VM's) that have duplicate SID's (none of them are DC's). I couldn't find any reason in reading your article why this would be a problem, but I was wondering if there is any chance for an issue with these machines when we do our upgrade.
I spent years doing Level II and Level III tech support for LANDesk specifically doing Operating System Deployment and imaging. In this time I found that software apps such as VPN software and including our LANDesk agent have unique values too.
You know what is needed now, is a plugin capability for Sysprep that vendors can use so when their software is installed, and Sysprep is run.
A vendor oculd let sysprep know about their software specific uinique values so that when sysprep runs, the plugin somehow informs sysprep of these files, registry, etc... So then sysprep can start handling the unique values of other pieces of software as long as those other vendors take time to write a sysprep plugin.
The plugin would install with the software, maybe add a registry key that sysprep would look at and it would delete every file, registry key, or maybe run a custom command the vendor provides (in case something complex is required) and sysprep would then do it.
I know if such a plugin capability existed, LANDesk would integrate with it right away.
Sysprep has always included a provider model for third-party applications:
I got to this post looking for SID details because a friend told me that he solved a Skype problem using newSID.exe.
If you have cloned machines with the same SID at the same network, Skype cannot communicate well. For example you can communicate with one contact but it fails with another. Simply run newSID.exe to correct.
I have a similar problem with Cybera (Cyber Cafe Administration) www.sourceforge.net/projects/cybera
Some client machines were created from the same image and Cybera stores each Cybera client MAC address, machine ID and SID at Cybera´s Server Windows registry.
Cybera server crashes when we turn on two or more machines with the same SID and works well if only machines with unique SIDs are connected.
Somehow SID really affects these applications.
Congratulations for your work!
ROTFL about so many people feeling so strongly about something they obviously don't understand and didn't even bother to read the full article (or couldn’t comprehend it).
Please, please tell me none of you ignorant ones are on my IT staff!
Thank you Mark for your info on Machine SIDs. I have been a user of NewSID for several years now. (Win XP) After reading ALL the responses to your article it is CRISTAL CLEAR that the real problem here is not whether Machine SIDs or any other SID can exist in duplication with out creating errors it is that SYSPREP was VERY BADLY written with no real understanding of the needs of the communities it is meant to be used in. The bottom line is this... We, IT support who need to deploy Windows systems, need to be in TOTAL control. We need to have a way to create an MASTER image and deploy it WITHOUT A SINGLE CHANGE, except the obvious i.e. Name of PC, the OS keycode for non Enterprise version of the OS, Join a domain on the fly (not needed but would be nice add on) and generate new SIDs, GUIDs or what ever MS say's MUST be unique to each system and NOTHING ELSE BY DEFAULT, let me repeat that last bit BY DEFAULT!!!!! NO changing of the Admin profile or account, NO OVERWRITING of the default profile we put so many hours in to creating JUST THE WAY WE WANT IT, NO destroying the Administrators’ profile which we, once again have put many hours in to tweaking JUST THE WAY WE WANT IT, no turning on or off ANY settings of any kind etc. These two profiles MUST NEVER BE BASED ON EACH OTHER. They have TOTALY different uses!!!!!! Per MS design. DUH. SO WHAT IS MS DOING MIXING THEM UP? Why being their usual arrogant "we can do no wrong" selves as usual. In fact I bet that they won't even read your blog, nor even vaguely consider the obvious need for a SYSPREP or similar utility that WILL NOT change my IMAGE except where obviously needed as stated above. If I am wrong about MS then here is their opportunity to prove it by admitting in public that they have COMPLETELY missed the boat on the needs of the customers and that before they do anything else they will IMEDATLY rewrite SYSPREP. If they had done this from the start, NewSID would have never been needed and would not now still be needed for Windows 7 etc. and ITs would not need to haggle over whether duplicate SIDs are or are not a problem.
So Mark, please respond AS AN MS representative on the subject of the NEED and the MS response to that need for a new version of SYSPREP. If they need help in laying out how it needs to work I am available for consultation and I guarantee you I will not sign off on anything that does not truly fix the problem and is as simple to use as checking or unchecking about 5 to 10 different check boxes before "sealing" the image for cloning. I figure that I could direct the team who wrote the current SYSPREP to modify the current version to do what it should have always done in about 5 days. It should not be rocket science to throw in a few check boxes and via them jump over the parts of SYSPREP that are screwing up all the images and messing with settings that SHOULD NEVER have been messed with in the first place. And this is really all DUH stuff that I am talking about. So what say yee Mark? Do I make sense? What does the Windows Team say now?
Thanks for that great write up. Though, I'm mainly suprised people are just figuring this out now. Although I use sysprep routinely, I've also deployed ghost images that weren't sysprep'd, and have never had an issue with SID duplication. Still better to dot the 'i's and cross the 't's though. Things are always changing, and you never know when non-conformity might bite you in the end.
@Bob +1 - Thank you for agreeing with me and 90% of the Corp world, MS for got about the Corp World years ago. I have tried and tried to get a hold of someone a MS that will fix this; if SYSPREP worked correctly there would be no need for Ghost or other imaging software. But MS always over writes your settings. We have policies on how things MUST be so we change the Default Profile and the Admin profile. But SYSPREP over writes those changes, so we have to use Ghost and NewSID or some other SID Changer. (I guess we don't need to NewSID now). But please fix SYSPREP. Thanks!!!
Mark I’m grateful to you for explaining SIDs (Machine & DC) in both a direct and easy to comprehend style. I also appreciate the efforts of other posters who provided comments, thereby permitting me to gain experience without its pain. After reading your post, I understand the Machine SID remains encapsulated within the executing OS (authority) and isn’t used by, at least Microsoft, as an interface between remote processes. However, there are instances mentioned in the reply posts where third party tools seem to rely on unique SIDs, as well as the vulnerability of NTFS formatted thumb and external drives
In addition to these situations, I’d like to catalog two other concrete examples where a unique machine SID would improve security. These situations reflect the same vulnerability as would an NTFS formatted thumb drive - more than one authority directly manages/employs the machine SIDs. They are:
• A multi-boot computer. This machine configuration explicitly defeats the machine SID encapsulation constraint, as each OS manages (but not concurrently) the NTFS ACLs of all drives physically attached to the machine. When building machines, I generally create a base image that includes a number of diagnostic utilities. This base image becomes a “Diagnostic” partition and evolves to become an end user’s “Primary” partition. These cloned partitions not only share the same Machine SID they also explicitly share, at least when first cloned, the same Account SIDs. Finally, since generating an RID involves simply incrementing the last one assigned, this mechanism easily generates overlapping Account SIDs between the different partitions.
• A situation where a drive is physically removed from one computer and installed into another where these computers were all cloned from one image and not SysPrep(ed)/NewSID . For example, pretending to work late one evening, a disgruntled employee decides to remove his manager’s hard drive and install it to his own computer. If this employee happens to know the password for the local administrator account (which might not have one), have the same local Account SID, or generate the same SID as the manager (by adding accounts to his image), the employee would not need to know his manager’s password and could then read/delete his manager’s files.
The situations above are not common, nor do they contradict Mark’s assertion. However, having a tool like NewSID would help to better secure the file system from casual attacks. Then again, having an easy to use tool like NewSID introduces the same vulnerability it was intended to solve….