Mark Russinovich’s technical blog covering topics such as Windows troubleshooting, technologies and security.
On November 3 2009, Sysinternals retired NewSID, a utility that changes a computers machine Security Identifier (machine SID). I wrote NewSID in 1997 (its original name was NTSID) because the only tool available at the time for changing machine SIDs was the Microsoft Sysprep tool, and Sysprep doesn’t support changing the SIDs of computers that have applications installed. A machine SID is a unique identifier generated by Windows Setup that Windows uses as the basis for the SIDs for administrator-defined local accounts and groups. After a user logs on to a system, they are represented by their account and group SIDs with respect to object authorization (permissions checks). If two machines have the same machine SID, then accounts or groups on those systems might have the same SID. It’s therefore obvious that having multiple computers with the same machine SID on a network poses a security risk, right? At least that’s been the conventional wisdom.
The reason that I began considering NewSID for retirement is that, although people generally reported success with it on Windows Vista, I hadn’t fully tested it myself and I got occasional reports that some Windows component would fail after NewSID was used. When I set out to look into the reports I took a step back to understand how duplicate SIDs could cause problems, a belief that I had taken on faith like everyone else. The more I thought about it, the more I became convinced that machine SID duplication – having multiple computers with the same machine SID – doesn’t pose any problem, security or otherwise. I took my conclusion to the Windows security and deployment teams and no one could come up with a scenario where two systems with the same machine SID, whether in a Workgroup or a Domain, would cause an issue. At that point the decision to retire NewSID became obvious.
I realize that the news that it’s okay to have duplicate machine SIDs comes as a surprise to many, especially since changing SIDs on imaged systems has been a fundamental principle of image deployment since Windows NT’s inception. This blog post debunks the myth with facts by first describing the machine SID, explaining how Windows uses SIDs, and then showing that - with one exception - Windows never exposes a machine SID outside its computer, proving that it’s okay to have systems with the same machine SID. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so MIcrosoft's support policy will still require cloned systems to be made unique with Sysprep.
Windows uses SIDs to represent not just machines, but all security principals. Security principals include machines, domain computer accounts, users and security groups. Names are simply user-friendly representations for SIDs, allowing you to rename an account and not have to update access control lists (ACLs) that reference the account to reflect the change. A SID is a variable-length numeric value that consists of a structure revision number, a 48-bit identifier authority value, and a variable number of 32-bit subauthority or relative identifier (RID) values. The authority value identifies the agent that issued the SID, and this agent is typically a Windows local system or a domain. Subauthority values identify trustees relative to the issuing authority, and RIDs are simply a way for Windows to create unique SIDs based on a common base SID.
You can use the Sysinternals PsGetSid tool to view a machine’s SID by running it with no command-line arguments:
Here, the revision number is 1, the authority is 5, and there are four subauthority values. At one point during the design of Windows NT, the machine SID might have been used for network identification, so in order to assure uniqueness, the SID that Setup generates has one fixed subauthority value (21) and three randomly-generated subauthority values (the numbers following “S-1-5-21” in the output).
Even before you create the first user account on a system, Windows defines several built-in users and groups, including the Administrator and Guest accounts. Instead of generating new random SIDs for these accounts, Windows ensures their uniqueness by simply appending a per-account unique number, called a Relative Identifier (RID), to the machine SID. The RIDs for these initial accounts are predefined, so the Administrator user always has a RID of 500:
After installation, Windows assigns new local user and group accounts with RIDs starting at 1000. You can use PsGetSid to view the name of the account for a specified SID, and here you can see that the local SID that has a RID of 1000 is for the Abby account, the name of the administrator account Windows prompted me to name during setup:
In addition to these dynamically created SIDs, Windows defines a number of accounts that always have predefined SIDs, not just RIDs. One example is the Everyone group, which has the SID S-1-1-0 on every Windows system:
Another example, is the Local System account (System), which is the account in which several system processes like Session Manager (Smss.exe), the Service Control Manager (Services.exe) and Winlogon (Winlogon.exe) run:
When an account logs on to a Windows system, the Local Security Authority Subsystem (LSASS -Lsass.exe) creates a logon session and a token for the session. A token is a data structure the Windows kernel defines to represent the account and it contains the account’s SID, the SIDs of the groups that the account belongs to at the time it authenticated, and the security privileges assigned to the account and the groups. When the last token that references a logon session is deleted, LSASS deletes the logon session and the user is considered logged off. Here you can see my interactive logon session, displayed with the Sysinternals LogonSessions utility:
And here you can see a token Lsass has created for the session in Process Explorer’s handle view. Note that number following the account name, 7fdee, matches the logon session ID shown by LogonSessions:
By default, processes inherit a copy of their parent process’s token. Every process running in my interactive session, for example, has a copy of the token that they inherited originally from the Userinit.exe process, the process Winlogon creates as the first of any interactive logon. You can view the contents of a process’s token by double-clicking on the process in Process Explorer and switching to the Security page of the process properties dialog:
When one of my processes opens an operating system object, like a file or registry key, the security subsystem executes a permission check that evaluates entries in the object’s access control list (ACL) that reference a SID included in the process’s token.
A similar check happens for remote logon sessions, which are the kind created by a “net use” of a remote computer’s share. To successfully connect to a share you must authenticate to the remote system with an account known to that system. If the computer is part of a Workgroup, then the credentials you specify must be for a local account on the remote system; for a Domain-joined system, the credentials can be for a remote system’s local account or a Domain account. When you access a file on the share, the file server driver on that system uses the token from the logon session for the permission check, leveraging a mechanism called impersonation.
The Microsoft-supported way to create a Windows installation that’s ready for deployment to a group of computers is to install Windows on a reference computer and prepare the system for cloning by running the Sysprep tool. This is called generalizing the image, because when you boot an image created using this process, Sysprep specializes the installation by generating a new machine SID, triggering plug-and-play hardware detection, resetting the product activation clock, and setting other configuration data like the new computer name.
However, some IT administrators install Windows on one of their systems, install and configure applications, then use deployment tools that don’t reset the SIDs of the copies of the Windows installations. The best practice up to now has been to run a SID-resetting utility like NewSID to change SIDs. These utilities generate a new machine SID, try to find all the locations on a system, including all the file system and registry ACLs, that contain copies of the machine SID, and update them to the new SID. The reason that Microsoft doesn’t support systems modified in this way is that, unlike Sysprep, these tools don’t necessarily know about all the places where Windows stashes away references to the machine SID. The reliability and security of a system that has a mix of the old and new machine SID can’t be guaranteed.
So is having multiple computers with the same machine SID a problem? The only way it would be is if Windows ever references the machine SIDs of other computers. For example, if when you connected to a remote system, the local machine SID was transmitted to the remote one and used in permissions checks, duplicate SIDs would pose a security problem because the remote system wouldn’t be able to distinguish the SID of the inbound remote account from a local account with the same SID (where the SIDs of both accounts have the same machine SID as their base and the same RID). However as we reviewed, Windows doesn’t allow you to authenticate to another computer using an account known only to the local computer. Instead, you have to specify credentials for either an account local to the remote system or to a Domain account for a Domain the remote computer trusts. The remote computer retrieves the SIDs for a local account from its own Security Accounts Database (SAM) and for a Domain account from the Active Directory database on a Domain Controller (DC). The remote computer never references the machine SID of the connecting computer.
In other words, it’s not the SID that ultimately gates access to a computer, but an account’s user name and password: simply knowing the SID of an account on a remote system doesn’t allow you access to the computer or any resources on it. As further evidence that a SID isn’t sufficient, remember that built-in accounts like the Local System account have the same SID on every computer, something that would be a major security hole if it was.
As I said earlier, there’s one exception to rule, and that’s DCs themselves. Every Domain has a unique Domain SID that’s the machine SID of the system that became the Domain’s first DC, and all machine SIDs for the Domain’s DCs match the Domain SID. So in some sense, that’s a case where machine SIDs do get referenced by other computers. That means that Domain member computers cannot have the same machine SID as that of the DCs and therefore Domain. However, like member computers, each DC also has a computer account in the Domain, and that’s the identity they have when they authenticate to remote systems.
Some articles on SID duplication, including this KB article, warn that if multiple computers have the same SID, that resources on removable media like an NTFS-formatted firewire disk can’t be secured to a local account. What they fail to mention is that permissions on removable media provide no security regardless, because a user can connect them to computers running operating systems that don’t honor NTFS permissions. Moreover, removable media tend to have default permissions that grant access to well-known SIDs, such as to the Administrators group, which are the same on all systems. That’s the fundamental rule of physical security and why Windows 7 introduced Bitlocker-to-Go, which enables you to encrypt removable storage.
The final case where SID duplication would be an issue is if a distributed application used machine SIDs to uniquely identify computers. No Microsoft software does so and using the machine SID in that way doesn’t work just for the fact that all DC’s have the same machine SID. Software that relies on unique computer identities either uses computer names or computer Domain SIDs (the SID of the computer accounts in the Domain).
It’s a little surprising that the SID duplication issue has gone unquestioned for so long, but everyone has assumed that someone else knew exactly why it was a problem. To my chagrin, NewSID has never really done anything useful and there’s no reason to miss it now that it’s retired. Note that Sysprep resets other machine-specific state that, if duplicated, can cause problems for certain applications like Windows Server Update Services (WSUS), so Microsoft’s support policy will still require cloned systems to be made unique with Sysprep
Stumbled upon this blog entry today, while checking to see if there was a new version available before I ghosted a few systems at a school and boy am I glad I did!
Every since I started working in schools and then working for myself, I have always used NewSID. Like many others, it always seemed to fix problems that reared there ugly heads and it became part of my 'Ghosting' process. In fact, I have advised so many people to do the same and I now have to direct them to this blog to show the error of my ways.
Anyway, I used SysPrep on a suite of 15 machines running Windows XP Pro SP3 after following this articles advice and it went nice and smoothly. Also, would you believe this fixed my problem with only one of the 15 machines ever showing up within WSUS.
So end result is two problems fixed and probably a lot more besides – so once again thanks and I urge all of you who still use NewSID to dump it and use SysPrep – it only took me about 10 minutes to get used to the utility and without disregarding Mark’s hard work in the past, it does seem to do the job more comprehensively.
P.S. Regarding some of the arguments about SIDS being important, yes they quite possibly are otherwise I would hope they wouldn’t still be lingering around, but who cares when the SysPrep tool assigns a new SID anyway if you ask it to. What is the problem?
Apparently, the - or, at least, a - problem is that NewSID allowed people to change the SID after the system was already up and running, while sysprep has to be used before deployment time (and can mess up some installed software).
Given that and my own experience, I would second the request from Nick Brown for a sysprep option to "just change the SID, do nothing else", alongside the option - or at least hopefully it would be an option! - to not change the SID at all. (I would also hope for a version of sysprep with this capability which is compatible with older versions of Windows, at least XP, but I'm not holding my breath over that one.)
ok Let's do the check list.
1. Ph'd check
2. Microsoft fellow check
3. Author of some the best windows books on the market. Check
4. Developer of the software in question (by the way given away for free). Check
5. On the Microsoft Core OS development team. Check
Yep you guys are right and he is full of it.No wonder Wintel IT Admins are held in such high regard.
A couple people posted things suggesting that this article is wrong... not that many people will read all the comments, actually getting to and reading this one! but..
As per KMS where someone else already said that it uses CMID, which is not SID or some such thing.. The OP re: KMS said "problem with non-sysprepped systems" - indicating the problem with these statements, as someone else mentioned -
...that MS still says to *use* sysprep, because it does other things that will cause some software problems if they are not done!
@MSDTC\DCOM\SQL\NLB (ect) Posts
MSDTC does not use the machine SID. It uses an internal UUID that is created when the computer starts for the first time with MSDTC running (i.e. before it is cloned) (there is a TechNet article on this somewhere). Most cloning software does not reset the MSDTC UUID Value (including Sysprep in an ESX VM Environment using the default sysprep options). You can verify this yourself by performing the following in a test environment with the issue (clusters are slightly more complicated but SQL connections are simple between two cloned servers with MSDTC installed prior to the cloning job):
1)from a command line: MSDTC.EXE -Uninstall
3)from a command line: MSDTC.exe -Install
This will fix your 'SID' like issues with MSDTC. We have had 100% success using the method for SQL errors in a cloned VM environment. The same should be true for all components that use MSDTC.
We had to update all of our cloned VM systems by removing MSDTC and reinstalling it where applicable (no other changes were needed to other applications). We also removed it from our master clone and scripted the install during a run once command. We have never seen the issue return and we have never run any re-SID on the servers.
The issue above with SQL only arrived when we needed multiple servers to talk to a single SQL server using MSDTC. There are no issues with our XP workstations using the same UUID to connect to the SQL server since the clients came from a different image; the UUID is already different from the server.
Please let me know if someone else has experienced otherwise (with details please). But the ESX (v3.5U4) flags for Sysprep did not reset the MSDTC UUID across images which utilized SysPrep and still generated 'identical SID' errors in the event log for MSDTC. Reinstalling the service is what fixed the internal UUID problem (also documented in a technet article I can no longer find). Setting different SysPrep flags might be another solution during the clone template function.
Sorry for the second post, but I don't like posting information without proper references. It seems that some of my information was incorrect and I have supporting data today...
MSDTC uses a CID (not a UUID) which is stored in the registry located at HKEY_CLASSES_ROOT\CID as well as registry data located: HKLM/Software/Microsoft/Software/MSDTC and HKLM/System/CurrentControlSet/Services/MSDTC.
If you Google "msdtc CID" you will come across several people who have had issues with SysPrep not replacing the CID value of services during the process (including myself). I do not know if this is normal functionality but it would explain the issue with communications issues. The solution is very similar to what I posted in my previous post. Uninstall MSDTC, delete the registry key's (please make backups first), reinstall MSDTC and verify your configuration settings.
User Community procedures for updating the MSDTC CID: http://blog.wadewegner.com/index.php/2007/08/13/warning-the-cid-values-for-both-test-machines-are-the-same/
Microsoft Support Procedures for reinstalling MSDTC on Server 2000 (very similar in 2003): http://support.microsoft.com/kb/867520
Microsoft MSDTC Troubleshooting tools (which would have correctly identified your problem as a CID issue): http://support.microsoft.com/kb/918331
I also need to correct my statement about not needing a Unique Machine SID for MSDTC to work correctly. While I have machines with the same machine SID and unique MSDTC CID's talking together, I found a BizTalk 2006 Server reference from the Microsoft MSDN site which states the following:
"Computers running the Windows operating system use security identifiers, or SIDs, to identify themselves. MSDTC functionality requires that the host operating system is assigned a unique SID. Disk duplicate images of Windows installations must be configured with the System Preparation tool (Sysprep) or the SID of the deployed operating system may not be unique and MSDTC functionality may be impaired. This can occur when using virtual hard disks to deploy an operating system to a virtual machine." Ref- Ensure that the operating system is assigned a unique security identifier (SID) , http://msdn.microsoft.com/en-us/library/aa561924(BTS.20).aspx
@Mark: Is it possible to get clarification on the SID requirement based on the MSDN article? Does MSDTC use the domain SID (when in a domain) or the machine SID? Is the statement still accurate with the current operating systems (my reference is a BizTalk Server 2006 article)? Should SysPrep change the CID values in the registry?
Mark, I am glad that you did the research and found this out. I am also very glad that there are so many people out there that are passionate about this topic. If the reader goes through this thread completely, they definitely will come away with a better understanding of how these identifiers work.
Even though NewSID was mainly used to generate new unique SID-s, there is need to be able to set specific (cloned/duplicate) SID for an installation. For example, in order to save EFS keys from no longer bootable system (e.g. due hardware failure without replacement available and system not bootable in virtualised environment), one needs to create an installation with identical SID, user identifiers and passwords in order to gain access and decrypt/export the users EFS keys (which are required to access EFS encrypted files on the disk). This is a scenario, where sysprep can not help and NewSID is the only "semiautomated" solution. Thus, retirement of NewSID is premature as it still has valid (and critical, though rare) use cases. It might be an option to create a data recovery helper tool specifically for setting machine SID and user account identifiers without the random ID generation feature to discourage using this tool in production environments.
Can you please add this comment you made to the main post:
"It appears many readers are confusing machine-specific state, computer Domain SIDs, and machine SIDs ..."
I'm not a trained systems administrator but have helped out people with small networks so all of this is very helpful to me. Thanks for posting this.
Consider users at home - many machines are never added to a domain.
Now consider that these users are apt to change their machine name occasionally.
Now imagine you're a cloud vendor that needs to identify each machine associated with a user's account that is faced with multiple machines named "Jane's Computer".
How else do you know which machine is which? Do you trust the user? (Did I mention they were users at home?)
Like all the applications we've been talking about, including WSUS, SMS, etc, you can create your own system identifier and place that on user's systems.
First, I have to admit, that when i saw this post back in oct. i was shocked as the rest of you. I trust Mark's words to the "T" on everything, but in this case my years of experience with Windows begs me to differ and contest Mark's findings.
The new for unique SID can be demonstrated easily with the use of Vmware workstation or such. Take a VM image of 2003 or 2008. Make two linked clones. login to one of them, use dcpromo and make it the first DC in a new domain. get onto the second one, and join it into the domain. reboot. now try to login to the domain from the second machine, and it will not work, and will complain of domain ID problems.
use newSID on the second machine and add it to the domain. Everything works flawlessly.
You obviously didn't read the post, at least not the entire thing. As I stated, the first DC in a domain must have a SID unique from the member systems.
In December, I posted a long comment about how NewSID has helped us and fixed many problems but it was never posted here. I did read the entire posting and there may have been some similar parts of my post but it was detailed to help others. I am disappointed that it was not posted.
I guess at this point we all would like to know what the verdict is: Mark will continue updating NewSID stating it is not needed and letting the end user decide to use it or not. Or will the windows community have to turn to 3rd party utilities to save them time and money ??
I feel that if NewSID is not needed then I will not use it in my cloning. However using it to fix problems where a SID change is needed or required to fix a problem the program should be available. The use of NewSID in ESF recovery (I have been there many times) is a God Sent. Server issues: there is no way I want to spend 10 days rebuilding a proprietary server just to change the SID because some Junior Admin did his part wrong, or the company out sourced the OS load and you don’t know you have and issue until it is too late. NewSID has save me many thousands of hours. I would hope that Mark would want to continue to help the Admin Community no matter how they use NewSID.
Mark thanks for all your hard work, your free software and the many articles and books you have written. They have helped me immensely over the years. I hope you will continue to update NewSID.
I must apologize. I overlooked that part. But before that please accept my kudos for responding back. You clearly are better at managing time than me.
I'm still sure that I faced many strange problems. I'll work on trying to replicate those problems and post back.