A couple weeks back, Taylor Higley asked a question on Twitter about Azure AD Password Sync, MD5 and FIPS compliance:
My reply was a bit cryptic and prompted replies from Eric Kool-Brown and Brian Arkills that pointed out that one-way hashes can't be decrypted (at least not without some brute forcing):
This blog post is a follow up to that conversation with more details now that I can take advantage of the luxury of having a more than 140 characters to use.
So, why does Password Sync fail to run on a Federal Information Processing Standard (FIPS) compliant machine and what is going on under the covers? This is because FIPS compliant systems bar the use of MD5 hash algorithms, so they block Password Sync when the tool tries to access MD5 functions.
Does Password Sync use MD5 functions for encryption? The answer is No. Here's what is actually going on under the covers.
User passwords are stored as a non-reversible hash in Windows Server Active Directory Domain Controllers (DCs). When our password sync agent attempts to synchronize the password hash from a DC over a secure RPC interface, the DC encrypts that password hash using an MD5 key. The MD5 key that the DC uses is derived from the RPC session key and a salt. Once this happens, the password hash is now wrapped in an MD5 encryption envelope. The password sync agent gets this encrypted password hash from the domain controller over the secure RPC interface.
The following is a code snippet how the MD5 hash key is generated.
byte ComputeMd5(byte sessionKey, byte salt)
byte data = new byte[sessionKey.Length + salt.Length];
Buffer.BlockCopy(sessionKey, 0, data, 0, sessionKey.Length);
Buffer.BlockCopy(salt, 0, data, sessionKey.Length, salt.Length);
using (MD5 md5 = new MD5CryptoServiceProvider())
Once the Password sync agent has the encrypted password hash it uses MD5CryptoServiceProvider to generate a hash key used for decrypting the envelope containing the password hash. At no point in time does the password sync agent have access to the clear text password. The password sync agent then secures the password hash by re-hashing it using a stronger SHA256 hash per RFC 2898 before uploading it to the cloud.
So when MD5CryptoServiceProvider is used in a FIPS compliant environment, it throws a System.InvalidOperation exception. This is because the MD5 hash is considered a weak hash and not recommended for use in a FIPs environment. However since it is not being used to do encryption, we believe this is a non-issue.
Password Sync can be enabled in a FIPS compliant system by locally disabling FIPS for the Directory Sync process. This can be done by adding <enforceFIPSPolicy enabled="false" /> in the miiserver.exe.config file. The miiserver.exe.config can be found at % ProgramFiles \Windows Azure Active Directory Sync\SYNCBUS\Synchronization Service\Bin\ miiserver.exe.config.
For more information here, refer to http://social.technet.microsoft.com/wiki/contents/articles/24961.password-sync-failing-in-fips-compliant-systems.aspx
I hope this is helpful and provides clarity for everyone interested in this issue.
As always, would love to get any feedback or suggestions you have!
Alex Simons (twitter: Alex_A_Simons)
Director of PM
Active Directory Team
(updated 7/3/2014 to fix a few typos and grammar issues)
This totally makes no ense. MD5 Encryption Envelop?
And no, there is no excuse for using MD5 as one-way for password hashing, it is still weak (easy to bruteforce and most likely even easier than that to collide)
This is a great post! Thank you for sharing your ideas. I always love learning new things because they widen my knowledge regarding certain things. Thanks and more power.
Is what is stored by Azure a verifier, or a hash of an NT hash which is not salted?
I am concerned with salting.
Neither the NT hash nor the LM hash is salted. Salting is a process that combines the password with a random numeric value (the salt) before computing the one-way function. Windows has never stored hashes in human-readable form, so there has never been a need
to salt them.
Windows also stores a password verifier on domain members when a domain user logs on to that domain member. This verifier can be used to authenticate a domain user if the computer is not able to access the domain controller. The password verifier is also commonly
called a cached credential. It is computed by taking the NT hash, concatenating the user name to it, and then hashing the result by using the MD4 hash function.
Like Bernd says, there is no such thing as an MD5 encryption envelope. As you note, AD does not store clear text passwords. The Windows client converts the initial user-entered plain-text password into a hash using the One-Way-Function (OWF) and that is
what is stored on the user object. Then when the user subsequently logs in, the Windows client runs the password through the OWF and sends the encrypted hash to the DC where it is decrypted and compared to the stored password OWF hash. Consequently DirSync
never has to mess with clear text passwords. Sure, you'd want to encrypt the password OWF hash when it is on the wire to AAD, but you'd use an encryption algorithm like RSA not a hash algorithm like MD5. Perhaps AAD is applying another round of hashing to
the password because the OWF is known to be weak, but why use MD5 at all, just use SHA256 directly.
BTW, my description is simplified for the sake of brevity. There are lots of good write-ups on the LM and NTLM OWFs including an excellent one by Jesper Johansson
The phrase "MD5 encryption envelope" seems to be a source of confusion here, and I'll admit that it perplexes me, too. First, Alex writes: "Does Password Sync use MD5 functions for encryption? The answer is No." Then the very next paragraph freely mixes
the terms "MD5" and "encryption." May we please have a more precise explanation of what's really going on?
Also, let me see if I have the flow straight --
1. The DC stores a hash of the password.
2. When asked to replicate this, the DC hashes the hash using the RPC session key and some salt. Where does the salt come from? Also, if I'm reading correctly, this what the article calls an "MD5 encryption envelope." I think, though, it's more accurate to
call it a hash of a hash, right?
3. The DC hands salted-and-hashed hash to the sync agent. The text states the agent "uses MD5CryptoServiceProvider to generate a hash key used for decrypting the envelope containing the password hash." What does this mean? It is the same "MD5 key" generated
by the DC in step 2? If so, wouldn't the sync agent need access to the same salt? Where does it get that from?
4. Before sending the password hash to the cloud, the sync agent re-hashes the hash with SHA256. OK, which hash is getting re-hashed? I think this is the salted-and-hashed hash handed to the DC at the beginning of step 3, right?
5. Wait...there is no step 5. We also need clarification around what happens at the other end. The text says the cloud receives the SHA256 has of (some) hash. Fine. What about that "MD5 hash key" generated in step 3? The text says it's for "decrypting the envelope
containing the hash." This is useless for the SHA256 hashed of (some) hash. So where does it get sent, and when, and when exactly does it get used?
One more thing. The recommendation to disable FIPS for directory sync isn't likely going to sit well with government customers. In his comment, Eric writes, "why use MD5 at all, just use SHA256 directly." Indeed -- why all this goo when a much simpler step
appears to be better?
Could we get a second opinion on this from a different cryptography expert? While I can maybe understand the justification of "compatibility issues," the rest of the justification about why this is sound security / cryptography is making much sense to
other security people.
From your description, MD5 actually *is* being used to cryptographically protect the user hashes, which are fairly sensitive data. Using MD5 in any layer of protecting data, with no setting that lets you change it to SHA, is just contrary to established security
best practices, regardless of how it is justified or spun. That applies to all cryptography -- whether you're talking encryption, one-way hashes, session keys, HMAC and checksums, etc.
The Windows / AD password hash is not secure and needs strong protection to keep attackers from getting the hash. The hash may be one-way, and yet that fact doesn't make disclosing the hash any safer. Attacks like "pass-the-hash" let attackers use the hash
itself without needing to know what the user's plain-text password is. The user's hash is also used in other cryptographic operations, so that knowing the hash gives you insight into part of the crypto key.
There are multiple contractual government mandates here (FIPS 140, FedRAMP, USGCB, etc.) and contractually Microsoft really is not supposed to be advising federal customers to weaken their security posture. You've got a way to disable FIPS -- so why not a way
to disable MD5 and use a more modern, secure cipher? Heck, Microsoft Update's use of cryptography was subverted by Flamer malware to compromise Windows computers, due to a few MS certificates that used MD5, and not for encryption but for signing. So stuff
like replacing MD5 with SHA does sometimes matter a great deal.
The Azure password replication service is relatively new and only talks to other DCs, right? What does it need to be compatible with? What else in the cloud would also be using a really funky, unusual, hard-coded to MD5-only crypto methodology like the one
The Wikipedia article on MD5 says it pretty well, and it's pretty damning -- MD5 should probably never have made it into Azure and shouldn't remain there if it can be helped.
Sorry for not replying on any of these. I didn't realize there had been any comments on this topic as it sat quiet for several weeks.
First, clearly my use of the word "envelope" was confusing. I was using it metaphorically trying to explain that the original password hash was now inside another layer of encryption. I apologize for my use of a metaphor in the context of an article where you
were looking for technical precision. To be technically precise, at that point the original password hash has been encrypted by the DC before handing it to the password sync agent which resides on the same server.
@Eric Kool Brown's description of how the original password has is created and used by Windows client is correct. As I explained in the article as that original password hash is replicated (using the DC replication protocol) from the DC to the Password Sync
Agent, the DC encrypts the hash. The encryption is performed by the DC with a key derived from the RPC session key by salting it. The key derivation is as follows [where SaltedEncryptionKey = MD5 ( RPC session Key, 128 bit random salt)]. The DC also passes
the salt to the sync agent using the replication protocol. Our sync agent decrypts the encrypted hash by deriving the key as described above. The password sync agent uses MD5 to perform the key derivation, as the derivation has to be identical to derivation
the DC performed (when it encrypted the data). And MD5 is the highest level available for this action in the DC replication protocol of existing Windows Server Active directory deployments. Once the decryption is done, the sync agent takes the resulting original
password hash and re-hashes it to a SHA256 hash by using PKDF2 key derivation algorithm as defined in RFC 2898. The Password Sync Agent then syncs that SHA256 hashed password hash over the wire to Azure AD.
Net Net: The only thing that crosses the wire on the way to Azure AD is a SHA256 hashed copy of original password hash. The password sync agent’s use of MD5 is strictly for replication protocol compatibility with the DC
@user: Once the SHA256 hashed copy of the original password hash reaches Azure AD, Azure AD encrypts the hash with AES algorithm before storing it in the cloud database.
Hope this helps to further clarify.