The reason for this problem is a mismatched shared secret.
The way CHAP works is that the password is MD5 hashed and sent to the server in a RADIUS request.
The server receives the RADIUS request, with the password hash, and then retrieves the password from SAM (System account manager), AD or NT4 Domain. Then proceeds to MD5 hash the password. And if the hashes match, the server will send an “access accept”.
This is all fine, IF the shared secret is used to calculate the message (request authenticator). When authenticating using MD5-CHAP, 2 different authenticators are present. One is for the request, and one for the response.
To calculated the message digest for the request a function similar to the following is used
RequestAuth = 16 Octet number.
That is Random, Unique on the lifetime of the shared secret. (but how it’s implemented is up the the hardware vendor)
However when calculating the Authenticator for the response, the following function is used:
ResponseAuth = MD5(Code+ID+Length+RequestAuth+Attributes+Secret)
As you can see, the secret is a major part of the response authenticator (for IAS/NPS), but not necessarily the same for the NAS implementation.
The result - A mismatched shared secret might go unnoticed for a while. Unless the admin gets complaints from users that they’re unable to login. When the admin checks the logs and discovers that they have indeed been issued “access accepts”. This should be the first place to look. Reset the shared secret on IAS and on the NAS.
Methods suffering from this issue: