Consider a scenario where users are not able to authenticate using FBA with LDAP to access the websites published through ISA server. In this scenario ISA server was part of one domain (contoso.com) and users who would access the website are part of another domain (fabrikam.com). FBA with LDAP is used on the web listener of the web publishing rule to authenticate the users from fabrikam.com’s domain controller. But these users are not able to authenticate using this method.
Note: More about Ldap authentication please refer http://technet.microsoft.com/hi-in/library/bb794854(en-us).aspx#ldap
To troubleshoot the issue took network traces while creating LDAP user set on the ISA server (reference http://technet.microsoft.com/hi-in/library/bb794854(en-us).aspx#LDAPUser), which failed with error, “access to LDAP server is denied.”
In the network captures we found:
1. LDAP Bind Request as below
2. LDAP Bind Response as below
Troubleshooting and Resolution
We tested the user credentials from a machine which is already part of the fabrikam domain and we were able to authenticate using same credentials. Then as per http://blogs.technet.com/b/isablog/archive/2008/04/17/isa-server-2006-form-base-authentication-problem-using-upn-logon-format-on-a-multiple-domain-environment.aspx we checked the HKLM\System\CCS\CONTROL\LSA\LMCompatibilityLevel on the ISA server and it was set to 0x2 (only allow LM and NTLM). Then checked same on the Domain Controller (which it was windows 2008 server) of the domain where users were located and it was set to 0x5(only allow NTLMv2 and block LM /NTLM).Since Domain controller only allows NTLM v2 it was not authenticating the request coming from the ISA server which was sending it with NTLM v1 as we can see in the NTLM challenge response in the network traces.
To resolve the problem we set LMCompatibilityLevel key on the ISA server to 0x3 (although we could also set it to 0x4) and restarted the ISA server. After that users were able to normally logon via FBA.
Author Suraj Singh Support Engineer Microsoft CSS Forefront Security Edge Team
Technical Reviewer Yuri Diogenes Sr Support Escalation Engineer Microsoft CSS Forefront Security Edge Team