This is Qasim Zaidi. First of all, welcome to my blog site.
Next, I am writing this blog (thanks to a colleague) since some of my customers are running into LDAP limitations which are now hardcoded in Windows Server 2008 and Windows Server 2008 R2. Though, ideally we would like to modify the paged queries but depending on the number of applications a customer might have, they might require months or even years to revamp their whole application coding strategy while in the meantime, they would also be upgrading their existing servers and domain controllers to Windows Server 2008 R2. So let's first see what we are talking about here...
According to http://support.microsoft.com/kb/2009267 titled Windows Server 2008 R2 or Windows Server 2008 domain controller returns only 5000 attributes in a LDAP response:
“An LDAP application may return less information when a query is sent to a Windows Server 2008 or Windows Server 2008 R2 domain controller than when sent to a Windows Server 2003 domain controller. The query results may appear truncated or incomplete. In some occasions you may not get any results.
If, for example, a LDAP application queries the members of a group, the Windows Server 2008 R2 or Windows Server 2008 domain controller only returns 5000 members, while the Windows Server 2003 domain controllers returns many more members…."
"… Hardcoded LDAP limitations have been introduced in Windows Server 2008 R2 and Windows Server 2008 to prevent overloading the domain controller”
What this means is that Windows Server 2008 R2 or Windows Server 2008 dictates MaxPageSize of 20,000 and MaxValRange of 5,000 therefore the maximum number of attributes a query can return is 5,000.
CAUTION: The below should be tested first for any impact on performance as stated in the above KB, and it is also recommended to use Paged Queries (RFC 2969), a standard which was introduced ~10 years ago.
To override the upper-limits introduced in Windows Server 2008/R2 and restore the old-style (no upper limit enforced behavior for LDAP Query Policy in Windows Server 2003), modify the dSHeuristic attribute in Active Directory. To do this, follow these steps:
1. Start ADSI Edit. To do this, open a command prompt in the Support Tools folder, type ADSIEDIT.MSC, press Enter
2. Right-click CN=Directory Service in the following location, and then click Properties: CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=forest root
3. Click the Attribute Editor tab, and then locate dSHeuristic in the Attributes list.Note By default, the value of this attribute is not set.
4. Click dSHeuristic, and then click Edit.
5. Type 000000000100000001 in the Value box, and then click OK. See Note below.
6. Restart the Active Directory Domain Service (NTDS) or the domain controller.
Note If a value has already been set for this attribute, incorporate the existing settings into the new value. When you do this, note the following:
· The tenth character from the left must be 1. Twentieth bit must be 2, and so on.
· The eighteenth character from the left must be 1.
· None of the other characters of the existing value should be changed. For instance, if the existing value is 0000002 then the new value should be 000000200100000001
Microsoft Warning If you use the ADSI Edit snap-in, the LDP utility, or any other LDAP client, and you incorrectly modify the attributes of Active Directory objects, you can cause serious problems. These problems may require you to reinstall Microsoft Windows Server OS, Microsoft Exchange, or both Windows and Exchange. Microsoft cannot guarantee that problems that occur if you incorrectly modify Active Directory object attributes can be solved. Modify these attributes at your own risk. For more, please see below
Windows Server 2008 R2 or Windows Server 2008 domain controller returns only 5000 attributes in a LDAP response http://support.microsoft.com/kb/2009267
Change the LDAP Policy using NTDSUTIL, please follow http://support.microsoft.com/kb/315071
Please also see fLDAPBypassUpperBoundsOnLimits (2 bytes): dSHueristic Attribue
wow, i did not think it was possible, yet, we were struggling to allow more than 5000 returns by a domain controller, you are absoultely right that it will take a lot of customers a long time to get to paged queries. In the meantime, its good to know there is a workaround we can use to breath. Thanks
Excellent Blog, thank you for sharing. I am sure we need to test our applications and OS requirements before migrating to Windows R2, this blog can help us buy more time.
good one indeed, we often don't see a blog like this from Microsoft. But I reckon, with workaround like this, why would companies force their coders to re-write their code or work according to the RFC query standard. Regardless, I am happy to see that there is a way, in the interim, atleast.
wooow, cool blog Qasim.
Yesssss, it worked
we were blocked with our wifi router that authenticate users on Activedirectory (Fortigate)
and the authentication module was unable to retrieve more than 5000 users in a group
in spite of the fact that we had modified the ntds-settings.
but in the microsoft article :
the bits start from 0 so th tenth bit is in fact the illeventh !
the bits you indicated starts from 1 so the tenth bit is the tenth :
are they wrong ?
i filled dsHeuristics with your version and it worked..
Hi benguesmia, per
DS-Heuristics is a Unicode string in which each character contains a value for a single domain-wide setting. The DS-Heuristics string takes the following format.
| <1> | <2> | <3> | <4> | <5> | <6> | <7> | <8> | <9> | <10> | <11> | <12> | <13> |
To provide data validation, each tenth character is set to the character number divided by ten. For example, the tenth character is '1'; the twentieth character is '2', and so on.
excellent, i was struggling with this and reading the comments plus your value, its working now. thanq again
good blog on how to defeat these limits, many thanks
Excellent stuff - way to go!
Note sure if my long, detailed post was lost. To summarize, I think the info posted regarding MaxPageSize is incorrect. Reference:
220.127.116.11.4.6 LDAP Policies
Great article, but as suggested in the midst of the text page limits are in place for a reason. It allows sharing of the service with other applications. For large corporations with lots of objects, you can hang a DC by setting this to a higher value and
letting an app query in large numbers. Something to keep in mind from a performance perspective. When programmers come to me with this complaint, I tell them to research paging thier query and they have a solution same day, no fuss. "Look my apps work!" is
great, until users complain they cannot log on.
Thanks for sharing.