Welcome to TechNet Blogs Sign in | Join | Help

Ask the Directory Services Team

Microsoft's official Enterprise Platform Support DS blog

News

  • Disclaimer: All postings are provided "AS IS" with no warranties, and confer no rights. This weblog does not represent the thoughts, intentions, plans or strategies of Microsoft. Because a weblog is intended to provide a semi-permanent point-in-time snapshot, you should not consider out of date posts to reflect current thoughts and opinions.

    Locations of visitors to this page
The Missing Links

Ned here again. Despite the title, today’s post is not about Australopithecus. I recently spent a few days in Redmond teaching part of the Microsoft Certified Master course. During the class… eh wait…

You’ve never heard of MCM? You should definitely check it out if you’re looking to take your career to the next level, and if you like really, really hard final exams.

Ok, back on track. During the class breaks one of the students asked an interesting question: “Why does the DFS client not return a complete list of DFS links in a referral request?”

If you’ve never looked at a very large DFS Namespace link referral list before, you might not have noticed this either. For most companies, there’s only one really big list – SYSVOL. As you know, the Domain System Volume (SYSVOL) share is actually a special-cased DFS link that only runs on domain controllers and doesn’t require any special set up other than DCPROMO to configure. So let’s consider the following:

Here I run DSQUERY on a Vista SP1 client with RSAT installed to list out all the DC’s in the domain:

Dsquery.exe computer "ou=domain controllers,dc=contoso,dc=corp,dc=proseware,dc=com”

"CN=CO1-NED-DC-99,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-99,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-17,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-20,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=CO1-NED-DC-90,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-26,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-22,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-13,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-02,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-62,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-08,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-05,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-77,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-06,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-03,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=CO1-NED-DC-01,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-16,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-25,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-72,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-27,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=CO1-NED-DC-60,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-65,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-61,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-04,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SIN-NED-DC-01,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-74,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-01,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-70,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-67,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-68,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-69,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=CLT-NED-DC-01,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-09,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-73,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SVC-NED-DC-03,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-66,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-18,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-12,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-15,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-14,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-11,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-23,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-19,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-21,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"
"CN=SRV-NED-DC-24,OU=Domain Controllers,DC=contoso,DC=corp,DC=proseware,DC=com"

For those that don’t feel like counting, that’s 45 DC’s in this domain. So then I run the DFSUTIL command to list out the link targets in this SYSVOL link:

Dfsutil.exe cache referral

<snipped out unrelated goo>

Entry: \contoso.corp.proseware.com\sysvol
ShortEntry: \contoso.corp.proseware.com\sysvol
Expires in 890 seconds
UseCount: 0 Type:0x1 ( DFS )
0:[\SRV-NED-DC-99.contoso.corp.proseware.com\sysvol] (ACTIVE TARGETSET)
1:[\SRV-NED-DC-03.contoso.corp.proseware.com\sysvol] (TARGETSET)
2:[\SRV-NED-DC-23.contoso.corp.proseware.com\sysvol]
3:[\SRV-NED-DC-73.contoso.corp.proseware.com\sysvol]
4:[\SRV-NED-DC-08.contoso.corp.proseware.com\sysvol]
5:[\SRV-NED-DC-60.contoso.corp.proseware.com\sysvol]
6:[\SRV-NED-DC-27.contoso.corp.proseware.com\sysvol]
7:[\SRV-NED-DC-13.contoso.corp.proseware.com\sysvol]
8:[\SRV-NED-DC-09.contoso.corp.proseware.com\sysvol]
9:[\SRV-NED-DC-21.contoso.corp.proseware.com\sysvol]
10:[\SRV-NED-DC-01.contoso.corp.proseware.com\sysvol]
11:[\SRV-NED-DC-70.contoso.corp.proseware.com\sysvol]
12:[\SRV-NED-DC-61.contoso.corp.proseware.com\sysvol]
13:[\SRV-NED-DC-01.contoso.corp.proseware.com\sysvol]
14:[\SRV-NED-DC-72.contoso.corp.proseware.com\sysvol]
15:[\SRV-NED-DC-68.contoso.corp.proseware.com\sysvol]
16:[\SRV-NED-DC-69.contoso.corp.proseware.com\sysvol]
17:[\SRV-NED-DC-17.contoso.corp.proseware.com\sysvol]
18:[\SRV-NED-DC-01.contoso.corp.proseware.com\sysvol]
19:[\SRV-NED-DC-02.contoso.corp.proseware.com\sysvol]
20:[\SRV-NED-DC-04.contoso.corp.proseware.com\sysvol]
21:[\SRV-NED-DC-74.contoso.corp.proseware.com\sysvol]
22:[\SRV-NED-DC-15.contoso.corp.proseware.com\sysvol]
23:[\SRV-NED-DC-06.contoso.corp.proseware.com\sysvol]
24:[\SRV-NED-DC-12.contoso.corp.proseware.com\sysvol]
25:[\SRV-NED-DC-26.contoso.corp.proseware.com\sysvol]
26:[\SRV-NED-DC-19.contoso.corp.proseware.com\sysvol]
27:[\SRV-NED-DC-62.contoso.corp.proseware.com\sysvol]
28:[\SRV-NED-DC-22.contoso.corp.proseware.com\sysvol]

So I have 45 DC’s, but I only see 29 DC link targets here. That’s weird. Maybe it’s just DFSUTIL. Let’s get a third opinion from a network trace like a good engineer always does:

Here’s the referral request packet from the client:

415 14.726426 {NbtSS:76, TCP:75, IPv4:74} 10.80.52.64
SRV-NED-DC-99.contoso.corp.proseware.com DFS DFS:Get DFS Referral Request,
FileName: \contoso.corp.proseware.com\sysvol, MaxReferralLevel: 4
- Dfs: Get DFS Referral Request, FileName: \contoso.corp.proseware.com\sysvol,
MaxReferralLevel: 4
MaxReferralLevel: 4 (0x4)
RequestFileName: \contoso.corp.proseware.com\sysvol

And here’s the response packet from the DC (i.e. the DFS root namespace server):

416 14.726426 {NbtSS:76, TCP:75, IPv4:74}
SRV-NED-DC-99.contoso.corp.proseware.com 10.80.52.64 DFS DFS:Get
DFS Referral Response, NumberOfReferrals: 29 VersionNumber: 4
- Dfs: Get DFS Referral Response, NumberOfReferrals: 29 VersionNumber: 4
PathConsumed: 68 bytes
NumberOfReferrals: 29 (0x1D)
+ ReferralHeaderFlags: 2 (0x2)
- ReferralEntries: Version:4
+ ReferralV4: Index:1 TTL:900 Seconds
+ ReferralV4: Index:2 TTL:900 Seconds
+ ReferralV4: Index:3 TTL:900 Seconds
+ ReferralV4: Index:4 TTL:900 Seconds
+ ReferralV4: Index:5 TTL:900 Seconds
+ ReferralV4: Index:6 TTL:900 Seconds
+ ReferralV4: Index:7 TTL:900 Seconds
+ ReferralV4: Index:8 TTL:900 Seconds
+ ReferralV4: Index:9 TTL:900 Seconds
+ ReferralV4: Index:10 TTL:900 Seconds
+ ReferralV4: Index:11 TTL:900 Seconds
+ ReferralV4: Index:12 TTL:900 Seconds
+ ReferralV4: Index:13 TTL:900 Seconds
+ ReferralV4: Index:14 TTL:900 Seconds
+ ReferralV4: Index:15 TTL:900 Seconds
+ ReferralV4: Index:16 TTL:900 Seconds
+ ReferralV4: Index:17 TTL:900 Seconds
+ ReferralV4: Index:18 TTL:900 Seconds
+ ReferralV4: Index:19 TTL:900 Seconds
+ ReferralV4: Index:20 TTL:900 Seconds
+ ReferralV4: Index:21 TTL:900 Seconds
+ ReferralV4: Index:22 TTL:900 Seconds
+ ReferralV4: Index:23 TTL:900 Seconds
+ ReferralV4: Index:24 TTL:900 Seconds
+ ReferralV4: Index:25 TTL:900 Seconds
+ ReferralV4: Index:26 TTL:900 Seconds
+ ReferralV4: Index:27 TTL:900 Seconds
+ ReferralV4: Index:28 TTL:900 Seconds
- ReferralV4: Index:29 TTL:900 Seconds
VersionNumber: 4 (0x4)
Size: 34 (0x22)
ServerType: Link targets returned or sysvol referral response
+ ReferralEntryFlags: 0 (0x0)
TimeToLive: 900 Seconds
DfsPathOffset: 34 (0x22) Offset:0x4C0
DfsAlternatePathOffset: 104 (0x68) Offset:0x506
NetworkAddressOffset: 2918 (0xB66) Offset:0x1004
ServiceSiteGUID: {00000000-0000-0000-0000-000000000000}
DfsPath: \contoso.corp.proseware.com\sysvol
DfsAlternatePath: \contoso.corp.proseware.com\sysvol
TargetPath: Index:1 \SRV-NED-DC-99.contoso.corp.proseware.com\sysvol

Note how the DC has returned 29 servers as well, so there was nothing wrong with DFSUTIL. The network trace shows how the referral indexes match up perfectly with what DFSUTIL shows, and the Index:1 value matches the position 0 in the DFS referral list we saw earlier. 29 is too weird of a number to be hard coded; developers like binary multiples. So what gives?

After source code review and a confirmation chat with the (awesome) lead developer for DFSN, I had my answer: This is expected, by design behavior. A Windows client will only request a 4 kilobyte buffer of links from the Namespace Root server. The 29 link list is not a hard-coded limit, but just happens to be how many links here in this domain will fit in a 4K referral request buffer used by the client. If the link paths were shorter then more would fit (and mine are pretty long here compared to most domains).

This buffer is used for performance reasons - not memory performance, but failover time performance. Since DFS lists are serialized and each server must be tried if the preceding server is unavailable, it was decided many years ago that only 4K would be allocated. Assuming a failover time of ~20 seconds (including retries) per link target, having an extremely long list of links would make the user experience very poor if the client had to fall through 500 servers and fail just because there was a temporary network issue on the client. If this many links are unavailable, it's highly likely that the network for this client is having catastrophic issues and even if a complete referral list was provided it is unlikely the link target requests would be fulfilled. And since until now this was never even documented, it seems like the decision worked out fine. :-)

For the curious:

1) No, this 4K buffer is not configurable.

2) The SPC referral buffer is 56K, so it's unlikely that it will ever be filled in all but the most gigantic forests. We’ve never heard of it happening in 10 years of AD, at least.

And to that student and his excellent question – Viele Grüße!

- Ned ‘N+1’ Pyle

Posted: Friday, March 06, 2009 5:00 PM by NedPyle
Filed under: ,
Anonymous comments are disabled
Page view tracker