• DCDiag reports “not advertising as time server”

    Just a quick post.

    Was onsite recently and had a DC that was not advertising is a time server. DCDiag confirmed it.

    Checked announceflags and it was set to 10 in hex (0x00000010) instead of 10 in decimal (0x0000000A).

    Changed registry value and use “w32tm /config /update” and all was well. Interestingly it appears to only read last value. So even he (0x0000001A) also works and advertises as time server. Presumably because it reads the last character “A” and thinks

    A = 10 = (0x8) + (0x2)

    and as per http://technet.microsoft.com/en-us/library/cc773263(WS.10).aspx#w2k3tr_times_tools_uhlp 

    AnnounceFlags
    Registry path

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config

    Version

    Windows XP, Windows Vista, Windows Server 2003, and Windows Server 2008

    This entry controls whether this computer is marked as a reliable time server. A computer is not marked as reliable unless it is also marked as a time server.

    • 0x00 Not a time server
    • 0x01 Always time server
    • 0x02 Automatic time server
    • 0x04 Always reliable time server
    • 0x08 Automatic reliable time server

    The default value for domain members is 10. The default value for stand-alone clients and servers is 10.

  • DCLocator and Closest Site info

    I’ve been doing some research to prepare for my upcoming first delivery of AD Troubleshooting workshop. The agenda of content includes DCLocator and Netlogon content. Hence this post.

    For the real detail on dsgetdcname and details on the nltest /dsgetdc flags, please see http://msdn.microsoft.com/en-us/library/ms675983(VS.85).aspx.

    I made a lab environment which consisted of a single domain forest and 1 x Vista and 1 x XP client. Here are my observations.

    If the Vista client’s IP address is from a subnet known by AD, a Windows Server 2008 DC will provide the closest site info details as well. It does not do this for XP. Presumably other down-level OS editions are also not given this detail but I didn’t check.

    Here is the LDAP filter one of the UDP based CLDAP ping performed by the Vista client.

    Filter: (&(DnsDomain=adatum.com)(Host=ADTMCLIENT1)(DomainGuid={AEF3F962-37E0-D147-8611-D8CA1AAED85E})(NtVer=16:00:00:00))

    And the response back from the Windows Server 2008 DC is as follows.

    Frame: Number = 86, Captured Frame Length = 200, MediaType = ETHERNET

    + Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-03-FF-0D-FA-00],SourceAddress:[00-03-FF-A5-9B-45]

    + Ipv4: Src = 10.1.1.2, Dest = 10.1.1.6, Next Protocol = UDP, Packet ID = 3904, Total IP Length = 186

    + Udp: SrcPort = LDAP(389), DstPort = 58266, Length = 166

    + Cldap: (CLDAP)Search Result Entry, MessageID: 1, Status: Success

    - NetlogonAttribute: LogonSAMLogonResponseEX (SAM Response to SAM logon request): 23 (0x17)

    - SamLogonResponseEx: ADTMDC1.adatum.com

    Opcode: LogonSAMLogonResponseEX

    Sbz: 0 (0x0)

    + Flags: 0x000013FD

    DomainGuid: {62F9F3AE-E037-47D1-8611-D8CA1AAED85E}

    DnsForestName: adatum.com

    DnsDomainName: adatum.com

    DnsHostName: ADTMDC1.adatum.com

    NetbiosDomainName: ADATUM

    NetbiosComputerName: ADTMDC1

    UserName:

    DcSiteName: HQ

    ClientSiteName: HQ

    Unknown: Binary Large Object (7 Bytes)

    + Version: 0x00000015 NT Version 5 Client

    + LmNtToken: Windows NT Networking: 0xFFFF

    + Lm20Token: OS/2 LAN Manager 2.0 (or later) Networking: 0xFFFF

    The italic Unknown: Binary Large Object (7 Bytes) has hex details corresponding to the following.

    05 53 69 74 65 32 00

    .Site2.

    Here is the site topology created using the ADTD Visio tool.

    AD Sites

    Currently Netmon 3.2 with latest parsers decodes as above. Wireshark as of v1.0.5 did not decode this.

    Please note that the above “Unknown: Binary Large Object (7 Bytes)” field is NOT available if the DC cannot see a subnet that the Vista machine belongs to. Netlogon.log on the client shows that the client did not use any try_next_closest_site flags. Yet the DC still presents the information. It can do this as the filter has a NtVer=16:00:00:00 string identifying the Client OS. XP for example would use NtVer=06:00:00:00. Hence a Windows Server 2008 DC will not respond with closest site info to them.

  • The case of the blocked FRS replication

    So I haven't written anything in quite a while and felt compelled to write something I recently ran into. A customer of mine is in the process of implementing Windows Server 2008 RODC in their branch offices. In order to do that, they are planning to in-place upgrade the W2K3 DCs to W2K8 at the hub sites.

    The customer did an in-place upgrade of their W2K3 DC in their lab and found that while the upgrade worked, the DC didn't appear to be completely functional. Specifically FRS replication appeared to be broken. Event 13508 was logged by the W2K8 DC reporting that it couldn't replicate. Similarly W2K3 DCs reported the W2K8 to be unavailable. The customer even turned the W2K8 firewall off to no avail. I asked for FRSDiag content from the customer and the results I got back were partial. There was no connstat.txt or any other content from ntfrsutl. Instead I had RPC errors scattered through the FRSDiag data.

    The customer in question has a very large global AD implementation and site to site communication is secured with firewalls in place. To ensure AD and FRS replication works through firewalls, the customer has configured static ports for AD and FRS replication. Firewall rules are also configured to ensure these ports are open and the replication traffic between DCs are allowed. http://support.microsoft.com/kb/224196/ and http://support.microsoft.com/kb/319553/ have details of how to lock these ports down.

    Analysis of the FRSDiag data showed that FRS was trying to listen on port TCP 49153. This had been enforced through Group Policies. However, it couldn't as the port was in use. In this case, we used "portqry -n <server> -e 135" to query the RPC endpoint mapper. This showed that the Event Log TCPIP was using the port. In W2K8 and Vista we made change to the ephemeral port range to start from 49152 (instead of 1025) and end at 65535 (instead of 5000). So the event log in W2K8 starts early on and grabs this port (49153). In W2K3 it would have grabbed something early in the 1025-5000 range. So the customer didn't see an issue until the upgrade to W2K8.

    Changing FRS to use a dynamic port (by deleting the reg key that enforced a static port) and restarting FRS fixed the issue temporarily. So we knew we had the cause figured out. The problem was the customer did not want to have to go and update firewall rules to allow the in-place upgrade of a W2K3 DC to W2K8 work. So he wanted to know whether we could have the Event Log use another port. From the research I've done, it does not appear to be possible to do this. However, in W2K8 you can move the ephemeral port range to start from another value instead of the default 49152. So I pointed the customer to the fine article at Ned's Askds blog (http://blogs.technet.com/askds/archive/2007/08/24/dynamic-client-ports-in-windows-server-2008-and-windows-vista-or-how-i-learned-to-stop-worrying-and-love-the-iana.aspx) and advised him to move the range to start from 49160 ( which freed 49152-49159 for customer requirements) and test!

    FRSDiag is such a great tool. For more good details on its usage, head on to Ned's blog for some quality articles such as How to get the most from your FRSDiag…and tips.

  • Broadcom IPV4 Large Send Offload

    I have run into a couple of scenarios where this setting has caused issues and hence decided to blog about it.

    1. Windows 2008 X64 based domain controllers didn’t replicate with each other. However they were pulling updates inbound from other DCs (that happened to be Windows 2000 Server based) fine. Running commands such as repadmin /bind <W2k8DCname>  and /replsum switches indicated replication failed with access denied. Additionally terminal server sessions –(remote admin mode) were dropped repeatedly within milliseconds of establishing a session. It turned out that the newly built Windows 2008 based DCs had Broadcom cards in them and the driver installer enabled the above setting. Once this was turned off, everything started working perfectly.
    2. Was at a customer recently that claimed when they made their Windows 2003 X64 DC in a single domain forest a GC, after a while the server was unresponsive to certain RPC traffic. once the server was removed from been a GC and rebooted, it would not cause issues. I didn’t believe them at first but then they demonstrated by configuring the DC to a GC role. Sure enough a few hours later the server was having issues. repadmin /bind <dcname> traffic once captured over netmon showed resets immediately. dir \\dcname\c$ and other SMB share access access such as SYSVOL and same DC worked. LDAP and GC traffic was perfect. However repadmin based commands like /bind, /replsum did not work. portqry commands to port 135 failed (unfortunately I don’t have error details at the moment). Once the above setting was turned off on the NIC and server was rebooted, no further recurrences were noted.

    So in case you run into issues where “RPC traffic is not responded to properly”, check and disable the above setting if configured on your server NIC. Please post a comment if you run into similar issues resolved by this setting.

    Thanks

    M

  • Recovering "Lost" ADAM Admin Access

    Consider the following scenario. You install ADAM/LDS as a particular user account. Lets assume its a normal domain user account with local administrative rights on a member server. So the install proceeds normally and you create the ADAM instance. Assume you use the same account to create other replicas of the configuration set.

    Now assume you delete the user account used for the install some time later after you've populated the ADAM instance. Lets say the account is deleted and garbage collected too :) The point is we don't have access to that account (SID to be precise) and the password that goes along with it.

    When an ADAM/LDS is installed using a certain account, that account is added to the ADAM/LDS configuration set's Administrators role. However, if no other Administrators are defined and the ADAM administrator account credentials are not available/usable, then you may find yourself with an ADAM install you "believe" you don't have full control for.

    But fret not for there is a way to recover from this scenario. The procedure is documented by Lee Flight at

    http://groups.google.co.uk/group/microsoft.public.active.directory.interfaces/browse_thread/thread/ade9cf248b0804b0

    Essentially we end up following the following steps.

    1. Open a command prompt or logon as an account with "Take ownership of files and other objects" right. I.e. by default local administrators of the server with the ADAM instance will do.
    2. Take ownership of the #ConfigurationNamingContext NC itself.
    3. Write an inheritable ACE at the config NC root which grants full control (GA) to a user/group of your choice. This could be your own credentials. We will be removing this ACE soon.
    4. Modify the CN=Administrators,CN=Roles,CN=Configuration,CN=<GUID> group and add the user/group from the previous step as a member. Else make whatever changes to the membership as you see fit.
    5. Remove the inheritable ACE configured at step3.

    From my experience I prefer to use adsiedit.msc to modify the membership of the groups. This is because it creates the Foreign Security Principal Objects (FSP) for me. If I use ldp or other tools I need to have the FSP ready.

    I also prefer the ldp.exe in ADAM for doing granular permissions. However, configuring permissions like GA is much easier using dsacls. Please remember to type the path as \\server:port\CN=Configuration,CN={GUID} instead of simply the DN without the \\server:port part.

    I've seen instances where customers that lost the credentials in someway or another have resorted to migrating from one ADAM instance to another or even sometimes rebuild. When the solution really is so easy.

    I personally find the above example as a good way of explaining how local admin rights (or more precisely certain security rights) can be "exploited" on a server. This is why one must be careful in granting local administrative rights to a user/service/group.

    HTH

    M