Blog - Title

July, 2013

  • DFS Replication in Windows Server 2012 R2 and other goodies, now available on the Filecab blog!

    Over at the Filecab blog, AskDS alum and all-around nice guy Ned Pyle has posted the first of several blogs about new features coming your way in Windows Server 2012 R2.  If you're a DFS administrator or just curious, go take a look!

    Ned promises more posts in the near future, and Filecab is near and dear to our hearts here in DS (they make a bunch of things we support), so if you don't already have it on your RSS feed list, it might be a good time to add it.

  • Roaming Profile Compatibility - The Windows 7 to Windows 8 Challenge

    [Editor's note:  Everything Mark mentions for Windows 8 clients here is also true for Windows 8.1 clients.  Windows 8 and Windows 8.1 clients use the same (v3) profile version, so the 8.1 upgrade will not prevent this from happening if you have roaming profiles in your environment.  Something to be aware of if you're planning to migrate users over to the new OS version. -David]

     

    Hi. It’s Mark Renoden, Senior Premier Field Engineer in Sydney, Australia here again. Today I’ll offer a workaround for an issue that’s causing a number of customers around the world a degree of trouble. It turns out to be reasonably easy to fix, perhaps just not so obvious.

    The Problem

    The knowledge base article "Unpredictable behavior if you migrate a roaming user profile from Windows 8 to Windows 7" - http://support.microsoft.com/kb/2748329 states:

    Windows 7 and Windows 8 use similar user profile formats, which do not support interoperability when they roam between computers that are running different versions of Windows. When a user who has a  windows 7 profile signs in to a Windows 8-based computer for the first time, the user profile is updated to the new Windows 8 format. After this occurs, the user profile is no longer compatible with Windows 7-based computers. See the "More information" section for detailed information about how this issue affects roaming and mandatory profiles.

    This sort of problem existed between Windows XP and Windows Vista/7 but was mitigated by Windows Vista/7 using a profile that used a .v2 extension.  The OS would handle storing the separate profiles automatically for you when roaming between those OS versions.  With Windows 7 and Windows 8, both operating systems use roaming profiles with a .v2 extension, even though Windows 8 is actually writing the profile in a newer format.

    Mark’s Workaround

    The solution is to use separate roaming profiles for each operating system by utilizing an environment variable in the profile path.

    Configuration

    File server for profiles:

    1. Create profile share “\\Server\ProfilesShare” with permissions configured so that users have write access
    2. In ProfilesShare, create folders “Win7” and “Win8”


     

    Active Directory:

    1. Create OU for Windows 7 Clients (say “Win7OU”) and create/link a GPO here (say “Win7GPO”)
    2. Create OU for Windows 8 Clients (say “Win8OU”) and create/link a GPO here (say “Win8GPO”)

    Note:As an alternative to separate OUs, a WMI filter may be used to filter according to Operating System:

    Windows 7 - SELECT version FROM Win32_OperatingSystem WHERE Version LIKE “6.1%” and ProductType = “1″

    Windows 8 - SELECT version FROM Win32_OperatingSystem WHERE Version LIKE “6.2%” and ProductType = “1″

    3. Edit Win7GPO

      1. Expand Computer Configuration -> Preferences -> Windows Settings
      2. Under Environment create an environment variable with
        1. Action: Create
        2. System Variable
        3. Name: OSVer
        4. Value: Win7

    4. Edit Win8GPO

      1. Expand Computer Configuration -> Preferences -> Windows Settings
      2. Under Environment create an environment variable with
        1. Action: Create
        2. System Variable
        3. Name: OSVer
        4. Value: Win8

    5. Set user profile paths to \\Server\ProfilesShare\%OSVer%\%username%\

     

    Clients:

    1. Log on with administrative accounts first to confirm creation of the OSVer environment variable

    2. Log in as users and you’ll observe that different user profiles are created in the appropriate folder in the profiles share depending on client OS

    Conclusion

    I haven't run into any issues in testing but this might be one of those cases where it's important to use "wait for network". My testing suggests that using "create" as the action on the environment variable mitigates any timing issues.  This is because after the environment variable is created for the machine, this variable persists across boots and doesn't depend on GPP re-application.

    You may also wish to consider the use (and testing) of a folder redirection policy to provide users with their data as they cross between Windows 7 and Windows 8 clients. While I have tested this to work with
    “My Documents”, there may be varying degrees of success here depending on how Windows 8’s modern apps fiddle with things.

     - Mark “Square Peg in a Round Hole” Renoden

     

     

  • Because TechNet didn't have enough Active Directory awesomeness already

    Time for a quick lesson in blog history.  There'll be a quiz at the end!  Ok not really, but some history all the same.

    Back a few years ago when we here at Microsoft were just starting to get savvy to this whole blog thing, one of our support escalation engineers, Tim Springston, decided to start up a blog about Active Directory.  You might have seen it in the past.  Over the years he's posted some really great insights and posts there that are definitely worth reading if you have the time.

    Of course, the rest of us decided to do something completely different and started up AskDS a little later.  Rumor has it that it had something to do with a high-stakes poker game (Tim *is* from Texas, after all), but no one is really sure why we wound up with two support blogs to be honest - it's one of those things that just sort of happened.

    Anyway, all this time while we've been partying it up over here on TechNet, our AD product team has been marooned over on MSDN with an audience of mostly developers.  Not that developers are bad folks - after all, they make the apps that power pretty much everything - but the truth is that a lot of what we do in Active Directory in terms of feature development is also targeted at Administrators and Architects and IT Pros.  You know, the people who read blogs on TechNet and may not think to also check MSDN.

    After a lot of debate and discussion internally, the AD product team came to the conclusion that they really should have a presence on TechNet so that they could talk to everyone here about the cool features they're working on.

    The problem?  Well, we sort of had a monopoly over here in support on AD-related blog names. :)

    Meetings were convened.  Conferences were held.  Email flew back and forth.  Their might even have been some shady dealings involving gifts of sugary pastries.  In the end though, Tim graciously agreed to move his blogging efforts over to AskDS and cede control of http://blogs.technet.com/ad to the Active Directory Product team.

    The result?  Everyone wins.  Tim's now helping us write cool stuff for AskDS (you'll see plenty of that in the near future, I'm sure), and the product team has already started posting a bunch of things that you might have missed when they were on MSDN.

    If you haven't seen what they're up to over there, go and take a look .  And as we get out of summer and get our people back from vacation, and, you know, roll a whole new server OS out the door, keep an eye on both blogs for updates, tips, explanations, and all manner of yummy AD-related goodness.

     

    --David "Wait, we get another writer for AskDS??" Beach

  • Interesting findings on SETSPN -x -f

    Hello folks, this is Herbert from the Directory Services support team in Europe!

    Kerberos is becoming increasingly mandatory for really cool features such as Protocol Transition.  Moreover, as you might be painfully aware, managing Service Principal Names (SPN’s) for the use of Kerberos by applications can be daunting at times.

    In this blog, we will not be going into the gory details of SPNs and how applications are using them. In fact, I’m assuming you already have some basic knowledge about SPN’s and how they are used.

    Instead, we’re going to talk about an interesting behavior that can occur when an administrator is doing their due diligence managing SPN’s.  This behavior can arise when you are checking the status of the account the SPN is planned for, or when you are checking to see if the SPN that must be registered is already registered in the domain or forest.

    As we all know, the KDC’s cannot issue tickets for a particular service if there are duplicate SPN’s, and authentication does not work if the SPN is on the wrong account.

    Experienced administrators learn to use the SETSPN utility to validate SPNs when authentication problems occur.  In the Windows Server 2008 version of SETSPN, we provide several options useful to identifying duplicate SPNs:

    -      If you want to look for a duplicate of a particular SPN: SETSPN /q <SPN>

    -      If you want to search for any duplicate in the domain: SETSPN /x

    You can also use the “/f” option to extend the duplicate search to the whole Forest. Many Active Directory Admins use this as a proactive check of the forest for duplicate SPNs.

    So far, so good…

    The Problem

    Sometimes, you’ll get an error running SETSPN -x -f:

    c:\>SETSPN -X -F -P
    Checking forest DC=contoso,DC=com
    Operation will be performed forestwide, it might take a while.
    Ldap Error(0x55 -- Timeout): ldap_get_next_page_s

    “-P” just tells the tool not to clutter the output with progress indications, but you can see from that error message that we are not talking only about Kerberos anymore. There is a new problem.

     

    What are we seeing in the diagnostic data?

    In a network trace of the above you will see a query against the GC (port 3268) with no base DN and the filter (servicePrincipalName=*)”. SETSPN uses paged queries with a page size of 100 objects. In a large Active Directory environment this yields quite a number of pages.

    If you look closely at network capture data, you’ll often find that Domain Controller response times slowly increase towards the end of the query. If the command completes, you’ll sometimes see that the delay is longest on the last page returned. For example, when we reviewed data for a recent customer case, we noted:

    ”Customer also noticed that it usually hangs on record 84.”

     

    Troubleshooting LDAP performance and building custom queries calls for the use of the STATS Control. Here is how you use it in LDP.exe:

    Once connected to port 3268 and logged on as an admin, you can build the query in the same manner as SETSPN does.

    1. Launch LDP as an administrator.

    2. Open the Search Window using Browse\Search or Ctrl-S.

    3. Enter the empty base DN and the filter, specify “Subtree” as the scope. The list of attributes does not matter here. 

    4. Go to Options:

     

    5. Specify an “Extended” query as we want to use controls. Note I have specified a page size of 100 elements, but that is not important, as we will see later. Let’s move on to “Controls”:

     


    5. From the List of Controls select “Search Stats“. When you select it, it automatically checks it in.

    6. Now “OK” your way out of the “Controls” and “Options” dialogs.

    7. Hit “Run” on the “Search” dialog.

     

    You should get a large list of results, but also the STATS a bit like this one:

     

    Call Time: 62198 (ms)

    Entries Returned: 8508

    Entries Visited: 43076

    Used Filter: (servicePrincipalName=*)

    Used Indices: idx_servicePrincipalName:13561:N

    Pages Referenced          : 801521

    Pages Read From Disk      : 259

    Pages Pre-read From Disk  : 1578

    Pages Dirtied             : 0

    Pages Re-Dirtied          : 0

    Log Records Generated     : 0

    Log Record Bytes Generated: 0

     

    What are these stats telling us?

    We have a total of 8508 objects in the “Entries Returned” result set, but we have visited 43076 objects. That sounds odd, because we used an Index idx_servicePrincipalName”. This does not really look as if the query is using the index.

     

    So what is happening here?

    At this point, we experience the special behavior of multi-valued non-linked attributes and how they are represented in the index. To illustrate this, let me explain a few data points:

     

    1. A typical workstation or member server has these SPNs:

    servicePrincipalName:
    WSMAN/herbertm5

    servicePrincipalName:
    WSMAN/herbertm5.europe.contoso.com

    servicePrincipalName:
    TERMSRV/herbertm5.europe.contoso.com

    servicePrincipalName:
    TERMSRV/HERBERTM5

    servicePrincipalName:
    RestrictedKrbHost/HERBERTM5

    servicePrincipalName:
    HOST/HERBERTM5

    servicePrincipalName:
    RestrictedKrbHost/HERBERTM5.europe.contoso.com

    servicePrincipalName:
    HOST/HERBERTM5.europe.contoso.com

     

    2. When you look at the result set from running setspn, you notice that you’re not getting all of the SPNs you’d expect:

    dn:CN=HQSCCM2K3TEST,OU=SCCM,OU=Test Infrastructure,OU=Domain Management,DC=contoso,DC=com
    servicePrincipalName: WSMAN/sccm2k3test
    servicePrincipalName: WSMAN/sccm2k3test.contoso.com

    If you look at it closely, you notice all the SPN’s start with characters very much at the end of the alphabet, which also happens to be the end of the index. These entries do not have a prefix like “HOST”.

     

    So how does this happen?

    In the resultant set of LDAP queries, an object may only appear once, but it is possible for an object to be in the index multiple times, because of the way the index is built. Each time the object is found in the index, the LDAP Server has to check the other values of the indexed attribute of the object to see whether it also matches the filter and thus was already added to the result set.  The LDAP server is doing its diligence to avoid returning duplicates.

    For example, the first hit in the index for the above workstation example is HOST/HERBERTM5“.

    The second hitHOST/HERBERTM5.europe.contoso.com kicks off the algorithm.

    The object is read already and the IO and CPU hit has happened.

    Now the query keeps walking the index, and once it arrives at the prefix WSMAN”, the rate of objects it needs to skip approaches 100%. Therefore, it looks at many objects and little additional objects in the result set.

    On the last page of the query, things get even worse. There is an almost 100% rate of duplicates, so the clock of 60 seconds SETSPN allows for the query is ticking, and there are only 8 objects to be found. If the Domain Controller has a slow CPU or the objects need to be read from the disk because of memory pressure, the SETSPN query will probably not finish within a minute for a large forest.  This results in the error Ldap Error(0x55 -- Timeout): ldap_get_next_page_sThe larger the index (meaning, the more computers and users you have in your forest), the greater the likelihood that this can occur.

    If you run the query with LDIFDE, LDP or ADFIND you will have a better chance the query will be successful. This is because by default these tools do not specify a time-out and thus use the values of the Domain Controller LDAP Policy. The Domain Controller LDAP policy is 120 seconds (by default) instead of 60 seconds.

    The problem with the results generated by these tools is that you have to correlate the results from the different outputs yourself – the tools won’t do it for you.

     

    So what can you do about it?

    Typically you’ll have to do further troubleshooting, but here are some common causes/resolutions that I’ve seen:

    1. If a domain controller is short on memory and encounters many cache misses and thus substantial IO. You can diagnose this using the NTDS performance counters in Performance Monitor.  You can add memory to reduce the IO rate and speed things up.
    2. If you are not experiencing memory pressure, the limiting factor could be the “Single-Thread-Performance” of the server. This is important as every LDAP query gets a worker thread and runs no faster than one logical CPU core can manage.  If you have a low number of logical cores in a system with a high amount of CPU activity, this can cause the threads to delay long enough for us to see an  nconsistent query return.  In this situation your best bet is to look for ways to reduce overall processor load on the domain controller – for example, moving other services off of the machine.
    3. There is an update for Windows Server 2012 which helps to avoid the problem:

    2799960          Time-out error when you run SETSPN.exe in Windows 8 or Windows Server 2012

    http://support.microsoft.com/kb/2799960/EN-US

    The last customer I helped had a combination of issues 1 and 2 and once he chose a beefier DC with more current hardware, the command always succeeded.  Another customer had a much bigger environment and ended up using the update I listed above to overcome the issue.

    I hope you have enjoyed this journey explaining what is happening on such a SETSPN query.

    Cheers,

    Herbert "The Thread Master" Mauerer