Blog - Title

May, 2011

  • Friday Mail Sack: Tuesday To You Edition

    Hi folks, Ned here again. It’s a long weekend here in the United States, so today I talk to you tell myself about a domain join issue one can only see in Win7/R2 or later, what USMT hard link migrations really do, how to poke LDAP in legacy PowerShell, time zone migration, and an emerging issue for which we need your feedback.

    Question

    None of our Windows Server 2008 R2 or Windows 7 computers can join the domain – they all show error:

    “The following error occurred attempting to join the domain "contoso.com": The service cannot be started, either because it is disabled or because it has no enabled devices associated with it.”

    image

    Windows Vista, Widows Server 2008, and older operating systems join without issue in the exact same domain while using the same user credentials.

    Answer

    Not a very actionable error – which service do you mean, Windows!? If you look at the System event log there are no errors or mention of broken services. Fortunately, any domain join operations are logged in another spot – %systemroot%\debug\netsetup.log. If you crack open that log and look for references to “service” you find:

    05/27/2011 16:00:39:403 Calling NetpQueryService to get Netlogon service state.
    05/27/2011 16:00:39:403 NetpJoinDomainLocal: NetpQueryService returned: 0x0.
    05/27/2011 16:00:39:434 NetpSetLsaPrimaryDomain: for 'CONTOSO' status: 0x0
    05/27/2011 16:00:39:434 NetpJoinDomainLocal: status of setting LSA pri. domain: 0x0
    05/27/2011 16:00:39:434 NetpManageLocalGroupsForJoin: Adding groups for new domain, removing groups from old domain, if any.
    05/27/2011 16:00:39:434 NetpManageLocalGroups: Populating list of account SIDs.
    05/27/2011 16:00:39:465 NetpManageLocalGroupsForJoin: status of modifying groups related to domain 'CONTOSO' to local groups: 0x0
    05/27/2011 16:00:39:465 NetpManageLocalGroupsForJoin: INFO: No old domain groups to process.
    05/27/2011 16:00:39:465 NetpJoinDomainLocal: Status of managing local groups: 0x0
    05/27/2011 16:00:39:637 NetpJoinDomainLocal: status of setting ComputerNamePhysicalDnsDomain to 'contoso.com': 0x0
    05/27/2011 16:00:39:637 NetpJoinDomainLocal: Controlling services and setting service start type.
    05/27/2011 16:00:39:637 NetpControlServices: start service 'NETLOGON' failed: 0x422
    05/27/2011 16:00:39:637 NetpJoinDomainLocal: initiating a rollback due to earlier errors

    Aha – the Netlogon service. Without that service running, you cannot join a domain. What’s 0x422?

    c:\>err.exe 0x422

    ERROR_SERVICE_DISABLED winerror.h
    # The service cannot be started, either because it is
    # disabled or because it has no enabled devices associated
    # with it.

    Nice, that’s our guy. It appears that the service was disabled and the join process is trying to start it. And it almost worked too – if you run services.msc, it will say that Netlogon is set to “Automatic” (and if you look at another machine you have not yet tried to join, it is set to “Disabled” instead of the default “Manual”). The problem here is that the join code is only setting the start state through direct registry edits instead of using Service Control Manager. This is necessary in Win7/R2 because we now always go through the offline domain join code (even when online) and for reasons that I can’t explain without showing you our source code, we can’t talk to SCM while we’re in the boot path or we can have hung startups. So the offline code set the start type correctly and the next boot up would have joined successfully – but since the service is still disabled according to SCM, you cannot start it. It’s one of those “it hurts if I do this” type issues.

    And why did the older operating systems work? They don’t support offline domain join and are allowed to talk to the Service Control Manager whenever they like. So they tell him to set the Netlogon service start type, then tell him to start the service – and he does.

    The lesson here is that a service set to Manual by default should not be set to disabled without a good reason. It’s not like it’s going to accidentally start in either case, nor will anyone without permissions be able to start it. You are just putting a second lock on the bank vault. It’s already safe enough.

    Question

    USMT is always going on about hard link migrations. I’ve used them and those migrations are fast… but what the heck is it and why do I care?

    Answer

    A hard link is simply a way for NTFS to point to the same file from multiple spots, always on the same volume. It has nothing to do with USMT (who is just a customer). Instead of making many copies of a file, you are making copies of how you get to the file. The file itself only exists once. Any changes to the file through one path or another are always reflected on the same physical file on the disk. This means that when USMT is storing a hard link “copy” of a file it is just telling NTFS to make another pointer to the same file data and is not copying anything – which makes it wicked fast.

    Let’s say I have a file like so:

    c:\hithere\bwaamp.txt

    If I open it up I see:

    image

    Really though, it’s NTFS pointing to some file data with some metadata that tells you the name and path. Now I will use FSUTIL.EXE to create a hard link:

    C:\>fsutil.exe hardlink create c:\someotherplace\bwaamp.txt c:\hithere\bwaamp.txt
    Hardlink created for c:\someotherplace\bwaamp.txt <<===>> c:\hithere\bwaamp.txt

    I can use that other path to open the same data (it helps if you don’t think of these as files):

    image

    I can even create a hard link where the file name is not the same (remember – we’re pointing to file data and giving the user some friendly metadata):

    C:\>fsutil.exe hardlink create c:\yayntfs\sneaky!.txt c:\hithere\bwaamp.txt
    Hardlink created for c:\yayntfs\sneaky!.txt <<===>> c:\hithere\bwaamp.txt

    And it still goes to the same spot.

    image

    What if I edit this new "”sneaky!.txt” file and then open the original “bwaamp.txt”?

    image

    Perhaps a terrible Visio diagram will help:

    hardlink

    When you delete one of these representations of the file, you are actually deleting the hard link. When the last one is deleted, you are deleting the actual file data.

    It’s magic, smoke and mirrors, hoodoo. If you want a more disk-oriented (aka: yaaaaaaawwwwnnn) explanation, check out this article. Rob and Joseph have never met a File Record Segment Header they didn’t like. I bet they are a real hit at parties…

    Question

    How can I use PowerShell to detect if a specific DC is reachable via LDAP? Don’t say AD PowerShell, this environment doesn’t have Windows 7 or 2008 R2 yet! :-)

    Answer

    One way is going straight to .NET and use the DirectoryServices namespace:

    New-Object System.DirectoryServices.DirectoryEntry(LDAP://yourdc:389/dc=yourdomaindn)

    For example:

    image
    Yay!

    image
    Boo!

    Returning anything but success is a problem you can then evaluate.

    As always, I welcome more in the Comments. I suspect people have a variety of techniques (third parties, WMI LDAP provider, and so on).

    Question

    Is USMT supposed to migrate the current time zone selection?

    Answer

    Nope. Whenever you use timedate.cpl, you are updating this registry key:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation

    Windows XP has very different data in that key when compared to Vista and Windows 7:

    Windows XP

     

    [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation]

    "ActiveTimeBias"=dword:000000f0

    "Bias"=dword:0000012c

    "DaylightBias"=dword:ffffffc4

    "DaylightName"="Eastern Daylight Time"

    "DaylightStart"=hex:00,00,03,00,02,00,02,00,00,00,00,00,00,00,00,00

     

    "StandardBias"=dword:00000000

    "StandardName"="Eastern Standard Time"

    "StandardStart"=hex:00,00,0b,00,01,00,02,00,00,00,00,00,00,00,00,00

     

    Windows 7

     

    [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation]

    "ActiveTimeBias"=dword:000000f0

    "Bias"=dword:0000012c

    "DaylightBias"=dword:ffffffc4

    "DaylightName"="@tzres.dll,-111"

    "DaylightStart"=hex:00,00,03,00,02,00,02,00,00,00,00,00,00,00,00,00

    "DynamicDaylightTimeDisabled"=dword:00000000

    "StandardBias"=dword:00000000

    "StandardName"="@tzres.dll,-112"

    "StandardStart"=hex:00,00,0b,00,01,00,02,00,00,00,00,00,00,00,00,00

    "TimeZoneKeyName"="Eastern Standard Time"

    The developers from the Time team simply didn’t want USMT to assume anything as they knew there were significant version differences; to do so would have taken an expensive USMT plugin DLL for a task that would likely be redundant to most customer imaging techniques. There are manifests (such as "INTERNATIONAL-TIMEZONES-DL.MAN") that migrate any additional custom time zones to the up-level computers, but again, this does not include the currently specified time zone. Not even when migrating from Win7 to Win7.

    But that doesn’t mean that you are out of luck. Come on, this is me! :-)

    To migrate the current zone setting from XP to any OS you have the following options:

    To migrate the current zone setting from Vista to Vista, Vista to 7, or 7 to 7, you have the following options:

    • Any of the three mentioned above for XP
    • Use this sample USMT custom XML (making sure that nothing else has changed since this blog post and you reading it). Woo, with fancy OS detection code!

    <?xml version="1.0" encoding="utf-8" ?>
    <
    migration urlid="http://www.microsoft.com/migration/1.0/migxmlext/currenttimezonesample">
    <
    component type="Application" context="System">
    <
    displayName>Copy the currently selected timezone as long as Vista or later OS</displayName>
    <
    role role="Settings">
    <!--
    Check as this is only valid for uplevel-level OS >= than Windows Vista –>
    <
    detects>
      <
    detect>
       <
    condition>MigXmlHelper.IsOSLaterThan("NT", "6.0.0.0")</condition>
      </
    detect>
    </
    detects>
    <
    rules>
    <
    include>
      <
    objectSet>
       <
    pattern type="Registry">HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation\* [*]</pattern>
      </
    objectSet>
    </
    include>
    </
    rules>
    </
    role>
    </
    component>
    </
    migration>

    Question for our readers

    We’ve had a number of cases come in this week with the logon failure:

    Logon Process Initialization Error
    Interactive logon process initialization has failed.
    Please consult the Event Logs for more details.

    You may also find an application event if you connect remotely to the computer (interactive logon is impossible at this point):

    ID: 4005
    Source: Microsoft-Windows-Winlogon
    Version: 6.0
    Message: The Windows logon process has unexpectedly terminated.

    In the cases we’ve seen this week, the problem was seen after restoring a backup when using a specific third party backup product. The backup was restored to either Hyper-V or VMware guests (but this may be coincidental). After the restore large portions of the registry were missing and most of our recovery tools (SFC, Recovery Console, diskpart, etc.) would not function. If you have seen this, please email us with the backup product and version you are using. We need to contact this vendor and get this fixed, and your evidence will help. I can’t mention the suspected company name here yet, as if we’re wrong I’d be creating a legal firestorm, but if all the private emails say the same company we’ll have enough justification for them to examine this problem and fix it.

    ------------

    Have a safe weekend, and take a moment to think of what Memorial Day really means besides grilling, racing, and a day off.

    Ned “I bet SGrinker has the bratwurst hookup” Pyle

  • Viewing ADLDS traffic with Netmon – where is my LDAP?

    Hi, its Linda Taylor here from the UK Directory Services Team! I have decided to make a return to the blog to show you a nice tip on how make Network traffic from ADLDS (Active Directory Lightweight Directory Services) look more readable…or in other words - to enable Netmon to parse it as LDAP.

    Note: Throughout this post I will refer to ADLDS but everything also applies to ADAM.

    Since I haven’t seen many customers run ADLDS on port 389 I can imagine that this will be useful to many. I will use port 50000 in my example, but you can do this for any port or even a number of different ports if you have different instances running on different ports.

    By default, Network Monitor will only parse traffic on port 389 as LDAP so for ADLDS we can edit the parser and add our desired port(s).

    Here is how to do that:

    1. Go to C:\Programdata\Microsoft\Network Monitor 3\NPL\Network Monitor Parsers\Base and open the properties of TCP.NPL.

    Uncheck the “read only” box.

    Note: ProgramData is a hidden folder by default.

    image

    2. Now you can edit the parser. To do this from inside Netmon:

    • Select the parsers Tab
    • Expand “Parser Files” on the left hand side
    • Navigate to TCP.NPL and select it (the parser file will open on the right)
    • Search for “389” and you will see a piece of code like this:

    Case 389:

             LDAP Ldap;

    image

    3. Now add another case statement with the port your ADLDS uses (for example, 50000) like this:

    Case 389:

    Case 50000:

              LDAP Ldap;

    image

    4. Save the parser and Reload it.

    Now the ADLDS traffic which previously showed as TCP will show up as LDAP and you can filter and look at it in the normal way.

    Note: Don’t forget to then go and check “read only” again on the TCP.NPL file.

    Finally, there is a more generic post on editing Netmon parsers here on the Netmon team blog:

    http://blogs.technet.com/b/netmon/archive/2006/10/04/npl-_1320_-the-power-behind-the-parsers.aspx

    Cheers!

    Linda “Blighty” Taylor, Escalation Engineer for Directory Services.

  • LINKS! USMT 4 forgot my LINKS!!

    Gary here, and in this post I am going to talk about USMT and the Links folder not migrating from Vista or Windows 7. This folder is new starting with Vista so it would not be an issue coming from XP to the higher versions of Windows.

    In case you are wondering what the Links folder is, it is the Favorite Links or Favorites for Explorer.

    image    image
    Windows Vista                                              Windows 7

    It also corresponds to the %userprofile%\Links folder with in the user’s profile.

    If you are using the migdocs.xml file, it uses the GenerateDocPatterns helper function to go collect most of the user profile folders. It is also worth noting that the miguser.xml file also has the same issue, but use of that file is discouraged regardless. The following table lists the profile folders gathered from each user’s profile directory as documented at “Migration XML Files” site for USMT.

     Variable

    Migdocs.xml

    Miguser.xml

    Description

    CSIDL_MYDOCUMENTS

    -or-

    CSIDL_PERSONAL

    X

    X

    In Windows Vista or Windows 7, the virtual folder representing the My Documents desktop item.

    CSIDL_MYPICTURES

    X

    X

    The file-system directory that serves as a common repository for image files. A typical path in Windows XP is C:\Documents and Settings\username\My Documents\My Pictures. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Pictures.

    FOLDERID_OriginalImages

    X

     

    The file-system directory that servers as a common repository for Windows Photo Gallery original images.  A typical path in Windows Vista or Windows 7 is C:\Users\Username\AppData\Local\Microsoft\Windows Photo Gallery\Original Images

    CSIDL_MYMUSIC

    X

    X

    The file-system directory that serves as a common repository for music files. A typical path in Windows XP is C:\Documents and Settings\User\My Documents\My Music. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Music.

    CSIDL_MYVIDEO

    X

    X

    The file-system directory that serves as a common repository for video files. A typical path in Windows XP is C:\Documents and Settings\username\My Documents\My Videos. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Videos.

    CSIDL_FAVORITES

    X

    X

    The file-system directory that serves as a common repository for the user's favorites. A typical path in Windows XP is C:\Documents and Settings\username\Favorites. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Favorites.

    CSIDL_DESKTOP

    X

    X

    The virtual folder representing the Windows desktop.

    CSIDL_QUICKLAUNCH

    X

    X

    The legacy Quick launch folder

    FOLDERID_Contacts

    X

     

    The file-system directory that serves as a common repository for the user's contacts. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Contacts.

    FOLDERID_Libraries

    X

     

    The file-system directory that serves as a common repository for the user's libraries. A typical path in Windows Vista or Windows 7 is C:\Users\Username\AppData\Roaming\Microsoft\Windows\Libraries.

    FOLDERID_Downloads

    X

     

    The file-system directory that serves as a common repository for the user's downloads. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Downloads.

    FOLDERID_SavedGames

    X

     

    The file-system directory that serves as a common repository for the user's saved games. A typical path in Windows Vista or Windows 7 is C:\Users\Username\Saved Games.

    CSIDL_STARTMENU

     

    X

    The file-system directory that contains Start menu items. A typical path in Windows XP is C:\Documents and Settings\username\Start Menu. A typical path in Windows Vista or Windows 7 is C:\Users\Username\AppData\Roaming\Microsoft\Windows\Start Menu.

    As you see, the list above does not include anything indicating the Link folder would be touched. To gather and restore the LINKS folder you can use the following sample. Please do not modify the migdocs.xml (or miguser.xml) file as you can easily include the additional XML file in the command line. In addition, it would prevent any potential conflicts if newer versions are released and overwrite your hard work.

    scanstate.exe c:\store /i:migapp.xml /i:migdocs.xml /i:miglinks.xml
    loadstate.exe c:\store /i:migapp.xml /i:migdocs.xml /i:miglinks.xml

    <?xml version="1.0" encoding="utf-8" ?>
    <!--
    Sample - migrates user Links (aka quicklaunch) folder from Vista/7 to Vista/7
    -->
    <!--
    This code is NOT necessary when going from XP to Vista/7
    -->
    <
    migration urlid="http://www.microsoft.com/migration/1.0/migxmlext/migratelinks"
    >
    <
    component type="Documents" context="User"
    >
    <
    displayName>Migrate Links Folder</displayName
    >
    <
    role role="Data"
    >
    <
    rules
    >
    <
    include
    >
      <
    objectSet
    >
       <
    pattern type="File">%CSIDL_PROFILE%\Links\* [*]</pattern
    >
      </
    objectSet
    >
    </
    include
    >
    </
    rules
    >
    </
    role
    >
    </
    component
    >
    </
    migration>

    With the use of the above information, you will be able to preserve the user’s Links folder from Vista/7 to Vista/7 migrations.

    Happy link migrations!

    Gary “look, a USMT post from someone besides Ned is possible” Mudgett

  • Certificate Authority disaster recovery steps when smartcard logon is required but no valid CRL can be published

    [Editor’s note: this is a reprinted post from the AD Troubleshooting Blog. If you’re not already a subscriber to that blog, you absolutely need to add it to your feed. Ingolfur is a Sr. Support Escalation Engineer in Sweden and a very smart dude - with rather odd hair - who deserves your attention. Make sure you keep this post in your back pocket; if you ever need it, you are going to need it in a hurry  – Ned]

    Consider the following disaster recovery scenario:

    • The CA has become temporarily unavailable, the current CRL and Delta CRL have expired and revocation checking is failing which is preventing smartcard logons.
    • You have the private/public key pair of the CA certificate available and want to quickly get a new valid CRL out for revocation checking to start succeeding again.

    For this scenario, as long as the private/public key pairs exist you can manually sign a CRL and publish it to get breathing room while you recover the original CA server installation. Even if it only exists in a PFX file and the original CA server is gone you should still be able to import the PFX file to another server and do the re-signing parts there - the key point is getting an updated valid CRL out that you can publish so that clients and domain controllers can locate CRL's so that CRL-checking will succeed again.

    Example: to sign a new CRL that is valid from the current time and 14 days into the future, you can run the following if the private key of the CA that signed the CRL exists locally:

    certutil -sign <old expired CRL file.crl> <new valid CRL file.crl> now+14:00 -2.5.29.46

    This will produce a new valid CRL file that you can then publish to the CDP locations that are defined on the issued certificates. The -2.5.29.46 option removes any existing Delta CRL from the new CRL so you don't have to worry about having to publish a new Delta CRL if any was present on the old CRL.

    How you publish the CRL depends on the CDP, for an HTTP CDP you would most likely need to manually copy the CRL file to the web server. For an LDAP CDP you should be able to use Certutil to publish the CRL.

    Example: to publish the CRL to the issuing SubCA object:

    certutil -dspublish <new valid CRL file.crl> SubCA

    This should publish the updated valid CRL to the issuing CA's object in Active Directory.

    Further details:
    http://blogs.technet.com/b/instan/archive/2008/12/08/requiring-smart-cards-for-logon-avoiding-the-outage-caused-by-expired-crl-s.aspx

    - Ingolfur Arnar Stangeland

  • Understanding USMT Printer Migration (or the lack thereof)

    Hi there folks, Ned here again. I've touched on USMT and printers before but it was a quick answer to a specific question in a Friday Mail Sack. Today I explain in detail how USMT migrates network-mapped printers and why it migrates nothing else. If you didn't understand this before, don’t worry… it's confusing as $&^%. :-)

    The way it all works

    We describe what USMT migrates here but we use some loose wording in this case: "Network printer mappings". Let's take the case of XP and what this means in practical USMT terms. There are only two USMT manifests for XP that mention printing:

    C:\usmt\X86\DLMANIFESTS>findstr /i /m "printer" *.man 

    PRINTING-SPOOLER-CORE-DL.MAN

    PRINTING-SPOOLER-NETWORKCLIENT-DL.MAN

    When you examine these files you find that "PRINTING-SPOOLER-CORE-DL.MAN" migrates a few general settings about your client's spooler, which has nothing to do with individual printers. However, the "PRINTING-SPOOLER-NETWORKCLIENT-DL.MAN" file is much more interesting:

    <rules context="User">
     <
    include
    >
      <
    objectSet
    >
       <!--

       DevModes2ListType.
       -->
       <
    pattern type="Registry">HKCU\Printers\DevModes2 [\\*]</pattern
    >
       <!--

       ListOfPrinterConnection.
       -->
       <
    pattern type="Registry">HKCU\Printers\Connections\* [*]</pattern
    >
      </
    objectSet
    >
     </
    include
    >
    </
    rules
    >

    If you look at a computer where you have a connected network printer, this will line up nicely:

    image

    image

    Oh yes, it's just that easy - whatever registry settings are here get copied. When your new Windows 7 computer connects to that print server, the driver will automatically download and you are back in business. The registry data for mapped network printers are just that - some pointers to a server who will do all the heavy lifting for you. Barely more than shortcuts, really.

    But what about those other printers - the ones connected via LPT, local Plug-and-Play, COM , TCP, and LPR ports?

    image

    image

    So you perform a test migration, restart the destination Windows 7 computer, and… eh. Only that network printer is migrated to the Devices and Printers applet.

    image

    Ah… but those were not the printer themselves - those DevModes2 entries are for printer preferences. That's why even the network printer is there: I had decided to print back to front, like a weirdo:

    image

    The actual printers for the Printers and Faxes applet are all stored - along with their driver, spooler, port, and other info - in a completely different registry key structure within:

    HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Print
    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Print\Printers
    HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Devices

    image

    image

    image

    And as you know now from examining that downlevel MAN file, USMT does not migrate those keys. And our practical repro reinforces this.

    What about Windows 7 though? Now armed with the other registry keys, we can search through our manifests for those entries and see what we find:

    C:\usmt\AMD64\REPLACEMENTMANIFESTS>findstr /m /i "SYSTEM\CurrentControlSet\Printers" *.man
    C:\usmt\AMD64\REPLACEMENTMANIFESTS>findstr /m /i "Control\Print\Printers" *.man
    C:\usmt\AMD64\REPLACEMENTMANIFESTS>findstr /m /i "NT\CurrentVersion\Devices" *.man

    Nothing there. How about the OS-included manifests?

    C:\Windows\winsxs\Manifests>findstr /m /i "Control\Print\Printers" *.manifest

    amd64_microsoft-windows-remoteregistry-service_31bf3856ad364e35_6.1.7600.16385_none_e55af7609d2857a8.manifest

     

    C:\Windows\winsxs\Manifests>findstr /m /i "NT\CurrentVersion\Devices" *.manifest


    amd64_microsoft-windows-printing-spooler-core_31bf3856ad364e35_6.1.7601.17514_none_3471a890d8284f57.manifest

    C:\Windows\winsxs\Manifests>findstr /m /i "CurrentVersion\Print" *.manifest


    amd64_microsoft-windows-fax-common_31bf3856ad364e35_6.1.7601.17514_none_6a2ab458674011dc.manifest

    amd64_microsoft-windows-p..-localprinting-home_31bf3856ad364e35_6.1.7600.16385_none_93a1ff364af0f5eb.manifest

    amd64_microsoft-windows-p..g-xpsdocumentwriter_31bf3856ad364e35_6.1.7601.17514_none_80fea45979a5d3f2.manifest

    amd64_microsoft-windows-p..ooler-core-localspl_31bf3856ad364e35_6.1.7601.17514_none_8e41636aa94da31c.manifest

    amd64_microsoft-windows-p..ooler-networkclient_31bf3856ad364e35_6.1.7601.17514_none_9799402887898e33.manifest

    amd64_microsoft-windows-printing-spooler-core_31bf3856ad364e35_6.1.7601.17514_none_3471a890d8284f57.manifest

    amd64_microsoft-windows-remoteregistry-service_31bf3856ad364e35_6.1.7600.16385_none_e55af7609d2857a8.manifest

    Ah, sweet data (slightly snipped up for readability - all we care about is the latest version of each file).

    The next step is to verify that these returned files care about USMT by looking for the string "USMT" that shows:

    <migration scope="Upgrade,MigWiz,USMT"

    It turns out the first one (*microsoft-windows-remoteregistry-service*.manifest) doesn't contain this so it’s irrelevant. The next one (*microsoft-windows-printing-spooler-core *) does, but it's only concerned about:

    <merge script="MigXmlHelper.SourcePriority()">
     
    <objectSet>
      <pattern type="Registry">HKLM\SYSTEM\CurrentControlSet\Control\Print\Providers [*]</pattern>
      <pattern type="Registry">HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Printers [DefaultSpoolDirectory]</pattern>
    </objectSet>
    </merge>

    Neither of those gets us all those other printers. Of the remaining ones, only these are for USMT:

    *microsoft-windows-p..ooler-core-localspl*
    *microsoft-windows-p..ooler-networkclient*
    *microsoft-windows-printing-spooler-core*

    The first one is for cluster printers (oh brother, how helpful that this is included for USMT), and the two others just reference what we've already seen so that Windows 7 can migrate those same network connected printers.

    Why? And now what do I do?

    This is all by design – the printer developers who wrote these manifests for USMT don’t automatically migrate all these varieties of local printers intentionally, because they can’t guarantee drivers and architecture will be the same between computers. The same way we don’t migrate video card drivers or other hardware. Connections to Windows Print servers are safe as the client is updated with a driver when connecting to the printer queue; even if the driver doesn’t exist now on the client, it soon will.

    As for other options: I'm not a printer guru, but we do have them. Our friends at the AskPerf blog spend much of their lives working on printer installation and migration issues. They tend to think that even network-mapped printers are a legacy of the past and that you should be doing things like Location Aware Printing, Active Directory Printing, and printer deployment using printui. If you talk to the Perf folks here through a support case, they will also bring up Group Policy based printer deployment or prnmgr.vbs. Between all these options, you can deploy network or local printers in automated fashions, regardless of USMT capabilities.

    A final note

    If you've already learned how to reverse engineer USMT behaviors no matter the component, you didn’t have to read this. You have transcended ordinary IT operations and now are a hot pepper-eating fire demon of awesomeness. 

    Until next time.

    Ned "Please think of the paper mill employees and certified tree farmers, and print this blog" Pyle

  • Friday Mail Sack: Goat Riding Bambino Edition

    Hi folks, Ned here again. I’m trying to get back into the swing of having a mail sack every week but they can be pretty time consuming to write (hey, all this wit comes at a price!) so I am experimenting with making them a little shorter. This week we talk AD PowerShell secrets, USMT and Profile scalability, a little ADUC and DFSR, and some other random awesomeness.

    Question

    Can you explain how the AD PowerShell cmdlet Get-ADComputer gets IP information? (ex: Get-ADComputer -filter * -Properties IPv4Address). Properties are always AD attributes, but I can not find that IPv4Address attribute on any computer object and even after I removed the A records from DNS I still get back the right IP address for each computer.

    Answer

    That’s an excellent question and you were on the right track. This is what AD PowerShell refers to as an ‘extendedAttribute’ internally, but what a human might call a ‘calculated value’. AD PowerShell special-cases a few useful object properties that don’t exist in AD by using other LDAP attributes that do exist, and then uses that known data to query for the rest. In this case, the dnsHostName attribute is looked up normally, then a DNS request is sent with that entry to get the IP address.

    Even if you removed the A record and restarted DNS, you could still be returning the DNS entry from your own cache. Make sure you flush DNS locally where you are running PowerShell or it will continue to “work”.

    To demonstrate, here I run this the first time:

    clip_image002

    Which queries DNS right after the powershell.exe contacts the DC for the other info (all that buried under SSL here, naturally):

    clip_image002[4]

    Then I run the identical command again – note that there is no DNS request or response this time as I’m using cached info.

    clip_image002[6]

    It still tells me the IP address. Now I delete the A record and restart the DNS service, then flush the DNS cache locally where I am running PowerShell, and run the same PowerShell command:

    clip_image002[8]

    Voila! I have broken it. :)

    Question

    Is there is a limit on the number of profiles that USMT 4.0 can migrate? 3.01 used to have problems with many (20+) profiles, regardless of their size.

    Answer

    Updated with new facts and fun, Sept 1, 2011

    Yes and no. There is no limit real limit, but depending on the quantity of profiles and their contents, combined with system resources on the destination computer, you can run into issues. If possible you should use hardlink migration, as that as fast as H… well, it’s really fast.

    To demonstrate (and to show erstwhile USMT admins a quick and dirty way to create some stress test profiles):

    1. I create 100 test users:

    image

    image

    2. I log them all on and create/load their profiles, using PSEXEC.EXE:

    image

    image

    3. Copy a few different files into each profile. I suggest using a tool that creates random files with random contents. In my case I added a half dozen 10MB files to each profile’s My Documents folder. You can’t use the same files in each profile, as USMT is smart enough to reuse them and you will not get the real user experience.

    4. I run the harshest, slowest possible migration I can, where USMT writes to a compressed store on a remote file share, with AES_256 encryption, from an x86 Windows 7 computer with only 768MB of RAM, while cranking all logging to the max:

    image

    This (amazingly, if you ever used USMT 3.01) takes only 15 minutes and completes without errors. Loadstate memory and CPU isn’t very stressful (in one test, I did this with an XP computer that had only 256MB of RAM, using 3DES encryption).

    5. I restore them all to another another computer – here’s the key: you need plenty of RAM on your destination Windows 7 computer. If you have 100 profiles that all have different contents, our experience shows that 4GB of RAM is required. Otherwise you can run out the OS resources and receive error “Close programs to prevent information loss. Your computer is low on memory. Save your files and close your programs: USMT: Loadstate”. More on this later.

    image

    This takes about 30 minutes and there are no issues as long as you have the RAM.

    image

    6. I bask in the turbulence of my magnificence.

    If you do run into memory issues (so far we’ve only seen it with one customer since USMT 4.0 released more than two years ago), you have a few options:

    a. Validate your scanstate/loadstate rules – despite what you may think, you might be gathering all profiles and not just fresh ones. Review: http://blogs.technet.com/b/askds/archive/2011/05/05/usmt-and-u-migrating-only-fresh-domain-profiles.aspx. Hopefully that cuts you down to way fewer than 100 per machine. Read that post carefully, there are some serious gotchas: such as once you run scanstate once on a computer, all profiles are made fresh afterwards for any subsequent scanstate runs. The odds that all 100+ profiles are actually active is pretty unlikely.

    b. Get rid of old XP profiles with DELPROF before using USMT at all. This is safer than UEL, as like I mentioned, once you run scanstate that’s it – it has to work perfectly on the first try, as all profiles are now “fresh”. (On Vista+ you instead use http://support.microsoft.com/kb/940017, as I’m sure you remember)

    c. Get more RAM.

    Question

    Is it possible in DSA.MSC to have the Find: Users, Contacts, and Groups default to finding computers or include computers with the user, contacts, and groups? Is there a better way to search for computers?

    Answer

    The Find tool does not provide for user customization – even starting it over without closing DSA.MSC loses your last setting. ADUC is a cruddy old tool, DSAC.EXE is the (much more flexible) replacement and it will do what you want for remembering settings.

    There are a few zillion other ways to find computers also. Not knowing what you are trying to do, I can’t recommend one over the other; but there’s DSQUERY.EXE, CSVDE.EXE, many excellent and free 3rd parties, etc.

    Question

    If I delete or disable the outbound connection from a writable DFSR replicated folder, I get warning that the “topology is not fully connected”. Which is good.

    image

    But if that outbound connection is for a read-only replica, no errors. Is this right?

    Answer

    It’s an oversight on our part. While technically nothing bad will happen in this case (as read-only servers - of course - do not replicate outbound), you should get this message in all cases (There are also 6020 and 6022 DFSR warning events you can use to track this condition). A read-only can be converted to a read-write, and you will definitely want an outbound connection for that.

    We’re looking into this; in the meantime, just don’t do it anywhere. :)

    Other Things

    Just to make myself feel better: “Little roller up along first. Behind the bag! It gets through Buckner!”

    • If you have parents, siblings, children away at college, nephews, cousins, grandparents, or friends, we have the newest weapon in the war on:
      1. Malware
      2. Your time monopolized as free tech support

    Yes, it’s the all new, all web Microsoft Safety Scanner. It even has a gigantic button, so you know it’s gotta be good. Make those noobs mash it and tell you if there are any problems while you go make a sandwich.

    • Finally: thank goodness my wife hasn’t caught this craze yet. She has never met a shoe she didn’t buy.

    Have a nice weekend folks.

    Ned “86 years between championships? That’s nothing… try 103, you big babies!” Pyle

  • Vista SP1 End of Support Reminder

    Hi folks, Ned here again with a quick public service announcement:

    Are you running Windows Vista Service Pack 1? SP1 support ends on July 12, 2011 - that's barely two months from now. Once that happens, computers running Vista SP1 do not get further security updates, MS Support will insist on SP2 installation before further troubleshooting, and you are generally in a world of hurt. Service Pack 2 has been out for two years; it's time to make it happen, IT pros.

    Obviously, an even better option is installing Service Pack 7.

    Chop chop!

    - Ned 'I still run NT 4.0 SP3 just to be a jerk" Pyle

  • Does USMT Migrate <This Goo>?

    Hi folks, Ned here again. Below are the two most common USMT questions I get asked these days:

    1. USMT is not migrating setting X for application Y. Is USMT broken or is this expected behavior?
    2. I am in the planning phase and have not yet tried, but will USMT migrate setting X for application Y?

    There are a million little knobs in Windows and all of its applications and it’s impossible to document them all with much specificity. Today I discuss how to determine this no matter what the registry setting or application. I also explain how USMT decides to migrate settings using component XML files. As long as you’re systematic in your investigation, you can’t go wrong. Perhaps this saves you a support case or delayed rollout someday.

    The Sample Scenario

    I’ll approximate a recent real-world example, where the customer is testing migration from Windows XP to Windows 7 and is having trouble with customized Internet Explorer Favorites. The Favorites migrate fine but the user’s customized sort order is not preserved. In my semi-fictionalized example, they are also migrating from IE7 to IE9 and from 32-bit to 64-bit, just to make things interesting.

    image
    The user’s custom IE7 favorites

    The key to solving any problem is a disciplined approach. You must remove distractions and carefully narrow down the genuine error conditions. Sir Arthur Conan Doyle said it best:

    “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.”

    Let’s walk through solving this USMT behavior case, and therefore, any USMT behavior case.

    Systematic Investigation

    Step 1 – Find the actual settings

    Naturally, you must discover where the settings really live. The easiest way is to let the Internet do the walking: search. Maybe someone has already documented this. Start with trustworthy spots like TechNet, MSDN, and the MS Knowledgebase, and then fan out to the woolier Internet.

    In this case, I’m not finding much, so I move to direct data analysis. There are a couple ways to do this with Microsoft tools:

    1. Process Monitor (download x86/x64)

    2. Windows System State Analyzer, included as part of the Windows Server Logo Tools (download x86, x64)

    These two utilities can monitor for changes and tell you what changed under the covers when you flipped some switch in the UI. They each have their advantages and limitations: WSSA is simple and shows changes without much modification, but is slower and sometimes unstable (I found it crashing on third party virtual hosts, but working fine on Virtual PC and Hyper-V). Process Monitor is fast, but complex and cumbersome. I demonstrate both below and you can decide – remember, my example is with Internet Explorer Favorites customization on XP so your steps will naturally vary. Before you get all bothered, there are plenty of snapshot comparison tools out there too; I can’t discuss their merits obviously or lawyers with sharks for arms will eat me.

    Scanning with Windows System State Analyzer

    1. I install WSSA and start it up (as an Administrator).

    2. I use the Tools menu to modify the options to monitor only registry and the user profiles directory. You can go as deep as the test user but some settings might exist in the All Users or Public profiles. I avoid scanning the entire drive, drivers, or services.

    image

    3. I start my application. In this case, I run Internet Explorer and open the Favorites menu (I want to be as close as possible to the change to avoid gathering unrelated data)

    4. I start a Baseline Snapshot and allow it to complete then move to section “Make the Change”.

    image

    Scanning with Process Monitor

    1. I Install ProcMon and start it up (as an Administrator).

    image

    2. I disable capturing and set the following filters to gather registry changes:

    Operation is regSetValue

    image

    Note: I may have to do this multiple times for various filters. For example, if I wanted to see file changes I’d switch my filter to instead:

    Operation is CreateFile
    Path begins with c:\documents and settings

    3. I start my application In this case, I run Internet Explorer and open the Favorites menu (I want to be as close as possible to the change to avoid gathering unrelated data)

    4. I clear the current data then start capturing.

    Make the change

    1. I alter my setting using the application. For example, here I move the AskDS blog link to the top of my favorites menu. Where it belongs. :)

    image

    2. I close the application to flush the change into the registry (there’s no guarantee that my change was immediately stored).

    Examine the Results with Process Monitor

    I stop capturing and look at the filtered results:

    HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\MenuOrder\Favorites\
    Order = <binary goo>

    image

    Examine the Results with Windows System State Analyzer

    I create a Current Snapshot, then compare to the baseline:

    menuorderssa
    click picture to see in large-o-vision

    Evaluate

    Isn’t that interesting? There’s a registry value named “Order” in our “MenuOrder\Favorites” key. I think we have a winner. I can confirm by looking up that specific key and yep, it’s the one. This also tells me that the key has been in use since (at least) Internet Explorer 4.0 and in every operating system since NT 4.0. The odds that this setting exists and works like this in Windows 7 and IE9 have gone way up.

    I take a quick look and yes, the same keys exist on my brand new test destination computer. A couple of red herrings out of the way – the OS and Browser version don’t matter.

    Sometimes you can understand the before and after settings, but this example isn’t pretty:

    image
    Yuck, REG_BINARY 

    Step 2 – Find the XML

    Ok, so now I need to find out if USMT migrates the Favorites customizations (I‘m sure it will migrate the Favorites themselves, that’s simply a shell folder in the user profile that we gather up). Luckily, much of the data migrated by USMT is defined in human-readable XML files. Since I now know the registry paths, I can examine them with confidence using FINDSTR.EXE within the USMT folder structure.

    Findstr.exe /s /i "currentversion\explorer\menuorder" *.*

    image

    See how FINDSTR returned the .man files containing data about those registry keys. Many application’s settings are migrated using these files rather than the migapp.xml (it is mainly for apps not included with Windows). There two kinds of manifests here: the DLManifests used on Windows XP and the ReplacementManifests used on Windows 7 and Vista.

    Since I’m on XP I’ll examine microsoft-windows-ie-internetexplorer-dl.man (using Visual Studio 2010 Express, as I previously described here). Right away, I see that these Favorites customizations absolutely will migrate.

    <include>

    <objectSet>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Internet Explorer\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\AutoComplete\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MenuOrder\Favorites\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Webcheck\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Ext\* [*]</pattern>

      <pattern type="Registry">HKCU\SOFTWARE\Microsoft\Search Assistant\* [*]</pattern>

      <pattern type="File">%CSIDL_LOCAL_APPDATA%\Microsoft\Internet Explorer\* [*]</pattern>

      <pattern type="File">%CSIDL_LOCAL_APPDATA%\Microsoft\Windows\History\* [*]</pattern>

      <pattern type="File">%CSIDL_APPDATA%\Microsoft\Windows\Cookies\* [*]</pattern>

      <pattern type="File">%CSIDL_APPDATA%\Microsoft\Internet Explorer\* [*]</pattern>

    </objectSet>

    </include>

    I chose wisely here but it might take you a little digging to find the migrating manifest. For example, the entire HKCU\Software\Microsoft\Windows key and its contents could be migrated and there would be no mention of the much deeper Favorites folder – my search would have failed in that case. If you don’t find it low, search high.

    Important note: Windows Vista and Windows 7 also include XML in the %systemroot%\winsxs\manifests folder, stored as .manifest files. You should examine these files with FINDSTR.EXE if you do not return any data when searching the USMT manifests and using a Vista or Windows 7 computer as the source machine. They are used automatically if there is no replacement manifest.

    There are a lot of manifests here and the ones that have a migration scope that includes “USMT” are the only ones that matter; the latest versioned one will be used if there are multiples:

    <migration scope="Upgrade,MigWiz,USMT">

    To find them all on your test computer, search your windows\winsxs\manifests folder like this:

    C:\Windows\winsxs\Manifests>findstr /m /i "usmt" *.manifest

    Step 3 – Validate the Processing

    This part should be easy: I migrate and logon as the test user to make sure the settings migrated successfully.

    But what if they didn’t?

    My investigation looked correct and I am confident that USMT will migrate these settings. I need to ensure that nothing else is blocking transfer, despite USMT’s best efforts. Again, the methodical approach works best.

    1. I run scanstate to gather the settings on my test XP computer.

    2. I run loadstate to restore the settings on my test Windows 7 computer.

    Critical note: remember, I am using the USMT manifests for both the scanstate and loadstate. If those manifests are not available, my settings will not migrate. If you look very carefully at the debug logs you will see the indications of failure:

    "Downlevel Manifests Folder is not present. System component settings will not be gathered."

     

    The ReplacementManifests folder used to service system component manifests is not present. OS settings migration will be done with system component manifests installed onto the system.

    The manifests might be missing because:

      • You forgot to copy them to the computer.
      • The manifest folders are not in the USMT working directory. For example, you are doing this (bad):

        C:\>c:\usmt\x86\scanstate.exe c:\store /i:migapp.xml /i:migdocs.xml

     

        C:\>\\server\share\scanstate.exe c:\store /i:migapp.xml /i:migdocs.xml

     

        C:\>c:\usmt\amd64\scanstate.exe c:\store /i:c:\usmt\xml\migdocs.xml /i:c:\usmt\xml\migapp.xml

    Instead of doing this (good):

    C:\usmt\x86\>scanstate.exe c:\store /i:migapp.xml /i:migdocs.xml

     

    C:\>NET USE U: \\server\share

    U:\>scanstate.exe c:\store /i:migapp.xml /i:migdocs.xml

     

    C:\usmt\AMD64>scanstate.exe c:\store /i:c:\usmt\xml\migdocs.xml /i:c:\usmt\xml\migapp.xml

    Notice how the executable is running within the folder path in the latter examples. There’s no USMT command-line to provide a path to the manifest folders! You can also run into this issue with SCCM or MDT as well, see:

    3. Before I log on to the test destination computer with my test user I load his user registry hive with regedit.exe and validate that the settings transferred:

    image

    image

    It looks good so I unload the hive. Don’t forget to do this!

    4. Then I logon as the migrated test user, but don’t start Internet Explorer. I examine the registry again and make sure it’s still unchanged. Why this step? The settings could be changing due to:

    • A logon script
    • A group policy
    • The act of logging on itself (as the first logon triggers a number of “personalization” steps that might override certain migrated settings).

    5. If I still didn’t know why the data was changing, I’d run Process Monitor in boot logging mode to see when and where the settings changed. That’s a theory though - I’ve never reached this stage using this methodical approach.

    Sir Arthur also said, “It is a capital mistake to theorize before one has data.”  Sherlock Holmes would have made a good Support Engineer.

    Final Note

    These techniques apply throughout USMT, not just with IE settings in a manifest file. I'm often asked about Office and migapp.xml, for example, and the answer is always the same - do the legwork above and you'll know. These techniques also apply to preventing migration of settings - once you know how USMT migrates it, making an unconditionalExclude to prevent it is easy.

    For those keeping score: the customer’s issue was the insidious missing manifest folders I pointed out in the validation phase. It happens to nearly everyone once… but never twice. :)

    Until next time.

    Ned “The game is afoot!” Pyle