SupportingWindows

  • Bcdedit Tips and Tricks For Debugging Part 1

    Hello everyone, my name is Sean Walker, and I am on the Platforms OEM team in Washington.   This article is for those people who have had a hard time switching from the old boot.ini configuration to the new BCD store (myself included). Doing the ...read more
  • Debugging a CLOCK_WATCHDOG_TIMEOUT Bugcheck

    Hi debuggers, Andrew Richards here for my first NT Debugging post. I thought I’d share a recent case that used a lot of discovery techniques to uncover the details of what was going on. Most bugchecks give you the information you need as arguments, but ...read more
  • Call Stacks for Pool Allocations

    Hello, it's the Debug Ninja back again for another NtDebugging Blog article.   For as long as I can remember user mode debuggers have had an easy way to get call stacks for heap allocations.   On more recent versions of Windows this has been ...read more
  • Where Did My Disk I/O Go?

    Hello, Mr. Ninja back again.   I recently discovered that although my team often tracks I/O from the file system through to the disk controller, we have never publicly documented the steps required to do this.   This seems like a great opportunity ...read more
  • Fixing an ICorDebugUnmanagedCallback induced hang

    Hi debuggers, Andrew Richards here with a NTDebugging post that is a little different to what is usually posted.   Instead of talking about debugging, I’m going to talk about an issue I just faced while writing a debugger.   This debugger work ...read more
  • My Kernel Debugger Won't Connect

    Hello ntdebugging readers, the Debug Ninja is back again with a quick blog this holiday season.   I recently encountered a situation where the kernel debugger could not connect to a Windows Server 2008 R2 system running in a Hyper-V virtual machine ...read more
  • Configuring a Hyper-V VM For Kernel Debugging

    Yesterday's blog prompted some questions about how to set up a debugger for a Windows OS running in a Hyper-V VM.   I was surprised that I wasn't able to find good, publicly available, Microsoft issued documentation for this configuration.   ...read more
  • Stop 0x19 in a Large Pool Allocation

    Hello all, Scott Olson here again to share another interesting issue I recently debugged with pool corruption and found that using special pool does not work with large pool allocations ( pool allocations greater than a PAGE_SIZE ).   Here is an ...read more
  • Identifying Global Atom Table Leaks

    Hi, it's the Debug Ninja back again with another debugging adventure.   Recently I have encountered several instances where processes fail to initialize, and a review of available resources showed that there was no obvious resource exhaustion.   ...read more
  • What Should Never Happen... Did

    Hi, this is Bob Golding; I wanted to write a blog about an interesting hardware issue I ran into. Hardware problems can be tricky to isolate. I recently came across one that I thought was interesting and gave an example of how to trace code execution ...read more
  • Debugging Backwards: Proving root cause

    Matt Burrough here again.   On rare occasions when debugging, we'll actually know (or strongly suspect) what the root cause of a problem is at the beginning of our analysis - but we still need to investigate to confirm our assertion.   The following ...read more
  • Performance issues due to Inactive Terminal Server Ports

    Good morning AskPerf!  There are several issues that have been associated with a high number of inactive Terminal Server ports.  Delayed logon times to RDP sessions, failure of printers to redirect, and slow server performance due to registry bloat from all the ports. These inactive TS ports accumulate because the Remote Desktop Services Device Redirector service creates a new port every time an RDP session is established, but the ports are not always recycled.  Every RDP session can possibly create a new port, and every ended session means a new inactive port. Performance degradation is known to occur when 250 or more TS ports exist in the registry. Increasingly large numbers of redirected devices will exacerbate performance delays.

    To eliminate any issues these TS ports may cause, we have a new Windows Server 2008 R2 hotfix.  The Hotfix can be downloaded here - KB 2655998 Long logon time when you establish an RD session to a Windows Server 2008 R2-based RD Session Host server if Printer Redirection is enabled.  This hotfix will prevent inactive TS ports from accumulating in the future, but you also have to clean up any currently inactive TS ports using the Fixit tool included in KB2655998.  The FixIT tool itself can also be downloaded directly from http://go.microsoft.com/?linkid=9799054.  The FixIt tool will remove the entries already accumulated under the TS ports registry key below:

    [HKLM\SYSTEM\CurrentControlSet\Control\DeviceClasses\{28d78fad-5a12-11d1-ae5b-0000f803a8c2}\##?#Root#RDPBUS#0000#{28d78fad-5a12-11d1-ae5b-0000f803a8c2}]

    Because the Hotfix and the FixIt perform two different functions, you must install the Hotfix and run the FixIt one time on your 2008 R2 Remote Desktop Server. 

    For down-level operating systems such as Server 2003, Server 2008 Terminal Servers and Windows 7 in a VDI scenario, we highly recommend a scheduled task be created to execute the Inactive TS Port FixIt on a bi-weekly basis, since the hotfix only supports Windows Server 2008 R2 at this time.

    The FixIt is an MSI package so you can run it silently with no user interaction required.  Follow these simple steps:

    1. Locate the MSI file in Windows Explorer and note the exact path to it

    2. Click Start and type in the following:

    msiexec /package “path” /quiet

    For example, if the MSI file MicrosoftFixit50833.msi is located in the “temp” folder of your C drive, then the path would be C:\temp\MicrosoftFixit50833.msi and the command would look like this:

    msiexec /package “C:\temp\MicrosoftFixit50833.msi” /quiet

    3. To add logging and check the results, use this command:

    msiexec /package "c:\temp\MicrosoftFixit50833.msi" /quiet /log c:\temp\MSI50833.log

    Additional Resources

    -Jess Cunningham

  • Unable to connect to WMI locally – “Win32: The system cannot find the path specified”

    Good morning AskPerf! Sanket Jagtap here from Platforms Team here to discuss an interesting issue I recently worked.  My customer had a Windows 2008 R2 Server, and could not connect locally to WMI.  When they ran “wmimgmt.msc” and tried to connect, the following error appeared:

    Failed to connect to <local computer> because "Win32: The system cannot find the path specified."

    image

    When this failed, the following events were recorded to the Application log:

    Log Name: Application
    Source: Microsoft-Windows-WMI
    Date: <Date/Time>
    Event ID: 28
    Task Category: None
    Level: Error
    Keywords: Classic
    User: N/A
    Computer: <Computer Name>
    Description: Failed to Initialize WMI Core or Provider SubSystem or Event SubSystem with error number 0x80070003. This could be due to a badly installed version of WMI, WMI repository upgrade failure, insufficient disk space or insufficient memory.

    Log Name: Application
    Source: Microsoft-Windows-WMI
    Date: <Date/Time>
    Event ID: 43
    Task Category: None
    Level: Warning
    Keywords: Classic
    User: N/A
    Computer: <Computer Name>
    Description: Windows Management Instrumentation ADAP failed to connect to namespace \\.\root\cimv2 with the following error 0x80070003

    We checked the WMI Security tab, and the Root namespace was missing, as well as under the Advanced tab:

    image

    image

    During the course of troubleshooting, we tried running “winmgmt /verifyrepository”, however it failed with a similar error:

    C:\>winmgmt /verifyrepository
    WMI repository verification failed
    Error code: 0x80070003
    Facility: Win32
    Description: The system cannot find the path specified.

    At this point, we decided to run the WMIDiag Tool (WMIDiag 2.1) to help troubleshoot this issue. The WMIDiag log reported the same error multiple times for most of the Root namespace:

    1843 12:43:16 (0) ** - Root, 0x80070003 - The system cannot find the path specified.
    Error Result: 0x80070003 ( -2147024893 )
    ID Defined as: ERROR_PATH_NOT_FOUND
    Message Text: The system cannot find the path specified.

    We then popped open the Registry and browsed to the following key:

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WBEM\CIMOM]

    We compared a working to a non-working registry and made the following discover:

    Working Machine

    clip_image006

    Non-Working Machine

    clip_image008

    The registry “Type” was set to REG_SZ instead of REG_EXPAND_SZ. We took a backup of the CIMON folder, then deleted the old “Repository Directory” value and created a new one using the REG_EXPAND_SZ type. After a restart, WMI now successfully connected to the Repository/Default Namespace.

    Unfortunately, we were unable to determine what changed this value, however it could have been a user change, some type of script, or caused by an application.

    Additional Resources

    - Sanket Jagtap

  • Print update rollup available for Windows 7 & Windows Server 2008 R2

    A new printing hotfix rollup was released that updates the core print spooler components localspl.dll, splwow64.exe, spoolsv.exe, win32spl.dll, and Winprint.dll.  This rollup is designed to reduce known issues such as printing performance, print spooler crashes, connectivity to print queues, and print driver installation. 

    To download the update and get more information on the fixes included in this rollup, please see KB article 2647753:  Description of an update rollup for the printing core components in Windows 7 and in Windows Server 2008 R2

    Many of the printing related support cases at Microsoft are resolved by one of the included hotfixes.   We highly recommend installing this hotfix rollup when troubleshooting any printing related issues. 

  • How the Clipboard Works, Part 1

    Recently I had the opportunity to debug the clipboard in Windows, and I thought I’d share some of the things I learned.   The clipboard is one of those parts of Windows that many of us use dozens (hundreds?) of times a day and don’t really think ...read more
  • Windows Performance Monitor Disk Counters Explained

    Hello AskPerf and Happy Friday!  This is just a quick post to let you know that Flavio, one of our Disk Experts on the Core team, has posted another informative blog post; this time on Performance Monitor Disk Counters.  Check it out and have a great weekend!

    Windows Performance Monitor Disk Counters Explained

    http://blogs.technet.com/b/askcore/archive/2012/03/16/windows-performance-monitor-disk-counters-explained.aspx

    -Blake Morrison

  • FIXED: Cluster Shared Volumes (CSV) in redirected access mode after installing McAfee VSE 8.7 Patch 5 or 8.8 Patch 1

    A fix for the above titled problem has been released.  If you are running into this problem, please downlaod and install the following fix on all Clusters running McAfee and wanting the updates they provide.

    2674551
    Redirected mode is enabled unexpectedly in a Cluster Shared Volume when you are running a third-party application in a Windows Server 2008 R2-based cluster
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2674551

    ===============

    Below is the information from the original post:

    There is an issue with Cluster Shared Volumes and McAfee VirusScan Enterprise that I wanted to pass along. When installing McAfee VSE 8.7 Patch 5 or 8.8 Patch 1, the CSV drives will go into redirected mode and will not go out of it.

    The reason for this is that the McAfee filter driver (mfehidk.sys) is using decimal points in the altitude to help in identifying upgrade scenarios for their product. The Cluster CSV filter only accepts whole numbers and puts the drives in redirected access mode when it sees this decimal value.

    When seeing this, if you run FLTMC from an administrative command prompt, you may see something similar too:

    C:\> fltmc

    Filter Name   Num Instances   Altitude   Frame
    ------------------------------------------------------
    CSVFilter          2          404900       0
    mfehidk                       329998.99  <Legacy>
    mfehidk            2          321300.00    0

    If you were to generate a Cluster Log, you would see the below identifying that it cannot read the altitude value properly.

    INFO [DCM] FsFilterCanUseDirectIO is called for \\?\Volume{188c44f1-9cd0-11df-926b-a4ca2baf36ff}\
    ERR mscs::FilterSnooper::CanUseDirectIO: BadFormat(5917)' because of 'non-digit found'
    INFO [DCM] PostOnline. CanUseDirectIO for C2V1 => false

    McAfee has released the following document giving a temporary workaround.

    Cluster Shared Volumes (CSV) status becomes Online (Redirected access)
    https://kc.mcafee.com/corporate/index?page=content&id=KB73596

    Microsoft is aware of the problem and currently working on a fix. When this fix is available, this will be updated and a new KB Article will be created with the fix.

    John Marlin
    Senior Support Escalation Engineer
    Microsoft Enterprise Platforms Support

  • Why is the CNO in a Failed State?

    Hello, my name is Steven Graves and I am a Support Escalation Engineer (SEE) in the Platforms Support group here at Microsoft. One of the technologies I support is Windows Server Failover Clustering. I’d like to take a couple of minutes to share some information on an issue I previously worked on. The customer wanted to create an Exchange 2010 DAG, which would be the first Windows Server 2008 R2 cluster in their environment and they were having issues bringing the CNO online after the cluster was created. The customer domain was originally 2003 and they had to add a 2008 R2 DC and update the schema in order to install Exchange 2010 DAG.

    For starters, since I knew the CNO was not coming online after creating the cluster I had the customer destroy the cluster, pre-staged a new computer object for the CNO then created a new cluster based on the name of the new CNO. After the cluster was created I noticed that the computer object was still disabled in AD and the following error message in the cluster log.

    clip_image001

    00000e80.00000d5c::2012/03/14-16:07:33.149 INFO [RES] Network Name <Cluster Name>: Trying to find computer account W2K8R2Cluster object GUID(cae8b3dcc60aa040bbcef250634427bb) on any available domain controller.
    00000e80.00000d5c::2012/03/14-16:07:33.306 WARN [RES] Network Name <Cluster Name>: Search for existing computer account failed. status 8007052E
    00000e80.00000d5c::2012/03/14-16:07:33.352 WARN [RES] Network Name <Cluster Name>: Couldn't get information from DC \\Info-dc3.infoimage.com. status 5
    00000e80.00000d5c::2012/03/14-16:07:33.352 INFO [RES] Network Name <Cluster Name>: Trying to find object cae8b3dcc60aa040bbcef250634427bb on a PDC.
    00000e80.00000d5c::2012/03/14-16:07:33.462 WARN [RES] Network Name <Cluster Name>: Couldn't get information about PDC. status 5
    00000e80.00000d5c::2012/03/14-16:07:33.462 INFO [RES] Network Name <Cluster Name>: Unable to find object cae8b3dcc60aa040bbcef250634427bb on a PDC.
    00000e80.00000d5c::2012/03/14-16:07:33.462 INFO [RES] Network Name <Cluster Name>: GetComputerObjectViaGUIDEx() failed, Status 8007052E.

    clip_image002Access is denied’

    In the System Event log you will see an ID 1207 that should be in synch with the time in the cluster log. The main thing to focus on is the “Unable to get the Computer Object using GUID”.

    Log Name: System
    Source: Microsoft-Windows-FailoverClustering
    Date: 3/14/2012 9:07:10 AM
    Event ID: 1207
    Task Category: Network Name Resource
    Level: Error
    Keywords:
    User: SYSTEM
    Computer: W2K8R2Cluster.Corp.com
    Description:
    Cluster network name resource 'Cluster Name' cannot be brought online. The computer object associated with the resource could not be updated in domain 'infoimage.com' for the following reason:
    Unable to get Computer Object using GUID.
    The text for the associated error code is: Logon failure: unknown user name or bad password.

    At this point, I’m pretty convinced there are some issues with the GPOs on the domain controllers but I still need to do my due diligence in troubleshooting the issue with the Cluster Network Name in a failed state.

    Since I pre-staged the CNO and it was still disabled after creating a new cluster, this gave me more evidence indicating an issue with the DC. I created a new OU and blocked inheritance in order to prevent any GPOs from being applied to the Node(s). I refreshed the GPO’s on the Node(s), confirmed there are no GPOs applied by running Gpresult /V from an Administrative CMD Prompt, but the Cluster Network Name still fails to come online. I’m convinced there is some issue with GPO’s on the DC but I’m not sure where to start looking.

    Next, I verified the permissions in AD on the CNO and, to be on the safe, I granted the CNO Full Control to the object and also confirmed that the CNO has the correct permissions to the OU(READ permissions on the OU should be sufficient rights to access the OU and get to the computer object). Despite this, the Cluster Network Name failed to come online.

    I moved on to check the DNS Host A record for the CNO not really thinking this is the issue but more or less making sure everything is in order. I came to find out a Host A record was not created for the Cluster Network Name because they do not have Dynamic Updates enable for DNS. I created the Host A record and checked off “Allow any authenticated user to update the DNS records with the same owner name.” I already knew the node was able to resolve the DC from the warnings in the cluster log but couldn’t get information from DC \\W2K8R2-DC.Corp.com. So it was not a name resolution issue trying to access the DC.

    At this point, I have gone through all the normal troubleshooting steps that generally resolve the ID 1207 and the CNO in a failed state from the cluster perspective. Now it’s time to engage Directory Services to take a deeper look at the DC configuration. After some time reviewing the Domain Controller configuration and GPOs the DS engineer narrowed it down to permission issues in the “Access this computer from the network” policy. The default permissions are pictured below.

    Access this computer from the network - This user right determines which users and groups are allowed to connect to the computer over the network. Since "Everyone" and "Authenticated Users" were missing from the settings, this meant that no computer would be able to access the domain controller.

    clip_image003

    Picture above shows the default permissions for the Access this computer from the network policy

    The DS engineer modified the “Access this computer from the network” policy in the Default Domain Controllers policy by adding Authenticated Users, refreshed GPOs by running GPUpdate /force, ran RSOP.msc to confirm the GPO is applied, and the CNO came online.

    Steven Graves
    Support Escalation Engineer
    Microsoft Enterprise Platforms Support

  • Our Team in Bangalore is Hiring - Windows Server Escalation Engineer

    Would you like to join the world’s best and most elite debuggers to enable the success of Microsoft solutions?   As a trusted advisor to our top customers you will be working with to the most experienced IT professionals and developers in the industry ...read more
  • How the Clipboard Works, Part 2

    Last time , we discussed how applications place data on the clipboard, and how to access that data using the debugger.   Today, we'll take a look at how an application can monitor the clipboard for changes.   Understanding this is important ...read more
  • Migrating Print Queues quickly using PRINTBRM, configuration files, and the Generic/Text Only Driver

    Good morning AskPerf! Digvijay here to chat about print server migration. All server admins have probably had an opportunity to migrate print servers. This process can be fast and easy when migrating between the same versions of operating systems; and can be done with a bit of effort moving between Operating Systems with different architecture (e.g., moving from 32-bit to 64-bit versions of Server 2008). However, things really get challenging when we have to migrate from an older Operating System to the latest Operating System on a different architecture. (e.g., from Windows 2003 Server x86 to Windows Server 2008 R2 x64). Fortunately, there are new tools such as the Print Management Console that ships with Server 2008 R2 or, if you prefer the command line interface, then you can use PRINTBRM.EXE. However, there can be some hurdles getting all the printers migrated successfully if we do not understand the Printer restoration process and fulfill necessary pre-requisites.

    Things to keep in mind during printer migrations:

    1. A print queue cannot function without a driver of the same server architecture (x86 or x64) on which it exists. 

    2. Unless the print processor is migrated successfully, the print queue will not show up.

    3. If you are migrating from a 32-bit Operating System to a 64-bit Operating System, it is important that you have 64-bit drivers pre-installed for all of your printers, along with the 32-bit drivers. (Remember that for a 64-bit client to print to print server hosted on 32-bit Operating System, you need the 64-bit version of the driver.)

    4. When a cross-architecture migration includes the migration of printer language monitors, an error will occur during the process of restoring the printers to the destination server using the PRINTBRM command-line tool. The reason for the error is that language monitor driver architecture must always be the same as the source server architecture. Therefore, when migrating from x86-based architecture to x64-based platforms, language monitor migration cannot be successful. An error posted to the event log will state that the source architecture is not the same as that of the destination server. More info about this behavior can be found in the Print Services Migration Guide.

    5. When using PRINTBRM, always run commands from an Administrative command prompt

    If performing a 32-bit to 64-bit migration and all the existing client machines are 32-bit, then you will have to install 64-bit drivers for all the 32-bit drivers before starting the migration. It is common to have third-party 32-bit printer drivers that do not have 64-bit equivalents. For those instances, we typically suggest using universal drivers if available but contacting the printer vendor for their input is the safest bet.

    During the Printer restore in cross-platform scenarios, if the backup file does not contain driver binaries for target server platform, PRINTBRM will attempt to install the drivers from the target server’s driver store if a matching driver is available. If a matching driver for the target platform is not available (either in the backup or on the server) then the print queue will not be created and there will be errors. See below:

    image

    image

    We will see relevant events in the event viewer also:

    image

    image

    image

    So to make it easy for everyone to understand, here is the straightforward way to successfully restore all the printers:

    1. Install 64-bit drivers on the Windows 2003 x86 Print server (source) or install all the required 64-bit drivers on the 2008/R2 Server (target). Keep in mind that the driver name string must be an exact match. If the names do not match exactly, the migration will treat it as if the driver is not present.

    2. Take a backup of the printers on the Windows 2003 source server using the Print Management Console or printbrm.exe run from the Windows 2008 R2 Server.

    3. Import the backup to the 2008 R2 server.

    NOTE We need to preinstall any driver that has different components when installed on x86 than when installed on x64. As long as the driver is installed then the printer installation will not be blocked unless there is a non-existent Monitor value in the Printer registry key.  If the printer (not the driver) requires this setting, then we will need to install the driver that uses this monitor on the x64 machine.  If there is no x64 monitor provided by the vendor or if the vendor provides a new x64 monitor that uses a different name, then you will need to add a printer using the x64 version of the driver on the new machine since the vendor providing the driver did not provide compatibility to the new OS.

    Keeping it simple

    To avoid most of these driver and component pitfalls, it’s often easiest to switch all the print processors and print monitors on the source server over to some basic defaults ones (eg: cleaning up the PRINT registry before starting with the migration). This will speed up the process and maximize the number of queues successfully migrated.

    Microsoft has provided a Spooler FixIT for Windows 2003/2008 to automate the spooler cleanup process. See our previous AskPerf blog entry for information about the functionality and usage of this tool.

    We also recommend removing old and unused drivers from the server before taking a backup.

    However, if you do not want to go through this lengthy process of fulfilling these prerequisites, are on a time crunch, want to quickly restore the print queue information on the target server and deal with the driver worries later, then read on.

    In order to quickly get the queues migrated to the new server, we can create them using the “Generic / Text only” printer driver without restoring any of original the drivers on the target. Remember that we need to have 64-bit drivers present either in the package or on the target server for the restore to finish and the print queue to be available for printing.

    Here are the steps to successfully migrate all of your print queues using the in-box “Generic/Text Only” print driver:

    1. On the source Windows 2003 Server, change all print processors to winprint using the following setprinter.exe command:

    SetPrinter.exe \\Servername 2 pPrintProcessor="Winprint"

    This can also be accomplished using WMI:

    wmic printer set PrintProcessor = WinPrint

    2. Install the latest update for Printbrm on the target 2008R2 server:

    KB 2636591  An update to improve the restore operation performance of Printbrm.exe in Windows 7 or in Windows Server 2008 R2

    3. Install a local printer on the target 2008 R2 server using the Generic / Text Only driver and share it to enable the firewall exceptions for the Print Services.

    4. From the Windows 2008R2 server, take a backup of the 2003 print server using the nobin switch

    (we are using the nobin switch since we don’t want to migrate any of the print drivers):

    Printbrm.exe –b –s \\servername –f nobin.printerexport –nobin

    5. Next, we have to modify the contents of the backup files. Expand the printerexport backup to a directory and the replace the BrmDrivers.xml, BRMLMons.xml and PProcs.xml files. The command to extract the files to a directory is:

    printbrm.exe  -r -d c:\temp\expand  -f nobin.printerexport

    image

    The Expanded directory will look similar to this:

    image

     

    • BrmDrivers – contains a list of every driver installed on the computer and the driver files for each driver
    • BrmForms – contains a list of all the installed forms
    • BrmLMons – will usually contain either Windows NT x86 or Windows x64 as the architecture and a list of port monitors and the port monitor files installed
    • BrmPorts – contains a list of all the installed printer ports
    • BrmPrinters – contains a list of all printers that have been installed
    • BrmSpoolerAttrib – contains information about the spooler directory path and also indicates whether or not the source server was a cluster server

    6. Now we need to edit the existing xml files and delete the information about unused print processors, language monitors and drivers in them and then create a new file without these components.

    NOTE You may ask if we already set all the printers to use WinPrint as default print processor using the earlier methods, then why do I need to do this? The answer is that when we take a backup using PRINTBRM, the tool copies all the files from their respective directories inside C:\Windows\System32\spool\. Hence even if we have replaced the information on the queues, these files will be backed up and at the restore operation, they will also be restored. So even though we took a backup using the nobin switch, the xml files contains information about all the drivers. When you attempt to restore, the server will try to look for these drivers and restore them. Also, the -nobin switch only omits the driver files, the Language monitor and the Print processors are still backed-up.

    For creating new BrmDrivers.xml, BRMLMons.xml and PProcs.xml files, we can use the following templates:

    BrmDrivers.xml

    ===========

    <PRINTERDRIVERS>
    </PRINTERDRIVERS>

    ===========

    BRMLMons.xml

    ==========

    <LMONS Arch="Windows x64"/>

    ==========

    PProcs.xml

    ========

    <PPROCS>

    </PPROCS>

    =========

    7. Once we have saved the new xml files in place, we will re-pack the backup using this command to get a new backup file that we will use to restore on the target server:

    printbrm.exe  -b -d c:\temp\expand  -f newbackupfile.printerexport

    8. Now that we have a clean printer backup, we will use the BRMConfig file to create all the printers using the “Generic / Text only” driver. The BRMConfig.xml file needs to have a mapping of all the drivers present on the source server that we want to restore with the Generic / Text Only driver.

    A sample BRMConfig file may look like this:

    =============

    <BrmConfig>

    <PLUGINS>

    </PLUGINS>

    <DriverMap>

    <DRV old="HP Universal Printing PCL 6" new="Generic / Text Only"/>

    <DRV old="HP Laserjet 5000 PCL 6" new="Generic / Text Only"/>

    <DRV old="HP Universal Printing PS 6" new="Generic / Text Only"/>

    </DriverMap>

    </BrmConfig>

    ==================

    For restoring all the printers from the backup, we need to mention all the existing drivers here.

    NOTE The name of the driver has to be an exact match.

    9. Once you have your brmconfig.xml file finished, run this command to start the restore:

    printbrm –r –s \\2008R2Servername  –f newbackupfile.printerexport -c c:\temp\brmconfig.xml

    NOTE We need to provide the absolute path of the BrmConfig.xml file here.

    10. If all the mentioned instructions have been followed, you will see all the print queues getting created on the server using the Generic / Text Only driver.

    image

    This is where we started from:

    image

    And this is what we have accomplished:

    image

    11. Verify that the printers are working fine by sending some test pages to the printers.

    12. After the queues are created on the target server, you can install the 64bit drivers for the printers as per your convenience and then switch the queues again.

    Simplifying the BRMConfig.xml file creation

    Considering most typical production scenarios, Print Servers you have hundreds of print queues and many print drivers, creating the BrmConfig.XML file for mapping each driver to Generic / Text Only driver can be real time consuming process. To ease the pain of creating this file yourself, there’s a BRMC tool that will come in handy. Download the tool from https://brmc.codeplex.com/ and run it from the directory where the BRMDrivers.XML is located. (So run it from the expanded location). Once the tool is run, and it creates a BRMConfig.xml file, pack the backup and then run the restore using the BrmConfig.XML file (to to step 8 and then go to step 10).

    image

    The BRMC tool creates the brmconfig.xml file that has the driver mappings. It also creates clean BrmDrivers.xml, BRMLMons.xml and PProcs.xml in the same directory. Be aware, the tool overwrites the original files, so save the original .printerexport file as a backup of BrmDrivers.xml, BRMLMons.xml and PProcs.xml.

    Troubleshooting

    If you encounter any errors during a Print Migration restore, check the event logs for possible causes. Common ones may be Print Processor is unknown, Unknown Port, or Print driver is unknown which usually means that the spooler was not cleaned properly on the source server when the initial backup was generated. Look for more in-depth troubleshooting of these kinds errors in a future blog.

    I hope this information will come in handy the next time you are working through a printer migration. Until next time…

    Addition Resources

    -Digvijay

  • Troubleshooting Boot Issues due to missing Driver signature (x64)

    Today, I am going to discuss how to troubleshoot a scenario where the system does not boot in normal mode; however, boots up fine upon selecting F8 -> Disable Driver Signature Enforcement.

    In 64-bit operating systems starting with Windows Vista, Windows will load a kernel-mode driver only if the driver is signed. You might get different fatal errors during the boot process depending on the driver that was blocked from loading and how it impacted the further processes. While some of the fatal system errors reference the driver on the blue screen, some may not.

    Here is an example of how we can diagnose the problem and fix it. When booting normally, you may receive this error and the machine halts:

    clip_image001

    This article explains this type of stop error.

    Bug Check 0xC000021A: STATUS_SYSTEM_PROCESS_TERMINATED
    http://msdn.microsoft.com/en-us/library/windows/hardware/ff560177(v=vs.85).aspx

     

    In decoding the stop error, you should look at the second parameter.

    Stop 0XC000021A FATAL SYSTEM ERROR
    (0X00000000 0XC0000428 0X00100588)  <--- second parameter indicates error code

     

    To translate the hex code, you can use the inbox SLUI.EXE tool or look up the error code from ntstatus.h file in the Windows SDK.

    Command Usage:

    slui.exe 0x2a 0xC0000428

    clip_image002

    Restart your computer and start pressing the <F8> key on your keyboard. On a computer that is configured for booting to multiple operating systems, you can press the <F8> key when the Boot Menu appears. On the Advanced Boot Options menu, select Disable Driver Signature Enforcement press <ENTER>.

    clip_image004

    If the server boots up after selecting this mode, we now know that it is some driver or module which is unsigned or being detected as unsigned that is preventing the system from booting up properly.

    Our next task is to find out the module name and devise a remedy. You would need to open Event Viewer and go to Applications and Services Logs -> Microsoft -> Windows -> CodeIntegrity -> Operational.

    Note: Check if you can see events like the ones depicted below.
    In case you get “access denied” while accessing the “operational” log create a folder on the root of the C drive and give Everyone full rights.
    Redirect the ETL file path to the newly created folder. Disable and enable the logging again.

    clip_image006

    Once logging is enabled, you will find the unsigned driver name from the events.

     

    Remediation...

    The driver can be a Microsoft inbox driver or a 3rd party driver. For drivers that have newer versions available and can be updated, that is the first thing to try. In most cases, this will fix the problem.

    However, you might encounter a scenario when there is no update available. Here are the steps to find out the required catalog file and place it accordingly.

    Files can be copied from another working server with similar file versions and extracted from the update that was the source of the file version.

    We can use the SIGCHECK.EXE tool to find the catalog file for a file.

    Sigcheck
    http://technet.microsoft.com/en-us/sysinternals/bb897441

     

    Step -1 : Verify if the driver is really Unsigned:

    Note: The full path of the driver file should be passed as a command line parameter

    clip_image008

    You can run the command as described in the screenshot above to make sure the driver is indeed being detected as unsigned.

     

    Step – 2 : Get the Signed driver from a working server:

    clip_image010

    The actual file and the catalog can be copied from a working server and placed in respective locations as depicted above. Once the copy completes, use the SIGCHECK tool to verify and make sure that the file is detected as signed and displays the name of the catalog. Reboot the server and ensure it comes up fine in the normal mode.

     

    Appendix - A

    Note:

    To find the catalog for the file, you can also enable verbose logging for CodeIntegrity on the working server.  This will log the name of the CAT file where the file hash was found during the verification.

    clip_image013

    Steps to enable CodeIntegrity verbose logging:

    i. Open Event Viewer
    ii. Select the View drop down menu and choose Show Analytic and Debug Logs from the list
    iii. Navigate to Applications and Services Logs -> Microsoft -> Windows -> CodeIntegrity -> Verbose
    iv. Right – click and select enable log

    More information on the Code Integrity can be found here:

    Code Integrity Diagnostic System Log Events
    http://msdn.microsoft.com/en-us/library/windows/hardware/ff539911(v=vs.85).aspx

     

    Appendix - B

    Note: The SIGNTOOL application can be used to get similar information.

    Signtool
    http://msdn.microsoft.com/en-us/library/windows/desktop/aa387764(v=vs.85).aspx

    Mentioned below is an example of this:

    C:\WinDDK\7600.16385.1\bin\amd64>SignTool.exe verify /a /v c:\windows\system32\win32k.sys

    Verifying: c:\windows\system32\win32k.sys
    File is signed in catalog: C:\Windows\system32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}\Package_7_for_KB2661001~31bf3856ad364e35~amd64~~6.1.1.0.cat
    Hash of file (sha1): DEDF7D339D8355A9875661559BD582FA46008AE5

    Signing Certificate Chain:
        Issued to: Microsoft Root Certificate Authority
        Issued by: Microsoft Root Certificate Authority
        Expires:   Mon May 10 04:58:13 2021
        SHA1 hash: CDD4EEAE6000AC7F40C3802C171E30148030C072

            Issued to: Microsoft Windows Verification PCA
            Issued by: Microsoft Root Certificate Authority
            Expires:   Wed Mar 16 03:35:41 2016
            SHA1 hash: 5DF0D7571B0780783960C68B78571FFD7EDAF021

                Issued to: Microsoft Windows
                Issued by: Microsoft Windows Verification PCA
                Expires:   Tue May 15 02:41:44 2012
                SHA1 hash: 5C616DC011E309DFCD15C0EA32494186654A2CDC

    The signature is timestamped: Mon Jan 16 15:24:23 2012
    Timestamp Verified by:
        Issued to: Microsoft Root Certificate Authority
        Issued by: Microsoft Root Certificate Authority
        Expires:   Mon May 10 04:58:13 2021    

    SHA1 hash: CDD4EEAE6000AC7F40C3802C171E30148030C072
            Issued to: Microsoft Time-Stamp PCA
            Issued by: Microsoft Root Certificate Authority
            Expires:   Sat Apr 03 18:33:09 2021
            SHA1 hash: 375FCB825C3DC3752A02E34EB70993B4997191EF            

                Issued to: Microsoft Time-Stamp Service
                Issued by: Microsoft Time-Stamp PCA
                Expires:   Fri Oct 26 02:12:17 2012
                SHA1 hash: FC33104FAE31FB538749D5F2D17FA0ECB819EAE5

    Successfully verified: c:\windows\system32\win32k.sys

    Number of files successfully Verified: 1
    Number of warnings: 0
    Number of errors: 0

     

    Below are articles you should review that discuss more details about this subject.

    Driver Signing Requirements for Windows
    http://msdn.microsoft.com/en-us/windows/hardware/gg487317

    Driver Signing Guidelines for ISVs
    http://msdn.microsoft.com/en-us/windows/hardware/gg463036


    Kernel-Mode Code Signing Walkthrough
    http://msdn.microsoft.com/en-us/windows/hardware/gg487328

    Installing an Unsigned Driver during Development and Test
    http://msdn.microsoft.com/en-us/library/windows/hardware/ff547565(v=vs.85).aspx

     

    Hope this helps and until next time, Take care.

    Parthiv Seth
    Support Escalation Engineer
    Microsoft Windows Server Core Team

  • Task Scheduler Error “A specified logon session does not exist”

    Good morning AskPerf!  Kapil Patry here from the Microsoft Platforms Support Team.  I am blogging today about an issue I recently worked.  This particular issue dealt with the creation of a task using Task Scheduler on a Windows 2008 R2 Server (can occur on Windows 7 as well).  When I attempted to create a scheduled task, the following error appeared:

    clip_image001

    Task Scheduler

    An error has occurred for the task <task name>. Error message: The following error was reported: A specified logon session does not exist. It may have already been terminated..

    This error came up whether I was logged on as a Local Administrator, Domain Administrator, or any other user that had rights to log on locally.

    After extensive research, I found that the above error will only occur if the following Security Policy is enabled and you select the “Run whether user is logged on or not” Security option on the General tab, when creating a new task:

    SECPOL.MSC | Security Settings | Local Policies | Security Options

    Network access: Do not allow storage of passwords and credentials for network authentication

    clip_image003

    To resolve this issue, simply Set this policy to Disabled:

    clip_image005

    The new version of Task Scheduler (Windows Vista onwards) uses Windows Credential Manager to store the credentials of the account that is specified to perform a task.  If the Network access: Do not allow storage of passwords and credentials for network authentication policy is enabled and applied, Credential Manager cannot store the credentials locally, thus this error message appears.

    NOTE you will not receive this error if the “Run only when user is logged on” Security option on the General tab is selected (we do not store passwords in this scenario).

    Addition Resources

    -Kapil Patry

  • Troubleshooting Memory Leaks With Just a Dump

    Hello debuggers, the debug ninja is back again.   Sometimes we have a scenario where a process is using a lot of memory, and the only data we are able to get at the moment is a user dump.   Ordinarily data from tools such as umdh or xperf would ...read more
  • Improve caching for CSR

    In previous versions of Windows, clients with remote connections would often initiate network traffic to retrieve remote information from a print server, resulting in a resource-consuming call to the server.  This behavior was more significant when Client Side Rendering (CSR) was used that caused increased communication with the remote server. Many applications use certain APIs to query the print spooler for print information at startup, at document creation time, and when displaying available fonts. This often resulted in visible delays or hangs in the applications from the user’s perspective. In Windows 7 and Windows Server 2008 R2, the performance of many spooler API calls has been improved through more aggressive caching that will reduce the remote server calls and improve end user performance.

    In general, the intent of the enhancements has been to decouple applications that use client print APIs (winspool.drv) from the print server to avoid application UI delays. Some of the APIs that are used by major client applications such as the print dialog and Microsoft Office have had their caching behavior improved. OpenPrinter, EnumJobs, and some forms of GetPrinterDriver are the main APIs that will see caching improvements.

    Overview of CSR caching behavior

    The CSR provider maintains a cache of print connection data for remote connections. This cache is persisted in the registry and loaded at provider startup.  The cache follows a “pull” model and in general operates asynchronously.  Calls requesting remote connection data are serviced from the local cache immediately where possible, and an asynchronous work item is queued that will synchronize with the remote server if necessary. Only in the case where the remote data is not cached locally will the call go over the network synchronously. The asynchronous synchronization with the remote server is done only periodically; most cache entries have an expiry time of 120 seconds. Cache coherency is enforced by checking Change IDs associated with both the local cache and the remote connection. If a request is made for cached data last accessed over 120 seconds ago, the remote Change ID will be compared to the local one, incurring a single RPC call of overhead. If they differ the cache will be updated from the remote server, incurring one or more RPC calls.

    Caching behavior of common APIs

    The table below shows the caching characteristics of common print APIs. The cache refresh interval for each caching mode is described in the next section.

    Caching behavior of key print APIs

    API

    Caching Behavior

    Caching Mode

    Changed from previous versions ?

    OpenPrinter

    Cached.  Client spooler still initiates network traffic for “Set” API calls and opens the remote printer handle when absolutely necessary.  OpenPrinter learns the server response per access mask; this it may block while populating the cache with results for masks it hasn't seen before. 

    Normal

    Yes

    EnumJobs

    Cached. Certain operations, like creating a job, cause an update of the EnumJobs cache.

    Volatile

    Yes

    GetPrinterDriver

    Cached when there is a printer connection to the target server. 

    Normal

    Yes, but only for level 8 calls

    EnumForms

    Cached for both server and printer handles when there is at least one printer connection to the target server. Not cached otherwise.

    Normal

    No

    EnumPrinterDrivers

    Not cached.

    Normal

    No

    EnumPrintProcessorDatatypes

    Cached.

    Normal

    Yes

    GetPrinter

    Cached when there is a printer connection to the target server.  Not cached otherwise.

    Normal

    No

    GetJob

    Not cached.

    Normal

    No

    EnumPrinters

    Cached for Level 4.  Level 4 only returns data when there are print connections.

    Normal

    Yes

    EnumPorts

    Not cached. 

    Normal

    No

    OpenPrinter2

    Cached based on flag.  Calls with ‘NO_CACHE’ flag will not be cached.  Calls without the ‘NO_CACHED’ flag will be cached.

    Normal

    No

    Registry keys for cache refresh intervals

    Two registry key values control the cache refresh behavior for print APIs. Caching mode “Normal” is controlled by the following registry key value and defaults to 120 seconds:

    HKLM\Software\Policies\Microsoft\Windows NT\Printers\SyncCacheAgeUser

    Caching mode “Volatile” is controlled by the following registry key value and defaults to 0 seconds:

    HKLM\Software\Policies\Microsoft\Windows NT\Printers\SyncCacheAgeVolatile

    -Bernd Hogen