Thoughts from the EPS Windows Server Performance Team
Useful Microsoft Blogs
In recent months we have seen an increasing volume of print cluster cases being opened due to poor performance. For example slow failover or spooler crashes. This article is provided as a guide for print server administrators to identify and resolve possible causes of these performance issues.
1. To increase print server stability, only the print drivers that are needed should be installed. Print drivers should be installed on the cluster print resource. Print drivers should not be installed on the local physical nodes of the cluster unless it is required by the printer vender. HP Universal Print Driver and the Xerox Global Print Driver and other Universal print drivers are a few of the print drivers that must be installed on the physical local nodes and on the Cluster print resource.
HP universal print driver configuration in clustered environment Best practices http://h20338.www2.hp.com/Hpsub/downloads/HP_Drivers_in_Cluster_Environments.pdf
From the HP web page above:
“When installing product specific drivers, it is recommended that the latest driver be used. The latest version of the driver can be found at hp.com. Also, the driver should be installed onto every node in the cluster prior to being installed onto the Virtual Print Server”
Xerox installation document: http://download.support.xerox.com/pub/docs/GLOBALPRINTDRIVER/userdocs
You will need an account with Xerox to access the link above and obtain the documentation referenced below.
“The X-GPD must be installed on the each node prior to being installed on the client”
2. There should not be multiple versions of the same HP Universal Printer Driver on the local nodes or the spooler resource, Example: versions (5.0) and (5.1) ,
The installed drivers can be found by using Regedit.exe:
HKLM\Cluster\Resources\<PrintGUID>\Parameters\Environments\ Windows NT X86\Drivers HKLM\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows NT X86\Drivers
3. Ensure the newest version of your Universal Print Driver which is installed on the spooler resource matches the version installed on the local nodes (should be at least 5.1 for HP UPD). Remove older versions of the UPD on the spooler resource and local nodes after changing all Print queues to the newest version. The removal can be done by going into the Print Server Properties under the Drivers tab.
Note: Print Server Properties can be accessed on 2008 and earlier by going into Printers and Faxes or Control Panel : Printers, right-click an empty area where the print queues are listed. Select Server Properties from the menu.
On Windows 7 And Windows Server 2008 R2, go into Printers and Faxes, select a print queue and the Server Properties option will be available in the toolbar.
If possible, all Version 2 (Kernel Mode Print Drivers) originally designed for NT 4 systems should be replaced with Version 3 print drivers. Version 3 drivers are recommended over Version 2 drivers because they support advanced printing features and improved performance. If a server stops responding as a result of a Version 3 driver problem, you only need to stop and restart the print spooler instead of restarting the server. Additionally, Version 3 drivers can reduce printer downtime, because Windows Server 2003 provides automatic recovery of the spooler service by default. If a Version 2 driver stops responding, you must reboot the server, taking all other services offline while the server restarts. In addition, Version 2 printer drivers operate in kernel mode (inside the operating system kernel memory space) and are designed for Windows NT 4.0. Because of this, if a Version 2 driver crashes, it can effect the stability of the whole kernel instead of just the spooler process.
There are 3 ways to determine if there are Version 2 drivers running in your environment:
A. Open Regedit.exe and check HKLM\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows NT X86\Drivers and if there is a Version 2 folder with drivers listed under it then you have some installed.
B. On a spooler resource you will need to check HKLM\Cluster\Resources\(GUID) of the spooler resource\Parameters\Environments\Windows NT X86\Drivers:
C. Alternatively you can check for version 2 drivers by opening Print Server Properties and clicking on the Drivers tab. Under the Version column Version 3 Drivers will show Windows 2000 and newer; however if it is a version 2 driver it will list Windows NT in the version column. Version 3 printer drivers operate in user mode (outside the operating system kernel memory space) and are designed for Windows 2000, Windows XP, and Windows Server 2003
5. Removing drivers on the spooler resource that are not currently assigned to a print queue can help spooler stability. As a general rule of thumb the fewer drivers in your environment the more stable and the faster the Cluster Print spooler can fail over. To cleanup unused drivers: Note: Do not do this if you are using Terminal Services with Printer Redirection.
1. Log in as Local Administrator 2. Open the Printers (Printers and Faxes) 3. Click Server Properties 5. Select the Drivers tab 6. Highlight all of the Drivers 7. Select remove (You will Need Admin Rights for this step) 8. This will only delete unused drivers, for any driver being used by a Print queue you will receive a popup stating the driver is in use and cannot be deleted. 9. Click past all of the popups and you will be left with only the drivers assigned to a print queue and all unused drivers will now be gone. 10. Open regedit (e.g. click Start, type regedit and press Enter) 11. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows NTx86\Drivers 12. Verify the Drivers are no longer listed in the registry. 13. Restart the spooler.
Note: An alternative to Microsoft Print Clustering is Windows 2008 R2 Clustered VM. You can cluster a Print Server VM and create Snapshots. It is a nice alternative given printer vendors do not always test their print drivers on a Microsoft Print Cluster. Clustered VMs is a new Hyper-V feature called Live Migration in Windows Server 2008 R2, which requires the failover clustering feature to be added and configured on the servers running Hyper-V. Hyper-V and failover clustering can be used together to make a virtual machine that is highly available, thereby minimizing disruptions and interruptions to clients. Live migration allows you to transparently move running virtual machines from one node of the failover cluster to another node in the same cluster without a dropped network connection or perceived downtime. In addition, failover clustering requires shared storage for the cluster nodes. This can include an iSCSI or Fiber-Channel Storage Area Network (SAN). All virtual machines are stored in the shared storage area, and the running virtual machine state is managed by one of the nodes.
Windows Server 2008 R2 & Microsoft Hyper-V Server 2008 R2 - Hyper-V Live Migration Overview & Architecture.
If you’re configuring a 2008 R2 Print cluster it is highly recommended the hotfix below is installed.
Stability update for Windows Server 2008 R2 Failover Print Clusters http://support.microsoft.com/kb/976571
The 5 Steps I have outline in this document may not resolve all of the issues in your clustered Environment, however If you are experiencing slow failovers or spooler resources crashing this may be a very good place to start.
Why is it that there is only ever a single print spooler on any given server? It would be far better if multiple spooler instances could be used and new printers grouped to use the spooler instance of choice. We run multiple multi-node print clusters to mitigate our downtime and thoroughly test print drivers, but even so there are instances when a spooler hangs due to a single fault with one print or device and requires an outage to all queues on that server. Perhaps this could be suggested as an enhancement?
What is the direction on PCL5, PCL5e and PCL6 drivers in a print server or print cluster? Great post!
Thanks for the tips. Why is it the vendors aren't encouraged more strongly to fix their stuff? These two little nuggets are so agitating:
" Print drivers should not be installed on the local physical nodes of the cluster unless it is required by the printer vender. HP Universal Print Driver and the Xerox Global Print Driver and other Universal print drivers are a few of the print drivers that must be installed on the physical local nodes and on the Cluster print resource. "
" It is a nice alternative given printer vendors do not always test their print drivers on a Microsoft Print Cluster."
These aren't small companies - can't partnerships/certification/WHQL or something be used to ensure their drivers actually line meet standards that have been in place for 3-4 years (I'm assuming OEMs like HP had early access to Longhorn)?
Hi Richard, in response to your comment: There are instances when a spooler hangs due to a single fault with one print or device and requires an outage to all queues on that server. Perhaps this could be suggested as an enhancement?
The enhancement you’re speaking of is actually included in the 2008 R2 Operating system "Print Driver Isolation” is included in 2008 R2 and is intended to fix that issue.
Print Driver Isolation allows a problem print driver to run in its own svchost so if the driver crashes the svchost it is running in is the only thing that goes down and the spooler process itself remain intact.
Note: Not all drivers support this functionality at this time as this is a new feature to 2008 R2.
As newer drivers are released by Vendors they are beginning to include the PDI functionality.
The Article below goes into the feature in depth.
you may find this (partially) addressed with the PrintIsolationHost feature. Please see www.sepago.de/.../printer-driver-isolation-in-windows-7-and-server-2008-r2
I know it's been a while since anyone has commented on the this post, but I support 45 printers on a newly created 2008 r2 print cluster. Intermittently I have users complain that it takes upwards of 20min before their job prints. Additionally on a regular basis we average 860 print jobs a day and I'm wondering if volume has anything to do with this issue. I attempted to install the hotfix, but since I've only recently setup my cluster, I assume that it's integrated into SP1 for Windows 2008 R2.
Thanks for your tips.
I have some similar problems with our print cluster. After we migrated the print cluster from win2003 x32 to win2008r2 X64, we found the failover took 2 min. I checked the event s by cluster management. It took 5s for the previous node to close the service. Then it took 5s to bring back the “storage” and “cluster group” in the next node. But after that we needed to wait 100s to let the cluster brought the service back. I didn’t find some error or warning events between them except the cluster couldn’t find the failed node.
Could you give me some suggestions?
I work in an environment where we have over 1000 queue's on our current 2008 R2 clustered print server. We are running the Xerox Global drivers. The problem we are encountering is that we need to update the drivers for fixes and new products. According to Xerox to update the drivers we would have to go through each queue that had that driver used and switch them to generic drivers. Then delete the driver and install the new version. Then go through and switch each queue back to use the new driver. that would be alot of work. Does anyone have a suggestion on how to handle this situation?