951557 (article not yet published)
Description of the Data Protection Manager 2007 hotfix rollup 3
Issues Resolved
- Recovery of SSL Sites does not work
- Consistency Check on compressed volume results in full Initial Replication
- Deleting old Recovery Points is not freeing space on Recovery Point Volume
- Consistency check does not complete in reasonable time
- User intent for Max Duration of Consistency Check job is not honored
- Increase verbosity in default DPM error logs*
- Restore security option for item recovery on the Recovery User Interface sometimes fails
This hotfix will not require reboot of the DPM Server. After installation on the DPM Server, there will be advice that Agent Updates are available in the DPM 2007 Administrator Conole. Installing the update from the console will indicate a reboot is necessary, but it is not. In other words, in the "Enter Credentials and Reboot option" check "Manually restart" but a restart is not required.
To confirm installation on the Protected Server, this event is logged in the Application log.
Event Type: Information
Event Source: MsiInstaller
Event Category: None
Event ID: 1022
Date: 5/14/2008
Time: 12:27:27 PM
User: NT AUTHORITY\SYSTEM
Computer: TX1MR2G
Description:
Product: Microsoft System Center DPM Protection Agent -
Update 'Microsoft System Center DPM Protection Agent Update -
KB951557' installed successfully.
This article is not yet public
contact Microsoft to receive this package
http://support.microsoft.com/common/international.aspx?RDPATH=dm;en-us;select&target=assistance&c1=509
Author:
Tom Acker
Microsoft Corporation Enterprise Support
Have you seen the new Microsoft Update Catalog? Check it out!
It gives you the ability to download as many updates as you need for various operating systems in a matter of minutes (or as fast as your download speed will allow).
http://catalog.update.microsoft.com/v7/site/home.aspx
Enter KB article number for the update into the search field and click Search.
The search results are then displayed. Click on the Add button next to the updates you wish to download.
Repeat as many times as you would like for as many different updates as you would like.
When you’re done, click on View Basket.
Remove any updates you no longer want or click Download.
Provide download location.
Receive nice pretty progress bars for each update which you can cancel at any time.
Then you’re done:
Install to the appropriate operating system at your leisure.
Enjoy!!
Author:
Charity Shelbourne
Support Escalation Engineer
Microsoft Corporation Enterprise Support
This blog entry covers a issue that some customers have encountered with the release of SP3 for Windows XP. The service pack supports the use of the /integrate command to integrate(or also called slipstream) SP3 into a Windows XP flat. It is supported to run the command with the following media
Windows XP Professional RTM
Windows XP Professional SP1
Windows XP Professional SP2
The problem arises if the /integrate command is run under Windows Vista or Windows Server 2008. The /integrate process will not report any error and appear to work but if you try to install from the media your product key may not be accepted. Luckily the fix is easy. Run the /integrate command under Windows XP or Windows Server 2003. For more information around this issue see http://support.microsoft.com/default.aspx?scid=kb;EN-US;950722
Author:
Scott McArthur
Support Escalation Engineer
Microsoft Corporation Enterprise Support
The DPM agent is a DCOM application - it cannot function correctly without proper DCOM configuration. This section covers the necessary configuration checks to make sure that DCOM is configured correctly.
Non-Clustered Member Servers:
As a rule, DCOM Settings for a working DPM 2007 Agent are evidenced by a lack of problems: You'll need to check Event Viewer, Add\Remove Programs, DCOMCNFG, and WBEMTEST in order to confirm the COM is working.
Event Viewer:
Make sure that there are no VSS or DCOM errors showing up in the event logs on the agent and the DPM server.
Here are some examples of what you might see on your DPM Server if DCOM needs to be checked out....
5/6/2008 8:46:46 AM 1 0 10006
DCOM N/A
<<DPMserverName>> 2148007941 <<DPM serverName fqdn>>
{C4EBD674-1457-4B79-BE30-B04735AED9D1}
Event Type: Error
Event Source: DPM-EM
Event Category: None
Event ID: 2
Date: 5/8/2008
Time: 9:02:00 AM
User: N/A
Computer: <<DPMserverName>>
Creation of recovery points for <<Protected ServerName>> on
<<Protected ServerName fqdn>> have failed.
The last recovery point creation failed for the following
reason: (ID: 3159)
The replica of <<Protected ServerName>> on
<<Protected ServerName fqdn>> is not consistent
with the protected data source. (ID: 30301)
DCOMCNFG:
Open DCOMCNFG, by clicking on Start\Run, and entering ‘DCOMCNFG’. Navigate down to
Console Root\Component Services\Computers\My Computer and open the properties of ‘My Computer’.
Note: If 'My Computer' has a red 'X' by it and the properties cannot be fully displayed, check the MSDTC (Distributed Transaction Coordinator) service to make sure it is running. If it doesn’t start, correct this problem before continuing. If the protected computer is a member server or workstation, try starting it and setting the service to 'Automatic' and rebooting. After logging onto the machine, open DCOMCNFG and 'My Computer' should appear without a red 'X' by it. If the system is a clustered server, make sure the cluster service is running as this will affect the MSDTC service's ability to run. MSDTC doesn’t, however, have to be a clustered resource in order for this to work correctly.
Check also for the existence of 2 DCOM Applications that are installed when the DPM Agent (recall that the DPM Agent is often called the "DPMRA") is installed on a server or workstation. Following are the settings for the DPMRA and WSSCmdletsWrapper DCOM applications.
‘DPM RA’ DCOM Application
Navigate through the "DCOM Config" tree to the DPM RA Service, as below, and select properties.
1. Under 'General', "Authentication Level - Default"
2. Under 'Location', only "Run application on this computer" should be checked
3. Under Security, verify that the "Launch and Activation Permissions" (select "Edit") include the machine account for the DPM Server as shown below.
4. Under Endpoints, leave "… default system protocols …" as the only entry.
5. Under Identity, the radio button "The system account (services only)." should be checked.
WSSCmdletsWrapper
This DCOM application appears under the same tree as above (My Computer / DCOM Config / WSSCmdletsWrapper / Properites). The related application is C:\Program Files\Microsoft Data Protection Manager\DPM\bin\WSSCmdletsWrapper.exe.
1. Under 'General', "Authentication Level - Default"
2. Under 'Location', only "Run application on this computer" should be checked
3. Under Security, you should see
4. Under Endpoints, leave "… default system protocols …" as the only entry.
5. Under Identity, the radio button, "The launching user" should be checked.
If you're concerned that the DCOM permissions have been modified, check the following registry key on both the protected server and the DPM server. There are up to 4 registry values added at the root of the OLE key.
HKLM\Software\Microsoft\Ole – check for the existence of the following value: ‘EnableDCOM’[REG_SZ] = ‘Y’ if not present, or if it is set to ‘N’ – either add the value or change it to ‘Y’. This is the registry storage location which tells if DCOM is enabled or disabled.
To reset the DCOM security, at the OLE registry key, delete all keys except for 'Default' and 'EnableDCOM'. Now reboot the server. When the server comes back up, the DCOM security will be at the default and you will need to open up DCOMCNFG in order to reconfigure it.
NOTE: Please backup the OLE key before you delete it. This will not cause the server to fail to boot but having a copy of the key will allow you to return it to its previous configuration if needed or desired.
The DPM Agent requires Distributed COM to work and does not require COM+ for agent installation or normal operation. Agents can be deployed on to systems that do not have the "Enable network COM+ Access" enabled within Add\Remove Programs.
Local DCOM Settings for DPM Agents are shown below.
1. Check to make sure that DCOM is enabled on the server that is going to be protected by DPM. This can be done in 2 ways. From within the UI, check the following in DCOMCNFG.
Open DCOMCNFG, by clicking on Start\Run, and entering ‘DCOMCNFG’. Navigate down to
Console Root\Component Services\Computers\My Computer and open the properties of ‘My Computer’.
On the "Default Properties" tab, "Enable Distributed COM on this computer" should be checked.
2. Check to make sure that the "Default Authentication Level" is set to "Connect" and “Default Impersonation Level" is set to "Identify".
3. The other option is to check the registry. Look for the key "HKLM\Software\Microsoft\Ole\EnableDCOM = Y"<REG_SZ> – check for the existence of this key and if not present, or if it is set to 'N' – either add the value or change it to 'Y'.
4. Back to the GUI on the Default Protocols tab - confirm that "Connection-oriented TCP\IP" is at the top. Also check the properties for DCOM port range to see if there are any restrictions (by highlighting "Connection-oriented TCP\IP" and selecting properties). Most likely, the screen will be blank, as below.
There are several registry settings that control the DCOM port restriction functionality. All of the named values listed below are located under the
HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\Internet
registry key. This registry key does not appear in the registry by default and must be created. If you find a need to restrict the DCOM ports, you should add the same port restrictions to both the DPM server and the protected server.
Note: You must reboot your machine any time you make changes to any of the following registry settings in order for them to take effect. Warning: Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall your operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk.
NAME TYPE
VALUE
Ports
REG_MULTI_SZ
Specify one port range per line. Example: 3000-4000 5141
DESCRIPTION: One or more port ranges. The options below determine the meaning of this named value.
PortsInternetAvailable REG_SZ "Y" (don't include quotes)
DESCRIPTION: Always set this to "Y".
UseInternetPorts REG_SZ "Y" or "N" (don't include quotes)
DESCRIPTION: If this value is set to "Y", then the Ports named value indicates which ports should be used for DCOM applications. If this value is set to "N", then the Ports named value indicates which ports should NOT be used for DCOM applications.
Examples:
1. Add the Internet key under: HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc
2. Under the Internet key, add the values "Ports" (MULTI_SZ),
"PortsInternetAvailable" (REG_SZ), and "UseInternetPorts" (REG_SZ).
In this example ports 5000 through 5100 inclusive have been arbitrarily selected to help illustrate how the new registry key can be configured. For example, the new registry key appears as follows:
Ports: REG_MULTI_SZ: 5000-5100
PortsInternetAvailable: REG_SZ: Y
UseInternetPorts: REG_SZ: Y
3. Restart the server. All applications that use RPC dynamic port allocation use ports 5000 through 5100, inclusive. In most environments, a minimum of 100 ports should be opened, because several system services rely on these RPC ports to communicate with each other.
You should open up a range of ports above port 5000. Port numbers below 5000 may already be in use by other applications and could cause conflicts with your DCOM application(s). Furthermore, previous experience shows that a minimum of 100 ports should be opened, because several system services rely on these RPC ports to communicate with each other.
Note: The minimum number of ports may differ from computer to computer and depends on the configuration of the computer.
To test and verify connectivity after making this registry change you can utilize the DCOMtest.exe tool. To download the tool and get additional details regarding the use and results you can leverage the following article.
KB259011 “SAMPLE: A Simple DCOM Client Server Test Application”
<http://support.microsoft.com/default.aspx?scid=kb;EN-US;259011>
For additional details on Using Distributed COM with Firewalls you can leverage the following links:
KB154596 “How to configure RPC dynamic port allocation to work with firewalls”
<http://support.microsoft.com/default.aspx?scid=kb;EN-US;154596>
<http://msdn2.microsoft.com/en-us/library/ms809327.aspx>
4. Check the COM Security tab for the following settings. The 'Edit Default' settings do not have to be modified.
On the COM Security tab, under Access Permissions, click Edit Limits.
Verify that the “Distributed COM Users” group is allowed both Local Access and Remote Access permissions. This is the group where the DPM Server's machine account has been added. This group is available after SP1 for Windows Server 2003 is applied, and after Windows XP SP2 has been applied.
after checking "Edit Limits" you'll see
6. On the COM Security tab, under Launch and Activation Permissions, click Edit Limits.
Verify that the Distributed COM Users group is allowed the following permissions:
· Local Launch
· Remote Launch
· Local Activation
· Remote Activation
Be sure to run secpol.msc and make sure that local security policy is not preventing network access.
1. Check ‘User Rights Assignment’ to confirm that the DPM Server’s machine account
or authenticated users is specified here.
2. Make sure that “Deny access to this computer from the network” does not include
the DPM Server's machine account, Everyone, Administrators, or Authenticated Users
as these will prevent the DPM Server from connecting.
Troubleshooting Security is beyond the scope of this article, but for a good reference, check out:
909887 Error message when you try to view a Web site that is hosted on Internet
Information Server 6.0 by using anonymous access: "401.1 Unauthorized: Logon failed"
http://support.microsoft.com/default.aspx?scid=kb;EN-US;909887
If Agent issues have been preventing consistency checks and/or replication, you may need to now reboot, uninstall the DPMRA, reboot, and reinstall.
Authors:
Victor Reavis
Tom Acker
Microsoft Corporation Enterprise Support
The following is a comprehensive list of operating systems that are supported as protected machines and/or meet the requirements to run System Center Data Protection Manager (DPM) 2007.
Important: DPM supports 32-bit and x64-bit operating systems. DPM does not support ia64-bit operating systems.
Wndows 2008 versions require pre-installation of KB950082
Hotfix for Windows Server 2008 support in DPM 2007
this article is not yet public
contact Microsoft to receive this hotfix
http://support.microsoft.com/common/international.aspx?RDPATH=dm;en-us;select&target=assistance&c1=509
Overviews of Windows editions are available at the following sites.
http://www.microsoft.com/windowsserver2008/en/us/editions-overview.aspx
Windows Server 2008
http://www.microsoft.com/windows/products/windowsvista/editions/default.mspx
Windows Vista
http://www.microsoft.com/technet/windowsserver/evaluate/features/compare.mspx
Windows Server 2003
Authors:
Thomas O'Malley
Christopher Allen
Support Escalation Engineers
Microsoft Corporation
Note: This is the second of a three part series for troubleshooting the DPM RA (Data Protection Manager Remote Agent). Please check out the earlier blog, and the next (all start with the same title).
Network security protocols between the DPM Server and the Agent can directly affect Agent communication as can DNS configuration and registration issues. Check these settings on each server to ensure that there are no network issues blocking the agent and server from communicating with each other.
1. Check SMB (Server Message Block) signing on the DPM and PS (Protected Server) servers and verify the following settings on both servers.
Windows XP and 2003 - local computer Group Policy
Workstation/Client:
Microsoft network client: Digitally sign communications (always) Security Setting: disabled
Microsoft network client: Digitally sign communications (if server agrees) Security Setting: enabled
Server
Microsoft network server: Digitally sign communications (always) Security Setting: disabled
Microsoft network server: Digitally sign communications (if client agrees) Security Setting: disabled
2. Check and make sure that Windows Firewall is disabled on both the DPM and PS servers. If it must be enabled, make sure that the following ports are open. (More information in 947682 - http://support.microsoft.com/kb/947682). Also, if it must be enabled, make sure it is enabled before installing the agent as the DPM installation process automatically configures the firewall exclusions to allow DPM agent communication.
a. 53 UDP
b. 88 TCP\UDP
c. 135 TCP
d. 137 UDP
e. 138 UDP
f. 139 TCP
g. 389 TCP\UDP
h. 5718 TCP (DPM)
i. 5719 TCP (DPM)
j. Add the DPMRA application to the exclusion list.
3. Make sure that there are no firewalls in between the DPM and protected servers and if there are, make sure the exceptions specified above have been applied.
4. If a Netlogon Event error is displayed in the System Event log or the below error in the DPM Administrator’s Console, make sure that the DPM Agent’s machine account exists in the domain and has not been reset. On the DPM Agent computer, check the System Event log for an Event 5721 from Netlogon indicating what the account problem is.
- If the machine account has been deleted, correct this by recreating the account and disjoining and rejoining the computer to the domain.
- If the machine account has been reset, then SID in the account will match the machine's SID and you can use NetDOM or NLTEST to reset the machine account's secure connection to the DCs.
Data Protection Manager Error ID: 318
Error: The agent operation failed because DPM was unable to identify the computer account for: <FQDN for agent machine>
Detailed Error Code: No mapping between account names and security IDs was done.
Recommended Action: Verify that both <FQDN> and the domain controller are responding. Then in Microsoft Management Console (MMC), open the Group Policy Object Editor snap-in for the local computer and verify the local DNS client settings in Local Computer Policy\Computer Configuration\Administrative Templates\Network\DNS Client.
5. If DCOM appears to be OK after checking WBEMTEST and DCOM sections (to follow, below) and we are still seeing what appears to be network communications issues, try pinging the DPM server from the protected server and vice versa by FQDN as well as IP. If this works, check to see if there have been any manual updates to the HOST file on the machines in question. Also check, using NSLOOKUP, to ensure the protected server is dynamically updating the DNS servers properly as this will also cause agent installation\communication to fail.
.
WBEMTEST:
WBEMTEST is a tool we can use to test connectivity between the DPM Server and the DPM Agent system. We will use WBEMTEST to test our DCOM connectivity.
Logon the DPM server using the same credentials you are using to deploy the DPM File agents. Try to connect using WBEMTEST from the DPM Server to the Server you plan on installing the DPM Agent onto. If you received the following error, please verify that the DCOM settings are correct.
Number: 0x80070005
Facility: Win32
Description: Access is denied
To test the DCOM connectivity via WBEMTEST, click on 'Start\Run' and type 'wbemtest' and hit enter. The following screen appears.
Click on the "Connect" button and the following screen will appear with 'root\default' in the 'Namespace' field. Type in the name of the DPM Agent machine and specify the 'root\default' namespace to connect to.
The UNC-style connection will initiate a DCOM connection from the DPM Server (local) to the DPM Agent Server (remote) and try to open up the 'default' namespace. If this fails, then there are still DCOM issues on the system.
A working example appears below.
Once a connection has been made, the button appear 'live' as in the screen shot above. Now click on the 'Enum Classes…' button at the top of the left-hand column.
Once this screen appears, you can specify a class name if you know one but the easiest confirmation is done by selecting the 'Recursive' button and clicking on OK. This will show all of the top-level classes in the Default namespace. Once you have this information, you have confirmed that DCOM works. The below screen shot is an example of what a successful recursive query will return.
Cluster Nodes:
The Cluster Service must be running in order for the DPM Agent installation to proceed. Without the Cluster service running, MSDTC will not be accessible and this will affect the agent installation of DCOM. There doesn’t need to be an MSDTC resource in the cluster for the DPM Agent installation to work.
All other settings for local security and DCOM are the same as the Non-Clustered Member Server settings above.
Author:
Victor Reavis
Support Engineer
Microsoft
Microsoft’s Data Protection Manager (DPM) version 2007 must have an agent installed on any server or workstation that it is going to protect. The installation of the DPM agent requires contact with many different components both locally and across the network. In this and the next two blogs, we will try to cover as thoroughly as possible, all of the local and remote contingencies that must be met in order for the DPM agent to be installed successfully and establish communication with the DPM Server.
These will include the specifics regarding Manual Agent Installation, Local Security on the DPM Agent, Network configuration, WBEMTEST, and DCOM configuration settings in a attempt to provide you with the most common causes of why DPM Agents either will not install or are not able to communicate with the DPM Server after installation. In this blog we'll cover:
- Manual Agent Installation
- Local Security on the DPM Agent
- Domain Controller requirements
Manual Agent Installation: When working on a DPM Agent Installation issue, consider some of the following tests to see where the problem may reside.
1. Try installing the agent manually on the server to be protected and try pushing it from the DPM Server. Do both error out or does one succeed when the other doesn't? If the manual agent installation succeeds and the push fails, then there is likely a problem on the network or with network communication between the DPM Server and the Protected Server or some necessary configuration in the environment with AD or Name Resolution (DNS) that is missing.
2. Try to add the agent server to the DPM Server using both SetDPMServer* as well as the PowerShell script (Attach-ProductionServer.ps1) on the DPM Server.
The SetDPMServer’s syntax is a follows:
SetDPMServer <DPM Server name>
for example,
SetDPMServer DPM_Server_01
*Note: The SetDPMServer tool is found on a DPM Agent under the ?:\Program Files\Microsoft Data Protection Manager\DPM\bin\ folder and it is run locally on the system.
The PowerShell script’s syntax below is followed by an example command.
Syntax:
Attach-ProductionServer.ps1 <DPM server name> <production server name> <user name > <password > < domain>
Example:
Attach-ProductionServer.ps1 dpmserver SQLProd1 Admin_VR P@ssw0rd1! Contoso
Use the above power shell script to attach a manually installed agent on a protected server to the DPM Server. Once complete, refresh the agent display in the DPM Console to see if the agent is showing healthy or if an error appears.
3. Disable everything non-essential using MSConfig to see if this will get around the installation issue. This would include anything 3rd party under the ‘Services’ or ‘Startup’ tabs. Clean the agent off of the server and attempt the installation again after the server is rebooted. The agent should be uninstalled using Add\Remove Programs.
4. Remove leftover DPM keys in the registry after removing DPM client using Add\Remove programs. Primarily, the DPMRA keys should be removed. A simple search through the registry should reveal the following locations:
HKLM\Software\Microsoft\Microsoft Data Protection Manager
HKLM\System\CurrentControlSet\Services\DPM** (DPMRA, DPMLA, DpmWriter)
After cleaning up the registry, also delete the "DPMRADmTrustedMachines" and "DPMRADCOMTrustedMachines" groups and remove the DPM server's machine name from the "Distributed COM Users" group. These groups are created during the DPM Agent installation and the DPM Server’s machine account is added to the "Distributed COM Users" group during the agent’s installation. If the local groups cannot be created or any of the three groups populated with the DPM Server’s machine account, then the agent cannot be managed by the DPM Server.
Local Security: In this section of the DPM Agent installation, we discuss the local security modifications that need to take place in order for the DPM agent’s installation to complete successfully and allow the DPM Server to manage the agent. We will discuss the local security settings on non-Domain Controllers first as this is the foundation from which we will make our modifications when discussing how the agent installation is different on Domain Controllers and Failover Cluster nodes.
Non-Domain Controllers:
The DPM Agent installation adds the following locally to the system.
1. DPMRADCOMTrustedMachines local group with the DPM Server’s machine account as the only member.
2. DPMRADmTrustedMachines local group with the DPM Server’s machine account as the only member.
3. The DPM Server’s machine account is added to the Distributed COM Users group.
Check the following settings on the machine where the DPM agent is being installed.
4. The ADMIN$ share on the Protected Machine must be accessible from the DPM Server using the account that you are planning to install the agent with.
5. Make sure that the Local Policy "Access this computer from the network" includes the <DPM Server Machine Account> account and\or "Authenticated Users"
This user right determines which users and groups are allowed to connect to the computer over the network. Terminal Services are not affected by this user right.
Default:
On workstations and servers:
Administrators
Backup Operators
Power Users
Users
Contoso\DPMServer < - - Must be added manually
On domain controllers:
Administrators
Authenticated Users
Contoso\DPMServer < - - Must be added manually
6. Make sure that the Local Policy "Deny access to this computer from the network" does not include the DPM Server's machine account, Everyone, Administrators, or Authenticated Users as these will prevent the DPM Server from connecting.
7. Delete, if it already exists, the leftover ‘Microsoft Data Protection Manager’ folder under C:\Program Files. If you cannot delete the folder, use a tool like Handle.exe or Process Explorer to see what has it locked.
8. Confirm that the PageFile is on the C:\ drive. Not a DPM requirement but has been seen on problem machines. For example, the agent install logs might show that the installation is trying to run on another drive on the system like D:\. After each failed attempt, there is a folder left behind with an alpha-numeric string but no indication that it is DPM related until you browse it’s contents.
9. Run the ‘Set’ command to see if the machine is logged into the domain. Log into the domain if you are not already. Specifically, check the ‘USERDOMAIN=’ value to confirm it contains a domain name and not the local system’s name.
Domain Controllers
Since there are no local accounts or groups on Domain Controllers, the location of the 'Distributed COM Users' group is found under 'Active Directory Users and Computers' snap-in. Look under the 'Built-in' node for this group. At the domain level, the machine account for the DPM Server must be added in order for the agent to be installed on Domain Controllers.
DPMRADCOMTrustedMachines and DPMRADmTrustedMachines local groups cannot exist on any Domain Controllers because they don't have a local accounts database. Manually create these groups in the domain with a "Global Group\Security Group" context and add the machine accounts for any DPM Servers that will manage the agents. It is recommended to only specify 1 DPM Server and have that DPM server protect all Domain Controllers to avoid agent issues.
Next, we'll discuss troubleshooting Networking and WBEMTEST followed by DCOM.
Author:
Victor Reavis
Support Engineer
Microsoft Enterprise Support
Introduction
Microsoft System Center Data Protection Manager (DPM) 2007 is a key member of the Microsoft System Center family of management products. DPM is the new standard for Windows backup and recovery, delivering seamless data protection for Microsoft application and file servers by using integrated disk and tape media. DPM performs replication, synchronization, and recovery point creation to provide reliable protection and rapid recovery of data for both system administrators and end-users.
For more information about DPM, please see http://technet.microsoft.com/en-us/library/bb795549.aspx
Basic Questions and Data Gathering for Troubleshooting DPM Issues
To help with troubleshooting DPM, I developed some basic questions and provided recommendations. As a DPM Administrator you would want to know all of the following:
• What release of System Center Data Protection Manager are you running (2006 or 2007)?
• Is this a fresh installation of DPM 2007 using the RTM bits or was this an upgrade from Beta2?
• What is the build number? You can click the circled "i" next to the Management tab on the DPM Administrator to view the version.
• Is the DPM server an x86 or x64 machine?
• What operating system version is running on the DPM server 32bit or 64bit?
• What operating system version is the protected machine running 32bit or 64bit?
• Are any DPM patches installed?
• When did the problem first start? You can select the Monitoring tab and review the Alerts/Jobs for details.
• Has it ever worked as expected?
• What changes were made just prior to the failures?
• Can you reproduce the problem? Note the exact repro steps.
• Are other protected data sources experiencing the same problem?
• Is the error specific to one type of data source? For example, Exchange jobs fail but SQL and SharePoint are successful.
• What is the application version that is experiencing the problem? Example, SQL 2000 SP4, SQL 2005, Exchange 2003…etc
• Is the protected data source running on a standalone server, domain controller or a cluster?
• Is the system that is experiencing the problem in the same domain as the DPM server?
• Do other protected data sources reside on the same machine? Are they also failing?
• Does the issue impact a single data source/single protected machine or multiple data sources/multiple protected machines?
• Are any other applications installed on the DPM server? DPM is intended to be a dedicated server.
• If jobs fail at a specific time, are there other scheduled services running at the same time? We have seen virus scans or third party backups causing DPM jobs to fail.
• Is the target machine experiencing on the same LAN as the DPM server or over a WAN? It’s always beneficial to know the location of the problem machine in relationship to the DPM server.
• What is the error message and ID in the Details pane? IMPORTANT, be sure to locate the first job that failed for a particular data source because it’s possible other failed jobs may be just a result of the initial failure.
If you need assistance resolving a problem with DPM, it is supported 24x7 by Microsoft Commercial Technical Support. For information on contacting Microsoft Technical Support, please see http://support.microsoft.com/contactussupport/?ws=support.
Here's information to have handy for your Support Professional
Data Gathering
• Get a screen shot of the error: this can be beneficial and supply a better understanding of the issue depending on the problem you’re troubleshooting. To locate a failed Job, click on the Monitoring tab, click the Jobs tab, locate and select the failed job so it’s highlighted. To get the complete error message, right-click in the Details pane and select “Copy details”. You can copy the details to a txt file or simply send you the details in an email. Additionally, you should check to verify if an Alert has been raised and view the Details for the Job.
• Is the issue regarding Archiving/Tape? If so, request the manufacturer's name and exact model of the tape drive and library/medium changer. Verify if the tape drive and medium changer are recognized correctly in Device Manager and know the driver details for each device.
• Check the event logs on DPM server and the target machine: Specifically around the time when the job fails. This can be especially useful in cases where an application specific failure has occurred. In our next DPM blog, we'll provide more information about events seen in the DPM Alerts Event Viewer Log.
• Data sources not being recognized when trying to add them to a Protection Group: Check the state of the application writer on the target/protected machine. You can run “vssadmin list writers” on the protected machine.
DPM Resources
• TechCenter: http://technet.microsoft.com/en-us/dpm/default.aspx
• System Center Data Protection Manager 2007 Product Documentation: http://technet.microsoft.com/en-us/dpm/bb931334.aspx
• DPM 2007 Error Code Catalog: http://technet.microsoft.com/en-us/library/bb795681.aspx
• Leverage the DPM TechCenter online resources: http://technet.microsoft.com/en-us/library/bb795539.aspx
Author: Tom O’Malley
Microsoft Enterprise Support
Support Escalation Engineer
1. Under no circumstance should any other resource be created in the Cluster Group other than the default ones which are Quorum, Cluster IP Address and Cluster Network Name. (ref: KB168948) If MS DTC is installed in the Cluster Group using the cluster’s resources and it’s not going to be used heavily by SQL, then this is the only exception that is supported as long as the MS DTC resource has the “Affect the group” box on the Advanced tab unchecked.
NOTE: Using the “Affect the group” box on the Advanced tab to uncheck this option before trying to bring any resource Online, is always a Best Practice tip to avoid unnecessary group failovers from one node to the other when the resource itself fails.
2. The most efficient way to create many file shares on a cluster is to create sub-folder shares, because this option can significantly reduce the number of resources and overhead. Using the same methodology as if you were creating user’s home folders. (ref: KB256926) This option also simplifies administration and disaster recovery. If you must use individual File Share resources for several hundred shares, it may be necessary to add more CPUs or memory to the server.
NOTE: When setting the permissions, do it on the File Share resource through Cluster Administrator to set share level permissions. Only domain level groups should be used in defining share level permissions, because local groups and user accounts do not reside on the other node, and the permissions will not take effect when the file share is failed over. The only exception to this is if all nodes in the cluster are domain controllers (which is not something we normally recommend due to the overhead on the DC). It is recommended for security granularity at the file level, to use NTFS permissions instead of share level permissions on a server cluster.
3. You can surpass the 26-drive-letter limitation, by using volume mount points feature. If you use the root (host) volume exclusively for mount points, the size of the host volume only has to be several MB. This reduces the probability that the volume is used for anything other than the mount points. This feature can also decrease the amount of time it can take to defragment a drive, backup/restore and run CHKDSK against a volume as oppose executing these tasks on a terabyte volume. (ref: KB280297)
4. The dependency hierarchy in cluster is important. Most resources should depend on Physical Disk and Network Name resource within its own group, especially Server Message Block (SMB) File Share resource which requires both a Physical Disk and Network Name dependencies. The only resource types that do not usually depend on another resource are the Physical Disk and IP Address resources. (ref: KB171791) The only exception of a Physical Disk having any kind of dependency, is when it’s a Mount Point that is dependent of the root Physical Disk coming Online first before the actual Mount Point. (ref: KB280297)
5. When configuring Microsoft Distributed Transaction Coordinator (MS DTC) on a cluster, make sure to create a separate MS DTC Group with its unique Physical Disk, Network Name and IP Address resources. (ref: KB301600).
NOTE: Before brining MS DTC resource online, you should confirm Enable Network DTC Access is enabled. (ref: KB817064)
Under some situations, applications that can heavily us MS DTC may require Enable Network COM+ Access to be enabled. (ref: KB817065)
6. You should not change the default privileges or set the Cluster Service Account (CSA) to be a member of the domain Administrators group. By giving the minimal possible user rights to the Cluster service account, you avoid potential security issues if that account is compromised. When you remove a required right from the CSA, you may cause unexpected behavior. (ref: KB269229) Before you apply more restrictive security settings to the Windows Server 2003-based and/or domain policies that apply to the cluster server nodes, you should read and apply certain recommended guidelines. (ref: KB891597)
7. When installing Cluster Services on the first node, make sure to read the article that talks about Recommended private "Heartbeat" configuration on a cluster server. Even though the article states the Private “Heartbeat” should be set at 10/Half Duplex, it is perfectly acceptable and supported to set at 100/Full Duplex. (ref: KB258750)
NOTE: Even though Auto-Detect/Auto-Sense is discouraged in a cluster environment, there are circumstances (like on brand new hardware such as certain blade servers) where vendors may ship network adapters that do not support manual settings. In these cases we have to suggest customer follow the adapter vendor's matrix, which sometimes they usually posted on their website. (ref: KB174812)
8. Make sure to download the latest drivers/firmware from the vendor’s website for the Cluster Solution. Especially, the latest drivers/firmware that will be used for the Cluster Solution such as network, fiber (HBA) and/or multi-path (MPIO) for the storage. (ref: KB814607)
9. Before creating a cluster, use the Microsoft Cluster Configuration Validation Wizard (ClusPrep) tool to validate that your system is configured properly by taking inventory of your system configuration and highlighting discrepancies in service pack levels, driver versions, etc.; evaluating and testing your network and storage configuration. (ref: KB933462)
10. Make sure to use a supported Cluster Solution from the Windows Server Catalog. (ref: KB309395 & KB828262)
For additional details regarding “Best practices for configuring and operating server clusters”, please click on the following link:
http://technet2.microsoft.com/windowsserver/en/library/2798643f-427a-4d26-b510-d7a4a4d3a95c1033.mspx
Author: Mike Rosado
Support Engineer
Microsoft - Windows Server - Enterprise Platforms Support - Core team (Setup, Cluster and Performance)
For those people that were unaware, the Windows group released a Product Activation Announcement regarding the changes and impacts to Windows Server 2008 method of activation. Since Windows Server 2008 uses the same activation method as Windows Vista, the error codes reported for Windows Server 2008 are the same used in Windows Vista as mentioned in the article below:
938450 How to troubleshoot Volume Activation error codes on Windows Vista-based computers
The new method of activation designed for volume license customers is Volume Activation (VA) 2.0. This method provides two types of customer-specific keys, namely Multiple Activation Key (MAK) and Key Management Service (KMS) Key. As stated in article ID: 938450, most of these errors would require customers contacting the Product Activation (PA) Call Center at either:
+1 (716) 871-2929 or toll free +1 (888) 571-2048 to validate, unblock and reissue the current KMS/MAK keys.
For more technical and planning information regarding Volume Activation 2.0, please click on the following link:
http://technet.microsoft.com/en-us/library/bb892849.aspx
Author: Mike Rosado
Support Engineer
Microsoft - Windows Server - Enterprise Platforms Support - Core team (Setup, Cluster and Performance)
Hello my name is Nitin Mishra and I am working with Platforms Core Team in Product Support Group. This article discusses an issue with latest Broadcom NC Series VBD drivers coming with DL360 G5 and DL380 G5, which prevent the Network Settings, provided in Unattend/sysprep answer file, to take effect. When attempting a unattended installation of Windows Server 2003 with this particular network card any settings specified in the unattend.txt will fail to apply due to how this network card uses a virtual bus driver (VBD). RESOLUTION
To use netset03.exe to specify network information as per KB 920293
Extract KB920293 and locate file netset03.exe. To extract the file to a location (e.g C:\Netset) use the following commandWindowsServer2003-KB920293-x86-ENU.exe /X
This will pop up the extraction wizard asking for a location to extract files. Once you have the netset03.exe file, create a file named netset.txt (You can choose any name).
Now copy all the adapter specific settings in this file and save.
Now you can use any one of the below mentioned methods to specify Network Settings:
1. Use [SetupParams] in unattend answer file to specify settings.
Create a Folder named Netset under $oem$\$1\Netset (This can be any name and not necessarily Netset). Copy netset03.exe and netset.txt file under
$oem$\$1\Netset Folder. Now add below entry in unattend.txt answer file.
[SetupParams]
UserExecute = “C:\Netset\Netset03.exe C:\Netset\Netset.txt”
You can add the above section just below [GuiUnattended].
OR
2. Use cmdlines.txt to specify settings.
Create a Folder named Netset under $oem$\$1\Netset (Again, this can be any name and not necessarily Netset). Copy netset03.exe and netset.txt file under $oem$\$1\Netset Folder. Now create another file named cmdlines.txt under $oem$ folder. Populate following entry in cmdlines.txt
[Commands]
“C:\Netset\Netset03.exe C:\Netset\Netset.txt“
If you want to install the NIC drivers along with this, you can use the following
[Commands]
“cmd.exe /c {location of nic driver executable} [/switch to make the install silent]”
“C:\Netset\Netset03.exe C:\Netset03.exe\Netset.txt”
The benefit of using cmdlines.txt is that it executes commands in a seqential order and the second command would wait for the first to finish before it can execute. In case of
SetupParams you can only run a single command. In case you’ve added more than one command, the second and subsequent will be ignored.
Note: In case you’re using Netset03.exe, make sure that you use PCIBusNumber, PCIDeviceNumber, PCIFunctionNumber to specify settings for each NIC. In case you’re using
the InfID to specify NIC settings you may see that NIC settings are not getting implemented. If you manually run that command after installation you will see below error:
Network Card {guid} not found in the answerfile.
Author: Nitin Mishra
Technical Lead
Platforms Core
It can be very frustrating getting a client machine to boot to ADS if things don't run perfectly and you aren't familiar with any of the troubleshooting methods. I thought it would be a good idea to go over some of the more common causes of PXE boot errors on the client machines and ideas on how to troubleshoot them.
PXE E53: No Boot FileName Recieved
and/or
PXE E55:ProxyDhcp service did not reply to request on port 4011
The first thing you need to check for is if the DHCP server is hosted on the same box that hosts ADS.
1. If DHCP and ADS are hosted on the same box you would need to configure both DHCP and ADSPXE service to be co-located on this box. For this purpose ADSDhcpconfig.wsf tool is provided along with ADS install files. you can locate this tool under %program files%\Microsoft ADS\bin
In order to co-locate ADS and DHCP services you need to run following on command prompt
adsdhcpconfig /add
Above command will give an output stating the ADSPXE and DHCP services are now configured to be co-located. Notice the DHCP scope after running this command. This commands enables scope option#60 and adds PXEClient string to DHCP scope.
If you are planning to move DHCP services to a different box and need to disable the co-location you can run the same tool again with remove switch.
adsdhcpconfig /remove
2. In case of cross subnet communication you need to ensure that the connecting router or switches have IPHelper enabled.
3. Ensure that there is no firewall blocking the port on ADS server.
4. Check if the client machine is added to ADS Console and you've taken control of the device.
5. If you've changed the IP of ADS server you can get this error message. You can contact Microsoft Support for getting ADSBind.exe tool to configure the ADS server to use the new IP.
6. If you've multiple NIC cards on ADS server, changing NIC card settings may leave stale entries in registry binding ADS services to previous MAC address. ADSBind.exe is your friend in such a situation.
More information on ADSBind.exe
On the controller run this command and reboot the controller:
adsbind.exe /A /B:a
This will set the services to the current IP address. Check the event log to make sure all services startup successfully
Note: If the ADS controller has multiple network cards its recommended to bind ADS to a single NIC only. The command you should use is:
adsbind.exe /N /B:a /ip:xxx.xxx.xxx.xxx
7. If none of the above resolve your issue, collect Netmon trace while reproducing the error.
You can download Netmon from below link
http://www.microsoft.com/downloads/details.aspx?FamilyID=18b1d59d-f4d8-4213-8d17-2f6dde7d7aac&DisplayLang=en
Author: Nitin Mishra
Technical Lead
Microsoft - Windows Server - Enterprise Platforms Support - Core team (Setup, Cluster and Performance)
Running a bit behind this week, but still getting our new articles posted. We have some interesting new Cluster articles coming out. I think they are worth looking over. Enjoy.
|
KB |
Title |
|
948874 |
Detection and deployment guidance for the February 12, 2008 security release |
|
948911 |
An external hard disk that is based on the IEEE 1394 standard is not detected on a Windows Vista-based computer |
|
948177 |
Windows Vista does not detect a DVD-R disc that was formatted two times |
|
945000 |
You may experience issues with the Windows Aero feature on a Windows Vista-based computer |
|
947247 |
Error message when you try to remove more than two images from a .wim file that contains four images in Windows Vista: "The data is invalid" |
|
947210 |
The upgrade process stops when you upgrade to Windows Vista by using a customized WIM file |
|
947711 |
GPT disks in a Windows Server 2003 server cluster are not migrated to a Windows Server 2008 failover cluster when you use the "Migrate a Cluster" wizard |
|
947709 |
How to use the “netsh advfirewall firewall” context instead of the “netsh firewall” context to control Windows Firewall behavior in Windows Server 2008 and in Windows Vista |
|
947710 |
Parallel SCSI support in Windows Server 2008 Failover Clusters has been removed |
|
947712 |