System Center Mobile Device Manager Team Blog : SCMDM

Troubleshooting device connection to the Device Management server

Jarrett Renshaw — Wed, 20 Aug 2008 00:25:00 GMT

You are enrolled. The device resets and shortly afterward you see the checked "V" icon, indicating you have a successful VPN tunnel to the MDM GAteway. Yes!

Now you wait for that PIN policy to come down, or that software package... and nothing happens.

Hmmm... are we REALLY connecting to the VPN server? The VPNDiag tool confirms that indeed we are.

At this point, the device has a VPN tunnel to the MDM Gateway, but something is wrong with the rest of the path to or from the Device Management server. (If VPN is not working, check out the Enrolled device cannot connect to Gateway topic the Technet troubleshooting documentation.)

If VPN is working but the client cannot connect to the DM, the client will not receive policy, software or Wipe Now requests. Until the device can contact the device management server and become managed, it remains in the Pending Enrollments node.

Troubleshooting

Test the DM server URL from the client and the DM server. You can find the URL in a number of ways to verify it is correct:

a. Device Status Viewer tool in the Client Resource kit. The URL is displayed on the home screen of the tool

b. \Deviceupdate.log on the device will show this if logging is enabled (see troubleshooting step 2 below)

c. The URL is also stored in the SCMDM2008DeviceManagement Active Directory SCP under ‘keywords’ > “url=”

d. You can easily enter the URL on the device or a test computer using the format:

https://DMServer.Mydomain.com:8443/MDM/TEE/handler.ashx

Enter this into Pocket IE on the device. You should get a “Choose a certificate” warning.

If the device cannot connect to the URL, you can also test this URL locally on the DM to narrow down the issue to connectivity or a problem with the server itself.

We’ll assume for now that the DM can connect to the URL using the localhost address so we know IIS and the server-side MDM services are working. Only devices cannot reach the DM. In this case, the issue is with the network, firewall, web certificate, or DNS.

The device might fail to contact the device management server for the following reasons:

1. It is necessary for the device to be able to resolve the FQDN of the DM server in DNS. Make sure there is a correct A record on the DNS server for the DM server. Depending on the network topology, DNS may be configured in different ways. If using the DNS server on the internal network, DNS traffic must be allowed on the internal firewall so the device can query the DNS server. If using a DNS server on the perimeter network, DNS must be configured to resolve the DM FQDN to the internal IP address of the DM.

2. The company firewall is blocking port 8443. For a list of ports required by SCMDM on the firewall see the planning guide.

3. The company proxy is blocking TCP port 8443 to the device management server. Allow tunneling on this port to allow the device to contact the device management server. This will only occur if the device connected successfully once before and received proxy policy. For details and resolution steps see Error 2147467259 When Synchronizing Policy in the Technet troubleshooting documentation.

4. A persistent route is needed on the MDM gateway to the corporate network through the back-end firewall, and another route is needed on the back-end firewall server to the VPN client address pool subnet through the MDM gateway. One simple method that may or may not satisfy your topology requirements is to configure the routes locally on the MDM Gateway and backend firewall servers as shown in this example:

Route #1 (on the GW): “route –p add <corporate subnet> mask <subnet mask> <Firewall IP>” which adds a route to the corporate network through the Back-end Firewall.

Route #2 (On the back-end firewall): “route –p add <Client pool subnet> mask <subnet mask> < Gateway IP>” which adds a route to the SCMDM 2008 client network through the SCMDM GW.

Note: If a Redirection Default Gateway is configured, MDM Gateway server will still prioritize any local static routes configured to particular destinations. If there is no such static route for the destination that you are trying to reach, then the packet will be forwarded to the Redirection Default Gateway.

5. It is also necessary to make sure the internal DNS server can access the Client VPN address pool on the MDM Gateway in order to forward resolved queries back to the devices. If the default gateway is, for example, the internal IP address of the back-end firewall, and the DNS server default gateway is not the internal IP address of the back-end firewall, a persistent (static) route may need to be configured as follows:

“route –p add <Client pool subnet> mask <subnet mask> < Firewall IP>” which adds a route to the SCMDM 2008 client network through the back-end firewall.

6. The DM’s web certificate subject name must match the FQDN of the DM server and chain to a valid root CA. Use the Best Practices Analyzer in the resource kit to check this.

These scenarios can vary, but the principle is the same

- Devices must have access to a DNS server to resolve the FQDN of the DM server

- There must be a route from the client VPN address pool on the MDM Gateway to the internal network

- There must be a route from the internal network to the Client VPN address pool on the MDM Gateway

- The internal DNS server must have access to the VPN client address pool subnet in order to respond.

- The DM certificate must be configured correctly and chain to the same root CA as the other servers – SCMDM only supports one root CA per instance.

Logging

Enable OMADM logging on the device using the Connect Now tool from the Client Tools in the MDM resource kit download. Issues contacting the DM server are logged in \deviceupdate.log.

The error codes in the log are Wininet errors, so you can look them up. Some that I have run across are:

Fail (-2147012889)

This error translates to ERROR_WINHTTP_NAME_NOT_RESOLVED and the symptoms are that we cannot connect to the internal network or the internet.

Check all the items above and make sure routing and DNS are configured properly. Also make sure port 8443 and Protocol 50 (IPSec ESP) are allowed both ways on the external firewall and Gateway Server’s Windows Firewall if it is enabled. This protocol allows browsing of internal and external web sites.

Failed sending an HTTP request to the server (0x80072f7d).

This can be caused by a problem with the certificate on the DM web site. Usually the log will show a successful connection to the server followed by the failure:

2008-07-17 15:58:38 omadmclient.exe: Establishing connection to https://dm.mdm.com:8443/MDM/TEE/Handler.ashx

2008-07-17 15:58:39 omadmclient.exe: [PID = 0x9e0b15d2] + Attempting to establish connection

2008-07-17 15:58:39 omadmclient.exe: [PID = 0x9e0b15d2] - Establishing connection

2008-07-17 15:58:39 omadmclient.exe: [PID = 0x9e0b15d2] + Transmitting package data

2008-07-17 15:58:41 omadmclient.exe: Failed sending an HTTP request to the server (0x80072f7d).

2008-07-17 15:58:41 omadmclient.exe: [PID = 0x9e0b15d2] - Transmitting package data FAILED (hr = 0x80072f7d)

Check the DM's web certificate and make sure the subject name matches the DM FQDN, the url= value in the SCMDM2008DeviceManagement SCP, and that the certificate is valid. Certutil is invaluable for verifying that all the certs are correctly deployed. A good resource for how to use certutil is http://blogs.technet.com/pki/archive/2006/11/30/basic-crl-checking-with-certutil.aspx

You can replace the certificate manually with a new one by using the MDM Certificate tool in the MDM Server resource kit or by following the intructions in the MDM product documentation at http://technet.microsoft.com/en-us/library/cc135742.aspx.

Remote Wipe Now and MDM Alerter troubleshooting

Dave Madison — Tue, 22 Jul 2008 03:07:00 GMT

Remote Wipe Now and MDM Alerter troubleshooting

A key security feature of SCMDM is the ability to wipe a device remotely. Often time is of the essence, so it is important to know if a wipe was successful or not. Here we will discuss how remote wipe works and how to troubleshoot it,

Introduction

Remote wiping of a device is a security feature that allows administrators to remotely send a command to an MDM managed device that causes it to erase all data and return to factory defaults. This is useful if the device has been lost or stolen. In a remote wipe, the flash and storage card data are overwritten, leaving only the base OS on the device and no user data. Administrators can also deploy the Self Service Portal which allows users to wipe their own devices.

For wipe operations, time is critical - that's why we tend to call the feature "Wipe Now" internally. Without the Now you just have a Wipe Soon operation. Wiping in 8 hours doesn't work great when your device and all its data are lost or stolen. Remote wipe commands in MDM depend heavily on networking between the server and the device and this is where problems can often occur.

In pilot environments administrators often run MDM Remote Wipe tests because it is easy to see it working. However, this is one area in deployment where it is easy to make mistakes and cause the Wipe to take longer than expected. Below are troubleshooting steps to help to determine the cause of a Wipe Now command taking longer than the expected ~~2-6 minutes~~. Failed to catch this before posting...expected time is 3-15 minutes.

How Remote Wipe Works

Before going into troubleshooting, here's a brief overview of how Wipe Now works.

An administrator or user submits a wipe request through the console, MDM Shell, or Self Service Portal.
The wipe request is stored in the DM Engine database for the device to pick up at its next scheduled OMA session

If we were to stop at this point it would be a "Wipe Soon" and the device would connect at its regular interval (4 hours, for example) and pick up the wipe. Wipes submitted are always submitted as a "Wipe Now" now command, and thus we have to go a step further

In parallel to adding the wipe request for retrieval, the wipe driver also calls the Alerter component to inform the device of a pending wipe request.
The Alerter sends an alert to the device over the Mobile VPN. The alert can only be sent through the VPN tunnel and thus Wipe Now requires VPN connectivity
The Alerter client on the device receives this Alert and immediately starts a management session with the Device Management server.
The device picks up its wipe request from the Device Management server, sends back an Acknowledgement that started the wipe, and starts the wipe process

Troubleshooting Wipes that are Taking Longer than Expected

From the moment a wipe request is submitted, a wipe should take approximately ~~2 - 6 minutes~~ 3-15 minutes] depending on the network and other factors. If it is taking longer than that, below are some troubleshooting steps that you can perform to determine the cause of the latent wipe request.

1. Verify that Management Sessions are operating as expected

Ensure that typical device management sessions, not related to wipe are working as expected. You can download the MDM Connect Now tool located in the SCMDM 2008 Resource Kit Client Tools package here: http://technet.microsoft.com/en-us/scmdm/cc304591.aspx Using this tool and associated documentation, you can verify that management sessions for devices are successful.

If management sessions are not working, you will need to fix this problem before devices can successfully receive the remote wipe command.

2. Verify the device is connected to the VPN

As discussed in the above "How Remote Wipe Works" section, remote wipe relies on the VPN being connected in order to send alerts to the device. The device may not be connected to the Mobile VPN for many reasons, but some of the most common are:

The device is switched off
The Mobile VPN is disabled (if users have the ability to disable it)
The user is roaming and VPN is off
The data connection is improperly configured
The user is temporarily out of service coverage area

In all the above cases, the device will not receive the Alert message as it relies on the VPN tunnel being up. When the device reconnects to the VPN, it will receive an Alert message or will start a management session immediately if it has missed its regularly scheduled session.

3. Verify the Gateway Server is not behind a Network Address Translator (NAT)

Once you have verified that management sessions are operating and the VPN was up on the device you were attempting to wipe, you need to check that the Gateway is not behind a Network Address Translator. MDM does not support locating a Gateway Server behind a NAT. There are several reasons for this requirement; one of the reasons has to do with the Alerter.

The reason the Alerter does not work with Gateway Servers behind a NAT is for security purposes. We'll talk about this some more, but for added security, the Alerter checks to ensure that the Alert it received is really from the MDM Gateway Server and not from a potential attacker.

The Gateway Server must have a public IP address and must not sit behind a NAT or the Alerter cannot verify the alert that the alert is valid. The Alerter discards invalid alerts. Ensure that your Gateway is not behind a NAT.

4. Check the Event Log on the MDM Gateway

On the MDM Gateway server, open up the "VPN Policy Engine" Event Log. Search for events 5507 and events 5506 in the log

Event 5506 indicates that the Alerter received a response from the device.

Event 5507 indicates that the Alerter sent a number of retries,but never heard from the device.

Further debugging information is needed for the following scenarios:

5507 events for every wipe issued

5507 event for a wipe issued in a controlled environment, where you know the device connected, online and available

Inconsistent 5507 and 5506 events

A mixture of 5507 and 5506 events are expected for normal operations as some devices may be offline, out of service range, or not connected for another reason. This is generally normal.

In our next post we will look at how to use the some client tools available in the resource kit, and how to use the Device Log to further narrow down any issues.

Marc McClure

Program Manager

Mobile Information Worker

Getting Started with SCMDM

Dave Madison — Mon, 07 Jul 2008 07:39:00 GMT

At the risk of stating the obvious, System Center Mobile Device Manager (SCMDM) is a Version 1 product. There are many opportunities that come with shipping a V1 product. The most important opportunity is for the team firs to identify the need for the product, it was clear to us that mobile devices were becoming an essential tool for information workers. In addition, the upcoming workforce will expect and demand that the majority of their information be accessible from their mobile device. In short, your mobile device is just as important as your laptop. That situation also presents opportunities for a company's IT departments. How do they manage those devices in the same manner they manage laptops and desktops? This, then, was the challenge we set out to solve with SCMDM. We worked closely with key enterprise customers, to identify what they needed--the ability to enforce policies, wipe devices, access data inside the firewall securely, and make Line of Business applications available on the mobile device. As we built the product and went through our milestones, we worked with our Technology Adoption Program customers to get specific technical feedback on the product in "the real world". These customers told us we had met their primary requirements and the product was ready to go. So, we launched SCMDM this past April 2008.

Three months have since passed, with many of our TAP customers continuing with their testing and pilot programs. But the momentum has yet to really get started. Because SCMDM supports Windows Mobile v. 6.1 and above, the real benefit of the product will start to take off over the next weeks and months as the 6.1 devices start to become commercially available. You will also be hearing more about SCMDM from your Microsoft account teams.

But don't wait for them; you can get started testing and piloting SCMDM. There is a 180-day Evaluation version available at http://technet.microsoft.com/en-us/evalcenter/cc339027.aspx. The perfect place to get the technical content you need to get started is The SCMDM Tech Center.

Many customers that are testing and piloting SCMDM have asked how to best integrate it with existing server technologies they may have. We've heard them and responded. We recently published a series of technical guides that explain how SCMDM can integrate with your existing solutions:

· Configuring External and Internal Firewalls for MDM

· Integrating MDM with Microsoft Exchange Server

· Deploying Mobile Device Manager in a Global Enterprise Environment

· Integrating Mobile Device Manager with Existing Web Sites or SharePoint Server

· Integrating Mobile Device Manager with Office Communications Server

We hope this gives you a good start in understanding and planning for SCMDM. While pointing you to all this great content is important in building your foundation, over the next weeks, months, and beyond you will start seeing posts from the product team that will discuss specific SCMDM technologies in depth. Your comments and feedback will be essential in ensuring we continue to deliver the best Mobile Device Management technology.

Welcome to the System Center Mobile Device Manager Product Team Blog

Dave Madison — Tue, 17 Jun 2008 02:50:00 GMT

Welcome to our System Center Mobile Device Manager 2008 blog. I encourage you to take this opportunity to connect to the Mobile Device Manager Product team, learn more about our products, and share your thoughts and opinions about enterprise mobility. Our team is looking forward to working with you to build a great community!

SCMDM 2008 launched in April 2008 after years of interactions with IT professionals. Mobile Device Manager is here today because of their vision of how enterprise IT could meet the needs of a mobile workforce. Mobile e-mail capability has become a given. Businesses want more—they want to give their employees access to the same line of business (LOB) applications and data available inside corporate networks. They want to manage mobile devices with the same infrastructure they use to manage PCs. Mobile Device Manager was designed to meet these needs of enterprises by integrating Windows Mobile with the Windows Server System.

Through this blog and other community efforts, we hope to provide you the information and insight you need to enable your Windows Mobile Devices to be more productive. I welcome your feedback and suggestions on how to improve our product.

Dieter Zirkler

Principal Group Program Manager