This is the final part of my round-up to date of FAQs and gotchas for native mode and Internet-based client management that I hope will help you be successful with these features.
Tips, tricks and hints covered:
Part 1 of 3
Part 2 of 3
Part 3 of 3
CRL Complications for Internet-Based Client Management
Strictly speaking, everything around the use of certificate revocation and the certificate revocation list (CRL) is external to Configuration Manager –it’s part of the PKI design and implementation. But it’s definitely something you need to factor in when deploying Internet-based client management, and I’m really struggling with the best way to incorporate this better into the documentation – struggling because the publication of the CRL is PKI-specific, design-specific, and is completely outside the control of Configuration Manager.
The documentation does say as part of the PKI deployment checklist “If you will use a Certificate Revocation List (CRL), publish it where all computers can locate it.” In a nutshell, this covers the requirement, but it has definitely been catching out customers who are deploying Internet-based client management. A number of customers didn’t realize that a CRL is published by default on a Windows Server 2003 CA and its location is by default added to each deployed certificate – which they then confirmed by checking the values for “CRL Distribution Points” in the Details tab of a deployed certificate.
For the November update, I updated the topic Prerequisites for Native Mode with some more information about CRL requirements – because although it’s catching out customers implementing Internet-based client management, it’s really just as relevant to native mode sites on the intranet:
So why is it catching out customers specifically when deploying Internet-based client management? Before I go any further, I have to ring the disclaimer bells and stress that you should get advice about CRL design and configuration options from PKI experts. This is completely external to Configuration Manager and we are not responsible for Microsoft Certificate Services or the documentation related to it. However, I’ll share with you what I know that has helped customers who fell into the CRL pit of problems.
First, in case you’re asking yourself just what is a CRL – a CRL (certificate revocation list) in its simplest form, is a list of certificates that a CA has issued but revoked. Usually, certificate revocation happens because the trustworthiness of the certificate or computer is now unknown (for example, a server that is suspected to have been compromised or a laptop that is stolen).
From Configuration Manager’s perspective, you can enable or disable certificate revocation checking on clients. It’s enabled by default in the Site Mode tab, which means if you don’t change this option, clients that can access site information from Active Directory Domain Services and clients that are installed using client push will be configured to check the CRL for each connection to a native mode site system.
If you install a client manually with /native: you have to specifically enable CRL checking by adding CRL or CRLANDFALLBACK (so the installation property becomes /native:CRL or /native:CRLANDFALLBACK). However, even if you manually install a native mode client without the CRL option, if the client can access site information from Active Directory Domain Services and CRL checking is enabled for the site, the client will be reconfigured so that it is enabled for CRL checking. This can leave you in an odd situation where some clients in the site check the CRL and some don’t. To check up on this setting, reference the Https State Flags value that’s returned through hardware inventory. For more information, see How to Identify Client Configuration Details for Native Mode and Internet-Based Client Management.
There is no similar option for the native mode site systems, but CRL checking is enabled by default through IIS and there is nowhere in the UI to disable it (you need to delve into the metabase and configure the CertCheckMode property). Disabling CRL checking is never recommended.
If the computer (Configuration Manager native mode client or native mode site system) performs CRL checking, and it can’t access the CRL, the connection will fail. This can be both good and bad – depending on what you’re trying to protect against. It’s good in that it prevents a computer from trusting another computer that has been marked as untrustworthy (for example, somebody has hacked into your Internet-based management point or somebody has stolen one of your laptops). It’s bad in that it can prevent two trusted computers communicating, and in worst case scenario an incorrectly deployed CRL can ironically result in a massive denial of service.
So how does CRL checking work in practice? When a CA issues a certificate, typically it includes in the certificate extensions a list of locations from which its CRL can be downloaded and its contents checked, so that computers that use the issued certificate know that the certificate hasn’t been revoked. For example, a native mode client attempts to connect to the Internet-based management point. During the mutual authentication process, the client not only checks that the server certificate is valid and successfully chains to a trusted source, but it also looks at the list of CRL locations in the certificate, and tries each location in succession until it successfully downloads the CRL (base CRL and any deltas). Then it checks that the certificate has not been revoked. Similarly, the Internet-based management point looks at the list of CRL locations in the client certificate, downloads the CRL and checks that the certificate has not been revoked. Both computers cache the CRL and use the cached version until they download a delta CRL or the CRL expires.
A CRL can be published to different locations and using different paths – for example, an LDAP path, an http path, and a file path. These are stored on the CA as CRL distribution points (CDPs) – not to be confused with Configuration Manager distribution points! Using the properties of the CA, Extensions tab, you can configure options for the default CDPs, and add and remove CDPs. Whenever you make any changes, you will be prompted to restart Certificate Services and newly issued certificates can then include the modified CDPs in the certificate extensions (if you selected the option to do so). Certificates that have already been deployed, obviously will not include the changes so you will need to redeploy these certificates if you want them to have the changed CDP information.
So far so good – for most native mode sites on the intranet all this will happen under the covers without any problems. But here’s the tricky bit for Internet-based client management: If your issuing CA is on the intranet, the CDP paths by default will refer to locations on the intranet which won’t be accessible to clients on the Internet, or to Internet-based site systems in the DMZ. These computers will have to use an http path to the CDP that is accessible from the Internet, and with a name that resolves on the Internet.
To walk through a couple of examples that explain how and where you might come across CRL problems:
Example One: You have just one issuing CA on the intranet, all your Internet-based site systems are in the intranet and configured for both intranet connections and Internet connections. These site systems are published to the Internet using ISA Server. This means by default, all the certificates are deployed with the CRL published in the intranet and the CDPs are listed in the certificates with intranet paths. Everything will work OK on the intranet, and even when you move clients onto the Internet everything will continue to work for a while. If the clients remain on the Internet however, the next time they try to download the CRL it fails because it cannot resolve the name in the http path. Without a valid CRL, a native mode connection is no longer possible. Take the client back on the intranet, it downloads the CRL, and off it goes again …
If the native mode clients are not configured for CRL checking, this scenario will not occur – but obviously that’s not the recommended security solution. However, in a testing environment it’s a viable way to confirm the problem. Another option is to add a new Internet http path for the CDP that clients can resolve, and publish an intranet CDP via the ISA Server. Alternatively, publish an additional CDP on a server in the DMZ (perhaps the ISA Server itself) so that clients that are on the Internet can connect to this public-facing CDP when they are on the Internet, and connect to the private CDP when they are on the intranet.
If you don't have a trusted connection to the server hosting the CRL for Internet access, you will need to manually copy the CRL (.crl and .crt files from the CA’s CertServ folder) such that the path and filename matches the http path you specified. And you will have to remember to do this every time before the CRL expires.
Example Two: You have one issuing CA on the intranet that automatically installs your native mode client certificates, and one issuing CA in the DMZ that installs native mode certificates for your Internet-based site systems in the DMZ, and they share the same root CA on the intranet. A client attempts a connection to the Internet-based management point. The client can access the CRL for the Internet-based management point, but the management point can’t download the CRL for the client. It references the CDPs listed in the client’s certificate but fails to resolve the intranet names (and doesn’t have access to the intranet). Native mode communication fails because in this scenario, it’s the server that can’t locate the CRL for the client certificates.
Again, you need to add a new CDP for the intranet CA so that it has an Internet path that can be resolved by the Internet-based servers in the DMZ, and publish the CRL where these servers can locate it from the DMZ.
I’ve come to the conclusion that when computers use certificate authentication on both the intranet and the Internet (the key scenario for Internet-based client management) – the design and implementation of CRLs are tricky! And because CRLs are cached, you might not think of checking your PKI infrastructure if everything has previously worked but suddenly stops. If you see the error message WINHTTP_CALLBACK_STATUS_FLAG_CERT_REV_FAILED in logs, assume CRL access problems.
Because new CDPs only get added to newly issued certificates, it certainly pays to carefully design the CRL implementation before you deploy your native mode certificates, rather than try and fix it afterwards. Do some research, lots of testing (keeping the computer in the same location for longer than your CRL validity interval – by default, 26 hours), and if necessary enlist help from PKI consultants.
You will need to factor in adding CDPs, name resolution, publishing onto a different server, and potentially firewalls for the http traffic. I’ve add these key factors to the topic Prerequisites for Native Mode in the November documentation update, with some additional information and links. But I’m currently at a loss how to add the CDP http requirement into the IBCM diagrams and ports information topic – because there are so many possible variations. We have no way of knowing the target computers for the CDP http connections that originate from the client and site systems when they download a CRL – it all depends on your PKI and CRL design.
“Can you update the step-by-step to include ….?”
The step-by-step was never intended to be a complete solution for any native mode deployment. It was intended to help administrators with no or little PKI knowledge get a basic native mode site up and running in a test environment as a proof of concept. With this intended audience, its aim was to step through the easiest deployment possible using the fewest number of steps.
As we’ve said many times before, and in the documentation, the Configuration Manager product doesn’t have any control over the design or implementation of the PKI requirements, so you shouldn’t expect to find this information in the Configuration Manager documentation. Native mode isn’t PKI – it simply uses PKI and the certificates it requires are just another external dependency. As an analogy, Configuration Manager site systems must reside in an Active Directory forest, but our documentation doesn’t include how to design and implement the forest, or how to deploy and configure domain controllers.
Native mode (and Internet-based client management in particular) wasn't designed for administrators who were inexperienced with PKI. The expectation from the product group was that customers would have their own PKI team with the knowledge and experience to implement the certificates required, or they would bring in external consultancy. It was expected that the topic Certificate Requirements for Native Mode would be the only documentation required.
As an ex-consultant, I have some PKI experience and guessed it would be pretty hard for most customers to interpret the certificate requirements topic and translate this information into a practical solution. So for customers willing to do their own research and testing, I provided some basic PKI information and links to the Microsoft PKI resources - but our documentation on PKI was never expected to be exhaustive and provide a complete solution for every native mode deployment.
From Deploying the PKI Certificates Required for Native Mode:
The public key infrastructure (PKI) certificates that are required for native mode in Configuration Manager 2007 must be created, installed, and managed independently from Configuration Manager 2007. This means that there is no single method of deployment for the required certificates and you will need to consult your particular PKI deployment documentation for the necessary procedures and best practices.
And there’s a similar statement in the step-by-step topic itself. Unfortunately, PKI is hard, partly because there are so many variations and it’s one of those the more you know, the more complex the subject becomes. Because of this, it’s almost impossible to distill PKI into a series of step-by-step procedures that would be suitable for multiple customers. And, native mode was specifically designed to be PKI-agnostic, not just for Microsoft Certificate Services.
The irony is, it’s the customers with the least PKI experience who fail to appreciate just how difficult it would be to extend the existing step-by-step, because they focus on only their particular requirement and can’t see the other possibilities. I can quite see why they want what they’re asking, but we have to provide documentation for 80% of our customers. Adding a particular variation wouldn’t meet that requirement – and we can’t document them all.
Here are some of the variations I’ve counted to date, and I’m sure there are others:
Even within these variations, there isn’t a single method of certificate deployment that would suit everybody. And if we were to document all the variations above into step-by-steps, we wouldn’t have time to document anything else – such as the new features coming out in the product. I don’t know if anybody counted the total number of steps in the existing step-by-step that covers the easiest possible deployment using the fewest number of steps …. but it just breaks into triple figures. Writing out each step for some of the more complicated variations would not be a short task to either write or read!
Then we get onto the thorny subject of deploying certificates for workgroup computers, or domain computers that don’t have access to Active Directory Domain Services because they are on the Internet. The bottom line is, deploying Microsoft PKI certificates for computers outside Active Directory Domain Services is not easy, and this was an accepted imitation for the product group when designing native mode. Certificate Services, as a Microsoft solution, is geared towards leveraging the centralized management that Active Directory Domain Services provides, with Kerberos authentication. And, to make things more awkward for administrators, installing a computer certificate requires local admin rights on the target computer – and it’s never good practice for a user to have local admin rights. Auto-enrollment of certificates using Group Policy gets around this, but Group Policy is not possible when the computer is on the Internet or it’s a workgroup computer.
Believe me, if there was a simple and secure certificate deployment method for computers outside Active Directory Domain Services to suit everybody, we would point customers towards it. But there isn’t, so for these computers you have to do your own research, testing, and weigh the pros and cons to find the best solution for your given circumstances. Your basic options using a Microsoft PKI are Web enrollment or using the rather unfriendly CertReq command line utility. Both of these have potential deployment problems and variations, mainly stemming from the fact that auto-enrollment is not possible because these computers are considered untrusted (no Kerberos authentication to a domain controller) and the local admin right requirement. I have read that you can also programmatically script certificate deployment, but I don’t know anybody who has done this.
Using Web enrollment could be your best option if you want a Do-It-Yourself style solution so that users request their computer certificate using instructions you provide. Or, you could use this option to request a certificate for another computer, export it with the private key and complete chain, and then install it on the target computer. However, be aware of limitations with Web enrollment and Vista computers and plan accordingly if you have Vista clients. CertReq is more suited to automated deployment but you will need to create a separate text file for each computer in order to specify the unique FQDN.
Your .inf text file for a client computer certificate request might look something like the following:
Subject = "CN=computer1.contoso.com"
;EncipherOnly = FALSE
Exportable = TRUE ; FALSE = Private key is not exportable
KeyLength = 1024 ; Common key sizes: 512, 1024, 2048, 4096,
; 8192, 16384
KeySpec = 1 ; Key Exchange
KeyUsage = 0xA0 ; Digital Signature, Key Encipherment
MachineKeySet = True
ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
ProviderType = 12
RequestType = CMC ; Omit entire section if CA is Enterprise
If you are requesting the certificate from a non-domain joined computer, you must use a standalone CA because an enterprise CA uses certificate templates that can be used only by authenticated computers. Additionally, you will need to manually approve each certificate request (although you can configure the CA to automatically approve every certificate request it receives, I’ll assume you don’t want to do this outside a test network for obvious security reasons).
In the topic Deploying the PKI Certificates Required for Native Mode we do list different options for deploying the certificates - which includes deploying the root CA and client certificates - with links to the Microsoft PKI documentation where these options can be researched further. If you can't find the information you need from the existing PKI documentation, we can pass that on to the UA folks in the PKI team.
I also get questions about how to install the Configuration Manager client when it’s on the Internet and what installation properties need to be defined because the client can’t access these from Active Directory Domain Services, or from client push. In the topic Decide How to Install Configuration Manager Clients for Internet-Based Client Management we have the procedure How to Install Clients on the Internet. This comes as close as we could to supplying an example Configuration Manager client installation by listing all the client installation options required - but of course, there might be other parameters you need to specify for your particular setup.
So to summarize this final and controversial point: Please don’t expect or wait for the Configuration Manager documentation to include certificate deployment step-by-steps for all the possible native mode and Internet-based solutions. Do your own research and testing, pool your knowledge and successes with the community, and enlist PKI consultancy where necessary. I fully encourage you to document and publish your own solutions that might also be suitable for other customers. But I have to warn you from experience that documenting PKI can be very thankless! Instead of somebody appreciating that your instructions get them 90% of the way, they berate you for the 10% that doesn’t fit their particular requirement – because they need a complete solution. PKI can be like a bottomless pit and I’ve discovered that no matter how much I write on it to help the majority of people, it’s never enough!
I hope you found these tips, tricks and hints helpful. If you have a question or feedback about the documentation, send e-mail to SMSDocs@Microsoft.com. Alternatively, rate the docs and provide feedback on specific topics (including your e-mail address in the feedback if you would like a response).
- Carol Bailey
This posting is provided “AS IS” with no warranties and confers no rights.
In troubleshooting an unrelated issue, I learned a lot about Certificate Revocation Lists (CRLs) and