If it's not necessary to use a Gateway server in a situation where mutual authentication is not possible, for instance a small number of agents in an untrusted domain, DMZ or workgroup, the agents can be configured to use certificate based authentication to a management server.

 

Here are the basic steps:

Management Server side:

  • Ensure the Management Server has a copy of the Trusted Root CA if it hasn't already (Really depends if the CA resides in the same domain, a foreign domain) in Computers certificate store.
  • Ensure a unique certificate with intended purpose Client Authenication/Server Authenication with the Subject name matching the FQDN of the Management Server has been issued with private key and then stored in the Computer Certificate store (Certificates (Local Computer)\Personal\Certificates in the Certificate snap-in). Make sure the certificate request has "Mark Keys as exportable" and "Store certificate in the local computer certificate store".
  • Run the appropriate platform version of MOMCertimport.exe. Ie. Run the 32-bit version on a 32-bit OS and a 64-bit version on a 64-bit OS :)  . I recommend the command line MOMCertImport /SubjectName XXX.YYY.ZZZ  , where XXX.YYY.ZZZ is the FQDN (alteratively of course XXX for a NetBIOS name in a workgroup). This negates having to export the certificate with private key and instead has MOMCertImport extract the certificate serial number directly from the computer certificate store and than store the serial number in the registry key location HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Machine Settings\ChannelCertificateSerialNumber . This is what the HealthService uses to determine which certificate to use.

 

Agent or Gateway Side:

  • Ensure the Agent or Gateway has a copy of the Trusted Root CA if it hasn't already (Really depends if the CA resides in the same domain, a foreign domain) in Computers certificate store.
  • Ensure a unique certificate with intended purpose Client Authenication/Server Authenication with the Subject name matching the FQDN of the agent has been issued. If the Gateway or Agent is in a workgroup just specify the NetBIOS name for the certificate subject as you won't have an FQDN. You can view this in the Certificates snap-in. Make sure the certificate request has "Mark Keys as exportable" and "Store certificate in the local computer certificate store".
  • Run the appropriate platform version of MOMCertimport.exe. Ie. Run the 32-bit version on a 32-bit OS and a 64-bit version on a 64-bit OS :)  . I recommend the command line MOMCertImport /SubjectName XXX.YYY.ZZZ  , where XXX.YYY.ZZZ is the FQDN (alteratively of course XXX for a NetBIOS name in a workgroup). This negates having to export the certificate with private key and instead has MOMCertImport extract the certificate serial number directly from the computer certificate store and than store the serial number in the registry key location HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Machine Settings\ChannelCertificateSerialNumber . This is what the HealthService uses to determine which certificate to use.

 

See the Security guide for more information about specifics on Certificate requests from Standalone as well as Enterprise CA's.

 

Final tip: Make sure you have the UI Settings\Security property set to Review new manual agent installations in pending management view enables with Auto-approve if necessary.