If you`re setting up a new agent/ Gateway installation which cannot communicate with the Management Server it`s always a good idea to also check the System Event Log and check for SChannel errors like:
Event ID: 36874- TLS 1.2 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported by the server. The SSL connection request has failed.
Event ID: 36888 - A fatal alert was generated and sent to the remote endpoint. This may result in termination of the connection. The TLS protocol defined fatal error code is 40. The Windows SChannel error state is 1205.
If these is a case, let`s check the certificates more closely regarding the signature hash algorithm:
512 has severe incompatibility issues, if the AD Team has not implemented any workarounds for this on the servers & CA it will not work.. since TLS 1.2 does not support SHA 512 -> http://technet.microsoft.com/en-us/library/dd560644(v=WS.10).aspx
It`s not OS related directly.. it applies both to WS2008, WS 2012 +R2.. it`s a just the TLS 1.2 protocol design. The reason why SHA 512 is not supported, is that it can cause higher CPU usage. However you can enable this from the registry, if it should be needed in your environment:
There are also updates available kb2975719 and kb2975331 which address this issue, depending on the Operating System.
Here a further documentation on supported ciphers by Schannel/TLS 1.2, SHA 512 is not there :) So the workaround would be to create certificates based on SHA 384 or SHA 256
+Cipher Suite Definitions
There is an update to this issue here;