I would like to share an issue occurring frequently with Exchange 2013 deployments. The behavior being reported by users is their Outlook clients cannot connect to any of the Exchange Web Services (OWA, EWS). Prior to exhibiting the behavior, there were no reported configuration changes made to the web services by the Exchange Admins, so what changed? The first place that you need to look at is that Exchange back end web site certificate bindings on port 444. Normally, if you check the IIS logs and the HTTP proxy logs you can see that you get Status code 500 when the connection proxy to the Exchange 2013 back end website. Another symptom that you see is the Exchange PowerShell console won't open a session to the server. The reason that this issue happens is due to the way connection flow happens in Exchange 2013. Below you can see the data flow when outlook client connects to exchange 2013 mailbox, and the explanation of the connection. I will cover the normal flow first and then we’ll go over why this breaks when missing or having a third party certificate bound to 444 on the Exchange Back End Web Site.
Exchange 2013 Mailbox
Outlook >>>>> 2013 CAS DWS:443
2013 CAS >>>> 2013 MBX:444
2013 CAS <<<< 2013 MBX:444
Outlook <<<<< 2013 CAS DWS:443
Step by Step Connection Flow
IIS authenticates the request and Microsoft.Exchange.FrontEndHttpProxy.dll running in the MSExchangeRpcProxyAppPool. HTTP Proxy determines the Mailbox Anchor, accessing Active Directory to look up the user, then maps to the user object that contains the mailbox GUID. As soon as we obtain the GUID, HTTP Proxy refers to Active Manager to locate the database where the mailbox resides. At this point, HTTP Proxy then builds the URL to the mailbox server that contains the user's mailbox GUID@Domain.com.
HTTP Proxy takes the authenticated request and creates a serialized blob (that contains SID, user groups, and more. Then a new header is attached that contains the Client Access server name, in order to use CSC (serialized access token authentication, also known as server to server authentication) and sends this new http request URL to the Mailbox server over SSL on port 444. The new URL contains the Mailbox server name so that the Mailbox server knows the request belongs to this server.
The request is sent to the Mailbox server's RPC Virtual Directory that is running in the MSExchangeRpcProxyAppPool by using by default, integrated authentication.
RPCProxy.dll on the Mailbox server is responsible for extracting the RPC packet from the tunnel and forwarding it to the local RPC Client Access Service on Port 6001.
Now that we went over the connection flow when a mailbox is located on an Exchange 2013 mailbox server, we can cover why this breaks and why is so important of not changing or having a third party certificate bound to port 444 on the Exchange Back End Web Site. When the client connects to the CAS, and HTTP Proxy determines where the mailbox is located, it builds an URL to the destination mailbox server on port 444. The URL is built using the hostname. Once we have the URL we try to connect on secure port 444 using SSL. In the process of the SSL handshake there is several validation task that have to take place before the connection is allowed. If the name being used on the URL is not in the certificate the request will fail and your client can’t connect.
But I see 200 status codes in the IIS logs…
The reason you ‘ll see the 200 status codes on IIS is due to the initial communication that is occurring on port 443 and on the Default website. Where the process breaks is during the second part of connecting to the mailbox server on port 444. If you look closely at the HTTP proxy logs, you will have two columns one for HTTP status and one for backend status. In that log you can see the 200 on the first hop but when passing it to the mailbox server on port 444 it fails with the 500 error. Another log that you can check is the IIS log for W3SVC2. In this folder you have the logs for all traffic going thru the Exchange Back End web site.
Screenshot 1 shows wrong binding configuration
Screen Shot 2 of Exchange Shell Error when certificate is misconfigured
Screenshot 3 after editing (corrected) the Exchange Back End Certificate Binding
Screenshot 5 and 6 of OWA behavior when certificate binding incorrect on the Exchange Back End Web Site
After logging in you get a blank page
The main point of this article is that if you have incorrect certificate bindings on the Exchange Back End web site, it can cause the web services on Exchange 2013 server not to work properly. This incorrect certificate binding will break the connection flow, causing clients to have a bad experience. Always make sure the Exchange Back End certificate bindings for 444 always is configured to use the self-signed certificate.
Thank you Mikhael Ortiz.
Thank you Mikhael
Thanks a lot for sharing
Thanks a lot Mikhael Ortiz . very simple way you have explained. :-)
Awesome Work Miguel!!
hello Mikhael. when default installation have finished Exchange Backend site choose certificate with server name which issued by domain ca authority. may we change it to self signed certificate or leave?
As long as the third party certificate contains the hostname is ok. If the third party certificate do not contain the hostname, then you need to revert it to the self signed certificate. Let me know if you have any other questions.
I did revert to the self signed certificate but now the Default Website won't start, claiming another site may be using the port. But all the bindings look correct, including the Microsoft Exchange certificate? Also, see this in the system events: The
World Wide Web Publishing Service (WWW Service) did not register the URL prefix
https://127.0.0.1:443/ecp for site 1. The necessary network binding may already be in use. The site has been disabled. The data field contains the error number. EAC and EMC still both unavailable?
@ Brian. If you only have one IP address bound to the NIC then you cannot specify the IP in the Default Web Site and the Backend Website. Instead use "unassigned" for port 444, and 443. Let me know if there are any other questions.