How it works on the wire: IIS HTTP Client certificate authentication

How it works on the wire: IIS HTTP Client certificate authentication

  • Comments 4
  • Likes

I collaborated with a colleague recently where the IE client was failing to authenticate to IIS and I was requested to analyze a few network traces collected while reproducing the issue. The main issue was that the client (IE) wasn’t able to successfully authenticate to IIS server when certificate authentication was configured on IIS side and was getting the HTTP error 403.7.

First of all, I asked my colleague to setup a lab environment with IIS client certificate authentication configured so that we should be able to see how it works under normal circumstances. After the requested logs were collected, we had a chance to analyze the working traffic. Also I should note that we used server certificate with its private key to decrypt the SSL traffic.

 

=> Here is how it works in summary:

1) Client establishes an SSL session to the server.

2) Client then sends the initial HTTP Get request inside this SSL tunnel

3) Server responds to that request with a new SSL re-negotiate request since client certificate authentication is configured on the IIS server side.

4) Client establishes a new SSL session inside the original SSL session. This time IIS server also asks the client to send a certificate (IIS server also provides the client with the list of possible CAs from which the client certificate should be obtained)

5) Client sends its certificate and then the second SSL negotiation finishes successfully and the server returns HTTP 200 response right after the second SSL negotiation is finished successfully.

 

=> The TCP 3-way handshake to the IIS server:

 

No.     Time                    Delta    Source                Destination           Protocol Ack number Info

     68 2013-12-26 13:03:51.654 0.000    10.10.10.16           10.10.10.15           TCP                 49685 > 443 [SYN] Seq=923564226 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1

     69 2013-12-26 13:03:51.654 0.000    10.10.10.15           10.10.10.16           TCP      923564227  443 > 49685 [SYN, ACK] Seq=4236088449 Ack=923564227 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1

     71 2013-12-26 13:03:51.657 0.003    10.10.10.16           10.10.10.15           TCP      4236088450 49685 > 443 [ACK] Seq=923564227 Ack=4236088450 Win=131400 Len=0

 

=> The initial SSL negotiation between the client and the IIS server:

     72 2013-12-26 13:03:51.657 0.000    10.10.10.16           10.10.10.15           TLSv1    4236088450 Client Hello

=> Please note that IIS server doesn’t ask the client send a certificate at the initial SSL negotiation as there’s no “Certificate request”  in the server response

 

     73 2013-12-26 13:03:51.657 0.000    10.10.10.15           10.10.10.16           TLSv1    923564384  Server Hello, Certificate, Server Hello Done

     74 2013-12-26 13:03:51.660 0.002    10.10.10.16           10.10.10.15           TCP      4236089827 49685 > 443 [ACK] Seq=923564384 Ack=4236089827 Win=130020 Len=0

     75 2013-12-26 13:03:51.660 0.000    10.10.10.16           10.10.10.15           TLSv1    4236089827 Client Key Exchange, Change Cipher Spec, Finished

     76 2013-12-26 13:03:51.664 0.004    10.10.10.15           10.10.10.16           TLSv1    923564710  Change Cipher Spec, Finished

     77 2013-12-26 13:03:51.666 0.002    10.10.10.16           10.10.10.15           TCP      4236089886 49685 > 443 [ACK] Seq=923564710 Ack=4236089886 Win=129964 Len=0

 

Note: To be able to see traffic in clear text from this point on, you need to decrypt the SSL traffic in a way or another.  I used the method that I mentioned in a previous blog post of mine:

http://blogs.technet.com/b/nettracer/archive/2013/10/12/decrypting-ssl-tls-sessions-with-wireshark-reloaded.aspx

 

=> The initial GET request (sent inside the SSL channel)

 

     78 2013-12-26 13:03:51.666 0.000    10.10.10.16           10.10.10.15           HTTP     4236089886 GET /1.aspx HTTP/1.1

    

=> Instead of sending an HTTP response back to the client, the IIS server asks the client to re-negotiate SSL:

     79 2013-12-26 13:03:51.666 0.000    10.10.10.15           10.10.10.16           TLSv1    923565216  Hello Request

     80 2013-12-26 13:03:51.668 0.001    10.10.10.16           10.10.10.15           TCP      4236089923 49685 > 443 [ACK] Seq=923565216 Ack=4236089923 Win=131400 Len=0

     81 2013-12-26 13:03:51.668 0.000    10.10.10.16           10.10.10.15           TLSv1    4236089923 Client Hello

=> This time the IIS server requires the client to send a certificate (as opposed to initial SSL negotiation) by adding “Certificate Request” to the response that it sends to client. It also includes the names of the accepted CA names from which a client certificate should be issued:

 

     82 2013-12-26 13:03:51.669 0.000    10.10.10.15           10.10.10.16           TLSv1    923565413  Server Hello, Certificate, Certificate Request, Server Hello Done

 

     83 2013-12-26 13:03:51.676 0.007    10.10.10.16           10.10.10.15           TCP      4236092088 49685 > 443 [ACK] Seq=923565413 Ack=4236092088 Win=131400 Len=0

     84 2013-12-26 13:03:51.684 0.008    10.10.10.16           10.10.10.15           TCP      4236092088 [TCP segment of a reassembled PDU]

 

=> The client chooses an appropriate certificate from its certificate store (it should be a client certificate that should be issued by one of those CAs pointed to by the server and it also should have “client authentication” key usage. And then it sends it to the server within the response packet:

 

     85 2013-12-26 13:03:51.684 0.000    10.10.10.16           10.10.10.15           TLSv1    4236092088 Certificate, Client Key Exchange, Certificate Verify

  

    86 2013-12-26 13:03:51.684 0.000    10.10.10.15           10.10.10.16           TCP      923567204  443 > 49685 [ACK] Seq=4236092088 Ack=923567204 Win=131328 Len=0

 

=> The second SSL negotiation is finished successfully at this point:

 

     87 2013-12-26 13:03:51.691 0.006    10.10.10.15           10.10.10.16           TLSv1    923567204  Change Cipher Spec, Finished

     88 2013-12-26 13:03:51.694 0.002    10.10.10.16           10.10.10.15           TCP      4236092178 49685 > 443 [ACK] Seq=923567204 Ack=4236092178 Win=131308 Len=0

 

=> Then the server returns the response to the initial HTTP Get request:

 

     89 2013-12-26 13:03:51.695 0.001    10.10.10.15           10.10.10.16           HTTP     923567204  HTTP/1.1 200 OK  (text/html)

 

 

Note: The web page includes the word “Hello”

 

=> Then the client tries to retrieve favicon.ico file which doesn't exist on the server and finally the session is gracefully terminated by the client:

     90 2013-12-26 13:03:51.701 0.005    10.10.10.16           10.10.10.15           TCP      4236092599 49685 > 443 [ACK] Seq=923567204 Ack=4236092599 Win=130888 Len=0

     91 2013-12-26 13:03:51.795 0.094    10.10.10.16           10.10.10.15           HTTP     4236092599 GET /favicon.ico HTTP/1.1

     92 2013-12-26 13:03:51.834 0.039    10.10.10.15           10.10.10.16           HTTP     923567582  HTTP/1.1 404 Not Found  (text/html)

     93 2013-12-26 13:03:51.844 0.009    10.10.10.16           10.10.10.15           TCP      4236094044 49685 > 443 [ACK] Seq=923567582 Ack=4236094044 Win=131400 Len=0

     94 2013-12-26 13:03:51.844 0.000    10.10.10.16           10.10.10.15           TCP      4236094044 49685 > 443 [FIN, ACK] Seq=923567582 Ack=4236094044 Win=131400 Len=0

     95 2013-12-26 13:03:51.844 0.000    10.10.10.15           10.10.10.16           TCP      923567583  443 > 49685 [FIN, ACK] Seq=4236094044 Ack=923567583 Win=130816 Len=0

     96 2013-12-26 13:03:51.851 0.007    10.10.10.16           10.10.10.15           TCP      4236094045 49685 > 443 [ACK] Seq=923567583 Ack=4236094045 Win=131400 Len=0

 

After that, we collected the same set of logs from the customer environment and we realized that the IIS server wasn’t returning the CA name of which issued the client certificate  to the client in the accepted CAs list even though that root CA certificate existed in the trusted root certificates store on the IIS server. So the IE client wasn’t sending the user certificate located at the user certificate store on its end.

Then we realized that the amount of root CA names were about 16 KB which seems to be an upper limit. After deleting some of the expired and unused root CAs from the IIS server the issue was resolved. You can find more details on this issue at the below article:

 

http://support.microsoft.com/kb/933430/EN-US Clients cannot make connections if you require client certificates on a Web site or if you use IAS in Windows Server 2003

 

The hotfix increases the Schannel security buffer to 16k. If you exceed this limit, you will still have issues that are described in the symptoms section of this article. This change has also been included with Windows Server 2008 and Windows Server 2008 R2. The workarounds described below will apply to Windows Server 2008 and Windows Server 2008 R2 as well.

 

But we were already hitting the 16 KB limit and hence the only solution seemed to be removing some expired or unused root certificates from the IIS server.

 

Hope this helps

Thanks,

Murat

Comments
  • Thanks,very helpful. quick question: when using WAP and ADFS to publish exchange with certificate/smart card does the "require certificate" supposed to be on "require or accept" on OWA virtual directory?

  • My CA is not in the accepted CA names list even though it's in the Trusted Root CA store on the server...

  • very informative. Thanks.

    have a look at this link also http://www.etechpulse.com/2014/04/aspnet-worker-process-iis-process-model.html

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment