SSL Handshake and HTTPS Bindings on IIS

Secure Socket Layer (SSL) also known as Transport Layer Security (TLS) is a cryptographic protocol which defines how 2 entities (client and server) communicate with each other securely. TLS is the successor of SSL. You can read more about it here: http://en.wikipedia.org/wiki/Transport_Layer_Security

These are the following protocols which are most commonly used:

SSL 2.0

SSL 3.0

TLS 1.0 (SSL 3.1)

TLS 1.1 (SSL 3.2)

TLS 1.2 (SSL 3.3)

SSL 2.0 had many security flaws which led to the development of its successor SSL 3.0. It is present only for backward compatibility. I have rarely seen anyone using this version and I would highly recommend against it.

As we know TLS/SSL is an application layer protocol. Below is a diagram depicting the TCP/IP model:

I am not going to discuss the SSL/TLS protocol in this post as it is beyond the scope of this topic. However I would be discussing SSL handshake in brief and relate it to IIS.

The above diagram makes it clear that TLS/SSL runs on top of TCP/IP like any other application layer protocol. Before we delve into SSL handshake we need to know something about TCP handshake too.

If we were to capture a network trace (or a TCP Dump) and look at the details available and analyze the details available; the IP Layer provides the TCP layer with IP Address of the client and server. The TCP layer contains the details about the source port and the destination port, TCP Flags and other details like checksum, Windows Size etc.

When the user launches a browser and punches in the web address, let’s say https://www.kaushalz.com, the client and the server would perform the TCP/IP handshake as seen below

So basically this is what is passed on from the TCP/IP layer to the application layer:

IP Address of the source and destination

Source Port and Destination Port

The host header is neither present in the IP or the TCP layer. This actually leads to a problem which was addressed via the introduction of SERVER NAME INDICATION (a TLS Extension).

Problem due to above Limitation

Before I describe the problem we need to understand a little about the server side bindings. When routing a HTTP request to a website the server determines which process the request to be routed based on the IP, PORT & the HOSTNAME. These 3 are always available to the server during a normal HTTP communication. So basically the combination of IP+PORT+HOSTNAME is used as a unique identity to route the site to a specific process. The server admin can have the same IP+PORT for all the HTTP websites and alter only the HOSTNAME and maintain the uniqueness throughout. Which also makes the server scalable.

However in case of SSL the server has access to IP & Port only. Since the HOSTNAME is not available, the server has to route the request to the process depending on IP+PORT. This limitation leaves the server handicapped, as it has to changes the design for websites running on HTTPS. Due to this, the uniqueness for the websites running on HTTPS is determined through combination of IP+PORT. In real world, having a separate IP for a website is not ideal due to hardware & monetary limitations. Also changing the port number for all SSL bindings may not be ideal as changing the port number to anything other than the default SSL port would require the client to specifically put out the port number in the request. As a result, the server is not scalable for HTTPS sites.

This was a protocol limitation and severely affected the scalability of the sites

This problem was addressed by introducing a TLS Extension called Server name Indication. The client sends the server the hostname it is requesting for as a part of the CLIENT HELLO in the form of TLS EXTENSIONS. You can read more about it here: RFC 3546 (Section 3.1)

Once the client has the Destination IP, it will send a TCP SYN to the server.

The Server responds with ACK to this SYN.

The client responds with an ACK to the ACK it received from the server. Now a TCP connection has been established between the client and the server. The client will now forward the requests to the Destination IP on port 443 (Default TLS/SSL port)

The control is now transferred to the SSL Protocol in the application layer. It has the IP & the Port information handy from previous steps. However, it still has no clue whatsoever about the hostname.

The client creates a TLS Packet called as CLIENT HELLO. This contains the following details:

SSL Protocol version

Session ID

List of Cipher Suites supported by the client.

List of CLIENT HELLO Extensions

The Client typically selects the most secure protocol version and sends it to the server. Below is a snippet from the RFC 3546:

Here the new “client_hello_extension_list” field contains a list of extensions. The actual “Extension” format is defined in Section 2.3.

In the event that a client requests additional functionality usingthe extended client hello, and this functionality is not supplied by the server, the client MAY abort the handshake.

Note that [TLS], Section 7.4.1.2, allows additional information to be added to the client hello message. Thus the use of the extended client hello defined above should not “break” existing TLS 1.0 servers.

A server that supports the extensions mechanism MUST accept only client hello messages in either the original or extended ClientHello format, and (as for all other messages) MUST check that the amount of data in the message precisely matches one of these formats; if not then it MUST send a fatal “decode_error” alert. This overrides the “Forward compatibility note” in [TLS].

If you were to capture a network trace (or a TCP Dump) this is how the CLIENT HELLO would look like:

The client sends a CLIENT HELLO to the server on the IP & Port it obtained during TCP handshake.

For this scenario I will consider IIS 7.5 as the SERVER entity. Upon receiving the CLIENT HELLO, the server has access to the following information:

IP Address (10.168.3.213)

Port Number (443)

Protocol Version (TLS 1.0)

List of Cipher Suites

Session ID

List of CLIENT HELLO Extensions etc.

The Server will first check if it supports the above protocol version and if any of the cipher suites in the provided list. If not, the handshake fails there itself.

The Server will now try to determine if there is an end point listening on the IP and PORT. If it finds an endpoint and if it is IIS, then the TCPIP.SYS driver moves the packet to the HTTP.SYS layer.

HTTP.SYS moves the request into the generic SSL Queue.

Until IIS 7.5 the SSL bindings were IP based i.e., IP+ Port and were associated with a certificate hash.

The HTTP.SYS tries to determine the certificate has corresponding to this IP+Port combination. It does so by enumerating the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\SslBindingInfo

From the above, the certificate hash corresponding to the IP+PORT combination is determined. Now the HTTP.SYS calls the CRYPTO API’s by passing on the cert hash to retrieve certificate blob, which calls the certificate store, finds the certificate and sends it back to the HTTP.SYS.

The extended server hello message format MAY be sent in place of the server hello message when the client has requested extended functionality via the extended client hello message specified in Section 2.1. The extended server hello message format is:

Here the new “server_hello_extension_list” field contains a list of extensions. The actual “Extension” format is defined in Section 2.3.

Note that the extended server hello message is only sent in response to an extended client hello message. This prevents the possibility that the extended server hello message could “break” existing TLS 1.0 clients.

The Server typically responds back with the following details:

SSL/TLS Protocol version.

One of the cipher suites from the list of cipher suites provided by client. (whichever is the most secure)

Certificate of the server (Without the private key of course)

List of SERVER HELLO Extensions.

(OPTIONAL)If the web app associated with this binding requires a Client Certificate for authentication then it would request the client to send the certificate. Here the IIS Sever would send the client the distinguished names of the list of TRUSTED ROOT CA it supports.

The Client uses the SERVER HELLO to perform SERVER AUTHENTICATION. This is described in detail here: http://support.microsoft.com/kb/257587.If the server cannot be authenticated, the user is warned and informed that an encrypted and authenticated connection cannot be established. If the server is successfully authenticated, the client proceeds to the next step.

NOTE: If you captured a network trace for a SSL Handshake you could see the details until SERVER HELLO, after that the encryption begins and nothing would be available and would make sense as the packets are encrypted.

The Client uses the data provided from the server to generate a pre-master secret for the session,encrypts it with the server’s public key (obtained from the server’s certificate), and then sends the encrypted pre-master secret to the server. If the server had requested for CLIENT CERTIFICATE, then client also signs another piece of data that is unique to this handshake and known by both the client and server. In this case, the client sends both the signed data and the client’s own certificate to the server along with the encrypted pre-master secret.

If the server had requested for client authentication, the server attempts to authenticate the client. If the client cannot be authenticated, the session ends. If the client is successfully authenticated, the server uses its private key to decrypt the pre-master secret, and then performs a series of steps (which the client also performs, starting from the same pre-master secret) to generate the master secret.

Both the client and the server use the master secret to generate the session keys, which are symmetric keys used to encrypt and decrypt information exchanged during the SSL session and to verify its integrity (that is, to detect any changes in the data between the time it was sent and the time it is received over the SSL connection).

The CLIENT & the SERVER send each other a message informing that future messages from them will be encrypted with the session key. It then sends a separate (encrypted) message indicating that its portion of the handshake is finished.

The SSL Handshake is done. The Client and the Server send each other messages which are encrypted/decrypted using the session keys generated in the previous step.

It is now that the Client sends the actual HTTP Request packet to the Server in the encrypted form.

The Server decrypts the request via the symmetric key and generates a response, encrypts it and sends it back to the client.

This continues normally for the entire session of secure communication. However, at any time either the client or the server may renegotiate the connection. In this case the process repeats again.

Below is a diagrammatic representation of the SSL Handshake:

Identifying problems during SSL Handshake

Eventually, once the handshake completes and the data exchange has been done, either both or one of the entities will eventually close down the connection gracefully. If there was a problem during the SSL Handshake then you there would be an exception raised within the SSL Layer (SSL ALERT PROTOCOL). These exceptions may or may not be fatal i.e. not all exceptions would cause the handshake to fail.

As we know there we can see details only until SERVER HELLO. Anything beyond this point is not visible. However, in case of a SSL ALERT we would see some notification which can be viewed in the network traces.

Servers also tend to propagate this information through some sort of server logging. On Windows we have SCHANNEL logging which throws a corresponding SCHANNEL event in the SYSTEM event logs. Watch out for these events. Below is a snippet of one such event:

Tags

Thanks for this excellent and detailed write-up. I really enjoyed reading it.

Just a small query : In the TLS/SSL Handshake steps, 1st step where it says – Client will try to resolve the hostname to an IPAddress via ARP'. I think ARP is used to convert IP to MAC. Should it not be DNS ?

Regards,

-Ashwin

6 years ago

Ashwin

small typo error.

For TLS 1.0,1.1,1.2, the corresponding SSL versions are shown as 3.1, instead of 3.1,3.2 & 3.3

Thanks for the correction.. yea it should be DNS.. However, the corresponding SSL versions for TLS 1.1 and TLs 1.2 would be SSl 3.2 and SSL 3.3 respectively..

Please capture a network trace and take a look at the major and the minor version. Here is a trace for one such trace:

=======================================================

– TLS: TLS Rec Layer-1 HandShake: Client Hello.

– TlsRecordLayer: TLS Rec Layer-1 HandShake:

ContentType: HandShake:

– Version: TLS 1.2

Major: 3 (0x3)

Minor: 3 (0x3)

…

…

=======================================================

5 years ago

Pedro Costa

Fantastic explanation! Flabbergasting!

Thank you

5 years ago

Dale Graunke

Great article. Thank you for posting it!

5 years ago

Mike Foster

Great article. I'll pass this on to my colleagues. Can you relate this scenario to a secure LDAP connection from client to server? Can the same vulnerability be exploited when the connection is not initiated over HTTPS but LDAPS (ports 636 and 3269)?

5 years ago

Jay Neiman

Very good article. I'm researching an issue with ServerXMLHTTP using MSXML6.DLL. It appears the ServerXMLHTTP client in the CLIENT HELLO is not sending a cipher supporting RC4. I've reviewed the registry under the SCHANNEL and no matter what I set for the client on the cipher side, the ServerXMLHTTP object is refusing to comply.BTW, it does appear to respect the Protocol registry settings. I suspect it is using a hardcoded FIPS setting. The result of this are sites that only accept RC4 ciphers (which they shouldn't) for TLS 1.0 and higher the ServerXMLHTTP is cannot complete the handshake.

In light of the POODLE vulnerability and sites dropping SSL support, we have seen a few situations where the only cipher these sites are support are RC4. It would be great to find a solution to continue using ServerXMLHTTP and connect to these few sites only supporting the RC4 cipher under TLS.

5 years ago

Suresh Kumar

Nice article.

I'm facing an SSL issue (event details are given below) with one of the web sites in a web server. We use F5 load balancer and have terminated the SSL certificate in the VIP and also configured the same certificate in the site. The SSL certificate uses SHA256 algorithm. When we access the website using servername:port or adding the server IP against the URL in the hosts, the site works fine. However, when we try to access through LB, it does not work and we could find the be below in the event logs.

Event ID :36874

An TLS 1.2 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported by the server. The SSL connection request has failed.

Event ID: 36888

The following fatal alert was generated: 40. The internal error state is 1205.) with one of our web servers.

Below is the handshake from F5 LB.

32 1 0.0003 (0.0003) C>S Handshake

ClientHello

Version 3.3

cipher suites

TLS_RSA_WITH_RC4_128_MD5

TLS_RSA_WITH_RC4_128_SHA

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_RSA_WITH_AES_256_CBC_SHA

TLS_RSA_WITH_3DES_EDE_CBC_SHA

TLS_RSA_WITH_DES_CBC_SHA

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA256

Unknown value 0xc013

Unknown value 0xc014

Unknown value 0xc012

Unknown value 0xff

compression methods

NULL

32 0.0008 (0.0005) S>

So, what needs to be configured at the web server end for the negotiation to happen without any issue.

We use Windows Server 2008 R2 OS with TLS 1.1 and 1.2 enabled in the registry.

The handshake failed with "The following fatal alert was generated: 40"

This indicates that the sender was unable to negotiate a Protocol/cipher suite. with the server. I would compare the cipher suites on LB & Server to see if they are supported.

You may also enable SCHANNEL logging on the Windows Server. Set the value of the EventLogging keyword to 7. Refer this KB article: support.microsoft.com/…/260729

5 years ago

Nagendra

I find this a great article for understanding the handshake. But Kaushal, will any of this handshake information available for .net application hosted in IIS? Specifically, can I know which SSL protocol the handshake has happened or the SSL protocol that client is using for the request in my asp.net application? Or do I need to write any extensions to get that information?

4 years ago

bigolrouter

I have an SSL/TLS issue that I think is related to a firewall being in-between the client and server. Is it normal during a TLS handshake not to see a SYN-ACK packet in response to a SYN packet? All that I see in my packet trace is a SYN packet, and then just ACK packets going back and forth.

I don't think I completely understand your question. For an SSL issue I would filter the network trace by SSL/TLS so that I can see relevant packets.

In case of SSL. firewall comes into picture if the port being used by SSL is blocked or if the hostname to which the SSL service connects to has been blocked. Nevertheless, you should first try to check if this is a SSL issue or a network issue.

4 years ago

madhu

Thanks, Kaushal for digging through the details and writing this blog.

4 years ago

Far

Hello Kaushal, a very nice blog about ssl handshake. just a query which i was not able to understand. In the client hello you see 2 protocol version i.e one is above Handshake.Protocol: Client hello, which shows TLS 1.0 and one is below that which shows TLS 1.0. In your case both are same but When i took wireshark traces I see above version as SSL 3.0 and below version as TLS 1.2. Do you know what are these 2 protocol version means ? why there are 2 protocol version and why they differ in some of the wireshark trace.

4 years ago

Ambajee

Hi, thanks for the explaination. Can you please tell if the Client is in SHA2 format and if the Server is in SHA1 format, will there be a fatal problem.

@Far, I haven't really come across a scenario where I have seen 2 protocol versions in the client hello. Would it be possible for you to upload your network trace somewhere and share it with me so that I can take a look?

@Ambajee, I'm perplexed by the query. There is no SHA2/SHA1 server/client in the ssl handshake. It is the cert that can be either SHA2 or SHA1. It again depends on the client as to whether they mandate SHA2 only. If the client has a policy for example lets say it mandates SHA2 and the server sends a certificate which is SHA1, then there will be a SSL alert and you will see a warning in your browser window for this.

Hi Kaushal,
Thanks for this great article.
Now a question: I have a secured restful wcf service which i hosted in IIS 8.5 on WS 2012 R2. I use self-signed certificate. After starting the service? everything works fine, but after a while (2 – 3 hours), after that service stops responding with an error like: SSLHandshake error. What is strange, that on WS 2008 R2 everything works fine. Do you ever meet something like that? Is it a certificate issue or some configuration? After rebinding the certificate in IIS service responds further… very strange behavior.

That is a strange behavior. SSL issues are pretty straight forward, however they could get tricky.
The behavior you described indicates that the SSL configuration is being changed, more specifically the IIS binding.
I have seen such issue with certain management softwares as they tend to overwrite system configuration to keep in sync with their configuration.

Hi Kaushal,
is there a way to capture the SSL session keys in IIS? or Client application?
I have a WCF client accessing WCF service using SSL. I want to decrypt the traffic, for that I need certificate private key and I also need SSL Session key. My WCF service are running in IIS and Client is a windows application.
In such scenario, how do I get SSL session key? any idea?

I have one more question: Which port number is used for CLIENT HELLO and SERVER HELLO transporting? From snippet of the network trace you provided to see, it is 80. But it seems should be 443 as to your description.

thanks

3 years ago

Rajashekar Naidu B

Hi Kaushal,

we are trying to host an mvc4 web application in windows server 2012R2, here we are facing a problem when a website hosted as HTTPS application. the application fails to fetch login credentials from the session.
If the application hosted as http no issues, once we test in our test server it goes fine.

It will be great to know solution from you, thanks in advance

3 years ago

andre E

I love it! Excellent article. Now we know all pros and cons of the form. Filling out forms is super easy with PDFfiller. Try it on your own here ID Blue Cross Form 5-123 and you’ll make sure how it’s simple.

3 years ago

Ziqing Wang

Hi Kaushal,

It is a great article. One thing I wanted to confirm is the order of the cipher suite. When pick which one to use, it depends on the order of the client cipher suite or the server cipher suite?

@Ziqing Wang
It is mostly client driven. The client will specify the order of cipher suites based on configuration. The server will scan it top to bottom, and select the first one that matches its list of cipher suites

2 years ago

Colin

Hi Kaushal,
great explanation.

We have a strange phenomenon we have observed when a windows 2012R2 Domain Client is authenticating users via LDAPS / Simple Bind to the AD (same OS / Version) via Port 636. Every 10 Minutes or so there is a 20 to 30 second pause in the TLS handshake after the cipher exchange when a connection is opened to authenticate another user.

Once the connection is correctly established and the user authenticated all following logins are <1Second for the next ca. 10 minutes, after which the next login takes approx. 30 seconds again.

Does anyone have an explanation what could be causing this pause? I have added a domain global policy to stop the checking of CRLs on the Microsoft website, which seems to have reduced the time from 30 secs to about 20 secs but that is still too long and we cannot see what the server is doing/trying to do in the meantime.

Interesting issue. I would have taken a ETW trace to see where the time is being spent. This requires deeper investigation. I would recommend you to open a support ticket with the AD team and let them investigate this.