We have a site with two wildcard certificates: *.foo.com and *.bar.foo.com. All of our subdomains resolve to the same IP address.

Newer browsers work fine with this, but IE7, for example, does not. My understanding of the problem is this: in older version of SSL (prior to TLS 1), the domain name itself is encrypted. Thus, when a request comes in to the server, there's a chicken-and-egg problem:

It needs to decrypt the request in order to find out the domain, but

It needs to know the domain in order to pick the right certificate to decrypt the request

I think this is the problem, and that it is also connected to the fact that the differnet subdomains point to the same IP.

I think that if we used different IP addresses for these subdomains, the server could differentiate requests that way and pick the correct certificate.

3 Answers
3

Your particulars are wrong (it is not a problem of encryption per se) but you have the right general idea.

In SSL/TLS, the client first connect to the server and sends a "ClientHello" message stating "I wish to do some SSL with you". This happens just above the TCP layer, and below anything which is related to HTTP; an HTTP request will be sent within the SSL tunnel only after the SSL tunnel has been created. SSL tunnel creation is done through the procedure called "the handshake", of which the ClientHello message is the first step.

During the handshake, the server sends his certificate to the client. The certificate contains the server name and his public key, and it is signed by a certification authority. The client will accept to use the public key only after having "validated" the certificate, a process which entails the two following steps:

The client must verify the signature by the certification authority over the certificate. It may have to validate the certification authority's own certificate before that, though; the process is recursive. Certificate validation is full of intricate details which I will skip here because they are not relevant to this discussion.

The client must check that the intended server name indeed appears in the certificate. Because that's the point of the certificate: to bind a public key (a mathematical object with which cryptography is done) to an identity (the server name); the certification authority issues the certificate (i.e. signs it) only after verified in some way that the public key is indeed the one for which the server controls the corresponding private key.

Once the client is reasonably sure that he has the correct public key, he can finish the handshake: depending on the negotiated cipher suite, this entails asymmetric encryption or digital signatures; either way, the server will use his private key. At the end of the handshake, both client and server end up with a shared session key with which more cryptography is going to happen, this time symmetric encryption.

Then, and only then, can "application data" be transfered through the tunnel. In the case of HTTPS, "application data" contains the actual HTTP request.

Now, the problem is the following: the client wants to contact a given server (with a name such as www.foo.com), but that intended server name is sent to the server only with the HTTP request. At the TCP level, only IP addresses exist. When the server needs to send his certificate, he only knows that it was contacted under some IP address, but it does not know under what name the client tries to locate it. Nevertheless, the server must send a certificate which contains the right name. When the server only hosts a single HTTPS server per public IP, that's easy: only one possible name. With several HTTPS servers sharing the same public IP, the server has to do some divination, which, on a general basis, is not possible for computers (it is already quite hard for augurs).

Possible solutions:

Embed several names in a certificate. The server has only one certificate, but it contains the names of all the HTTPS servers which share the public IP. This is supported in X.509 as the Subject Alt Names extension. This requires client browsers to actually support standards the way they should, so this is not a given. This page claims that most browsers support it, but I think this warrants at least some extensive testing (especially when there are several names in the extension and these names contain the "wildchar" *).

Enhance the protocol with an extra exchange at the beginning. This is known as the STARTTLS option. In an HTTP context, this means that the client would first sent a special HTTP request which contains the server name, with a special command (namely "STARTTLS" instead of a normal "GET") to indicate the willingness of the client to engage into an SSL/TLS handshake right away on the same connection. This has been proposed at some point for HTTP, but rejected because it does not work well with existing HTTP proxies. STARTTLS is used in other protocols such as SSMTP or IMAPS, where an actual connection is always available.

Have the client send the intended server name early enough in the handshake, e.g. by smuggling it somewhere in the ClientHello message. This is known as Server Name Indication. This is what the HTTPS world is aiming at.

The biggest issue with SNI is that it is not supported by IE on Windows XP (it requires IE 8+, but also a Vista or newer version; because the actual SSL code used by IE comes from the OS, not from IE itself). So you have to somehow induce your clients with WinXP to switch to Firefox, Chrome, or some other browser which knows about the SNI and does the SSL itself instead of relying on the implementation provided by the OS.

A fantastic and educational answer. Thank you so much! I'll be referring back to this in the future. :) Once I can verify that it solves our current problem, I'll mark it as accepted.
–
Nathan LongJan 10 '12 at 15:22

1

Option 2 (STARTTLS-like system) is more or less RFC 2817, which is implemented in Apache Httpd since version 2.1. I'm not sure any browser support it though.
–
BrunoJan 10 '12 at 16:11

What a pity that they chose option 3. I would have preferred option 2 for increased privacy. But of course SNI is faster and in most cases you don't need that kind of privacy.
–
CodesInChaosJan 17 '12 at 23:07

Yes. It is still the standard practice to have a separate IP for each SSL-enabled domain. The Server Name Indication extension to SSL looks to be fairly impressively adopted. The saturation isn't enough for most people driving sites with notable traffic to abandon the unique IP arrangement yet, though.

From the application protocol point of view, TLS belongs to a lower layer. This means that the TLS handshake is usually (except in the STARTTLS case) performed before the application protocol can start. The name-based virtual server feature being provided by the application layer, all co-hosted virtual servers share the same certificate because the server has to select and send a certificate immediately after the ClientHello message. This is a big problem in hosting environments because it means either sharing the same certificate among all customers or using a different IP address for each of them.

There are two known workarounds provided by X.509:

If all virtual servers belong to the same domain, a wildcard certificate can be used. Besides the loose host name selection that might be a problem or not, there is no common agreement about how to match wildcard certificates. Different rules are applied depending on the application protocol or software used.

Add every virtual host name in the subjectAltName extension. The major problem being that the certificate needs to be reissued whenever a new virtual server is added.

In order to provide the server name, RFC 4366 Transport Layer Security (TLS) Extensions allow clients to include a Server Name Indication extension (SNI) in the extended ClientHello message. This extension hints the server immediately which name the client wishes to connect to, so the server can select the appropriate certificate to send to the client.