Tag Archives: OpenSSL

There are many, sorry very many, different SSL libraries today that various programs may want to use. In the open source world at least, it is more and more common that SSL (and other crypto) using programs offer build options to build with either at least OpenSSL or GnuTLS and very often they also offer optinal build with NSS and possibly a few other SSL libraries.

So, a common problem should be able to find a common solution. What if we fixed this in a way that would be possible for many projects to re-use? What if one project’s ability to select from 9 different provider libraries could be leveraged by others. A single SSL API with a simplified API but that still provides the functionality most “simple” SSL-using applications need?

Marc Hörsken and I have discussed this a bit, and pidgin/libpurble came up as a possible contender that could use such a single SSL. I’ve also talked about it with Claes Jakobsson and Magnus Hagander for postgresql and I know since before that wget certainly could use it. When I’ve performed my talks on the seven SSL libraries of libcurl I’ve been approached by several people who have expressed a desire in seeing such an externalized API, and I remember the guys from cyassl among those. I’m sure there will be a few other interested parities as well if this takes off.

What remains to be answered is if it is possible to make it reality in a decent way.

Upsides:

will reduce code from the libcurl code base – in the amount of 10K or more lines of C code, and it should be a decreased amount of “own” code for all projects that would decide to use thins single SSL library

will allow other projects to use one out of many SSL libraries, hopefully benefiting end users as a result

should get more people involved in the code as more projects would use it, hopefully ending up in better tested and polished source code

There are several downsides with a unified library, from the angle of curl/libcurl and myself:

it will undoubtedly lead to more code being added and implemented that curl/libcurl won’t need or use, thus grow the code base and the final binary

the API of the unified library won’t be possible to be as tightly integrated with the libcurl internals as it is today, so there will be some added code and logic needed

it will require that we produce a lot of documentation for all the functions and structs in the API which takes time and effort

the above mention points will deduct time from my other projects (hopefully to benefit others, but still)

Some problems that would have to dealt with:

how to deal with libraries that don’t provide functionality that the single SSL API does

how to draw the line of what functionality to offer in the API, as the individual libraries will always provide richer and more complete APIs

What is actually needed to get the work on this started for real? And if we do, what would we call the project…

PS, I know the term is more accurately “TLS” these days but somehow people (me included) seem to have gotten stuck with the word SSL to cover both SSL and TLS…

I recently mentioned the new schannel support for libcurl that allows libcurl to do SSL natively without the use of any external libraries on Windows.

This “getting native support” obviously triggered Nick Zitzmann who stepped up and sent in Secure Transport support – the native API for doing SSL on Mac OS X and iOS. This ninth supported SSL library is now called ‘darwinssl’ in the curl code base. There have been some follow-up commits too to cleanup things and make use of that API for providing the necessary function calls when doing NTLM too etc.

This functionality is merged in to curl’sÂ master git repository and will be part of the upcoming curl 7.27.0 release, planned to hit the public at the end of July 2012.

It could be noted that if your for example build curl/libcurl to also support SCP and SFTP, you’d be linking with libssh2 for that and libssh2 is still relying on a crypto library that is either OpenSSL or gcrypt so you may in fact still end up linking with a 3rd party crypto library… Nick mentioned in a separate mail how he has looked into making libssh2 use the Secure Transport API, but that he faced some issues regarding big numbers which made him hesitate and consider how to move forward.

This story begins with a security flaw in OpenSSL. OpenSSL is truly a fundamental piece of software these days and I would go so far and say that lots of our critical infrastructure today is using it and needs it. Flaws in OpenSSL literally affect entire societies or at least risk doing so if the flaws can be exploited.

SSL/TLS is a rather old and well used protocol with many different implementations, both client and server side. In order to enhance how OpenSSL works with older SSL implementations or just those that have different views on how to implement things, OpenSSL provides an API call to tweak behaviors. The SSL_CTX_set_options function. In the curl project we’ve found good use of it for this purpose, and we use the generic define SSL_OP_ALL to switch on all “rather harmless” workarounds that OpenSSL offers. Rather harmless, that’s what the comment in the header file says.

Ok, enough background and dancing around the issue.Â The flaw that ignited my idea to write this blog post was a particular mistake made within SSL a long time ago within the code handling SSL 3.0 and TLS 1.0 protocols when speaking this protocol with a peer that could select the plain-text (see this explanation) – the problem is a generic one with the protocol so different SSL libraries would approach it differently. Ok, so OpenSSL fixed the flaw back in the days of 0.9.6d (we’re talking May 9th 2002). As a user of a library such as OpenSSL it always feels good to see them being on top of security problems and releasing fixes. It makes you feel that you’re being looked after to some extent.

Shortly thereafter, the OpenSSL developers discovered that some broken server implementations didn’t work with the work-around they had done…

Alas, on July 30th 2002 the OpenSSL team released version 0.9.6e which offered a way for programs to disable this particular work-around. By switching this off, it would of course make the protocol less secure again but it wouldÂ inter-operateÂ better with faulty servers. How do you switch off this security measure? By using the SSL_CTX_set_options function setting the bit SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS.

Ok, so far so good. But the next step is what changed everything from fine to not so fine anymore: they then added that new bit to the SSL_OP_ALL define.

Yes. In one blow every single application out there that use SSL_OP_ALL suddently started switching off this security measure as soon as they were recompiled against this version of OpenSSL. This change was made in 2002 and this is still like this today. It fixed the security problem from OpenSSL’s aspect, but the way the bit was later added to the SSL_OP_ALL define it was instead transferred to affect many programs.

In curl’s case, we were alerted about this flaw on January 19th 2012 and it resulted in a security advisory.Â I did a quick search for SSL_OP_ALL on koders.com and it is obvious that there are hundreds of programs out there still using this bitmask as-is. In the curl project we enabled the SSL_OP_ALL approach for the first time in the 7.10.6 release we did in July 2003. It was wrong already at the time we started using it. It turns out we’ve been enabling this flaw for almost nine years.

In the GnuTLS camp however, they simply stopped doing their work-around for this as soon as they started supporting TLS 1.1 due to the problems the work-around caused to some servers. This since TLS 1.1 isn’t vulnerable to the problem. OpenSSL 1.0.1 beta was released on Janurary 3 2012 and is the first OpenSSL version ever released to support TLS newer than 1.0… The browsers/NSS seem to have mitigated this problem in a different way and there’s a patch available for OpenSSL to implement the same work-around but there’s been no feedback on how or if it will be used.

At the end of the year I thought it would be interesting to have a look back over the past twelve months and see what the biggest changes in the curl project were and what the most important bug-fixes were etc. I’ve turned into a little mini series of blog posts. The top-3s. First out, the top-3 changes.

Top-3 changes

In total I counted 29 notable changes brought during 2011. The most significant ones in my view are:

Change 1 – fancier protocol support in proxy strings

This may sound trivial, and the code certainly is, but with this change suddenly a lot of applications that use libcurl got better proxy support without having do anything at all. Previously the application would have to set what protocol type the proxy it would use is, even though libcurl has supported having the proxy specified as an environment variable for ages.

Having only the proxy name was useful but limiting. With this new change you can specify proxy type or rather proxy protocol by prefixing the proxy name like “socks4://magic-proxy.example.com” if you want a SOCKS4 proxy. libcurl now supports socks4, socks4a, socks5 or socks5h used as prefixes as well as http.

Change 2 – allow sending “empty” HTTP headers

Another minor change in code but possibly larger impact in usefulness for applications. We introduced a way for applications to change internal headers and to add new ones ages ago. We (or rather I) then made the choice that if you’d provide a header with only the name and a colon, that is with no contents on the right side, it would delete the internal header. That was not a clever move as later on people have wanted to add “blank” or “empty” headers that look exactly like that, but libcurl has then refused to.

There have been some more or less hackish work-arounds to trick libcurl into allowing an empty header, but now finally we introduce a nice and clean way for applications to pass in these kinds of empty headers:

Pass in an empty header that instead of a colon has a semicolon! This is an otherwise illegal header that wouldn’t make sense, but libcurl will use that as a trigger that an empty header should be used and it will then replace the semicolon with a colon and things will be fine.

Change 3 – Added support for cyassl and axTLS

Proving that libcurl moves forward and into more and more markets, the number of supported SSL libraries grew to 7 this year. The all new cyassl backend that replaced the previously done yassl backend that was using cyassl’s former OpenSSL emulation layer. Now we’re using the native and pure API and things much cleaner. The possibly smallest available TLS library axTLS also got support.

Not all backends and not all SSL libraries are the same or support the same set of features, but then libcurl is used in many different scenarios and use cases and this way we offer more options to more users to craft libcurl for their particular needs. Our internal SSL backend API has managed quite well and proves to have been a worthy change. Adding support for yet another SSL library within libcurl is actually not a lot of work.

Change 4 – TSL-SRP

You may think number 4 of a top-3 list is weird, but I couldn’t cut it off here! =) TLS-SRP has been waiting in the shadows for so long and all of a sudden two of the major SSL libraries have support for it in released versions and libcurl got support for using these features in both libraries during 2011.

At the end of July 2009, Scott Cantor contacted us in the curl project and pointed out a security flaw in libcurl (in code that was using OpenSSL to verify server certificates). Having read his explanation I recalled that I had witnessed the discussion on the NSS list about this problem just a few days earlier (which resulted in their August 1st security advisory). The problem is basically that the cert can at times contain a name with an embedded zero in the middle, while most source code assumes plain C-style strings that ends with a zero. This turns out to be exploitable, and is explained in great detail in this document (PDF).

I started to work on a patch, and in the mean time I talked to Simon Josefsson of the GnuTLS team to see if GnuTLS was fine or not, only to get him confirm that GnuTLS did indeed have the same problem.

So I contacted vendor-sec, and then on the morning of August 5 I thought I’d just make a quick check how the other HTTPS client implementations do their cert checks.

So, Internet Explorer and Firefox were vulnerable. NSS and GnuTLS were. (OpenSSL wasn’t, but then it doesn’t provide this verifying feature by itself) (lib)curl, wget, neon, serf were all vulnerable. If that isn’t a large amount of the existing HTTPS clients then what is? I also think that this shows that it would be good for all of us if OpenSSL had this functionality, as even if it had been vulnerable we could’ve fixed a busload of different applications by repairing a single library. Now we instead need to hunt down all apps that use OpenSSL and that verify certificate names.

Quite clearly we (as implementers) have all had the same silly assumptions, and quite likely we’ve affected each other into doing these sloppy codes. SSL and certificates are over and over again getting hit by this kind of painful flaws and setbacks. Darn, getting things right really is very very hard…

(Disclaimer: I immediately notified the neon and serf projects but to my knowledge they have not yet released any fixed versions.)

yassl is said to be Yet Another SSL library and I’ve been told that for example it is the preferred library used by the mysql camp. I got interested in this several years ago when I learned about it since I thought it was fun to see an alternative implementation of OpenSSL that still offers the same API.

Since then, I’ve amused myself by trying to build and run curl with it like every six months or so. I’ve made (lib)curl build fine with yassl (and its configure script also detects that it is an OpenSSL API emulated by yassl), but I’ve never seen it run the entire curl test suite through without failing at least one test!

I asked the mysql guy about how yassl has worked for them, but he kind of shrugged and admitted that they hadn’t tried it much (and then I don’t know really who he spoke for, the entire team or just he and his closest friends) but he said it worked for them.

Today I noticed the yassl version 1.9.6 that I downloaded, built and tried against curl. This time curl completely fails to build with it…

Let me also point out that it’s not like I’ve not told the yassl team (person?) about these problems in the past. I have, and there have been adjustments that have been meant to address problems I’ve seen. I just can’t make curl use it successfully… libcurl can still be built and run with OpenSSL, GnuTLS or NSS so it’s not like we lack SSL library alternatives.

The same team/person seems to behind another SSL lib called Cyassl that’s aimed for smaller footprint systems and I’ve heard whispers about people trying to get libcurl to build against that and it surely is going to be interesting to see where that leads!

As the primary curl author, I’m finding the comments here interesting. That blog entry “Teaching wget About Root Certificates” is about how you can get cacerts for wget by downloading them from curl’s web site, and people quickly point out how getting cacerts from an untrusted third party place of course is an ideal situation for an MITM “attack”.

Of course you can’t trust any files off a HTTP site or a HTTPS site without a “trusted” certificate, but thinking that the curl project would run one of those just to let random people load PEM files from our site seems a bit weird. Thus, we also provide the scripts we do all this with so that you can run them yourself with whatever input data you need, preferably something you trust. The more paranoid you are, the harder that gets of course.

On Fedora, curl does come with ca certs (at least I’m told recent Fedoras do) and even if it doesn’t, you can actually point curl to use whatever cacert you like and since most default installs of curl uses OpenSSL like wget does, you could tell curl to use the same cacert your wget install uses.

This last thing gets a little more complicated when one of the two gets compiled with a SSL library that doesn’t easily support PEM (read: NSS), but in the case of curl in recent Fedora they build it with NSS but with an additional patch that allows it to still be able to read PEM files.

While I think the author Steven J. Vaughan-Nichols is mostly correct about his criticism, I think he’s being far too specific and trying to pinpoint Debian and claiming that to be a single specific bad distro (and his additional confused complaint on Firefox vs Iceweasel just made the article lose focus).

As someone who’s involved in a bunch of projects that are being packed by a range of Linux distros, I can’t but to disagree. This habit of changing packages without passing the changes upstream is wide-spread and not limited to changes done by maintainers since it also includes mere bug reports. It is something that just about every distro is doing to at least some extent. It varies from package to package and over time, but given an overview I honestly can’t say that there’s a single specific distro that is worse than the others. It is a disease that follows the distros and we must all help out to exterminate it.

Of course, the upstream projects also need to be aware of this and help pushing packagers of their software to behave.