Reviewers

Assignees

Labels

Projects

Milestone

16 participants

Disclaimer: I am not a security expert. This PR is mostly common sense, but should still be reviewed by the community before being merged, especially those well-versed in security.

This PR contains the following security fixes, and attempts to address issue #1074:

Self-update and packagist are now forced to use SSL.

This introduces a hard dependency on the openssl extension when using
self-update or packagist. Users should not be using either without
utilizing a proper SSL connection anyway.

Keeps a trusted CA bundle file in the composer home directory.

Attempts to copy over the operating system CA bundle if available.

Upon encountering an untrusted certificate, the user is prompted if
they would like to trust the certificate and the CA in the future. If
they choose yes, the root CA for that cert is added to the CA bundle
in the composer home directory.

- Self-update and packagist are now forced to use SSL.
- This introduces a hard dependency on the openssl extension when using
self-update or packagist. Users should not be using either without
utilizing a proper SSL connection anyway.
- Keeps a trusted CA bundle file in the composer home directory.
- Attempts to copy over the operating system CA bundle if available.
- Upon encountering an untrusted certificate, the user is prompted if
they would like to trust the certificate and the CA in the future. If
they choose yes, the root CA for that cert is added to the CA bundle
in the composer home directory.
- Expired certificates generate an extra warning and prompt.
- Invalid common names generate an extra warning and prompt.

- Removed $key var that was used for a certificate cache implementation
which I've since removed.
- Moved whitespace outside of Symfony console color tags.
(Side note: the color tags aren't working for me in gnome-terminal for
some reason...)

Unfortunately the git protocol has no support for TLS or X.509 client certificate verification, AFAIK. Defaulting to HTTPS, and allowing users to trust Github's cert/CA ensures that users are not vulnerable to MITM. This way, if a MITM occurs, the user will get a prompt that A) the certificate they're being served is not valid for GitHub.com and B) that the CA has changed to an untrusted party. If a user continues at this point, it's their responsibility. Composer has done everything it can to protect them. By defaulting to git://, a MITM could occur silently.

If we could promote and enforce the use of signed tags, this problem would go away entirely.

Note that the github protocols stuff is used only for downloading packages from "source" (as opposed to dist), which means typically it is dev versions of packages, and not tags. Now assuming you get MITM'd, I think the git process would prompt for user acceptance of the new signature, and it'd block there because that is hidden from the user, and then it'd time out after a while.

That's not completely true, because you changed http urls to https above (getcomposer.org & packagist). In those cases a downgrade to http is possible. Maybe it can just always try to downgrade and if it fails with http then fail hard.

Also if you keep http in the chain, if https fails it'd then fallback on http, so that kind of defeats the whole point. I think either we force https (but I know for a fact that this won't work for some people.. that's why we have this chain thing in place) or we try https first, then git and http and try to read the https failures to see if it failed because of MITM (in which case abort) or because of an expected failure (then try next in the chain)?

If we really need to support those users, here's how it should work for them: The first time they run composer, it will check for openssl immediately. If it's not found, it'll throw a nice warning explaining that they are strongly encouraged to enable openssl before using composer and explain the implications of not doing so. The prompt will then ask them if they wish to put composer into an "insecure mode", if they say no, composer quits, and on next run the same prompt will show up. If they say yes, we put an 'insecure_mode': true, entry into their config.json file in their composer home dir. If that's enabled, a warning will still be thrown on each composer run, "WARNING: Composer is running in insecure mode!", etc, but it will not stop at the openssl check. We'll also make fallback to http:// and git:// dependent on insecure_mode being on. Additionally, we can have an environment variable, COMPOSER_INSECURE_MODE which would do the same thing if it's set.

I think this PR is a great step towards making Composer more secure. I cannot help but notice though that a lot of the comments seem defensive, seemingly preferring less secure options. Given that things like these were not built in from the start, and esp. stuff like gpg signing was not included from the start, I have to wonder if it's secure by design, or that the intention is now to patch it up, hoping we cover enough to make it more or less secure?

There was simply no expectation of security at the beginning. To be entirely sure about the security of the code you run you will have to review it yourself. When using 3rd party libraries you always place some trust into those 3rd party libraries. The work being done now, is to make it more difficult for an attacker to get between you and that 3rd party. The responses aren't defensive, they are trying to make sure composer continues to actually work. Because what is the point of making it more secure if you can then no longer use it? Hence we need to come up with secure and working solutions like the proposed option for switching to a mode in which communication is not necessarily secure if that is not an option available on the respective system.

@naderman I totally understand your points. That said, your quote, "To be entirely sure about the security of the code you run you will have to review it yourself. When using 3rd party libraries you always place some trust into those 3rd party libraries," leaves this open: you need to be able to trust that the 3rd party library whose code you are reviewing is actually from the 3rd party you tried to obtain it from. That's the primary driver of this PR, from my standpoint as primarily a user, and secondarily somebody providing packages for others.

I'm not sure this is a safe way to check the CN. I'd have to look into it more, but iirc the cert used to verify the connection is not always the last in the chain. I'm concerned that a self-signed cert after a valid cert for another domain, could be added to the end of the chain, pass the openssl verify, and then get to here and also pass this check. ... I might be wrong though and would have to look into the specifics of how openssl is validating the chain.

I'm not sure myself at the moment. For those wondering why this is being done here, PHP doesn't check for both the CN and SAN entries, it only checks the CN. As a result, PHP's internal handling can fail a connection even on a valid certificate for the current Host which is a serious reliability issue since SANs have been accepted practice for years now. As a result, we need to increase reliability through some manual CN/SAN checking.

Yeah -- I wasn't 100% on this part. I'd appreciate if someone more familiar with SSL could double check both how I'm assuming the CA / top certificate and also how I get and verify the CN / alternative names. The fact that I'm skip the intermediary certificates is almost certainly wrong; I'd appreciate any pointers or links o docs/specs that would give a clue how to do that part properly.

IIRC openssl uses a scramble algorithm, (ie can it find any path to valid), and the order of the certs is almost irrelevant. See Moxie's TACK proposal for how they add a second chain to a TLS connection for a good practical example of this.... I'd have to source out where that algorithm is in code, but yes, I think this might be exploitable.

This verify operates on a separate stream context it seems. Verifying a connection unrelated to the actual transfer? This will probably lead to a client-profiling attack. (Eg let the cert check hit the real server, intercept the 2nd request)... this might need more review.

Ah, you're right about it being a string -- unfortunately it still seems not to take into account the alternative names in the certificate. I tried with 'CN_match' => $this->fileUrlParts['host'], and it still fails for getcomposer.org and packagist.org because the CN is for dl.getcomposer.org and dl.packagist.org -- we need to check the alternative names, and it seems PHP doesn't do this. Perhaps I'm doing something wrong.

This is a PHP bug or feature omission. You can't rely on the CN_match setting reliably. For getcomposer.org, the CN is actually dl.getcomposer.org (mismatch from PHP's perspective since it ignores the valid SAN entry for getcomposer.org).

Just FYI, the dl.* should go away as soon as get my hands on a wildcard cert for the domains. It's probably a good idea to handle this case though, because the cert is valid for both dl.getcomposer.org and getcomposer.org, and works just fine in all browsers for example.

@Seldaek The cert you have is perfectly valid. PHP just hasn't been updated to support Subject Alternative Names in line with common practices - something we noticed recently when digging into PHP's SSL/TLS support in depth. The current cert also works fine with Curl.

I'm pretty sure we'll need the whole chain rather than just the peer certificate to validate this properly. I should be able to do it without the second context in a way that each request is validated as to not allow for the client profiling attack mentioned above (great find, by the way).

Shouldn't need it. The peer_cert has already been validated by openssl. See the C code. If you make sure to do the SAN check on the same CTX and make sure that verifypeer and allowselfsigned are set correctly, then you can be sure of the peer validation. Only check left to do is CN and SAN on the peer_cert. You dont need the chain (and really shouldnt look at it)

Oops, I didn't mean we need it to validate the cert. You're right, that's done by openssl. But in the event that it's not trusted yet, won't we need the chain to add the proper one to the trusted CA bundle before trying the request again?

Oh you're trying to override a peer validation failure too? Rather than have the users accept an insecure bundle, why not have them configure a secure bundle? (eg prompt the user to apt-get install ca-certificates ) I think manually adding CA roots might be a bad idea, especially given that they may need to be revoked.

That's actually a good point... I have no clue how that would work on Windows though.

On *nix, we can just check the common paths like /etc/ssl/certs/ca-certificates.crt, /etc/ssl/ca-bundle.pem, /etc/pki/tls/certs/ca-bundle.crt and /usr/share/ssl/certs/ca-bundle.crt and we have most everyone covered.

My original thought was that we show the user the sha1 fingerprint and strongly encourage them to verify that against what they get in their browser. (For a simpler process, we could even provide an https://getcomposer.org/ssl-fingerprint and https://packagist.org/ssl-fingerprint) but you're right that this could be bad if the certificates need to be revoked; ideally the root certs are installed via some mechanism that auto-updates.

The less than ideal solution if the host has no trust might be if its the composer url itself that is failing to validate (ie have no ca bundle at all), then bundle the current known composer public key in the phar. (chances are folks are going to experience this error shortly after downloading the latest version anyway) Then I'd add a new command, "composer build-ca", which will grab a known-good cacert.pem from https using the pre-known key. Install that for composer-use only, and go from there.

Again, this is just there for usability reasons, without it presently if you don't have unix utils installed on windows, you can't download composer via CLI. This one might be solvable by shipping a windows MSI installer though, at which point I would happily remove the eval hack.

As for what @EvanDotPro suggested (adding CA roots) AFAIK it's only added to ~/.composer/trusted-ca-bundle.crt, which is composer specific and will not affect anything else. Obviously it is still a problem if it contains revoked CAs, so I guess the build-ca or similar command would be a good solution and could be done as part of the installer script as well.

@StormTide Do you have any links/resources about the fingerprint forgery you mentioned on twitter? Trying to think of how to improve the approach used in Sslurp. Seems to be quite a chicken-or-the-egg problem.

@EvanDotPro it really depends on the type of hash being used and the circumstances of the validation. Such attacks are known as preimage attacks and vary depending on the hash etc. Generally speaking they're pretty hard to accomplish in a sha1 hash cert. Since you're sha1'ing the cert manually, this might be ok, but generally speaking the cert fingerprint is not guranteed to be sha1, and could be a preimageable hash. (remember certs can sport many hash and signing algorithms)

Interesting stuff. So I'm trying to figure out how to allow verify_peer on the initial connection to mxr.mozila.org when we don't have any valid CA bundle yet. See my comment here. Thanks for all your help reviewing this, by the way!

Yeah, it's a bit tricky if the initial bundle can't be grabbed from a secure source. Any way getcomposer.org could perhaps host a secure copy and occasionally grab an updated copy and it's hash for distribution? Single point of failure, yes, but Composer can quickly establish HTTPS to getcomposer.org with an initial PEM from the installer so a secure route is now established? It's convoluted, I know!

@Freeaqingme if you look at the comments here, you may notice that we've for many people deeply interested in security struggling to figure out how exactly to do validate an ssl connection in php. That is part of the reason the network security hasn't been addressed before: it's a huge pain and php is clearly not helping. As for the defensive comments, I was just trying to find a balance between security requirements and usability. Some people just can not enable openssl for various reasons (often bad ones IMO, but we can only nudge in the right direction), and it should not mean they can't use composer at all. That said, I'm glad we now have so many people interested in fixing this for good.

No one should be using composer over http the way it is today without package signing. IMO It shouldnt even be an option. It's like voluntarily installing a backdoor on your machine.

The difficulty here comes from trying to use the built-in openssl stuff which, yes, requires some understanding of SSL to use... cURL could handle this easily, safely and securely with default options on the other hand.

@Seldaek It's understandable that many people can't enable openssl, I know how policy and bureaucracy can get in the way of things. To address this, however, I believe composer should be secure by default and only be insecure if the user specifically configures it to be that way and throws lots of warnings in the users face when they do.

@PHLAK That's what the PR is aiming for. A lot of what is now being discussed (several of us being security knowledgeable) is working through PHP's obstacle course. It's native SSL/TLS handling sucks compared to Curl which is defaulted to be secure and usually has a CA bundle PEM pre-configured on package repos.

@PHLAK, @padraic, The nice thing is, with sslurp, Composer will be able to build and keep up-to-date the exact same CA bundle PEM that cURL uses -- it builds from the same source and in the same way as cURL's mk-ca-bundle.pl, except in a much more secure way (funny that cURL's build script fetches the CA bundle over HTTP).

@HosipLan, I'll eventually get around to cleaning the code -- right now we are focusing on actual security and integrity of the data. If it's a problem for you right now, I'd gladly accept a pull request.

@sandermarechal You would also need to include the CN_match option in the SSL Context (to ensure the valid certificate is for the Host being requested over HTTPS). Secondly, you also need to replace all instances of file_get_contents() across Composer. Recommended security practice is to disable the allow_url_fopen setting in php.ini that Composer currently requires to be enabled. This means switching from file_get_contents() to a non-file op using sockets.

As mentioned on Twitter, I'm looking to finally finish this up this month now that I have the time. I need some help reviewing Sslurp, as I want to make sure the method it uses for establishing a trusted certificate bundle is well-reviewed and agreed to be secure before I integrate it into Composer. I also want to look at fixing the installer script as well, because if they get a compromised copy of Composer in the first place, all this work is for nothing.

@till unfortunately, I shortly after SunshinePHP, I began to get pretty busy again. It's looking like there's beginning to be a renewed interest in this again though, and I have a new client interested in implementing composer into their workflow, so maybe with enough poking and prodding from the community, I can try to find some time to wrap this up once and for all. I haven't really gotten any official 👍's on Sslurp from known security professionals yet though, which would certainly be helpful.

@EvanDotPro would it make sense to crowdfund this for you? Because this is really important issues which has to be solved. I believe a lot of developers would help you to get your time paid so you can find some time to do it.

Also there are many groups in PHP world which would benefit from this pull request, like Zend, Sensio and everyone who uses composer to install their dependencies which are then used on production servers (this is major issue, because lot of people uses CI to automatically install dependencies and they then deploy these to production directly).