In Debian Security Advisory 1571, also known as CVE-2008-0166 (New openssl packages fix predictable random number generator), the Debian Security Team disclosed a vulnerability in the openssl package that makes many cryptographic keys that are used for authentication (e.g. through SSH) or signing (e.g. web server certificates) potentially vulnerable.

End User Summary

all key types that were generated using openssl (this includes RSA and DSA keys)

compromise of other keys or passwords that were transmitted over an encrypted link that was set up using weak keys. Note that this last point means that passwords transmitted over ssh to a server with a weak dsa server key could be compromised too; see the Debian project's reaction to this.

The following cryptographic tools are unaffected:

cryptsetup (neither LUKS nor the regular dm-crypt use openssl, the openssl keyscript - which is not used in any default installations - does use openssl, but only to encrypt the key, not to actually generate the key that is used to encrypt the partition, the encryption of the key may therefore be less strong than expected but the key itself is not)

GNUTLS

GnuPG

Identifying Weak Keys

Characteristics of potentially vulnerable keys:

Generated since 2006-09-17

Generated with Etch, Lenny or Sid (Sarge is not vulnerable)

Generated using 'openssl', 'ssh-keygen', or 'openvpn --keygen' (GnuPG and GNUTLS are not affected)

In addition, any DSA key must be considered compromised if it has been used on a machine with a 'bad' OpenSSL. Simply using a 'strong' DSA key (i.e., generated with a 'good' OpenSSL) to make a connection from such a machine may have compromised it. This is due to an 'attack' on DSA that allows the secret key to be found if the nonce used in the signature is known or reused.

To fix this, first aptitude update && aptitude upgrade to install the new version of the openssl and libssl0.9.8 packages (the vulnerability is fixed in version 0.9.8c-4etch3 for etch and version 0.9.8g-9 for lenny/sid). You probably want to also pick up the new openssh packages that include the blacklist of known weak keys, but you will need to aptitude dist-upgrade for that in order to install the new openssh-blacklist package.

If you choose not to use the above aptitude command, note that all of the following packages must be upgraded (they all come from the same source package):

openssl

libssl0.9.8

libssl-dev

Then, regenerate and distribute any potentially vulnerable keys. Instructions for how to regenerate the keys for these applications are below. You can also test to see if keys are vulnerable using the dowkd.pl utility as described below.

How weak?

The broken version of OpenSSL was being seeded only by process ID. Due to differences between endianness and sizeof(long), the output was architecture-specific: little-endian 32bit (e.g. i386), little-endian 64bit (e.g. amd64, ia64), big-endian 32bit (e.g. powerpc, sparc). PID 0 is the kernel and PID_MAX (32768) is not reached when wrapping, so there were 32767 possible random number streams per architecture. This is (2^15-1)*3 or 98301.

Non-broken OpenSSL seeds from PID and /dev/urandom.

Application Details

Asterisk

Asterisk uses RSA keys as an optional authentication method for IAX2 and for DUNDI. Keys are SSL public/private key pairs. The Asterisk package does not generate keys automatically and most users don't seem to use them. You should probably know if you use such a key.

BIND9

To regenerate your rndc key, do the following. (This is what the postinst script does as well)

rndc-confgen -r /dev/urandom -a

> I don't know if this is neccessary or not though... ronalde: According to the changelog for bind9 in Debianrndc-confgen in Debian uses /dev/urandom since March 2002 (before then /dev/random was used); I guess rndc-keys aren't affected.

Keys for DNSSEC or DynamicDNS are probably weak too and should also be recreated through the use of dnssec-keygen(1). Exactly what parameters to use depends on how you are using the keys. See Secure DDNS Howto for some examples for DDNS.

Then the keys must be distributed out to each host prior to running csync2 again.

cyrus imapd

To find out which certificates/keys are in use, see the directives whose names contain "key_file" or "cert_file" in /etc/imapd.conf.

Generate new private keys and certificates as shown in "Generic PEM Generation" above, then restart the service:

invoke-rc.d cyrus21 restart

if you are using version 2.1 (if you are using another version, please examine /etc/init.d for the correct service name)

dovecot

Generate a new PEM as shown in "Generic PEM Generation" above (making sure that it is placed in the correct place according to your dovecot configuration, then restart dovecot: invoke-rc.d dovecot restart .

ftpd-ssl

Generic PEM Generation

This is just a reminder for those who generate PEM encoded certificates. Your site probably has other policies in place about how to manage keys which you should follow. Additionally, you may need to get the certificates signed again by a 3rd party Certificate Authority rather than by using a self-signed certificate as shown below:

OpenSSH (Server)

Updated packages for openssh that have a blacklist of known weak keys are now available; see DSA 1576 for more information. Installing these packages on hosts with weak keys will cause the ssh server to regenerate its keys. Weak user keys being used for a connection will also be rejected where possible.

Note that you will have to use aptitude dist-upgrade (or apt-get dist-upgrade) to install these packages rather than just upgrade because this update will cause the new package openssh-blacklist to be installed.

You can also update your openssh-server keys manually.

rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server

Note that in either case, your users will see a "IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!" warning when they next log on to your ssh server because the key has changed. They will need to edit $HOME/.ssh/known_hosts to remove the offending line before continuing; checking that the key fingerprint is correct, of course (the fingerprint of your new key can be found with ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key). You can remove the key from known_hosts by running "ssh-keygen -R hostname"

StrongSWAN

OpenVPN

If you're using x509 certificates, you need to create a whole new CA if you generated the CA key with a broken OpenSSL. Even if you CA key isn't compromised, some of the keys of the OpenVPN clients might be. In that case you need to revoke all the certificates for those keys and add the CRL to your OpenVPN configuration. See the OpenVPN HOWTO for more information about revocation.

postfix

Generate a new PEM as shown in "Generic PEM Generation" above (making sure that it is placed in the correct place according to your postfix configuration, then restart postfix: invoke-rc.d postfix restart .

ssl-cert

telnetd-ssl

tinc

Tor Onion Router / Hidden Service Keys

encfs

Analysis of the impact is still ongoing. Details known so far: encfs uses the RNG from libssl to create an internal encryption key with some post-processing applied. Checking of that key against a pre-calculated blacklist like with dowkd.pl might be possible when it's ready.

In the meantime, encrypted filesystems that might be created with a broken version of libssl shall be considered vulnerable against offline attacks. Any copy of encrypted data located in unthrusted environment might become readable, sooner or later. If the date of creation is unknown, checking of the originating version in output of the encfsctl command against the changelog of encfs package may provide some hints but is not absolutely reliable either because there was no strong dependency on certain openssl version.

To regain security, try following:

Create another encfs file system with encfs after upgrading to the latest (fixed) openssl library

Mount the original FS and copy the data into the new location

Umount the old filesystem. Destroy the contents of the old encrypted files using the shred command. (If the files have been moved rather than copied than make sure that the remaining space on the particular block device or filesystem is rewritten with random data).

xrdp

Kerberos (MIT and Heimdal)

If you were using MIT Kerberos, you would be fine so far as I can tell since MIT Kerberos has its own crypto layer and its own randomness functions. Heimdal uses OpenSSL as its crypto layer. Given that, it may well be possible to brute-force the session key of any captured GSSAPI-encrypted traffic and decrypt it retroactively.

If you're using Heimdal, you should also change all long-term random keys (such as any key in generated keytab files) that were generated using the vulnerable version of OpenSSL.

This can be done using cpw -r <principal> within kadmin. Take into account that this principals have been randomed-key assigned and should be regenerated as well.

pwsafe

First issue: the random seed of the DBs is now the same. More exactly, the random, the salt and the iv. The random is fixed at creation of the DB. Apparently the seed & iv are renewed at every edition of the DB and depend only on a fresh random. Data encryption depend only on hash and seed/iv so even with different random, if the fresh random for seed is the same, the encrypted data is the same. A practical attack against a pwsafe DB is e.g. to construct a generic rainbow with the possible random seeds and your favorite character class. Once done it could attack any pwsafe DB generated with a broken OpenSSL or even edited with a broken OpenSSL (to be confirmed).

Second issue: all nice passwords proposed by pwsafe when creating a new entry are now broken! So are all the accounts you created around based on those passwords. No matter the DB/user/group/... you are using. Of course no matter how decently you seeded the RANDFILE ~/.rnd (cf man). To rekey your pwsafe, create a new one, import the old one then delete the old one:

Copy the key files on the controller and the backup controller and the cert file on all the nodes of your cluster then restart SLURM with the command:

invoke-rc.d slurm-llnl restart

SSL Certificate Reissuance

If you paid good money to have a vulnerable key signed by a Certificate Authority (CA), chances are your CA can re-issue a certificate for free, provided all information in the CSR is identical to the original CSR. Create a new key with a non-vulnerable OpenSSL installation, re-create the CSR with the same information as your original (vulnerable) key's CSR, and submit it to your CA according to their reissuance policy

Comodo: Comodo customers should log into their Comodo accounts to submit their new CSR for a free re-issue. Comodo is offering free replacement SSL certificates to any online businesses affected by the security flaw even if the original certificate was provided by another company. Non Comodo customers should visit http://www.instantssl.com/ssl-certificate-support/debian/ssl-certificate-contact.html to apply for their free replacement SSL certificate.

?GoDaddy: http://help.godaddy.com/topic/234/article/867?GoDaddy calls the process "re-keying", while they call the act of sending you the same signed certificate as your original order a "reissuance". Via the web interface re-keying is only possible within 30 days of the initial order but email support (ra@godaddy.com) with the ?GoDaddy account # and certificate(s) domain name(s) and they will issue re-key credits valid for 7 days.

ipsCA: Generate a new CSR as if you are purchasing a new certificate, follow through the procedure up until you get to the point where you are required to pay with your credit card. At that point contact support via their email and let them know that you are requesting a revocation and re-issue and include the ticket number of your new CSR request.

CAcert: This is a cost free certification authority. Simply revoke your old certificates and add new ones. (The key has to be created on a fixed machine and ONLY the certification request has to be uploaded!) At the moment the certificate generation will take some time as it seems that many users are re issue there certificate.

Testing keys using ssh-vulnkey

In the latest openssh-client packages for etch, ssh-vulnkey is included along with a blacklist of known weak keys. This tool appears to be similar to dowkd.pl (below) although it is restricted to checking ssh keys (not that dowkd.pl can do much more than that anyway).

Run ssh-vulnkey as a user to check your host and your own keys.

Run ssh-vulnkey -a as root to check all users keys (as well as the host key).

Note: about output is one line per key not linewrapped as shown here in the wiki

Note: openssh-blacklist-0.1.1 provided for Debian Etch does not include 4096bit or 1024bit RSA fingerprints. Instead, grab the openssh-blacklist-0.3 package from Debian Unstable and backport it, or grab ready-made debs for Etch from http://love.hole.fi/atte/openssh-blacklist/

Testing keys using dowkd.pl

The security team has prepared a utility for you to test your keys to see if they have a fingerprint that matches a list of known-weak finger prints. Download dowkd.pl.gz (gpg signature) Attention: This version is not the same than the original tool released by Florian Weimer. So the patches below will not apply!

Changes to the original version:

Added copyright note

Improved Note

Added verbose output

Better check for keytype/length matching

More robust

Integrated PEM support (Available with the file option)

Integrated support of the patches below (different implementation)

Added support other key lengts

As I (Klaus Ethgen) can see the tool do what it should (Version sha1:027adef3b86991661394b6ab6159d75dde3c3087 0.9). But please check yourself. Also if someone make update of the script please tell it on this page to not confuse users.

Note that dowkd.pl may produce false negatives. As of 2008-05-19, the current version of dowkd.pl prints warnings if a false negative is likely, except if the key in question has been generated on a big-endian architecture. If unsure, regenerate your key material.

PEM keys (SSL certificates)

The SSL server (and client) certificates, aka PEM keys, may be used in various tools (Apache + mod SSL, etc.).

The Ubuntu team has created a package which will verify if PEM files are vulnerable. I used it against PEM keys created on a known vulnerable system and the utility highlighted them as vulnerable. The package is called "openssl-blacklist". The package is available here. It can be installed on etch using dpkg -i --force-all.

I have as yet not determined how accurate this method is, but it seems to work reliably. Someone more familiar with the workings of ssh-keygen can confirm the reliability of this method. Scripting this test should be easy enough.

If the code for generating the key in your PEM file used OpenSSL's random number generator even slightly differently than ssh-keygen from openssh, it's likely to have keys that are equally-weak but not contained in that blacklist. I wouldn't recommend drawing any conclusions from this method. -- ?MarshRay

Can you tell us how you tested this? We are unable to reproduce your results as none of the keys we generate with the known-weak version of openssl are flagged as being weak by dowkd.pl or dowkd.pl with RichiH's patch (below). Details of how those certificates were generated are below; are you doing the same? -- StuartPrescott

someone who actually understands this tool and its output, please insert details here!

I don't know what kind of information is being requested, but it seems fairly easy. On first invocation of the tool, it generates a DB file from the "blacklist" contained in the __DATA__ section of the script. If the DB exists, it is opened for reading. Once the DB exists, it uses ssh-keygen to retrieve the fingerprint of the public key presented to it by the executor of the script, and compares the fingerprint that it retrieves to the blacklist. If there is a match, then the script prints 'weak key'.

This is perhaps irrelevant now -- an earlier version of this tool didn't say "weak key" but just printed the key name. It also completely failed to parse correctly formed authorized_key files. Perhaps it has now been sufficiently refined as to now be understandable. This page has been a work in progress throughout the day and getting the bones of the content here so that it could then be added to by others was the priority. -- StuartPrescott

I created a patch to parse PEM files. fw told me to look at the Modulus of the RSA key, so I am doing that. Even though I tested more than 20 certificates generated on an affected machine, I did not find a single weak key. This could mean that

the list is incomplete

I am looking in the wrong place

the certificates are not, in fact, affected by this. Note that I did not generate them myself as I do not have any vulnerable hosts left

As I manually verified the results against, I do not think there is an error in my patch. It is worth noting that none of the certs showed any hits when I converted them into SSH format. Please test this and report back to me!

When we were throwing test cases around on #debian (approx 2008-05-14 02:00 UTC for those who have logs), I threw together this script to run on an unpatched etch box. None of the keys were marked as weak by dowkd.pl when tested by either method even though the keys were generated by a known weak version of openssl. I can only conclude that these methods of testing this class of key for weakness must be incorrect. -- StuartPrescott

I think the existing blacklist works only for keys generated with ssh-keygen and a keylength oh 2048 bits. I tried to produce vulnerable keys on a vulenrable system with ssh-keygen and did not get any weak keys as per dowkd.pl for 1024 bits while 2048 bits worked every time. I also tried to generate 2048 bit keys with openssl genrsa and had no success either. I had a look at the generated keys and found that ssh-keygen uses 35 as public exponent while openssl uses 65535. So probably the blacklists don't fit again. Unfortunately this means we cannot use the script for checking keys generated with openssl until we get updated blacklists or a description how these were generated. -- Joachim Ring

The Patch for authorized_keys2 and known_hosts (dowkd.pl.ke.patch.gz) is not needed anymore. -- Klaus Ethgen

I have hacked a little shell script using the tables from openssl-blacklist from ubuntu which allows the user to check the ssl key of a webhost from remote by just calling "chksslkey hostname". You can find it here: chksslkey.tar.bz2 -- Michael Holzt

I threw together a little script that combines everyone elses and lets you test a cert, ssl rsa key, pem file, or remote https against the blacklist's. You can find it at http://www.gokickrocks.us/wp-content/uploads/2008/05/audit-ssl.tar.gz. Super easy to use, just take a peek at the README. Basic usage is just "chkssl testtype fileorurl". -- Florian Hines

Scope of the blacklists

The current official blacklists (in "openssh-blacklist") cover RSA-2048 and DSA-1024 keys as generated on 32-bit little-endian, 64-bit little-endian and 32-bit big-endian systems. That's sufficient to cover users who used the default key length, but not if some of your users decided they wanted a longer keylength. For non-default keys, see "openssh-blacklist-extra" which contains RSA-1024, RSA-4096, and (an empty) DSA-2048 for all three architectures.

I have made a blacklist for RSA-4096 keys as generated on 32/64-bit little-endian systems and posted it at http://www.red-bean.com/~maxb/ . The 64-bit keys I generated, the 32-bit ones I took from the metasploit site linked earlier on this page. The format is full fingerprints:

suitable for appending to dowkd.pl, if you also edit the body of the script so that it tries to check 4096 bit keys too

or, if you remove the first 12 characters from each line, suitable for use with ssh-vulnkey

(There are also RSA-1024 and RSA-1023 versions there, potentially useful for the paranoid evaluation of older keys.) --?MaxBowsher

I used MaxB's RSA-4096 fingerprint list for my unofficial openssh-blacklist-0.1.2, but it seems there is now openssh-blacklist-0.3 in Debian Unstable which includes these, and RSA-1024 which is also missing from the Etch package. I backported it for Debian Etch, grab it from http://love.hole.fi/atte/openssh-blacklist/ or backport yourself. --Atte Peltomäki

Technical Summary

(If you want to add more technical details that an end-user doesn't need to know or isn't likely to understand, please add them here rather than making the above summary impossible for the average user to understand.)

Causes

It is important to understand that this problem was caused by trying to remove valgrind warnings related to the use of uninitialised memory within the openssl libraries. This was done to try to make it easier to debug C applications that use the openssl libraries which is a good thing to do.

A discussion of why this change was made can be found at #363516 and also on the openssl-dev list. Judging from the discussion there, the main culprit seems to be a misunderstanding about which is the right list to ask this question on, followed by misleading answers from the list.

A bit more detail

In an effort to clear up confusion about this bug, here's a bit more technical description. This was caused by an overzealous, well-intentioned elimination of code that was believed to have no impact on security. (Please do note from the above links that this was discussed with the !OpenSSL team and that no objections were raised at the time.)

The second of these is in ssleay_rand_bytes where buf is used as an output buffer. It had already been marked as a bad idea when -DPURIFY was in effect, because Purify (and valgrind, naturally) dislike this use of an output buffer as input. This use of MD_Update is dubious but shouldn't hurt as long as the mixing function of the PRNG is "sufficiently good". The removal of this call to MD_Update should not meaningfully alter the entropy available in the pool -- that is, OpenSSL does not depend upon uninitialized memory for its correct operation.

The first call, however, is in ssleay_rand_add where buf is used as an INPUT buffer, to add entropy to the pool. Failing to call MD_Update there means that the pool will never actually get the entropy intended for it.

Acknowledgements

This document was put together by the folks on #debian to help with user support. It would have been nice to have had the chance to prepare this (and the key rollover page) in the 5 days between the commit being made to pkg-openssl and the DSA being released. Particular credit to themill, dondelelcaro, stew, Maulklin, iobound for reading, writing and suggesting bits for this page. No thanks to the people who keep using the wiki's GUI editor and keep generating 15 to 20 KB diffs from when it randomly reformats the entire page when you only change one line.