Share this story

A widely used cryptographic algorithm used to secure sensitive websites, software, and corporate servers is weak enough that well-financed criminals could crack it in the next six years, a cryptographer said.

The prediction about the SHA1 algorithm, posted recently to a hash function mailing list sponsored by the National Institute of Standards and Technology, is based on calculations its author and fellow cryptographers admit are rough. The back-of-the-envelope math also incorporates several assumptions that are by no means certain. At the same time, the ability to carry out a reliable "collision attack" on SHA1 would have catastrophic effects on the security of the Internet.

Similar collision attacks on the weaker MD5 algorithm provide an example of how dire and widespread the resulting harm could be. The Flame espionage malware, which the US and Israel are believed to have unleashed to spy on sensitive Iranian networks, wielded such an exploit to hijack Microsoft's Windows Update mechanism so the malicious program could spread from computer to computer inside an infected network. Separately, in 2008, a team of computer scientists and security researchers used the technique to forge a master secure sockets layer certificate that could authenticate virtually any website of their choosing.

SHA1 is considerably more resistant than MD5 to collision attacks, in which two different plaintext sources generate the same ciphertext, or digital signature. As a result, SSL certificate authorities, software companies, and most other security-minded organizations have discontinued use of MD5 in favor of SHA1, or better yet SHA2, which is believed to be stronger still. (Just this week, NIST designated an algorithm known as Keccak to be SHA3.) Cryptographers have long presumed these more advanced algorithms will suffer the same fate as MD5, as computers' processing speeds become ever faster. With SHA1 a staple in digital certificates that certify the authenticity of websites, commercial software, and credentials used to administer corporate servers, a practical attack on it anytime soon would come with dire consequences.

"When it does happen, it's going to be a disaster, because SHA1 is everywhere," said Matthew Green, a professor specializing in cryptography at Johns Hopkins University. "You could be Microsoft, you could be Google, if you were able to get an attack on SHA1."

SHA1 and other hash algorithms generate a digital fingerprint that in theory is unique for each different file or text input they sign. When the underlying plaintext is altered in even miniscule ways, the signature changes. The algorithms are used cryptographically to prove that no unauthorized changes have been made to websites or software code. Collision attacks undermine this assurance by allowing attackers to forge the cryptographic signatures provided by the algorithm.

Based on the rough calculations of Jesse Walker, an Intel employee and a designer of a SHA3 runner-up algorithm known as Skein, SHA1 may fall sooner than many expected. He reached that conclusion based on the continuing growth of computing speed and the advent of cloud services such as Amazon's EC2, which allow people to rent commodity servers for as little as 4 cents per hour.

"A collision attack is therefore well within the range of what an organized crime syndicate can practically budget by 2018, and a university research project by 2021," Walker wrote, according to a transcript included in a blog post by fellow cryptographer Bruce Schneier. Walker went on to say his assumptions didn't take into consideration the use of graphical processing units and other optimized hardware, so "the need to transition from SHA-1 for collision resistance functions is probably more urgent than this back-of-the-envelope analysis suggests."

Walker's estimate is also based on the assumption a collision attack exploiting SHA1 would be based on a technique laid out earlier this year by Marc Stevens, a cryptographer from the Centrum Wiskunde & Informatica in Amsterdam. If attackers came up with a more efficient method for bringing about a collision attack—as the world-class cryptographers behind Flame did—that might also be a factor that would accelerate the downfall of SHA1. The estimate also assumes that Moore's Law, the observation that computing speed capacity doubles roughly every 18 months, will continue for the foreseeable future.

"The point is," Schneier concluded, "that we in the community need to start the migration away from SHA-1 and to SHA-2/SHA-3 now."

Promoted Comments

No it is not. Just webpages and browsers need to move to TLS 1.2. TLS 1.2 supports SHA-2 hashes. It's been around for years. I implemented a solution using it in a private EFT terminal implementation years ago.

Firefox and Chrome ironically are towing the chain and sad to say but IE and Opera lead in this regard. Websites aren't being migrated because browsers don't support it and browsers don't support it because there aren't any webpages that use it. Really browsers should lead here but they are pretty irresponsible.

The problem is that SHA1 is embedded into many higher-level security protocols, which are embedded into many commercial products. The most notable example is SSL/TLS, used to secure web sessions and VPN tunnels.

The process to change is very long and difficult. It starts with the standards bodies updating the standard to include one of the newer SHA-2 algorithms (or preferably, SHA-3). Then, device and software manufacturers have to incorporate that standard into new products. These new products take (often) years to roll out into the consumer space and gain a decent adoption rate. The year 2018 isn't all that far off, in this context.

Software, such as web browser and the like, are fairly easy to upgrade with a patch. Devices are a lot harder. There are crypto algorithms baked into the firmware of modems, routers, switches, etc all over the infrastructure, to secure administration channels and other secure tunnels. Often, even if you could upgrade the firmware to include stronger crypto, the hardware on the device isn't able to perform the calculations in a reasonable timeframe due to their complexity.

Then comes the problem of backwards compatibility. What do you do with all the devices that can't effectively use the stronger crypto? What happens when mom-and-pop's router tries to connect to the network and the device on the other end requests SHA-3/RSA-3072 to secure the connection but the device only supports SHA-1/RSA-1024? Do you disallow them on the network? Subsidize a replacement device?

For background, I help major F-500 companies (and smaller) gain their FIPS 140-2 validations on IT products you use every day. SHA-1 is already 'deprecated' according to the standard (along with RSA-1024) so progress is being made. The transition schedule can be found here. The testing/validation methods for SHA-3 will be developed shortly, and with that, I believe many companies will begin to incorporate it quickly. However, FIPS allows the support of older algorithms for backwards-compatibility. So, ultimately, the problem has to be addressed at the standard-level, and vendors have to be prepared with products as soon as those standards are finalized.

This is a real problem, and the transition schedule of algorithms is an important issue to the vendors, and ultimately, to the general public. I appreciate Ars raising awareness on the issue.

No it is not. Just webpages and browsers need to move to TLS 1.2. TLS 1.2 supports SHA-2 hashes. It's been around for years. I implemented a solution using it in a private EFT terminal implementation years ago.

Firefox and Chrome ironically are towing the chain and sad to say but IE and Opera lead in this regard. Websites aren't being migrated because browsers don't support it and browsers don't support it because there aren't any webpages that use it. Really browsers should lead here but they are pretty irresponsible.

The problem is that SHA1 is embedded into many higher-level security protocols, which are embedded into many commercial products. The most notable example is SSL/TLS, used to secure web sessions and VPN tunnels.

The process to change is very long and difficult. It starts with the standards bodies updating the standard to include one of the newer SHA-2 algorithms (or preferably, SHA-3). Then, device and software manufacturers have to incorporate that standard into new products. These new products take (often) years to roll out into the consumer space and gain a decent adoption rate. The year 2018 isn't all that far off, in this context.

Software, such as web browser and the like, are fairly easy to upgrade with a patch. Devices are a lot harder. There are crypto algorithms baked into the firmware of modems, routers, switches, etc all over the infrastructure, to secure administration channels and other secure tunnels. Often, even if you could upgrade the firmware to include stronger crypto, the hardware on the device isn't able to perform the calculations in a reasonable timeframe due to their complexity.

Then comes the problem of backwards compatibility. What do you do with all the devices that can't effectively use the stronger crypto? What happens when mom-and-pop's router tries to connect to the network and the device on the other end requests SHA-3/RSA-3072 to secure the connection but the device only supports SHA-1/RSA-1024? Do you disallow them on the network? Subsidize a replacement device?

For background, I help major F-500 companies (and smaller) gain their FIPS 140-2 validations on IT products you use every day. SHA-1 is already 'deprecated' according to the standard (along with RSA-1024) so progress is being made. The transition schedule can be found here. The testing/validation methods for SHA-3 will be developed shortly, and with that, I believe many companies will begin to incorporate it quickly. However, FIPS allows the support of older algorithms for backwards-compatibility. So, ultimately, the problem has to be addressed at the standard-level, and vendors have to be prepared with products as soon as those standards are finalized.

This is a real problem, and the transition schedule of algorithms is an important issue to the vendors, and ultimately, to the general public. I appreciate Ars raising awareness on the issue.

"SHA1 and other hash algorithms generate a digital fingerprint that in theory is unique for each different file or text input they sign."

This seems a bit confusing. Hashing and signing are two different things.

Also, at some point in the near future, can Ars do an article looking at SHA-3?

Edit:

As a friend pointed out, my comment about the hashing/signing thing could be read as a bit snarky. I didn't mean to criticize, just to point out that it could be slightly misleading to those who don't know the subject.

No, just that being a criminal is a prerequisite for being capable of cracking it.

What!?

From the article: "A collision attack is therefore well within the range of what an organized crime syndicate can practically budget by 2018, and a university research project by 2021."

In other words, the first group to whom SHA1 collisions will be available is a group of well-financed criminals, followed in a few years by other, more law-abiding crackers. No one ever said cracking crypto made you a criminal.

Instead of focusing on aggressive acts of war like Stuxnet and Flame, the US Government would do a lot better to focus on a much better defensive approach, and switch everyone over to SHA-3, and other protection mechanisms. This is my problem with increasing the budgets of the "cyber divisions". They are using that money mostly for offensive stuff, and then whine about how some cyber-attack could threaten national security.

Assuming that the general infrastructure of the Internet can be migrated within the advised time frame, does this mean that companies will have to finally have to get rid of Windows XP?

I wouldn't imagine that Microsoft would be willing to issue SP4 to patch IE6, etc. By the way, I continue to be amazed at the number of companies in America that are purchasing new computers, wiping them and installing WinXP, which when it came out was awesome, but is now simply impeding the progress towards a more secure, functional computing environment worldwide.

https://blogs.technet.com/b/srd/archive ... ected=true (e.g., sentence "The attacker can then apply the collision algorithm [...] to create a forged certificate that removes the critical Microsoft Hydra extension and still matches the MD5 hash of the legitimate certificate signed by the CA.")

I'm surprised that none of you anal-retentive pedants commented on this before me: Moore's law has never been about the speed of the processor, it has always been about the number of transistors in the processor. Just because the two tend to coincide does not make them the same.

simast is applying the classic "firetrucks are red => a red truck is a firetruck" logical fallacy, I think. The article only states criminals will be able to crack.

More accurately well funded criminals would have the budgets and desire to build or pay for the processing power to crack SHA-1 once the cost for that processing power drops enough to make it worth their while. This group of individuals have a motive and money that makes them likely one of the first groups to attempt this behind governments and possibly large corporations with low ethics (corporate espionage, we will call this criminals as well).

Instead of focusing on aggressive acts of war like Stuxnet and Flame, the US Government would do a lot better to focus on a much better defensive approach, and switch everyone over to SHA-3, and other protection mechanisms. This is my problem with increasing the budgets of the "cyber divisions". They are using that money mostly for offensive stuff, and then whine about how some cyber-attack could threaten national security.

This doesn't fully jive with history... The NSA and DoD have been leaders in funding and designing cryptography. The have a need to both improve "our" cryptography while defeating that of "others".

I always assumed the best researchers were the criminals. Someone working at a company is already getting paid, their motivations are promotions and recognition, a continued paycheck. A criminal is trying to get paid in the first place, and those who find the fault first are likely to cash in the most.

I always assumed the best researchers were the criminals. Someone working at a company is already getting paid, their motivations are promotions and recognition, a continued paycheck. A criminal is trying to get paid in the first place, and those who find the fault first are likely to cash in the most.

The point isn't that criminals are better researchers. The point is that criminals are the group that will first be able to recoup the significant investment required to develope a reliable exploit. (And while we're at it, we should remember nation states are similarly positioned.)

Researchers at universities are probably more sophisticated than criminals, but their return on investment isn't as high as a criminal organization, which is why the Walker estimates university researchers won't develop something until a few years after the criminal organizations.

"SHA1 and other hash algorithms generate a digital fingerprint that in theory is unique for each different file or text input they sign."

This seems a bit confusing. Hashing and signing are two different things.

Also, at some point in the near future, can Ars do an article looking at SHA-3?

Edit:

As a friend pointed out, my comment about the hashing/signing thing could be read as a bit snarky. I didn't mean to criticize, just to point out that it could be slightly misleading to those who don't know the subject.

I'm a fan of Dan's articles; keep up the great work!

I didn't read your comment as snarky, but I appreciate the clarification anyway, secretknight42. Thanks.

Writing articles that are both concise and technically accurate can be a challenge at times. Yes, hashing a password is different than signing a document or piece of software. But both activities rely on an algorithm's ability to generate a unique hash digest of the plaintext source, no?

I'm surprised that none of you anal-retentive pedants commented on this before me: Moore's law has never been about the speed of the processor, it has always been about the number of transistors in the processor. Just because the two tend to coincide does not make them the same.

Edit: typo

You'd have probably gotten more up votes had you not called us all a bunch of anal-retentive pedants.

Let's look at the source (or at least, an easily readably copy of the source):https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html

That shows that it would cost certain amounts of money to find a collision in certain years. 2018 is one of those years. 2012 is also one of those years. The finality in the Ars title is in conflict with *one* actual source they are referencing.

Secondly, that's the amount it costs to find a collision, which is not exactly a magic "now I can read all your SHA1 encrypted data" bullet.

Thirdly, there are a bunch of comments against Schneier's blog which provide a bit of sanity to this.

All I'm saying is, give the original a source a read as well as this Ars article for a bit of perspective.

"The point is," Schneier concluded, "that we in the community need to start the migration away from SHA-1 and to SHA-2/SHA-3 now."

Good. This is what will drive up computational intensity which will in turn drive up the hardware industry and we will get even more powerfull hardware as a lowest common denominator thus enabling even cheaper crypto hacking, or exactly the opposite of what they want to do by implementing more complex hashes.

This article should identify that this is not referencing stored passwords, which should not rely on SHA-1 hashing as they can already be cracked. It is best to use b-crypt instead, which is pretty easy to implement and there are free bcrypt scripts available on the internet.

"The point is," Schneier concluded, "that we in the community need to start the migration away from SHA-1 and to SHA-2/SHA-3 now."

Good. This is what will drive up computational intensity which will in turn drive up the hardware industry and we will get even more powerfull hardware as a lowest common denominator thus enabling even cheaper crypto hacking, or exactly the opposite of what they want to do by implementing more complex hashes.

A widely used cryptographic algorithm used to secure sensitive websites, software, and corporate servers is weak enough that well-financed criminals could crack it in the next six years.

Cracking a cryptographic algorithm makes you a criminal?

No, just that being a criminal is a prerequisite for being capable of cracking it.

Wow. Just wow. So you're saying every scientist who was digging into crypto algorithms was a criminal? Or that they had no business doing so unless they were? If legitimate scientists didn't research things like cryptography then the criminals would outpace law enforcement because no scientist would risk it! And no criminal would openly advertise problems with widely used systems because that's where they can make their money.

"The point is," Schneier concluded, "that we in the community need to start the migration away from SHA-1 and to SHA-2/SHA-3 now."

Good. This is what will drive up computational intensity which will in turn drive up the hardware industry and we will get even more powerfull hardware as a lowest common denominator thus enabling even cheaper crypto hacking, or exactly the opposite of what they want to do by implementing more complex hashes.

What are you suggesting, we move back to XOR?

I was not suggesting anything, merely stating what is happening in the industry -- cat and mice chase, but it ain't Tom and Jerry.

Just 2 years ago, AES256 encryption/decryption was possible on Nehalem with ~300MB/sec, nowadays anyone with Core i5 can rack up ~3GB/sec because of AESNI extensions. That is a tenfold increase in crypto performance. If key derivation and hashing for new computationally intensive algorithms are not accelerated by the same amount they will be very soon or our laptop's battery life will be back in stone age once they start using them. It is a kind of race where you can't win, you just have to be better than the last person.

But in practice it can be almost impossible, beacuse (in layman's terms) it is a one way encryption system. Once something has been "encrypted" with sha1, it is totally impossible to decrypt it.

So, if you encrypt a bunch of data with sha1, and then erase the original data, then it is impossible to stop using sha1 because you do not know what the original data was. It's gone.

An example is passwords, chances are your password for arstechnica is not stored in the database, instead a sha1 hash of your password is stored. In order to switch to something else they would have to ask you to type your password again. Sometimes that is not possible.

Basically, there is no need to switch away from sha1 right now, but if you are building a new system now, do it wish sha2 instead. High security military or banking systems might want to switch right away however.

I was not suggesting anything, merely stating what is happening in the industry -- cat and mice chase, but it ain't Tom and Jerry.

Just 2 years ago, AES256 encryption/decryption was possible on Nehalem with ~300MB/sec, nowadays anyone with Core i5 can rack up ~3GB/sec because of AESNI extensions. That is a tenfold increase in crypto performance.

Speeding up the encryption capabilities of comodity hardware does not have any impact on the encryption capabilities of hardware used by criminals and the government. It is the speed of their specialty hardware that is important, not the speed of the CPU in your personal computer or web server.

igor.levicki wrote:

If key derivation and hashing for new computationally intensive algorithms are not accelerated by the same amount they will be very soon or our laptop's battery life will be back in stone age once they start using them. It is a kind of race where you can't win, you just have to be better than the last person.

[/quote]

Twenty milliseconds of hard math every now and then is not going to have any impact on battery life. You're seriously ignorant if you think encryption has any apreciable impact on the speed and power requirements of a typical PC.

A widely used cryptographic algorithm used to secure sensitive websites, software, and corporate servers is weak enough that well-financed criminals could crack it in the next six years.

Cracking a cryptographic algorithm makes you a criminal?

I'm not sure how you got "everyone who does this is a criminal" out of that. You may have to explain that to me. After rereading it a few times, the only interpretation I can get from it is that if a criminal is well-financed, it's feasible that they could start cracking SHA1 hashes in the next six years.

That, of course, says nothing about whether everyone who does it is a criminal. Just that given a criminal who is motivated and has a reasonable amount of cash lying around, it'll become possible that they could do this. As opposed to now, where no criminal could do this, as the computing power simply doesn't exist. Presumably even a well financed nation couldn't do it either.

In general, md5 hacks have been of the form, "I can match your hash by adding X new bytes," or some such thing. Adding Length alone would seem to stymie any such attacks. By adding these other measures, it would seem to make a hacker's job much more difficult, wouldn't you agree.

I think in general sites hosting files where they need to give out a, "And its hash is this..." that a prefix stating the schema of the "multi-hash", which would make the scheme future proof. Sound good?

"The point is," Schneier concluded, "that we in the community need to start the migration away from SHA-1 and to SHA-2/SHA-3 now."

Good. This is what will drive up computational intensity which will in turn drive up the hardware industry and we will get even more powerfull hardware as a lowest common denominator thus enabling even cheaper crypto hacking, or exactly the opposite of what they want to do by implementing more complex hashes.

What are you suggesting, we move back to XOR?

I was not suggesting anything, merely stating what is happening in the industry -- cat and mice chase, but it ain't Tom and Jerry.

Just 2 years ago, AES256 encryption/decryption was possible on Nehalem with ~300MB/sec, nowadays anyone with Core i5 can rack up ~3GB/sec because of AESNI extensions. That is a tenfold increase in crypto performance. If key derivation and hashing for new computationally intensive algorithms are not accelerated by the same amount they will be very soon or our laptop's battery life will be back in stone age once they start using them. It is a kind of race where you can't win, you just have to be better than the last person.

Forgive me if I'm misunderstanding your point, but I'm taking both of your posts as context.

From what I understand, you're saying that more powerful systems drive more complex crypto which drives more powerful systems, which is counter to the point of improving security. And in addition, it increases power draw in order to do so. Correct?

In response to that: The cost of "more complex crypto" is that for the user, it may result in a multiplicative increase in time to decrypt/encrypt, right? That may sound bad, but the benefit is that for a person trying to break an encryption the time is EXPONENTIALLY larger. The gap between a legitimate user and an illegitimate hacker/criminal/curious individual becomes significantly wider.

Consider this: I found a reference that says SHA1 takes approximately 5.8 cycles per byte of data... and SHA3 (Keccak) takes 12.5 cpb. However, SHA1's resulting hash size is 160-bits, while SHA3's hash size is 512-bits. (Theoretically) each bit you add doubles the amount of effort it takes to find a collision. So, by a little over doubling the time it takes to hash something, you've made it 2^352 times harder to find a collision. That is an _enormous_ number. It's actually so big that no one actually tries to brute force these algorithms... they must expend human brain power to find weaknesses in the algorithm or in the way it's used. And that kind of resource is much more expensive and scarce.

Here's a fun fact about trying to naively brute force a collision with SHA3: If you could turn every atom in the universe (10^80) into a computer capable of finding a collision in SHA1 within a second (and such a process was applicable to SHA3), it would take > 2.9x10^18 years for all of such a system to find a collision in SHA3. That's 290 million-trillion years.

Quick edit: Keccak is configurable, so, while it has a configuration for 512-bits, it doesn't necessarily have to always emit a 512-bit hash. From seeing the comical numbers above, it should show you that it's not necessary to run it at 512-bits from a security standpoint.