Great catch, and a good proposal for a fix. Pushing the activation height out to allow existing hardware to enter obsolescence prior to activation may help reduce miner resistance. It may also avoid legal threats from those currently abusing. If miners still resist, the threat of an earlier activation height is a good negotiating tool.

Raystonn

On 5 Apr 2017 2:39 p.m., Gregory Maxwell via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org> wrote:A month ago I was explaining the attack on Bitcoin's SHA2 hashcash whichis exploited by ASICBOOST and the various steps which could be used toblock it in the network if it became a problem.

While most discussion of ASICBOOST has focused on the overt methodof implementing it, there also exists a covert method for using it.

As I explained one of the approaches to inhibit covert ASICBOOST Irealized that my words were pretty much also describing the SegWitcommitment structure.

The authors of the SegWit proposal made a specific effort to not beincompatible with any mining system and, in particular, changed thedesign at one point to accommodate mining chips with forced payoutaddresses.

Had there been awareness of exploitation of this attack an effortwould have been made to avoid incompatibility-- simply to separateconcerns. But the best methods of implementing the covert attackare significantly incompatible with virtually any method ofextending Bitcoin's transaction capabilities; with the notableexception of extension blocks (which have their own problems).

An incompatibility would go a long way to explain some of themore inexplicable behavior from some parties in the miningecosystem so I began looking for supporting evidence.

Reverse engineering of a particular mining chip has demonstratedconclusively that ASICBOOST has been implementedin hardware.

On that basis, I offer the following BIP draft for discussion.This proposal does not prevent the attack in general, but onlyinhibits covert forms of it which are incompatible withimprovements to the Bitcoin protocol.

I hope that even those of us who would strongly prefer thatASICBOOST be blocked completely can come together to supporta protective measure that separates concerns by inhibitingthe covert use of it that potentially blocks protocol improvements.

The specific activation height is something I currently don't havea strong opinion, so I've left it unspecified for the moment.

This proposal inhibits the covert exploitation of a knownvulnerability in Bitcoin Proof of Work function.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in thisdocument are to be interpreted as described in RFC 2119.

==Motivation==

Due to a design oversight the Bitcoin proof of work function has a potentialattack which can allow an attacking miner to save up-to 30% of their energycosts (though closer to 20% is more likely due to implementation overheads).

Timo Hanke and Sergio Demian Lerner claim to hold a patent on this attack,which they have so far not licensed for free and open use by the public.They have been marketing their patent licenses under the trade-nameASICBOOST. The document takes no position on the validity or enforceabilityof the patent.

There are two major ways of exploiting the underlying vulnerability: Oneobvious way which is highly detectable and is not in use on the networktoday and a covert way which has significant interaction and potentialinterference with the Bitcoin protocol. The covert mechanism is noteasily detected except through its interference with the protocol.

In particular, the protocol interactions of the covert method can block theimplementation of virtuous improvements such as segregated witness.

Exploitation of this vulnerability could result in payoff of as much as$100 million USD per year at the time this was written (Assuming at50% hash-power miner was gaining a 30% power advantage and that miningwas otherwise at profit equilibrium). This could have a phenomenalcentralizing effect by pushing mining out of profitability for allother participants, and the income from secretly using thisoptimization could be abused to significantly distort the Bitcoinecosystem in order to preserve the advantage.

Reverse engineering of a mining ASIC from a major manufacture hasrevealed that it contains an undocumented, undisclosed abilityto make use of this attack. (The parties claiming to hold apatent on this technique were completely unaware of this use.)

On the above basis the potential for covert exploitation of thisvulnerability and the resulting inequality in the mining processand interference with useful improvements presents a clear andpresent danger to the Bitcoin system which requires a response.

==Background==

The general idea of this attack is that SHA2-256 is a merkle damgard hashfunction which consumes 64 bytes of data at a time.

The Bitcoin mining process repeatedly hashes an 80-byte 'block header' whileincriminating a 32-bit nonce which is at the end of this header data. Thismeans that the processing of the header involves two runs of the compressionfunction run-- one that consumes the first 64 bytes of the header and asecond which processes the remaining 16 bytes and padding.

The initial 'message expansion' operations in each step of the SHA2-256function operate exclusively on that step's 64-bytes of input with noinfluence from prior data that entered the hash.

Because of this if a miner is able to prepare a block header withmultiple distinct first 64-byte chunks but identical 16-bytesecond chunks they can reuse the computation of the initialexpansion for multiple trials. This reduces power consumption.

There are two broad ways of making use of this attack. The obviousway is to try candidates with different version numbers. Beyondupsetting the soft-fork detection logic in Bitcoin nodes this haslittle negative effect but it is highly conspicuous and easilyblocked.

The other method is based on the fact that the merkle rootcommitting to the transactions is contained in the first 64-bytesexcept for the last 4 bytes of it. If the miner finds multiplecandidate root values which have the same final 32-bit then theycan use the attack.

To find multiple roots with the same trailing 32-bits the miner canuse efficient collision finding mechanism which will find a matchwith as little as 2^16 candidate roots expected, 2^24 operations tofind a 4-way hit, though low memory approaches require morecomputation.

An obvious way to generate different candidates is to grind thecoinbase extra-nonce but for non-empty blocks each attempt willrequire 13 or so additional sha2 runs which is very inefficient.

This inefficiency can be avoided by computing a sqrt number ofcandidates of the left side of the hash tree (e.g. using extranonce grinding) then an additional sqrt number of candidates ofthe right side of the tree using transaction permutation orsubstitution of a small number of transactions. All combinationsof the left and right side are then combined with only a singlehashing operation virtually eliminating all tree relatedoverhead.

With this final optimization finding a 4-way collision with amoderate amount of memory requires ~2^24 hashing operationsinstead of the >2^28 operations that would be require forextra-nonce grinding which would substantially erode thebenefit of the attack.

It is this final optimization which this proposal blocks.

==New consensus rule==

Beginning block X and until block Y the coinbase transaction ofeach block MUST either contain a BIP-141 segwit commitment or acorrect WTXID commitment with ID 0xaa21a9ef.

(See BIP-141 "Commitment structure" for details)

Existing segwit using miners are automatically compatible withthis proposal. Non-segwit miners can become compatible by simplyincluding an additional output matching a default commitmentvalue returned as part of getblocktemplate.

Miners SHOULD NOT automatically discontinue the commitmentat the expiration height.

==Discussion==

The commitment in the left side of the tree to all transactionsin the right side completely prevents the final sqrt speedup.

A stronger inhibition of the covert attack in the form ofrequiring the least significant bits of the block timestampto be equal to a hash of the first 64-bytes of the header. Thiswould increase the collision space from 32 to 40 or more bits.The root value could be required to meet a specific hash prefixrequirement in order to increase the computational work requiredto try candidate roots. These change would be more disruptive andthere is no reason to believe that it is currently necessary.

The proposed rule automatically sunsets. If it is no longer neededdue to the introduction of stronger rules or the acceptance of theversion-grinding form then there would be no reason to continuewith this requirement. If it is still useful at the expirationtime the rule can simply be extended with a new softfork thatsets longer date ranges.

This sun-setting avoids the accumulation of technical debt dueto retaining enforcement of this rule when it is no longer neededwithout requiring a hard fork to remove it.

== Overt attack ==

The non-covert form can be trivially blocked by requiring thatthe header version match the coinbase transaction version.

This proposal does not include this block because this methodmay become generally available without restriction in the future,does not generally interfere with improvements in the protocol,and because it is so easily detected that it could be blocked ifit becomes an issue in the future.

==Backward compatibility==

==Implementation==

==Acknowledgments==

==Copyright==

This document is placed in the public domain._______________________________________________bitcoin-dev mailing listbitcoin-***@lists.linuxfoundation.orghttps://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

If this is the underlying reason why SegWit is being delayed... that is pretty deplorable.

Probably too late now for bitcoin, but maybe it would be good to pre-mix the block header bits around before it even enters the SHA256 hash. Not sure if best to use a hardcoded map, or to make the map with the tx merkle root as a seed. Depends on how hard it is to find good nonce (etc) bit location collisions.

Maybe gmaxwell's solution is good enough for this particular problem... but the above recommendation might help improve bitcoin's available remaining puzzle difficulty.

Another thing that could be done is increase the number of times SHA256 is performed... but now we are really talking about altering the PoW algorithm. Correct me if I'm wrong: The more number of times its performed, the less any patent-able pre or post calculation skipping/caching have an effect on efficiency.

Post by praxeology_guy via bitcoin-devAnother thing that could be done is increase the number of times SHA256 isperformed... but now we are really talking about altering the PoWalgorithm. Correct me if I'm wrong: The more number of times itsperformed, the less any patent-able pre or post calculationskipping/caching have an effect on efficiency.

The more complex that the PoW algorithm is, the more likely it is thatsomeone finds a unique and special method for optimizing it that they areable to patent. And the more difficult it is to create specialized hardwareto run that algorithm, meaning that there will be fewer players who areable to do so profitably (higher fixed costs).

If you want to talk about changing the PoW algorithm, you really want to belooking to simplify it so that it's more obvious (not that you can ever becompletely sure) that there are no hidden or unexpected optimizations thatsomeone could patent.

We can even do a lot better than SHA. Cryptographic hash functions need tobe collision resistant, and collision resistance is the property thatusually breaks. Preimage resistance and partial preimage resistance (andsecond preimage resistance) is generally easier to protect - to the best ofour knowledge, md5 would actually still be a secure PoW function today.

It's bitterly ironic to me that so much research and effort has been putinto making asic-resistant PoW algorithms when in the long runasic-resistance only leads to problems like these - single parties who havefound significant optimizations and not shared them, completely destroyingany chance of a level playing field and giving themselves a centralizedmonopoly - a result that is supremely unhealthy for the rest of thecommunity.