This proposal inhibits the covert use of a known optimization in BitcoinProof of Work function.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in thisdocument are to be interpreted as described in RFC 2119.

==Motivation==

Due to a design oversight the Bitcoin proof of work function has a potentialoptimization which can allow a rational miner to save up-to 30% of theirenergycosts (though closer to 20% is more likely due to implementation overheads).

Timo Hanke and Sergio Demian Lerner applied for a patent on thisoptimization. The company "Sunrise Tech Group, Llc" has offered to licenseit to any interested party in the past. Sunrise Tech Group has beenmarketing their patent licenses under the trade-name ASICBOOST. Thedocument takes no position on the validity or enforceability of the patent.

There are two major ways of taking advantage of this optimization, asdescribedby the patent:One way which is highly detectable and is not in use on the networktoday and a covert way which has significant interaction and potentialinterference with the Bitcoin protocol. The covert mechanism is noteasily detected except through its interference with the protocol.

In particular, the protocol interactions of the covert method can block theimplementation of virtuous improvements such as segregated witness.

The use of this optimization could result in a big payoff, but the actualsum depends on the degree of research, investment and effort put intodesigningthe improved cores.

On the above basis the potential for covert use of this optimizationin the covert form and interference with useful improvements presents adanger to the Bitcoin system.

==Background==

The general idea of this optimization is that SHA2-256 is a merkle damgardhashfunction which consumes 64 bytes of data at a time.

The Bitcoin mining process repeatedly hashes an 80-byte 'block header' whileincriminating a 32-bit nonce which is at the end of this header data. Thismeans that the processing of the header involves two runs of the compressionfunction run-- one that consumes the first 64 bytes of the header and asecond which processes the remaining 16 bytes and padding.

The initial 'message expansion' operations in each step of the SHA2-256function operate exclusively on that step's 64-bytes of input with noinfluence from prior data that entered the hash.

Because of this if a miner is able to prepare a block header withmultiple distinct first 64-byte chunks but identical 16-bytesecond chunks they can reuse the computation of the initialexpansion for multiple trials. This reduces power consumption.

There are two broad ways of making use of this optimization. The obviousway is to try candidates with different version numbers. Beyondupsetting the soft-fork detection logic in Bitcoin nodes this haslittle negative effect but it is highly conspicuous and easilyblocked.

The other method is based on the fact that the merkle rootcommitting to the transactions is contained in the first 64-bytesexcept for the last 4 bytes of it. If the miner finds multiplecandidate root values which have the same final 32-bit then theycan use the optimization.

To find multiple roots with the same trailing 32-bits the miner canuse efficient collision finding mechanism which will find a matchwith as little as 2^16 candidate roots expected, 2^24 operations tofind a 4-way hit, though low memory approaches require morecomputation.

An obvious way to generate different candidates is to grind thecoinbase extra-nonce but for non-empty blocks each attempt willrequire 13 or so additional sha2 runs which is very inefficient.

This inefficiency can be avoided by computing a sqrt number ofcandidates of the left side of the hash tree (e.g. using extranonce grinding) then an additional sqrt number of candidates ofthe right side of the tree using transaction permutation orsubstitution of a small number of transactions. All combinationsof the left and right side are then combined with only a singlehashing operation virtually eliminating all tree relatedoverhead.

With this final optimization finding a 4-way collision with amoderate amount of memory requires ~2^24 hashing operationsinstead of the >2^28 operations that would be require forextra-nonce grinding which would substantially erode thebenefit of the optimization.

It is this final optimization which this proposal blocks.

==New consensus rule==

Beginning block X and until block Y the coinbase transaction ofeach block MUST either contain a BIP-141 segwit commitment or acorrect WTXID commitment with ID 0xaa21a9ef.

(See BIP-141 "Commitment structure" for details)

Existing segwit using miners are automatically compatible withthis proposal. Non-segwit miners can become compatible by simplyincluding an additional output matching a default commitmentvalue returned as part of getblocktemplate.

Miners SHOULD NOT automatically discontinue the commitmentat the expiration height.

==Discussion==

The commitment in the left side of the tree to all transactionsin the right side completely prevents the final sqrt speedup.

A stronger inhibition of the covert optimization in the form ofrequiring the least significant bits of the block timestampto be equal to a hash of the first 64-bytes of the header. Thiswould increase the collision space from 32 to 40 or more bits.The root value could be required to meet a specific hash prefixrequirement in order to increase the computational work requiredto try candidate roots. These change would be more disruptive andthere is no reason to believe that it is currently necessary.

The proposed rule automatically sunsets. If it is no longer neededdue to the introduction of stronger rules or the acceptance of theversion-grinding form then there would be no reason to continuewith this requirement. If it is still useful at the expirationtime the rule can simply be extended with a new softfork thatsets longer date ranges.

This sun-setting avoids the accumulation of technical debt dueto retaining enforcement of this rule when it is no longer neededwithout requiring a hard fork to remove it.

== Overt optimization ==

A BIP for avoiding erroneous warning messages when miners use the overtversionof the optimization was proposed several years ago, in order to deter thecovertuse of the optimization. But that BIP was rejected.However, in light of the current discoveries, that BIP could bereconsidered.

The over optimization does not generally interfere with improvements in theprotocol.

==Backward compatibility==

==Implementation==

==Acknowledgments==

Greg Maxwell <***@xiph.org> for the original report, which containedseveral errors that were corrected in the present proposal.

Post by Sergio Demian Lerner via bitcoin-dev<pre>BIP: TBDLayer: ConsensusTitle: Inhibiting a covert optimization on the Bitcoin POW functionStatus: DraftType: Standards TrackCreated: 2016-04-07License: PD</pre>==Abstract==This proposal inhibits the covert use of a known optimization inBitcoin Proof of Work function.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in thisdocument are to be interpreted as described in RFC 2119.==Motivation==Due to a design oversight the Bitcoin proof of work function has apotential optimization which can allow a rational miner to save up-to30% of their energycosts (though closer to 20% is more likely due to implementation overheads).Timo Hanke and Sergio Demian Lerner applied for a patent on thisoptimization. The company "Sunrise Tech Group, Llc" has offered tolicense it to any interested party in the past. Sunrise Tech Grouphas been marketing their patent licenses under the trade-nameASICBOOST. The document takes no position on the validity orenforceability of the patent.There are two major ways of taking advantage of this optimization, asdescribedOne way which is highly detectable and is not in use on the networktoday and a covert way which has significant interaction and potentialinterference with the Bitcoin protocol. The covert mechanism is noteasily detected except through its interference with the protocol.In particular, the protocol interactions of the covert method canblock the implementation of virtuous improvements such as segregatedwitness.The use of this optimization could result in a big payoff, but theactual sum depends on the degree of research, investment and effortput into designingthe improved cores.On the above basis the potential for covert use of this optimizationin the covert form and interference with useful improvements presentsa danger to the Bitcoin system.==Background==The general idea of this optimization is that SHA2-256 is a merkledamgard hashfunction which consumes 64 bytes of data at a time.The Bitcoin mining process repeatedly hashes an 80-byte 'blockheader' while incriminating a 32-bit nonce which is at the end ofthis header data. This means that the processing of the headerinvolves two runs of the compression function run-- one that consumesthe first 64 bytes of the header and a second which processes theremaining 16 bytes and padding.The initial 'message expansion' operations in each step of theSHA2-256 function operate exclusively on that step's 64-bytes ofinput with no influence from prior data that entered the hash.Because of this if a miner is able to prepare a block header withmultiple distinct first 64-byte chunks but identical 16-bytesecond chunks they can reuse the computation of the initialexpansion for multiple trials. This reduces power consumption.There are two broad ways of making use of this optimization. Theobvious way is to try candidates with different version numbers.Beyond upsetting the soft-fork detection logic in Bitcoin nodes thishas little negative effect but it is highly conspicuous and easilyblocked.The other method is based on the fact that the merkle rootcommitting to the transactions is contained in the first 64-bytesexcept for the last 4 bytes of it. If the miner finds multiplecandidate root values which have the same final 32-bit then theycan use the optimization.To find multiple roots with the same trailing 32-bits the miner canuse efficient collision finding mechanism which will find a matchwith as little as 2^16 candidate roots expected, 2^24 operations tofind a 4-way hit, though low memory approaches require morecomputation.An obvious way to generate different candidates is to grind thecoinbase extra-nonce but for non-empty blocks each attempt willrequire 13 or so additional sha2 runs which is very inefficient.This inefficiency can be avoided by computing a sqrt number ofcandidates of the left side of the hash tree (e.g. using extranonce grinding) then an additional sqrt number of candidates ofthe right side of the tree using transaction permutation orsubstitution of a small number of transactions. All combinationsof the left and right side are then combined with only a singlehashing operation virtually eliminating all tree relatedoverhead.With this final optimization finding a 4-way collision with amoderate amount of memory requires ~2^24 hashing operationsinstead of the >2^28 operations that would be require forextra-nonce grinding which would substantially erode thebenefit of the optimization.It is this final optimization which this proposal blocks.==New consensus rule==Beginning block X and until block Y the coinbase transaction ofeach block MUST either contain a BIP-141 segwit commitment or acorrect WTXID commitment with ID 0xaa21a9ef.(See BIP-141 "Commitment structure" for details)Existing segwit using miners are automatically compatible withthis proposal. Non-segwit miners can become compatible by simplyincluding an additional output matching a default commitmentvalue returned as part of getblocktemplate.Miners SHOULD NOT automatically discontinue the commitmentat the expiration height.==Discussion==The commitment in the left side of the tree to all transactionsin the right side completely prevents the final sqrt speedup.A stronger inhibition of the covert optimization in the form ofrequiring the least significant bits of the block timestampto be equal to a hash of the first 64-bytes of the header. Thiswould increase the collision space from 32 to 40 or more bits.The root value could be required to meet a specific hash prefixrequirement in order to increase the computational work requiredto try candidate roots.

Root value pow - Does this mean that every miner would be penalized inthis way regardless of the actual number of transactions in the block?

Post by Sergio Demian Lerner via bitcoin-devThese change would be more disruptive andthere is no reason to believe that it is currently necessary.The proposed rule automatically sunsets. If it is no longer neededdue to the introduction of stronger rules or the acceptance of theversion-grinding form then there would be no reason to continuewith this requirement. If it is still useful at the expirationtime the rule can simply be extended with a new softfork thatsets longer date ranges.This sun-setting avoids the accumulation of technical debt dueto retaining enforcement of this rule when it is no longer neededwithout requiring a hard fork to remove it.== Overt optimization ==A BIP for avoiding erroneous warning messages when miners use theovert versionof the optimization was proposed several years ago, in order to deterthe covertuse of the optimization. But that BIP was rejected.However, in light of the current discoveries, that BIP could bereconsidered.The over optimization does not generally interfere with improvementsin the protocol.==Backward compatibility====Implementation====Acknowledgments==several errors that were corrected in the present proposal.==Copyright==This document is placed in the public domain.