Some thoughts about the activation mechanism for soft forks. In the past we used IsSuperMajority and currently use BIP9 as soft fork activation methods, where a supermajority of hashrate triggers nodes to begin enforcing new rules. Hashrate based activation is convenient because it is the simplest and most straightforward process. While convenient there are a number limitations with this method.

Firstly, it requires trusting the hash power will validate after activation. The BIP66 soft fork was a case where 95% of the hashrate was signaling readiness but in reality about half was not actually validating the upgraded rules and mined upon an invalid block by mistake[1].

Secondly, miner signalling has a natural veto which allows a small percentage of hashrate to veto node activation of the upgrade for everyone. To date, soft forks have taken advantage of the relatively centralised mining landscape where there are relatively few mining pools building valid blocks; as we move towards more hashrate decentralization, it's likely that we will suffer more and more from "upgrade inertia" which will veto most upgrades.

Upgrade inertia in inevitable for widely deployed software and can be seen for example, with Microsoft Windows. At the time of writing 5.72% of all Microsoft Windows installations are still running Windows XP, despite mainstream support ending in 2009 and being superseded by 4 software generations, Vista, 7, 8 and 10.

Thirdly, the signaling methodology is widely misinterpreted to mean the hash power is voting on a proposal and it seems difficult to correct this misunderstanding in the wider community. The hash powers' role is to select valid transactions, and to extend the blockchain with valid blocks. Fully validating economic nodes ensure that blocks are valid. Nodes therefore define validity according to the software they run, but miners decide what already valid transactions gets included in the block chain.

As such, soft forks rules are actually always enforced by the nodes, not the miners. Miners of course can opt-out by simply not including transactions that use the new soft fork feature, but they cannot produce blocks that are invalid to the soft fork. The P2SH soft fork is a good example of this, where non-upgraded miners would see P2SH as spendable without a signature and consider them valid. If such an transaction were to be included in a block, the block would be invalid and the miner would lose the block reward and fees.

So-called "censorship" soft forks do not require nodes to opt in, because >51% of the hash power already have the ability to orphan blocks that contain transactions they have blacklisted. Since this is not a change in validity, nodes will accept the censored block chain automatically.

The fourth problem with supermajority hash power signaling is it draws unnecessary attention to miners which can become unnecessarily political. Already misunderstood as a vote, miners may feel pressure to "make a decision" on behalf of the community: who is and isn't signalling becomes a huge public focus and may put pressures onto miners they are unprepared for. Some miners may not be in a position to upgrade, or may prefer not to participate in the soft fork which is their right. However, that miner may now become a lone reason that vetoes activation for everyone, where the soft fork is an opt-in feature! This situation seems to be against the voluntary nature of the Bitcoin system where participation at all levels is voluntary and kept honest by well balanced incentives.

Since miners already have the protocol level right to select whatever transaction they prefer (and not mine those they don't), it would be better if a miner could chose to not participate in triggering activation of something they won't use, but, without being a veto to the process (and all the ire they may have to experience as a consequence).

The alternative discussed here is "flag day activation" where nodes begin enforcement at a predetermined time in the future. This method needs a longer lead time than a hash power based activation trigger, but offers a number of advantages and perhaps provides a better tradeoff.

Soft forks are still entirely optional to use post activation. For example, with P2SH, many participants in the Bitcoin ecosystem still do not use P2SH. Only 11% of bitcoins[2] are stored in P2SH addresses at the time of writing. Miners are free to not mine P2SH transactions, however, the incentives are such that miners should still validate transactions so they don't accidentally include invalid transactions and cause their block to be rejected. As an additional safety measure for well designed soft forks, relay policy rules prevent non-standard and invalid transactions from being relayed and mined by default; a miner would have to purposefully mine an invalid transaction, which is against their own economic interest.

Since the incentives of the Bitcoin system rely on self validation, economic nodes (miners and users) should always remain safe by ensuring their nodes either validate the current rules, or, they can place their network behind a full node that will filter out invalid transactions and blocks at the edge of their network (so called firewall or border nodes).

A user activated soft fork is permissive. Miners do not have to produce new version blocks and non-upgraded miners' blocks will not be orphaned as was the case with IsSuperMajority soft forks (e.g. BIP34, BIP66, BIP65-CLTV) which made it a compulsory upgrade for miners.

BIP9 "versionbits" soft fork activation method is also permissive in so far as non-upgraded miners are not forced to upgrade after activation because their blocks wont be orphaned. A recent case was the "CSV" soft fork that activated BIP68, BIP112 and BIP113. As such, the CSV soft fork allows non-upgraded miners to continue mining so long as they didn't produce invalid blocks.

Miners always retain discretion on which transactions to mine. However, regardless of whether they actively include transactions using the new soft fork feature, or not, the incentive for hash power to upgrade in order to validate is strong: if they do not, they could be vulnerable to a rogue miner willing to waste 12.5BTC to create an invalid block, which may cause non-validating miners to build on an invalid chain similar to the BIP66 incident. Validation has always had a strong requirement.

A user activated soft fork is win-win because it adds an option that some people want that does not detract from other peoples' enjoyment. Even if only 10% of users ever wanted a feature, so long as the benefit outweighed the technical risks, it would not be rational to deny others the ability to opt-in.

My suggestion is to have the best of both worlds. Since a user activated soft fork needs a relatively long lead time before activation, we can combine with BIP9 to give the option of a faster hash power coordinated activation or activation by flag day, whichever is the sooner. In both cases, we can leverage the warning systems in BIP9. The change is relatively simple, adding an activation-time parameter which will transition the BIP9 state to LOCKED_IN before the end of the BIP9 deployment timeout.

Miners have no business deciding on new features. Maybe in the past it was assumed that miners would go along in the best interests of users, but now it's clear that miners have become too political and keen to push their own agenda.

Perhaps lowering miner activation to 51% + getting direct support from all major exchanges and wallet providers + 95% full node support + long flag day delay + better education with users and miners about using border nodes for miners opting out + 95% developer support = a safe Soft Fork.

I think 95% is not archieveable - independent from soft or hardfork or what about the change would be. I think we are already too big and diversified for this. imagine a society where you have a government which is only able to work if more than 95% of the citizen voted for it (in freedom). Sounds not possible anymore.

I think 95% is not archieveable - independent from soft or hardfork or what about the change would be. I think we are already too big and diversified for this. imagine a society where you have a government which is only able to work if more than 95% of the citizen voted for it (in freedom). Sounds not possible anymore.

I would suggest over 95% support is achievable for a consensus critical bug like we saw in 2013. This solution may make it far easier to activate a SF , but yes supermajority HFs are still extremely difficult. I also believe that we should try and get segwit activated as it might be the last SF we see as bitcoin begins to ossify.

I don't think that we would even agree all on a 95% fix for a bug in bitcoin... i have the feeling that there are already too many enemies which try to kill or damage bitcoin.. so 75% around would be reachable i think.

We have reached 95% consensus among the miners many times in the past.

I don't see this as anything more than an attempt to activate a contentious fork.

There is good evidence that those opposing it are in the minority(Even many "Big Blockers" support segwit - Gavin/Garzik, ect...). Miners controlling most hashpower are only a handful of people and one miner Bitmain may control over 50% . Thus this proposal would not be about getting a lower level of consensus to activate a SF but an accurate and high level of 95% of the community. 95% hashpower doesn't necessarily mean 95% economic user support.

the strange to me is why three days after this proposal not a single bitcoin developer except from Jameson Lopp bitgo eng has not tell us his opinion about this. It seems that all of them keep mysterious silent...

Yeah, would also love to hear some opinions.. Maybe they are discussing it at the moment internally about how to cope with the current situation and if a flag day is possible (and working).

i hope so. Because i am afraid that they avoid to take position to this. I have ask to slack and to irc and in irc i have ask gmaxwell what is his opinion in this and never get an answer even a single word from them. My most fear is that they avoid to do it because maybe they afraid that Jihan will get angry

This is a good idea, if done carefully. I really dislike giving miners any decision-making power, for the reasons you mentioned and also because they are employees of the network -- the only parties paid by the Bitcoin system. I proposed similarly forcing the P2SH softfork when miners were (for a time) blocking that softfork, though miners stopped blocking it before too long.

This has the potential to get messy, though. Without miners, you have to be really sure that a large majority of the economy is going to enforce the new rules, since otherwise a prolonged split could develop. There's a worrying situation today where both mining and verification is centralized, even though Bitcoin really requires that at least one of them is decentralized. So there'd still be the ridiculous need to engage in lobbying, though in this case it'd be with big centralized verifiers like BitGo and bc.i rather than with centralized miners. (Verifiers are a lot less centralized than miners, though, which makes somewhat better.) If the softfork activates without a substantial majority of the economy enforcing it, then it could end up splitting the network/currency long-term, which would be the same sort of disaster as a contentious hardfork. There are a lot of economic reasons why this is an unlikely outcome, but it could happen if the softfork isn't done carefully. Before something like this is added for any particular softfork, I'd like to see a detailed investigation into how much of the economy is committing to support it; how much is actually doing verification rather than just blindly trusting miners; the popularity and expected behavior of each end-user wallet software; etc.

Another consideration is that if miners are expected to keep producing very long invalid chains under the new rules, then this sort of softfork has almost exactly the same costs as a hardfork. So if this is considered a likely outcome, then it'd may be better to just hardfork and make some other useful hardfork changes at the same time.

The flag day would probably need to be at least 1 year in the future to be reasonably safe. 2 years would be better. Backports would be needed for all versions that are actually being used, including the #bitcoin-assets 0.3.x version. I wonder if there's any reasonable way to handle the (hopefully very unlikely) scenario where a softfork needs to be canceled partway into the wait time.

I'd like to see a detailed investigation into how much of the economy is committing to support it; how much is actually doing verification rather than just blindly trusting miners; the popularity and expected behavior of each end-user wallet software; etc.

I'd like to see a detailed investigation into how much of the economy is committing to support it; how much is actually doing verification rather than just blindly trusting miners; the popularity and expected behavior of each end-user wallet software; etc.

PS: the only truly reliable and safest to some extend (not for the forkers obv) way to market pull such "improvement" is commit hardfork and see how the shitcoin goes.

just pick a day, and fork off.

and why a whole blockchain system has to wait when or if Jihan wake up in a good mood, or to be his panda happy or whatever to enable a tech improvement? if bitcoin ecosystem want it it can bypass him. And for sure he has more to loose than anyone else here.