Recently Gavin merged support for BIP 70, 71 and 72 into Bitcoin-Qt. BIP 70, aka the payment protocol, is intended to be a big change to how people use Bitcoin. A lot of questions about it come up repeatedly, so here's an attempt to answer the most common ones.

What is the payment protocol?

Old timers will remember that Bitcoin v0.1, the first version ever released, allowed you to pay people in two ways. One was by entering a Bitcoin address. The other was to enter an IP address. If you entered an IP address, your computer would connect to the node on that IP and run a rudimentary payment protocol that involved asking the node for a fresh public key, then uploading the payment transaction directly.

Satoshi thought this payment protocol would be how people mostly used Bitcoin. It had some advantages, like guaranteeing a fresh address. He originally intended addresses to be used only for when the recipient was offline. But he was wrong - addresses quickly came to dominate and hardly anyone used the pay-to-IP feature. Eventually it was removed.

Why did it fail?

One reason is that it wasn't very secure, anyone who could MITM your internet connection (like someone sharing your wifi) could steal money sent this way. Another more practical reason is that back then there were no online shops that accepted Bitcoin. In fact Bitcoin was a Windows-only pure GUI app with no JSON-RPC API so nobody could even build online shops. So you never knew if the person you were trying to pay would be online or not at the time, meaning giving an address was more reliable. Also: NAT, firewalls, etc.

But it wasn't a bad idea and so now we're bringing it back, but with a better design. The hope is that over time this will come to replace Bitcoin addresses for most usages. But don't worry. Addresses aren't going to disappear. They'll still be around if you want them.

How does it work?

It's a way to format a small piece of data (think file) that contains instructions on how to pay someone (a PaymentRequest message), and then a formal specification of how to satisfy those instructions by submitting a Payment message. The data is designed to be extensible in future so we can add lots of useful features.

When clicked, your wallet app will download the request URL and read the PaymentRequest it gets. The request message contains another URL, which is where to submit the finished transaction to (instead of/as well as broadcasting it to the P2P network).

Who is implementing the new protocol?

Support will be in the next Bitcoin-Qt release, and is being worked on for MultiBit/Android Bitcoin Wallet, CoinBase and CoinPunk. BitPay also committed to implementing it, so all merchants who use their services will get support automatically. If you're planning to support it, let me know and I'll add you here!

What's wrong with Bitcoin addresses?

Lots. They lead to privacy leaks, they are inflexible and tough to extend with new features, they aren't authenticated and they're one way only.

Why do they lead to privacy leaks?

The standard way to use an address is one address per payment received (if your wallet makes that easy to do). This can lead to privacy leaks in the following scenario.

Say you work for a small coffee shop which accepts Bitcoin payments of say 0.01 coins for a coffee. Over the month lots of these small payments get accepted. At the end of the month you get your salary, so you give your employer a fresh address. They make a payment to that address by collecting together lots of the coffee payments and generating one giant salary-sized output.

You then go to the pub and your friend says, hey mate, you owe me $10 for losing that bet we made last week (or whatever). So you send him $10 worth of Bitcoins, of course your wallet goes and selects your salary output to do that. Your friend can now look at the transaction he received and see that it came from a single large, salary-sized looking output. Now they know how much you earn.

A better way would be if you gave your employer 100 addresses, and they could then pay you with 100 transactions that re-allocate small numbers of coffee payments to your keys. Then the linkage would be much smaller. But in practice nobody does this.

How does the payment protocol solve that?

The request message allows you to request the payment be spread over multiple outputs (which don't have to be pay-to-address type outputs at that point, they could contain any script, like multi-signature scripts). It also lets the payer submit multiple transactions as part of satisfying the same payment. When an app generates a payment request (e.g. to get your salary), it knows how much money it's requesting and it can spread the money over many outputs. They payer might still generate one mega-transaction, but it's smarter for the payer to try and match up what it's got most closely with what's requested, to maximize privacy and minimise leakage.

Why does the sender submit transactions directly to the recipient?

Currently to pay someone you broadcast transactions to the P2P network and the recipient waits for them to arrive. This works OK, but can be inflexible. For instance, what if the sender doesn't have an internet connection? What if they're behind an evil firewall that only allows web traffic? What if the recipient has a deal with a specific miner and doesn't want to broadcast the transactions at all?

In the new payment protocol, the request contains instructions on how to submit the payment. Typically, the wallet will be given an HTTPS URL to submit the data too, but alternatives are easily possible. For instance, the Android wallet app already supports using Bluetooth to send transactions directly to the recipient (when both sides use Android). That's great when you're travelling/roaming and might not have an internet connection. The payment protocol will let us standardise that behaviour instead of it being specific to that one app.

What other features does it add?

In v1 of the protocol:

Refunds. The sender can submit some refund addresses to the recipient at the same time as submitting the transactions. Now if the seller wants to give the buyer a quick refund, they can do so. No more problems with trying to "guess" the users address and sometimes getting it wrong due to shared wallets. Bitcoin-Qt generates a refund address for every payment done this way, but you'll only see them if you do actually get a refund.

Memos. Clickable links can already contain short strings or labels, but their length is tightly limited by what browsers will accept and what fits in a QRcode. BIP 70 allows payment requests to contain arbitrarily long memos, which can be used to describe what you are buying. This is super useful when the payment request is authenticated ....

What is authentication about?

Bitcoin is a difficult project partly because we're moving money around with general purpose computers that can be hacked or get viruses. VISA and MasterCard have moved everyone (outside the USA) to special-purpose hardware like chip cards and dedicated readers which can't have random apps installed on them. Our approach has to be a bit different, but dedicated hardware is still coming. The Trezor device is a mini computer just for Bitcoin that plugs into a "real" computer via USB. It has a display and a couple of buttons. It holds your private keys. When you want to make a payment, the details of the payment are shown on the screen and if they're what you want, you press OK and things get signed.

There's an obvious flaw with that design when combined with Bitcoin as it is today. The "details of the payment" you'll see on screen will look like this:

Pay 0.5 BTC to 1EZEqFBd8yuc9ir2761987q7k3VcALC8YQ?

Where did that address come from? Well, it came from your computer. The thing that probably has a virus on it. That virus could have swapped out the address sent by the online shop you're using for one owned by the virus author, and because they're just random numbers you'd never know. Although the virus can't empty your wallet immediately, it can still steal payments when you make one, and avoiding that is the whole point of using special hardware!

The payment protocol allows recipients to sign their requests under an "identity". An "identity" is just some arbitrary string, that was itself signed by some "certificate authority". The authority vends signed statements that say (simplifying a bit) "I, Bob Smith, believe address 1EZEqFBd8yuc9ir2761987q7k3VcALC8YQ is owned by Mike Hearn". Now if your Trezor happens to trust that Bob Smith validates identities reliably, it can display:

Pay 0.5 BTC to Mike Hearn?

and that message can't be tampered with by any virus. As you can see this is much safer.

How do receipts work?

When a payment request is signed it's not just the addresses. The memo is also signed. By keeping around the signed payment request, and the transactions you created to satisfy it, you obtain a mathematical proof of payment. The merchant cannot dispute what was paid for because the memo field contains a description, nor can they dispute that payment was delivered (unless they claim they lost control of their private keys, of course).

This is great because it fills in a missing piece for low-trust third party dispute mediation using multi-sig transactions. Currently if you tried to implement that, you'd run into the problem of a buyer saying "seller did not deliver!!" and the seller saying "the buyer never paid me!" or more problematically, buyer saying "seller gave me something different to what I asked for" and the seller saying "I gave him exactly what we agreed on".

With receipts, it suddenly becomes much easier to figure out what happened in a transaction and recompense the right party.

Do payment requests have to be signed?

No. Without a signature they are basically just a container for addresses and memos.

Do payment requests get stored in the block chain?

No. Payment requests and responses are transmitted directly between buyer and seller. They are not recorded publicly.

What about other features, like tips?

The payment request and response messages use a format called protocol buffers, which is easy to extend. I compiled a list of future feature ideas here:

None of those exist today, but before they can be created we need somewhere to stash the data they require. The payment protocol is the perfect place.

How do identities work?

The protocol can be extended with multiple methods, but for v1 the only one that works is X.509 certificates. This is the same scheme used for SSL.

Essentially, anyone can get a certificate attesting to some kind of identity. Often it's just an email address or website domain name (e.g. bitcointalk.org has a certificate containing its domain name). It can also be a legal name like "Michael C Hearn" or "Mt Gox Ltd". In fact, if you have a passport with a chip in it then you already have a certificate containing your passport data, but it's not very useful because there's no private key associated with it.

Sometimes certificates are paid for and other times they are free. The certificate authorities never get your private key. Instead you generate a private key locally, then ask the CA to sign your public key. At the same time you prove your identity, if your identity is an email address then you typically receive a clickable link there for instance. If it's your name you might have to submit some paperwork.

Does this make Bitcoin less anonymous or private?

No. People always know an identity of who they're paying already. That doesn't have to be a legal identity. It can be a forum handle or pseudonym too, so even on the Silk Road this holds true - you know you're paying "Drug Dealer Dan" or whoever. But you basically never send money into the void with absolutely no idea of who it's meant to go to (outside of special protocols we're not going to address here).

Knowing the identity of who is paying you is less common, but the payment protocol doesn't establish two-way identity. Only the entity requesting payment has the option to sign (and of course they don't have to).

Can I use the PGP web of trust?

No, that's not supported in v1. However anyone can become a certificate authority, in fact Apple ships a GUI for creating and running one with every copy of MacOS. But you would have to convince people to trust you, which is the hard part. The "web of trust" doesn't solve that problem - you would still have to convince people to trust you for the purpose of signing keys.

That said, it's imaginable that trusted community members might create their own CA's and start issuing identities. For example, BitcoinTalk could issue certificates where the identity is your forum username. If wallet authors/hardware wallet designers decided to trust theymos for that purpose, you could see forum handles appear on your Trezor.

Isn't SSL horribly broken?

No, not really. Some people say it is, but there are no better systems - the X.509 PKI (public key infrastructure) reflects the result of many years of evolution and improvement. What's more, it's still being improved. Because it underpins SSL there's a lot of rich and powerful entities who want to see it get better, like Google, which pays people to work on it.

Let's make that concrete. One of the most common concerns with the X.509 PKI is that there are too many certificate authorities. This is still an improvement on the old days (early 90's) when there was only one CA, but having hundreds means that any CA can issue a cert that every user will accept. What stops a CA issuing a bogus cert to an imposter, like a hacker or a government .... or just another customer?

Currently the act of issuing a cert leaves no traces so this is a real concern, although there are remarkably few known cases of it happening given the enormous size and scale of the PKI. Google is developing a solution called "certificate transparency":

It's a public post-unforgeable audit log of all certificates issued. It works a little bit like the Bitcoin block chain, in fact. Each certificate can contain a proof (some Merkle branches) linking it to an entry in the public log. Once a CA has started using the system, if a certificate is found signed by that CA which does not contain such a proof it can be rejected as invalid. This means the CA can no longer issue certificates in secret. By watching the public audit logs, you can spot if a CA issues a certificate in your name .... or even in a name that's just confusingly similar to yours (so it is a way to fight against phishing as well).

I still prefer the web of trust

You shouldn't. The PKI is the result of evolving a web of trust style system over many years, as it got real usage. It looks the way it does because of its solutions to the problems the raw web of trust model has.

For instance, let's say Bob starts signing keys using GPG. His signature is worth very little unless I happen to know and trust him. In practice as a new Bitcoin user who just saw it on CNN, I don't know Bob. However, maybe I do trust the guys making the Trezor because they are a real company and they live in a country with sane laws, etc, they have lots of happy customers, so that gives me a starting point. Then if stick and slush trust Bob, and Bob has signed the key of bitcoinstore.com then I have a chain of trust to the store.

Bob has become a certificate authority!

How trustworthy is Bob, really? Does he keep his private key on a computer running a warez copy of Windows XP that is full of malware? It would be nice if we could formalise the "stick and slush trust Bob" part of the above description. Otherwise what stops Mallory turning up and demanding that Trezor trusts him too?

A good way to resolve this conundrum is to come up with a set of best practices, like keeping your private key inside a hardware security module, and setting some rules around when Bob/Mallory should sign a public key. To increase trust in the system and stop Mallory just claiming he follows the rules when really he doesn't, we might want to create a formal audit system and an auditor organisation that verifies these guys are following the rules.

Eventually as Bob and Mallory get more professional and trusted, they'll discover it's sort of hard to do it in their spare time so they'll create companies and start charging fees. They'll compete in the open market. After a long time, some of them will discover that for the most basic kind of key signing (emails and domain names) it can be entirely automated and done for free.

That's StartSSL.

As the number of trusted parties goes up and they handle more and more key signings, eventually Bob or Mallory might get hacked or pressured by the government. It'd be nice if everyone knew what keys Bob and Mallory had signed, in a more scalable way than just relying on everyone to upload all their keys to the MIT key server.

That's certificate transparency.

I hope you can see now why the PGP web of trust would eventually end up being pretty similar to the regular PKI, if it got big enough.

I'd like to note that splitting initial payment into 10 addresses does not help with privacy much. When you run out of money, your next payment will aggregate several "change" outputs thus linking you to the very first salary payment. Real privacy can only be achieved when you link several outputs that have very big distance between each other in blockchain. For that you'd need real mixing.

Also, when you pay to the same person from different chunks into which your salary was split, he now can detect that you own all those chunks. If the salary was not split from well-mixed (very far from each other) outputs by your employer, it'll be easy to estimate your whole salary size just by seeing a couple of chunks connected together.

The chunks aren't linkable, that's the idea. So you can't estimate the salary, you can only get a lower bound.

Edit: to clarify, no mixing is required or involved. The coins that were used to pay your salary have never been linked because they came from different people to start with. If you can avoid ever linking them in the process of paying your employee, mixing is not necessary to improve privacy a lot.

Today the payment request system is intended for online shops and payment processors like BitPay. Gavin wrote some PHP that can be used to generate and serve the signed requests server side.

But in future wallets might create payment requests for their users too. On Android this will be a seamless upgrade - the QRcodes people scan today will start including Bluetooth MAC addresses and the protocol will run over that. We've already got a prototype launched in the Play Store that doesn't use BIP70 but something much more primitive. No pairing is involved, the user experience is seamless.

On the desktop you'd probably fill out a form in your wallet GUI (for the price, the memo, etc) and then drag and drop an icon onto an email you're composing, IM window, web form etc. Anywhere that lets you transfer files basically. The request would be sent to the other side and opened by their wallet.

For signing, anyone can get X.509 certs for an email address for free from StartSSL. It's not super convenient to sign up, but there's no technical reason it can't be just one or two clicks. Once you got a certificate, most operating systems have a "key store" function built in which will let you install the certificate and encrypt your private key for you, often linked to whether the screen is unlocked or not. Android does this as well. Wallets can then talk to the OS to get the key needed for signing.

For receiving asynchronous payments that you didn't explicitly request, I think we'll see web sites pop up that host payment request files for you. Whenever you receive money via the uploaded request your wallet would notice and then upload a fresh one. This would be useful for tips, etc, that currently leak a lot of private data.

The request message allows you to request the payment be spread over multiple outputs (which don't have to be pay-to-address type outputs at that point, they could contain any script, like multi-signature scripts). It also lets the payer submit multiple transactions as part of satisfying the same payment. When an app generates a payment request (e.g. to get your salary), it knows how much money it's requesting and it can spread the money over many outputs. They payer might still generate one mega-transaction, but it's smarter for the payer to try and match up what it's got most closely with what's requested, to maximize privacy and minimise leakage.

This is just abusing the blockchain. It won't work when tx fee gets higher in the future

Hardly. Making payments with good privacy is the whole point of Bitcoin, isn't it?

No. I believe the whole point of Bitcoin is "A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution". Privacy is just a secondary objective.

Back to the topic. Unless the salary is paid with 100 completely unrelated transactions, your friend could still figure out your salary. This will create a massive overhead and is obviously not sustainable.

In addition, it would be a stupid idea to ask your boss to pay directly to a hot wallet on your smartphone. Firstly, it's unsafe. Secondly, you boss can see all your transactions. The right way is to ask your boss to send to your salary wallet, and use some coin-mixing scheme to move the money to your spending wallet.

Hardly. Making payments with good privacy is the whole point of Bitcoin, isn't it?

No. I believe the whole point of Bitcoin is "A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution". Privacy is just a secondary objective.

Back to the topic. Unless the salary is paid with 100 completely unrelated transactions, your friend could still figure out your salary. This will create a massive overhead and is obviously not sustainable.

In addition, it would be a stupid idea to ask your boss to pay directly to a hot wallet on your smartphone. Firstly, it's unsafe. Secondly, you boss can see all your transactions. The right way is to ask your boss to send to your salary wallet, and use some coin-mixing scheme to move the money to your spending wallet.

I think he's saying that payment protocol would generate new destination addresses so you would not leak other wallet addresses to your boss.

Probably there is a happy spot between consolidation of transactions/leaking information and overwhelming the blockchain. I'd imagine that this would be both configurable and have defaults that get adjusted with each client release based ultimately on the ratio of the rate of growth of the blockchain vs. rate of growth in storage space per dollar.

In some cases the payment won't be broadcast to the network but sent directly to the merchant ("What if the recipient has a deal with a specific miner and doesn't want to broadcast the transactions at all?"). In these cases, I could immediately double spend those outputs. How is this prevented?

Yes, thezerg has it right. But I really wish people would back up arguments about scalability with numbers. I think that kind of approach to privacy will scale fine. Prove me wrong.

Yes I mean SSL private key.

The merchant is responsible for broadcasting the transaction if they think you might double spend. You're not responsible for anything except handing it off to the merchant. If the tx is valid they can then propagate in parallel with starting to provide the service (pouring your coffee for example).

So you want to use the OpenPGP WoT to authenticate payment protocol requests... here's how you do it:

1) Set pki_type in PaymentRequest to 'openpgp'

2) Set signature in PaymentRequest to a detached OpenPGP signature of the serialized_payment_details

3) Set pki_data to an empty string. Why an empty string? Because OpenPGP signatures already specify the key doing the signing; if you don't already have that key there's a good chance the user never verified it anyway.

4) (client-side) Verify the payment request by checking that the signature is valid. If you don't already have the key, you should tell the user the PaymentRequest can't be used because you don't know who signed it; don't let them pay the request unless the key is already imported and GnuPG or whatever trusts it and don't do that for them.

As for implementing this, you can use the GPGME library for C/C++ code (like the reference client) On Python I've used the python-gnupg library with success. Dunno much about Java, but I'm sure you can find something.

But remember that crypto doesn't exist in a vacuum - the payment protocol is mostly useful for multi-factor wallets, where you need to authenticate the same payment request on multiple devices. If you want your hardware wallet to make use of OpenPGP-signed payment protocol requests, it needs to support OpenPGP and you need to securely import your web-of-trust database into it. You're probably going to find that something like the following is far simpler, and almost as likely for users to use securely:-----BEGIN PGP SIGNED MESSAGE-----Hash: SHA256

Donations to the Keep Bitcoin Free project are accepted at Peter Todd'sdonation address:

Yes, thezerg has it right. But I really wish people would back up arguments about scalability with numbers. I think that kind of approach to privacy will scale fine. Prove me wrong.

Yes I mean SSL private key.

The merchant is responsible for broadcasting the transaction if they think you might double spend. You're not responsible for anything except handing it off to the merchant. If the tx is valid they can then propagate in parallel with starting to provide the service (pouring your coffee for example).

Transactions

If you make multiple payments in a row you might wind up not having any confirmed coins to pay the merchant with, which is why the payment protocol lets a Payment message contain more than one transaction. Wallet software will need to determine what unconfirmed transactions are required to complete a given payment, and give the merchant the whole lot. Merchant-side you need to make sure your software considers the payment complete provided the requested txout scriptPubKey:value exists in the blockchain - it's quite possible that the transaction that actually ends up paying you is not the one the customer actually gave you. Note how this also means that every order must use a separate scriptPubKey for that reason.

Right, thanks for the correction. In the first deployment wallets will still be treating 1 tx == 1 payment. The ability to request multiple outputs and submit multiple transactions will get activated over time as wallets get smarter.

Please add Coinpunk to the list of clients that intend to support this. Thanks!

I need time to fully digest the implications of this, it's very interesting (and thank you for using protocol buffers here), but I'm initially a little skeptical as to whether this is a good approach. I think depending on central-server hot wallets has been demonstrated to be a dangerous approach to implementation of Bitcoin systems, as evidenced by the theft of millions of USD worth of Bitcoin over the last few years. I do feel like this system could potentially be just as dangerous as addresses, if not more so.

For example, as a merchant I could create a receive address on an offline computer, which is a design where your Bitcoins are very unlikely to be stolen. The trade-off is "privacy" of course, but from my chair, I'm way more concerned with security than I am with privacy. And the way I'm seeing most Bitcoins stolen right now are from server break-ins.

I also have concerns about increasing the complexity of Bitcoin transactions. This definitely increases the education curve, which is probably one of the main reasons why addresses became more popular: they are simple to understand.

It also introduces a non-Bitcoin system as a dependency, which may have far reaching consequences. What if that SSL certificate is revoked, or the government raids and confiscates the domain? If you need the domain to receive a transaction, that is quite a dependency introduced.

Again, very interesting, and I do like the CA approach, but I do have concerns about whether it is an improvement on the address system. It's my hope that there won't be a de-prioritization of the latter for the former.

My apologies in advance if I have not understood something, I have only had a chance to briefly skim over this so far and it's very possible I've misunderstood some of the machinations here and am talking nonsense.

Right, thanks for the correction. In the first deployment wallets will still be treating 1 tx == 1 payment. The ability to request multiple outputs and submit multiple transactions will get activated over time as wallets get smarter.

Merchants accepting multiple conflicting transactions is important for CoinJoin, because not all your inputs might be yours and thus it's possible they could get double-spent. You should give the merchant two transactions in this case, one being the CoinJoin tx spending yours and their inputs to their and the merchants txout, and another spending just your input. (second tx having a slightly lower fee so it's not the preferred one by miners)

Of course this does degrade privacy slightly as the merchant can figure out which inputs were yours, but the blockchain record remains anonymized and the next payment you make will come from inputs whose inputs a step back are anonymized.

Another example is multiple payments in a row: suppose I'm Alice and I first pay Bob, and then Charlie using the payment protocol. When I pay Bob I create and sign a transaction, tx1, and give it to Bob in my Payment message. That transaction has two txouts, one that pays Bob, and the other being my change address.

But when I pay Charlie I should actually create two transactions: the first is being a modified version of tx1 - tx1b - that double-spends tx1, (potentially adding new inputs) and adds Charlie's requested address as an additional txout. The second transaction, tx2, then depends on the change output of tx1 and pays Charlie that way. Normally the merged tx1b will be what confirms, efficiently paying both Bob and Charlie in a single transaction, but if tx1 confirms first Charlie still gets paid with tx2. (the order of transactions in the list given to the merchant in this case is tx1b, tx1, tx2)

Of course, if any of the above actually gets implemented I'll be pleasantly surprised.

You can still keep keys offline when using payment requests. It doesn't change anything in that respect.

With regards to increased complexity, yes it's a concern but for now it's only going to be merchants who deal with this (and it's optional of course). In future it'll be used more widely as well, but I think we can come up with simple user interfaces for that.

If a government seizes your domain then having to create unsigned payment requests is really the least of your problems. But at any rate, it means your payment requests wouldn't validate and you'd have to stop signing them (thus making your users less secure).

A future version of the payment protocol can allow the receiver to give the sender a BIP32 seed, and a total amount requested value. The receiver then considers the payment satisfied once outputs with addresses derived from that seed exist in the blockchain with a total value >= the total amount requested.

This means a merchant with debts to someone else, like their employees or suppliers, can get PaymentRequests from whomever they are in debt to and in effect have their customers pay those debts directly.

If those suppliers in turn have debts to other parties the process can repeat again. This time the supplier to the merchant just gives the merchant a list of addresses to pay, addresses that happen to belong to whomever the supplier owes a debt too. (maybe their employees?) Now two steps in the whole payment process have been skipped, and the customers of the merchant could very well be actually paying into wallets owned by employees of the merchant's suppliers!

In the business world 'Net 30 days' accounts are very commonly used, so this kind of arrangement could actually be quite practical and useful. Everyone in between saves on transactions fees, there would be a lot less demand for blockchain space, and privacy is improved for everyone.

Of course, it'd be a bit surprising if you went to pay a bill on Thursday night and the moment you paid it your wallet told you your employer had paid the exact same amount as part of your upcoming Friday paycheck...

It just sounds too complicated. Raised level of complexity normally will have unforeseeable consequence if something went wrong. Maybe this feature could be modularized and called from the official protocol