It seems that when you send a bitcoin transaction, all the coins in the sending address are spent in that transaction, divided into the amount that you intended to send, and "change", which goes back to you, but at another (newly created) receiving address.

That makes it a bit difficult to track your balance on block explorer (especially since the new receiving address does not seem to be displayed in the Bitcoin client anywhere).

Does it have it have to work like this, or is this a specific implementation detail of the client software?

Also, do you have to wait for change to be confirmed before you can spend it again?

I have this same issue. The question I have, does the spec (github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) dictate what is supposed to happen? Or is it up to the wallet to take the best guess as to what to do? And secondly, if wallets are allowed to do whatever they want, how can an app traverse all the addresses that might be used by some other wallet to transfer funds to? It seems nearly impossible because it seems like a wallet could do whatever it wants. It might hide your funds in some obscure place if it doesn't follow any conventions.
– Melbourne DeveloperJan 8 '18 at 0:06

3 Answers
3

"Accounts" are used for the convenience of people to track their funds. This is primarily used to track the source of funds. Since this is just for your tracking, you can move Bitcoins from one account to another just by moving a number from one column to another. No transactions are needed. (This is like when you know you owe your son $25 for allowance, and you have $200 budgeted for groceries.)

"Addresses" are used to receive Bitcoins in transactions. The coins are sent to an address. The client associates each address with an account and adds received funds to that account. This is simply done for convenience to allow people to track indirectly which address funds were sent to. But you can have any number of addresses associated with the same account.

Change comes from the way Bitcoins are spent. To spend a certain number of Bitcoins, you must pull in Bitcoins from transaction outputs to accounts you control. Note that in the spending part, it doesn't matter what address this is or what account that address is associated with. When you spend Bitcoins from a particular account, that just means you debit that account for the amount you send. It doesn't mean the funds come from addresses associated with that account. Remember, the association between addresses and accounts is for receiving only, not sending. (Like when you spend money on groceries, it's not like you have specific bills for groceries. You just have an amount budgeted.)

So when you pull in transaction outputs, you form a pile of Bitcoins big enough for the number you are trying to send. Usually, it won't be exact since you must claim an entire output. So the excess forms the 'change'.

Since there is no address associated with sending Bitcoins, there is no particular address the change should be sent to. So, to preserve anonymity, the client creates a new one just to receive the change from this transaction. Since this address isn't really associated with an account and shouldn't be used to receive any more Bitcoins (because that would senselessly tell people the same recipient got the coins as got this change) the client does not display it.

Because the client manages coins in a particular way, it doesn't make sense to try to view coins it is managing with any kind of explorer. It's specifically trying to obscure the fact that all the coins are related. Those kinds of services are intended to monitor recieved funds, not managed funds.

The information in David's answer is correct, but it may not answer the actual question -- it's unclear whether the question is about change in general, or specifically sending change to a new address.

If the latter, nothing needs to be added. If, however, the question was about the practice of change in general, then yes, it is necessary.

The reason for this is that an output, when used as an input, must be spent in its entirety. Say someone sends you 10 BTC, that 10 BTC is a single output. You cannot spend part of that coin, the same way you couldn't slice off a part of a physical coin and have it maintain its value.

The reason for this is that an output is actually a script, and to spend an output you simply broadcast the solution to this script. That output, in its entirety, becomes an input -- and if you were to not include change, the difference between the inputs and outputs would all go to transaction fees.

What is client-specific is how to handle this change: the Satoshi client sends it to a new address, while other clients may simply send the change back to one of the output addresses.

The change is an output like any other, which means that you do need to wait for confirmations -- but you can still use that change to send another transaction immediately, you'll just need to wait for the first transaction to confirm before the second one can.

Does it have it have to work like this, or is this a specific implementation detail of the client software?

Yes, this is implemented by the Bitcoin protocol itself, but it's only best practice not to re-use addresses. Since we are dealing with a pseudo-anonymous cryptocurrency would make sense to increase "anonimity" the more we can do.
You can however force your wallet to send "change" to a static and permanent address without generating a new change address each time you make a transaction. This will reduce substancially your privacy and transactions history will not be "obfuscated" as it is intended to be.

Also, do you have to wait for change to be confirmed before you can spend it again?

No, you can spend the change coins with 0 confirmations also.

Except coinbase coins (i.e. fresh mined coins which needs 100 blocks to mature) you can spend coin with 0 confirmations from command line wallets and many others. This is permitted by the protocol itself but many clients may not allow users to broadcast txs until the inputs has 1 confirmation at least, or even more.

REMEMBER that spending 0 confirmations coins from untrusted source can result in transaction pruning from the network mempool due to possible "pseudo-double spending" attack. In the case of spending your own change with 0 confirmations you are trusting yourself as the original sender so this is not a problem (unless also the original transaction which generated the "change" coins was with 0 confirmations from an untrusted sender.)