I use OpenSSL to check ECDSA validity. I believe Zathras has an implementation you could probably use since he is also using something Microsoft(y) to create his code, excuse my ignorance I'm not sure what language you are using Zathras.

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Once we agree on the output of these keys I think it's safe to try and broadcast a message

I use OpenSSL to check ECDSA validity. I believe Zathras has an implementation you could probably use since he is also using something Microsoft(y) to create his code, excuse my ignorance I'm not sure what language you are using Zathras.

I'll speak to this: feel free to use anybody's code. Just please be clear that you are doing so and don't try to take credit for it when payout time comes. If other people build on Zathras' code, the contest rules state that his payout gets bigger (how much it gets bigger is subjective, based on our collective feelings about how much his code sharing helped achieve the contest goals)

Redundancy is helpful too - if you write your own version of the code, the differences will help find bugs. The payouts in the last contest reflected this, with four people getting a share of the pot with a lot of overlap between their work.

I agree with vokain about the work you guys are doing - I couldn't be happier.

I was trying to collect all the great ideas from you all, and implement them in mastercoin-tools send and parse.It includes:1. using compressed pubkeys.2. using only valid ecdsa points.3. obfuscating the data.4. ignore all previous multisig experiments

The padded dataHex is 0100000000000000010000000000bc614e0000000000000000000000000000

After obfuscation (using sha256 of the string '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX') and adding 02 at the beginning and a random tail, it becomes 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4

Within 4 iterations of searching for a valid pubkey (each time adding 1), a valid one is found.

A transaction is created and signed.

A parsing test shows the same values.

On blockchain.info, the transaction looks this way. Note that the input used for the transaction was too small to include also change (the change was less than the dust limit), so all the under-dust change got added to the fee. The fee then increased to 0.0001228 instead of the requested 0.0001.

It will probably show up on Masterchest as well since I can't find anything wrong with it.

Please understand that we are currently working hard on a new encoding standard which requires us to reparse the data a lot, during these times our sites might not miss some transactions or perhaps not display them correctly. This is all very cutting edge software so things might go wonky from time to time.

Sorry guys have to be quick - back to back meetings today.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

Quote

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

Could I please suggest the parties involved in this transaction just hold for a little, while the development team have a discussion around Class A transaction validity.

Is the contest actually running now btw? FYI I'm (and think Tachikoma is also) still spending my time to properly achieve the goals of the last contest (storing data in the blockchain) as I don't feel that's actually finished yet (though we're close).

I use OpenSSL to check ECDSA validity. I believe Zathras has an implementation you could probably use since he is also using something Microsoft(y) to create his code, excuse my ignorance I'm not sure what language you are using Zathras.

.NET yep. The most recent changes haven't made their way into the masterchest library yet as they're in a separate codebase (you can still use the encodetx function to build the cleartext mastercoin packet but that function also builds the entire transaction, not just the packet so you'll have to pull the relevant data out of the final raw transaction hex). I'll have a modified version up with new functions for encoding/decoding reflecting the suggested amendments over the weekend.

For ECDSA validity (I'll also put a check function into the masterchest library) I took my inspiration from the casascius bitcoin address utility - have a read through some of the pubkey and ECDSA stuff to point you in the right direction.

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Once we agree on the output of these keys I think it's safe to try and broadcast a message

The padded dataHex is 0100000000000000010000000000bc614e0000000000000000000000000000

After obfuscation (using sha256 of the string '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX') and adding 02 at the beginning and a random tail, it becomes 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4

Within 4 iterations of searching for a valid pubkey (each time adding 1), a valid one is found.

A transaction is created and signed.

A parsing test shows the same values.

On blockchain.info, the transaction looks this way. Note that the input used for the transaction was too small to include also change (the change was less than the dust limit), so all the under-dust change got added to the fee. The fee then increased to 0.0001228 instead of the requested 0.0001.

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:

With each packet: * For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash. * For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash. * We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.Rinse & repeat.

Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!

So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

Quote

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.

How about we do the following.

Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.

All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.

Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

Transaction type is 0

Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.

I will make some time today to see if this change would affect any existing transactions but I highly doubt it.

Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:

With each packet: * For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash. * For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash. * We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.Rinse & repeat.

Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!

So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed, let's just use the sending address for all packages. This should go into the spec.

Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.

19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95

Quote

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.

Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:

Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.

So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.

I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.

None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

Completely agree the guy was not at fault at all, just unlucky. As rough as it sounds though it has to be the rules of the protocol that define transaction validity regardless of whether it's right or wrong from a particular point of view. I'm with definitely on board that this should be a valid transaction though and I'm confident if we present the right set of changes to include fringe cases that JR will be open to adopting them and dropping things like the perfect sequence invalidation (which would then make this transaction valid).

As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.

How about we do the following.

Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.

All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.

Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

Transaction type is 0

Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.

I will make some time today to see if this change would affect any existing transactions but I highly doubt it.

I definitely support only allowing simple sends, I have this in the amendment:

Quote

NOTE: Class A transactions are restricted to the ‘simple send’ transaction type only. All other Mastercoin transaction types are supported by Class B transactions only. Client implementations should utilize Class B for all transaction types, including ‘simple send’.

Also agree with re-instating the requirement for outputs to be for the same amount, I actually really liked this This requirement would only be for Class A transactions as you note.

If someone gets really unlucky and creates a transaction where the change amount equals the rest of the output, and then also gets ambiguous sequence numbers, we could use the decode & peek method - I'm just trying to consider ways this could be abused but I can't think of any (since we would throw the transaction out if we found 2 decode-able packets in a Class A transaction).

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

All protocol transactions should have the same output amount. If one output is different, that is the change address.

If all outputs are the same, then look at sequence numbers:

If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)

If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!

If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

...

It was originally stated that all outputs should be for the same amount & the odd one out was change. Then we needed bigger outputs and larger fees, so that requirement was made more of a convenience and we started to use outputs with different amounts, making checking for the odd one out as change no longer reliable and change was identified with sequence numbers instead.

It's all in the interpretation - just because the 'odd one out is change' method was no longer reliable as the outputs no longer had to be the same amount, that doesn't mean I can't use it to decode and validate a transaction in some cases. In other words, my implementation is being too strict/literal. I thus consider this transaction valid and will amend my code to reflect this

I definitely agree we should make the outputs be the same amount a requirement for Class A. If you guys & JR adopt the amendment we'll now have (logically) two classes of transaction, so we can make this really clear for others since we can apply rules to one class but not the other. We can make matching output values a requirement for Class A ('original') transactions but not Class B ('multisig'). We can apply the change-goes-to-sender rule to Class B and not Class A. Add peek & decode to Class A etc. Helps me keep things clear anyway

We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed, let's just use the sending address for all packages. This should go into the spec.

Excellent, using the sender address for all transactions will simplify the coding (and lessen the bugs).