Plissken wrote:Great news!
They bruteforced it or detected the algorythm?

The algorithm has been known for quite some time now. A pair of 512-bit RSA keys along with an MD5 hash function were used to digitally sign the disks.
A network of distributed computing was utilized to sieve the RSA keys. So, technically, you could say the keys were "bruteforced".

Been looking into RSA encryption a bit to understand how to put these to use. I have absolutely no previous experience with encryption, so these are some notes on what I've found from Google searches.

If Altmer goes ahead and creates an image compiler that does this, then that's cool. Otherwise, I will (extremely slowly) do more research as I have time with the intention of (eventually) coding a small app to do disc encryption. So here's what I know so far:

With RSA encryption, there is a series of equations for determining public and private keys. The input and output variables are p, q, N, d, and e. It starts with p and q, which are arbitrary prime numbers (preferably large and of similar size). N is p * q. e is a value that is "relatively prime" to/coprime with (p-1)*(q-1) (in other words, e and (p-1)*(q-1) share no divisors but 1).

e and N constitute the public key.

The last value, d, comes from the equation e*d = 1 mod (p-1)*(q-1).

d and N constitute the private key.

So that's what all those different numbers are all about in the file that tport released. For us, we really care about d, e, and N, since those are the "magic values" that we can use to encrypt and decrypt data. But how do we do that?

Well, we need to convert our message (our data) to a number (m) and create an encrypted version of it (c) based on the following equation:

c = m^e mod N

Since we know e and N, we can calculate an encrypted message given an input message m.

When we have an encrypted message (c), we can find the decrypted message (m) with the following equation:

m = c^d mod N

So there are a few things to consider still, before we can apply this to the 3DO. The following are questions that I personally still don't understand:

1. Why do encryption examples always use the public key to do the encryption, and the private key to do the decryption? This seems backwards.

2. What exactly is the difference between encrypting and signing? Which is the 3DO doing? Or is it doing both?

3. Why are there two key sets for 3DO encryption/signing?

4. How do we get a number for the message in encrypting/signing 3DO data? Presumably, this is where the MD5 hash function comes in, but I don't know what that function is.

5. What files do we need to encrypt/sign?

Anyway, these are some opening thoughts. Hopefully there will be more to come.

RSA signing is interesting. It works roughly like this (in 3DO terms):

* The 3DO Company wants to send my 3DO a signed message (an approved game)
* The 3DO Company will take my unsigned game and run a hash function on it to produce a number.
* They will then take this number, raise it to the power of (d mod n) and attach it to the message as a signature.
* My 3DO system will read the message and produce the same hash as the 3DO Company originally did.
* To verify the signature, my 3DO system will raise the signature value to the power of (e mod n). Due to some crazy math I don't understand, this has the effect of decrypting the signature value
* My 3DO then compares the decrypted hash to the hash it calculated itself. If these values are the same, then that confirms the 3DO Company's identity and verifies that this is an authorized message/game. (That's because only their private key would create a message that could be decrypted with the given public key)

So, that's half a step closer to understanding what's happening. I still don't understand why there are two keys, though. Maybe one is for encrypting and one is for signing? But what is actually being encrypted?

Also, what is the hashing algorithm? I'm skeptical that it's MD5 because MD5 doesn't produce a number as its output, which is required for this sort of encryption. Since every 3DO system would have to perform this hashing, the algorithm must be stored in the system BIOS. Maybe we could reverse engineer it to find it?

Maybe... Eventually... But at this point, who the hell knows if I'll actually get that far?

Anyway, mini update: I found the location in the FZ-1 BIOS where the public keys (N) are stored. Key #1 starts at offset 0x00014F49 and Key #2 starts at offset 0x00014F8d. They're stored in hexadecimal. How is this useful? It probably isn't. But it's nifty to look at.

In the meantime, I'm installing cygwin so that I can use some Linux tools (specifically, binutils and objdump) to maybe disassemble the FZ-1 BIOS. Maybe then I can relearn some assembly and figure out the hash function? We'll see.

Applying this to the orbatak iso enabled it to boot to the banner screen, but it went to the 3DO logo and startup screen after that. My guess is there's a problem with the signatures, since the key for it hasn't been fixed.

I had to replace the boot_code, os_code and banner screen with those from the homebrew CD.