Problems with Bruce Schneier's "Solitaire"

Readers of Neal Stephenson's "Cryptonomicon" will be
familiar with the cipher "Solitaire"
(called "Pontifex" in the book), which was designed by
cryptologist Bruce Schneier specifically for the purposes of
the book. It is intended to be the first truly secure hand
cipher, and requires only a pack of cards for encryption and
decryption.

As yet, the technical information about the design of the
cipher has not been made available, but I decided to go
investigating, and I've now written a fast "C" implementation
of the cipher suitable for collecting statistics about the
CPRNG at its heart. I have found two interesting facts:

The CPRNG state machine is not reversible,
contrary to what the operational notes claim: the initial
step in which a joker is moved to the top if it is on the
bottom cannot be reversed. This is surprising since
non-reversible CPRNGs tend to have shorter periods and can
more easily exhibit bias.

And indeed, the output of the CPRNG is very biased. The
output of each step of the CPRNG is a number from 0 to 25;
you would expect successive outputs to be the same around one
time in 26, but my experiments show that the frequency is
closer to 1/22.5.

I'm making the source code for the tests I ran available to
the public domain. Note that the problems reported in earlier
versions are not present in this version; it turns out that it
was only my development version that had the problems anyway
and the one available for download was fine.

Perl implementation of Solitaire
modified to do coincidence counting and take the same user
interface as the C version, though of course the C version is
about 250 times faster. This is useful for debugging and
verifying the C version.

Normal distribution probability
calculator, for when your statistical significance gets
so high you fall off the end of your normal tables.
Unfortunately, the significance of the bias I've detected in
Solitaire (passphrase CRYPTONOMICON, 9999999 samples, 444745
coincidences) falls off the end of what this program can
calculate, at 98.87 standard deviations from the mean...

I believe I now know the main reasons why this bias arises:
when the value of the top card is the same in two successive
rounds, an event you would expect to occur with probability
just under 2%, the probability that the output letter is also
the same is very high, around 34%. This is I assume because
when this happens it's likely that many other cards are also in
the same positions across the rounds. This isn't the sole
source of the bias but it's by far the largest component. Below
is the output from my latest version of c-sol, instrumented to
measure the correlations between "coincidences" (two successive
outputs being the same) and "top matches" (the value of the top
card being the same on two successive rounds); everything it
measures shows some bias, so there's more to be found than what
I've discovered so far.