An Introduction to Primality Testing

July 8, 2012

I will explain two commonly-used primality tests: Fermat and
Miller-Rabin. Along the way, I will cover the basic concepts of
primality testing. I won't be assuming any background in number
theory, but familiarity
with modular
arithmetic will be helpful. I will also be providing
implementations in Javascript,
so familiarity
with it will also be helpful. Finally, since Javascript doesn't
natively support arbitrary-precision arithmetic, I wrote a simple
natural number class
(SNat) that
represents a number as an array of decimal digits. All algorithms
used are the simplest possible, except when a more efficient one is
needed by the algorithms we discuss.

Primality testing is the problem of determining whether a given
natural number is prime or composite. Compared to the problem of
integer
factorization, which is to determine the prime factors of a given
natural number, primality testing turns out to be easier; integer
factorization is
in NP and
thought to be
outside P
and NP-complete,
whereas primality testing
is now
known to be in P.

Most primality tests are actually compositeness tests; they involve
finding composite witnesses, which are numbers that, along
with a given number to be tested, can be fed to some easily-computable
function to prove that the given number is composite. (The composite
witness, along with the function, is a certificate of
compositeness of the given number.) A primality test can either
check each possible witness or, like the Fermat and Miller-Rabin
tests, it can randomly sample some number of possible witnesses and
call the number prime if none turn out to be witnesses. In the latter
case, there is a chance that a composite number can erroneously be
called prime; ideally, this chance goes to zero quickly as the sample
size increases.

The simplest possible witness type is, of course, a factor of the
given number, which we'll call a factor witness. If the
number to be tested is \(n\) and the possible factor witness is \(a\),
then one can simply test whether \(a\) divides \(n\) (written as \(a
\mid n\)) by evaluating \(n \bmod a = 0\); that is, whether the
remainder of \(n\) divided by \(a\) is zero. This doesn't yield a
feasible deterministic primality test, though, since checking all
possible witnesses is equivalent to factoring the given number. Nor
does it yield a feasible probabilistic primality test, since in the
worst case the given number has very few factors, which random
sampling would miss.

The simplest useful witness type is a Fermat witness,
which relies on the following theorem of Fermat:

(Fermat's little theorem.) If \(n\)
is prime and \(a\) is not a multiple of \(n\), then
\[
a^{n-1} \equiv 1 \pmod{n}\text{.}
\]

Thus, a Fermat witness is a number \(1 \lt a \lt n\) such that
\(a^{n-1} \neq 1 \pmod{n}\). Conversely, if \(n\) is composite
and \(a^{n-1} \equiv 1 \pmod{n}\), then \(a\) is a Fermat
liar.

Let
n =
and
a =
.

If \(n\) has at least one Fermat witness that is relatively prime,
then we can show that at least half of all possible witnesses are
Fermat witnesses. (Roughly, if \(a\) is the Fermat witness and \(a_1,
a_2, \ldots, a_s\) are Fermat liars, then all \(a \cdot a_i\) are also
Fermat witnesses.) Therefore, for a sample of \(k\) possible
witnesses of \(n\), the probability of all of them being Fermat liars
is \(\le 2^{-k}\), which goes to zero quickly enough to be
practical.

However, there is the possibility that \(n\) is a composite number
with no relatively prime Fermat witnesses. These are
called Carmichael
numbers. Even though Carmichael numbers are rare, their
existence still makes the Fermat primality test unsuitable for some
situations, as when the numbers to be tested are provided by some
adversary.

Another useful witness type is a non-trivial square root of
unity \(\mathop{\mathrm{mod}} n\); that is, a number \(a \neq \pm
1 \pmod{n}\) such that \(a^2 \equiv 1 \pmod{n}\). It is a theorem of
number theory that if \(n\) is prime, there are no non-trivial square
roots of unity \(\mathop{\mathrm{n}}\). Therefore, if we do find one,
that means \(n\) is composite. In fact, finding one leads directly to
factors of \(n\). By definition, a non-trivial square root of unity
\(a\) satisfies \(a \pm 1 \neq 0 \pmod{n}\) and \(a^2 - 1 \equiv 0
\pmod{n}\). Factoring the latter leads to \((a+1)(a-1) \equiv 0
\pmod{n}\), which means that \(n\) divides \((a+1)(a-1)\). But the
first condition says that \(n\) divides neither \(a+1\) nor \(a-1\),
so it must be a product of two numbers \(p \mid a+1\) and \(q \mid
a-1\). Then \(\gcd(a+1, n)\)[1]
and \(\gcd(a-1, n)\) are factors of \(n\).

Finding non-trivial square roots of unity by itself doesn't give a
useful primality testing algorithm, but combining it with the Fermat
primality test does. \(a^{n-1} \bmod n\) either equals \(1\) or not.
If it doesn't, you're done since you have a Fermat witness. If it
does equal \(1\), and \(n-1\) is even, then consider the square root
of \(a^{n-1}\), i.e. \(a^{(n-1)/2}\). If it is not \(\pm 1\), then it
is a non-trivial square root of unity. If it is \(-1\), then you
can't do anything else. But if it is \(1\), and \((n-1)/2\) is even,
you can then take another square root and repeat the test, stopping
when the exponent of \(a\) becomes odd or when you get a result not
equal to \(1\).

To turn this into an algorithm, you simply start from the bottom
up: find the greatest odd factor of \(n-1\), call it \(t\), and keep
squaring \(a^t\) mod \(n\) until you find a non-trivial square root of
\(n\) or until you can deduce the value of \(a^{n-1}\). In fact, this
is almost as fast as the original Fermat primality test, since the
exponentiation by \(n-1\) has to do the same sort of squaring, and
we're just adding comparisons to \(\pm 1\) in between squarings.

The original idea for the test above is from Artjuhov, although it
is usually credited to Miller. Therefore, we call \(a\) an Artjuhov witness[2] of \(n\) if it shows \(n\) composite by
the above test.

Let
n =
and
a =
.

If \(n\) is an odd composite, then it can be shown (originally by
Rabin) that at least three quarters of all possible witnesses are
Artjuhov witnesses. Therefore, for a sample of \(k\) possible
witnesses of \(n\), the probability of all of them being Artjuhov
liars is \(\le 4^{-k}\), which is stronger than the bound for the
Fermat primality test. Furthermore, this bound is unconditional;
there is nothing like Carmichael numbers for the Artjuhov test.

isProbablePrime called
with hasFermatWitness is the Fermat primality
test, and isProbablePrime called
with hasArtjuhovWitness is the Miller-Rabin primality
test. The latter is the current general primality test of
choice, replacing
the Solovay-Strassen
primality test.

In this case, for sufficiently large \(b\), the Fermat primality
test is acceptable, since Carmichael numbers are so rare and we're the
ones generating the possible primes to be tested.[3]

There are other primality tests, but they're less often used in
practice because they're
either less
efficient or more
sophisticated than the algorithms above, or they require \(n\) to
have specialproperties.
Perhaps the most interesting of these tests is
the AKS
primality test, which proved once and for all that primality
testing is in P.