With a given set of patterns,
the goal of redundancy reduction without loss of information is this:
Generate a new set of pattern vectors (the code) such that (a) for
each pattern from the original set
there is exactly one pattern
from the code
(represents) and (b) the
components of code patterns are ``less dependent on each other''.
To make (b) more precise, let us measure the
degree of the redundancy of a random variable
with components by

where

denotes the entropy of variable .
Ideally, the result of redundancy reduction
is a ``factorial code''.
With a factorial
code, the code components are statistically independent:

(throughout this paper, the subscript of a symbol representing
a vector indexes the -th component of the vector).