In the field of unsupervised learning
several information-theoretic objective functions (OFs)
have been proposed to evaluate the quality of sensory codes.
Most OFs focus on properties
of the code components --
we refer to them as code
component-oriented OFs, or COCOFs.
Some COCOFs explicitly favor near-factorial, minimally
redundant codes of the input data
[2,18,23,7,24]
while others favor local codes
[22,3,16].
Recently there has also been much work on COCOFs encouraging
biologically plausible sparse distributed codes
[20,10,25,9,6,8,12,17].

While COCOFs express desirable properties of the code itself they
neglect the costs of constructing the code from the data.
E.g., coding input data without redundancy may
be very expensive in terms of information required to
describe the code-generating network, which may need
many finely tuned free parameters.
We believe that one of sensory coding's objectives
should be to reduce the cost of code generation through
data transformations, and postulate that an important scarce
resource is the bits required to describe the
mappings that generate and process the codes.

Hence we shift the point of view and
focus on the information-theoretic costs of code generation.
We use a novel approach to unsupervised learning called
``low-complexity coding and decoding''
(LOCOCODE [15]).
Without assuming particular goals such as data compression,
subsequent classification, etc.,
but in the spirit of research on minimum description length (MDL),
LOCOCODE generates so-called lococodes
that (1) convey information about the input data,
(2) can be computed from the data by a low-complexity mapping (LCM),
and (3) can
be decoded by an LCM. We will see that by minimizing coding/decoding costs
LOCOCODE can yield efficient, robust, noise-tolerant
mappings for processing inputs and codes.

Lococodes through regularizers.
To implement LOCOCODE we apply regularization
to an autoassociator (AA) whose hidden layer
activations represent the code.
The hidden layer is forced to code
information about the input data
by minimizing training error;
the regularizer reduces coding/decoding costs.
Our regularizer of choice will be
Flat Minimum Search (FMS) [14].