According to our analysis LOCOCODE
attempts to describe single inputs with as few
and as simple features as possible.
Given the statistical properties of many visual inputs
(with few defining features),
this typically results in sparse codes.
Unlike objective functions of previous methods,
however, LOCOCODE's does not contain an explicit term
enforcing, say, sparse codes --
sparseness or independence are not
viewed as a good things a priori.
Instead we focus
on the information-theoretic
complexity of the mappings used for coding and decoding.
The resulting codes typically compromise between conflicting
goals. They tend to be sparse and exhibit
low but not minimal redundancy --
if the cost of minimal redundancy is too high.

Our results suggest that
LOCOCODE's objective may embody a general principle of unsupervised
learning going beyond previous, more specialized ones.
We see that there is at
least one representative (FMS) of a broad class of algorithms
(regularizers that reduce network complexity)
which (1) can do optimal feature extraction as a by-product,
(2) outperforms traditional ICA and PCA on visual source separation tasks,
and (3) unlike ICA does not even need to know the number
of independent sources in advance.
This reveals an interesting, previously ignored
connection between regularization and ICA,
and may represent a first step towards unification of
regularization and unsupervised learning.

More.
Due to space limitations,
much additional theoretical and experimental analysis
had to be left to a tech report (29 pages, 20 figures)
on the WWW: please see
[15].

Acknowledgments.
This work was supported by DFG grant
SCHM 942/3-1 from ``Deutsche Forschungsgemeinschaft.''