Introduction

The first step toward discovering general principles of sensory processing is to determine the correspondence between neural activity patterns and sensory stimuli. We refer to this correspondence as a "neural code." The Information Bottleneck and the Information Distortion methods [1, 3] approach the neural coding problem by finding an optimal clustering of paired stimulus/response observation (X; Y) by solving a constrained optimization problem, with both equality and inequality constraints, in hundreds to thousands of dimensions. The method of annealing has been used to solve this optimization problem: starting at an uninformative solution, one tracks this solution as an annealing parameter varies. The solutions undergo a series of rapid changes with the increase of the annealing parameter (Figure 1). We relate the changes to bifurcations or phase transitions in a dynamical system. The form of the bifurcations is dictated by the subgroup structure of SN[2]. As a consequence of this symmetry, generically only pitchfork-like and saddle node bifurcations are possible. The purpose of this contribution is to describe these bifurcations in detail, and to indicate some of the consequences of the bifurcation structure. The results are then applied to the neural coding problem.

Figure 1

The bifurcations of the solutions (q*,β) to the Information Distortion problem [1]. For a mixture of 4 well-separated Gaussians, the behavior of D(q) = I(X; T) as a function of β is shown in the top panel, and some of the optimal solutions q*(T; Y) are shown in the bottom panels.

We have been able to answer several questions about these bifurcations:.

1. There are N - 1 symmetry breaking bifurcations observed when continuing from the initial solution because there are only N - 1 subgroups in the chain SN→ SN-1→...→ S2 → S1.

3. Symmetry breaking bifurcations are generically pitchfork-like and derivative calculations predict whether the bifurcating branches are subcritical or supercritical, as well as their stability. Symmetry preserving bifurcations are generically saddle nodes.

4. A local solution to the optimization problem does not always bifurcate through a symmetry breaking bifurcation.

5. The bifurcations of solutions dictate the convexity of the curve in Figure 1. In particular, a subcritical bifurcation of solutions at I0 implies that the rate-distortion curve R(I) changes convexity in a neighborhood of I0. This is in contrast to the rate distortion curve in information theory, R(D), for which D(q) is linear in q.