Self Organising Maps

A map is a lower dimensional discretised representation of some input space. The self organising map (SOM) is a kind of artificial neural network. It differs in the sense of using a neighbourhood function which preserves the topological properties of the input space. This makes the SOM applicable for visualisation of high-dimensional data.

The SOM consists of a grid of nodes. Each node holds a randomised weight vector of the same dimension as the training/input data. The nodes are connected with another through a neighbourhood function, e.g. the Gaussian neighbourhood function.

During the training of the SOM, input vectors are assigned to the closest node based on some kind of distance metric. The weight vector of all nodes is updated towards the assigned input vector by
\[ w_i(t+1) = w_i(t) + \Theta(d(v_i,BMU),t)\gamma(t)(In(t)-w_i(t)) \]
where the weight vector \( w_i \) is updated dependent on the neighbourhood scaling \(\Theta(v_i,t)\) which depends on the distance between the updating neuron \(v_i\) and the Best Matching Unit (\(BMU\)), a decreasing learning coefficient \(\gamma(t)\) and the difference between Input Vector (\(In(t\)) and the present weight vector \(w_i\). The training is done sequentially with a great number of input vectors. During each step all nodes are by some degree moved due to the neighbourhood function. This leads to similar features mapped close together and dissimilar features apart. This way the SOM forms a semantic map over the feature space of the input data with more nodes pointing to feature reach parts of the input space. This characteristic can be used for visualisation by calculating an U-matrix (unified distance matrix). The U-matrix consists of the euclidean distance between neighbouring weights which can be represented in a grey scale picture where spots of light colour show clusters of near weights and therefore of the higher dimensional input data. After the training phase, input data can be mapped.

A Python implementation of a Self Organising Map applied to images

The following Python code is a simple implementation of a SOM to map pictures to a reduced colour space. The neighbourhood function is a simple radius function with a constant weight within the radius. The training is done with sampled pixels from the input image.