Kohonen Self-Organizing Map

The Kohonen Self-Organizing Map (SOM) designed by Tuevo Kohonen [6] is a variation of the traditional Artificial Neural Network. It is a third generation neural network, meaning that many of its functional characteristics are thought to mirror those found in biological fact.

An SOM consists of a collection of nodes of neurons that are each connected to every other node and each node has associated with it a set of input weights w. The SOM also has associated with it a metric for determining which nodes are in the neighborhood N of a given node.

When the network is presented with a vector x_i at its input, it computes the neural response s_j of the node j using the formula:

$$s_{j}~=~~w_{j} \cdot x_{i}$$ (1)

Normalize both w_j and x_i before computing the dot product, s_j, and refer to the node that produces the largest value of s as node k. Since the dot product of the normalized w_k and x_i vectors is the cosine of the angle between them, we can conclude that the winning node is the one with the weight vector closest to the input vector in its spatial orientation. We can then say that node k giving the largest s is closest to recognizing the input vector. We allow the nodes to learn by applying a $\triangle w$ to their weights using the formula:

$$\triangle w_{k} = \alpha~( x_{i} - w_{k} )$$ (2)

where $\alpha$ is a constant in the range [0,1] called the learning constant. The learning process is applied to the maximum response neuron and neurons in its defined neighborhood.

This training process can be described by the following algorithm:

1.

A cycle: for every input vector x_i

(a)

Apply vector input to the network and evaluate the dot products of the normalized weights on each node and a normalized input vector. Call these dot products s.

(b)

Find the node k with the maximal response s_k.

(c)

Train node k, and all the nodes in some neighborhood of k, according to the learning equation above.

(d)

Calculate a running average of the angular distance between the values of w_k and their associated input vectors.

(e)

Decrease the learning rate, $\alpha$.

2.

After every M cycles, called the period, decrease the size of the neighborhood N.

3.

Repeat steps 1-2 for some finite period of time or until the average angular distance calculated above is below a certain tolerance.

The effect of this process is to train the SOM to classify the input vectors into groups that will be characterized by particular values of w_k. In our simulations we used 10 input vectors and M, the number of cycles in a period, was also 10.