Hyper Artificial Intelligence by Vlady Madejczyk

Example of Hopfield network

This is just simple example which you can download on your computer. Just unzip the file and that is it. Actually the example is for shapes, but can be modified for a different input signal.

link for download - example of Hopfield Neural Network - shape recognition

screenshot

hopfield neural network, education

Explanation

Well, actually it is not that complicated. At first we have to change complete shapes into vectors, to "explain" to Hopfield network the shape possibilities. It is preffered to have orthogonal shape vectors in the process. In the above example the 7x7 shape matrix (on the left side) is changed to 49 elements vector (all columns are appended) and put to the top box. When we have enough number of complete shape vectors in the top box, we change them from binary to bipolar (2*x-1) and build weights matrix. Building weights matrix: vectors from the top box are taken one by one, treated with a kind of bipolar function, multiplied by their transposes (every time we get 49x49 corellation matrix), and the corellation matrixes for the vectors are added to each other to give weights matrix. It is a bit like we explain to the 49x49 weigths matrix how every shape looks like. In other words the 49x49 matrix is a "brain" of Hopfield network. When weights matrix is ready, we can put an incomplete shape to the left side and just click the button = RECOGNIZE = to get a guess on the right side. During one click of the button there are about 5 thousands operations (*, +). This time the vectors from the left side are multipied directly by the weights matrix and treated by a threshold function (in the example for negative value the result is 0, for positive value the result is 1), to give the final 49 elements (7x7) vector which is at the same time the final image.

A bit intuitive explanation how the network works: imagine yourself three vectors in 3d space, vector A, vector B and vector C, and imagine that they points to two places close to each other. But which of the B and C is closest to the place pointed by vector A? Maybe it is easy to guess when we have 3d space and short vectors, but in 49d space the situation is difficult. Lets put it this way: every shape is like vector. Lets assume that we have 10 vectors already in the memory of the network, 10 for 10 complete shapes, that means that weights matrix is ready. And now we try to guess a new incomplete shape. The new shape is a vector, and the network will find to which one of the 10 vectors already in memory, the new shape is most similar. Obiously this is not strict mathematical language, but I hope it is easier now to grasp the idea.