Recurrent neural networks with complex dynamics as
a medium for
information processing
Dmitry A. Kuminov
Institute of Radio
Engineering and Electronics of the Russian Academy of Sciences, Moscow,
Russia
Due to recent investigations of the nature of electroencephalograms of human and animal brain, development of qualitative understanding of information processes in brain, experimental analysis and mathematical modeling of biological subsystems performing information processing, the fundamental role of deterministic chaos and complex dynamics in information processing by biological systems becomes more and more clear.
It should be noted, that in most papers connected with a use of chaos and complex dynamics in neural networks, the neural network itself plays a role of a "black box" which is trained according to certain rules in order to obtain a desired response to a certain signal at its input.
The aim of this report is to demonstrate an ability of an application of complex dynamics and chaos to storing and processing information in recurrent neural networks, used as a hardware realization of simple mathematical models of a kind of 1-D piecewise linear maps of a segment into itself [1-3]. Evidently this concept substantially differs from the presentation of a neural network as a "black box".
At the beginning of the report we give a brief summary of the earlier obtained results on the problem of storing and retrieving information using 1-D maps.
In the next part of the report a realization of 1-D map dynamics by a recurrent neural network with a single hidden layer is discussed. The network training for a single production of the sequence of the points describing the map function is shown to reduce to simple calculation formulas.
The third part is devoted to storing and retrieving pictures using recurrent neural networks. Here a presentation of color pictures as strings is considered. Further an example of storing and retrieving pictures and information retrieving from the network, is shown.
In the final part additional opportunities coupled with a transition from information storing on stable limit cycles to storing on unstable limit cycles and realization of global chaotic regime in the neural network, are discussed. In particular, a possibility of a use of neural networks with such dynamics for image recognition is shown.