Spiking Memristive Architectures for Learning to Learn

Contemporary artificial intelligence (AI) and machine learning applications often imply heavy computational loads with current technology. However, there is a growing demand for low-power autonomously learning AI systems that are employed in the field. This scenario includes applications in mobile devices, autonomous mobile robots and vehicles, edge devices, and the internet of things, to name just a few obvious examples. The SMALL project will investigate low-power solutions for this alternative application scenario. Inspired by the architecture of the human brain, the system will be based on networks of so- called spiking neurons, neurons that communicate by sending short voltage pulses via synaptic connections. Today, neural networks are implemented in digital hardware. In contrast, we are developing hardware that implements the operation of spiking neurons using analog voltages and the specific properties of the computational substrate. This design principle naturally leads to low-power systems. Another essential innovation of the system is its memory-technolgy, which will not consist of standard memory elements, but rather of memristive memory. In contrast to usual memory technology, this novel type of memory does not only store bits but analog values, with the promise for extremely high memory density and further improvements of memory efficacy. In the field applications often demand online adaptation of such systems, which usually necessitates slow and complex training procedures. To overcome this problem, we will investigate the applicability of learning to learn (L2L) to spiking memristive hardware. In other words, the system will be trained in the laboratory to quickly learn a task in the envisioned application domain. In the application itself, the system then quickly adapts to the task at hand. In summary, the goal of the SMALL project is to build versatile and adaptive low-power small size AI machinery based on spiking neural networks with memristive synapses using L2L. As a proof of concept, we will deliver an experimental system in a real-world robotics environment.

Contemporary Artificial Intelligence (AI) applications often rely on methods that imply heavy computational loads. However, there is a growing demand for low-power autonomously learning AI systems that are employed "in the field". We investigated in this project options for learning in low-power unconventional hardware that is based on so-called spiking neural networks (SNNs). SNNs are a biologically inspired neural network type with energy-efficient spike-based communication between neurons. Our networks were implemented in so-called neuromorphic hardware, which is hardware that implements essential ingredients of biological neural networks. As synaptic connections between neurons, we considered nano-scale electric circuit elements, so-called memristors. Memristors are can be used to implement large synaptic arrays very efficiently. However, their behaviour is also noisy, which necessitates novel training schemes. "In the field" applications often demand online adaptation of such systems, which typically implies hardware-averse training procedures. To overcome this problem, we investigated the applicability of "learning to learn" (L2L) for the neuromorphic hardware. In an initial optimization, the hardware is trained to become a good learner for the target application. Here, arbitrarily complex learning algorithms can be used on a host system with the hardware "in the loop". In the application itself, simpler algorithms - that can be easily implemented in neuromorphic hardware - provide adaptation of the network in hardware. In this project, we developed novel variants of spiking neural networks that are significantly more powerful and versatile than previous models. These networks are able to learn various demanding tasks such as the evaluation of mathematical expressions. In particular, we considered spiking neural networks with brain-like memory systems which enables them for example to learn from single examples and answer questions about previously presented stories. We also developed novel learning algorithms that enable the training of neuromorphic hardware with low-precision synaptic weights as well as hardware with highly unreliable memristive synapses. Furthermore, we established paradigms to utilize learning-to-learn for neuromorphic systems. The final outcome of the project was an experimental system in a real-world robotics environment. An industrial robot arm was controlled by a spiking neural network with memristive synapses. Using the learning-to-learn approach, the robot was able to learn a novel arm trajectory after a single exposure to a demonstrated arm movement. This provides a proof of concept that learning-to-learn can be applied to neuromrophic systems with memristive synapses.