Neuromorphic Processor Design for Ultra-Low-Power Embedded Vision Applications

Abstract

The exponential growth of intelligent edge devices in use in applications like autonomous navigation, real-time surveillance and intelligent robotics is driving the need to energy efficient embedded vision systems with the capability to process complex cognitive tasks in a power-sensitive and latency-sensitive environment. In this paper, a new architecture of neuromorphic processor specifically tuned towards ultra-low-power embedded vision applications is as presented. Based on how the human brain is efficient in sensory processing, the proposed processor uses the proposed processor uses event-driven spiking neural networks (SNNs) and in-memory computing (IMC) paradigm to reduce data movement and power consumption. Architecturally, this focuses on the assembly of digital leaky integrate and fire (LIF) neurons into a programmable processing element (PE) network, with a PE comprising a local memory and computation to serve asynchronous, spike-based, processing. Also, the processor also includes resistive RAM (ReRAM)-based crossbar arrays, allowing analog in-memory multiply-accumulate (MAC) operations and decreasing the energy expenditure of older von Neumann architectures significantly. The custom compiler framework implements a temporally encoded visual stream as created by event-based sensors, e.g. Dynamic Vision Sensors (DVS), to the Neuromorphic hardware, with latency-based scheduling and dynamic spike routing. Specifically to test the feasibility of the proposed design, large-scale simulations and FPGA-based experiments with real-scale vision benchmarks were performed, such as MNIST-DVS, CIFAR10-DVS and N-MNIST. Since the power of the processor is lowered up to 4 times and the inference latency is 2.5 times smaller than the traditional edge AI platforms (Google Edge TPU and Intel Loihi 2), our experimental results clearly demonstrate that the processor offers up to 2.5X lower energy consumption, in addition to lower inference latency and competitive classification accuracy. Results indicate a promising potential of integrating biologically inspired neural architecture and novel memory technologies to design scalable low-power vision processors to support AI at the edge. This work forms a baseline of future work in adaptive neuromorphic systems and integration with neuromorphic sensors that can facilitate end-to-end real-time perception with ultra-low energy footprints in large-scale distributed embedded applications.