Deep binarized photonic reservoir computing for ultrafast multimedia signal processing

Abstract

Abstract We present a deep photonic neural network architecture based on ultrafast binary optical modulation using a digital micromirror device (DMD), optical scattering in a random medium, high-speed photodetection with a CMOS sensor, and a time-multiplexed multilayer structure. Operating at Gigabit-per-second (Gb/s) optical information throughput, our system based on the reservoir computing (RC) framework achieves state-of-the-art performance across various multimedia tasks, including video, image and speech recognition. We show that the careful optimization of key physical intra- and inter-layer hyperparameters can significantly enhance the deep photonic RC system’s ability to extract relevant temporal and spatial features by balancing memory retention and dynamical response of individual layers. This approach paves the way for highly scalable hierarchical photonic reservoir computing systems for high-throughput real-time multimedia signal processing.