CIRO7.2: A Material Network with Circularity of -7.2 and Reinforcement-Learning-Controlled Robotic Disassembler

Abstract

The competition over natural reserves of minerals is expected to increase in part because of the linear-economy paradigm based on take-make-dispose. Simultaneously, the linear economy considers end-of-use products as waste rather than as a resource, which results in large volumes of waste whose management remains an unsolved problem. Since a transition to a circular economy can mitigate these open issues, in this paper we begin by enhancing the notion of circularity based on compartmental dynamical thermodynamics, namely, , by defining a time-window formulation to derive a physics-inspired value from the time-varying definition. Then, we model a thermodynamical material network (TMN) processing a batch of 2 solid materials of criticality coefficients of 0.1 and 0.95, with a robotic disassembler compartment controlled via reinforcement learning (RL), and processing 2-7 kg of materials contained in a PC desktop. The tested RL algorithms are the soft-actor critic (SAC), the truncated quantile critics (TQC), and the twin delayed deep deterministic policy gradient (TD3), and each algorithm is enhanced through hindsight experience replay (HER). Subsequently, we focus on the design of the robotic disassembler compartment using state-of-the-art RL algorithms and assessing the algorithm performance with respect to (Figure 1). The simulations show that the highest circularity is -2.1 achieved in the case of disassembling 2 parts of 1 kg each, whereas it reduces to -7.2 in the case of disassembling 4 parts of 1 kg each contained inside a chassis of 3 kg, which is the most complex task of those considered. Finally, a sensitivity analysis highlights that RL performance has a positive correlation with circularity, with the impact increasing with material criticality and mass. In contrast with the data-intensive material flow analysis (MFA), we approach circularity as a design of material flows to be optimized with respect to and create a compartmental network tunable through 12 parameters depicting its physical and temporal properties. Furthermore, our approach captures the impact that very fast dynamics occurring in less than 1 second has on the whole system circularity, whereas MFA studies usually depict only slow dynamics, e.g., variations from year to year . This work lies in the emerging research topics of circular intelligence and robotics (CIRO). Source code is publicly available.