CAMBRIDGE, MASSACHUSETTS — Researchers from MIT and Symbotic have developed a deep reinforcement learning system that automatically prioritizes and reroutes warehouse robots to prevent traffic jams and increase efficiency. The system achieved approximately 25 percent higher throughput than existing methods in simulation tests based on actual e-commerce warehouse layouts.

The hybrid system uses artificial intelligence to determine which robots should be prioritized at any given moment based on developing congestion patterns. It can also reroute robots in advance to avoid bottlenecks and adapts quickly to changing conditions through a planning algorithm that provides real-time instructions to the robots.

The researchers designed a neural network model that observes the warehouse environment and decides how to prioritize robot movements. They trained this model using deep reinforcement learning, a trial-and-error method in which the system learns to control robots in simulations that mimic actual warehouses. The model receives rewards for decisions that increase overall throughput while avoiding conflicts. "By interacting with simulations inspired by real warehouse layouts, our system receives feedback that we use to make its decision-making more intelligent," said Han Zheng, a graduate student in the Laboratory for Information and Decision Systems at MIT.

The system captures long-term constraints and obstacles in each robot's path while considering dynamic interactions between robots as they move through the warehouse. By predicting current and future robot interactions, the model plans to avoid congestion before it occurs. After the neural network determines which robots should receive priority, a planning algorithm tells each robot how to move from one point to another.

"There are a lot of decision-making problems in manufacturing and logistics where companies rely on algorithms designed by human experts," said Zheng. "But we have shown that, with the power of deep reinforcement learning, we can achieve super-human performance." Zheng also noted, "In this setting, we don't have an exact prediction of the future. We only know what the future might hold, in terms of the packages that come in or the distribution of future orders."

"This hybrid approach builds on my group's work on how to achieve the best of both worlds between machine learning and classical optimization methods," said Cathy Wu, Class of 1954 Career Development Associate Professor in Civil and Environmental Engineering and the Institute for Data, Systems, and Society at MIT. "Pure machine-learning methods still struggle to solve complex optimization problems, and yet it is extremely time- and labor-intensive for human experts to design effective methods."

The researchers tested their trained system in simulated warehouses that differed from those used during training. Because industrial simulations were too inefficient for the problem, the researchers designed their own simulated environments to mimic actual warehouses. On average, the hybrid learning-based approach achieved 25 percent greater throughput per robot than traditional algorithms and a random search method in simulations, measured by the number of packages delivered per robot.

The research appears in the Journal of Artificial Intelligence Research. The paper's authors include Han Zheng and Yining Ma from MIT's Laboratory for Information and Decision Systems, Brandon Araki and Jingkai Chen from Symbotic, and Cathy Wu.