CAMBRIDGE, MASSACHUSETTS — Researchers from MIT and the MIT-IBM Watson AI Lab developed a rapid prediction tool that estimates the power consumption of AI workloads on specific processors or accelerator chips in seconds. The tool, called EnergAIzer, was presented at the IEEE International Symposium on Performance Analysis of Systems and Software.

Traditional modeling techniques for predicting energy consumption can take hours or days to yield results, while EnergAIzer produces reliable power estimates in a few seconds. Conventional methods involve breaking a workload into individual steps and emulating how each GPU module is utilized at each step, and AI workloads such as model training and data preprocessing are large enough that detailed simulations can take hours or days.

To speed up power prediction, the researchers used less-detailed information by leveraging repeatable patterns in AI workloads. AI algorithm developers often optimize programs to distribute work across parallel processing cores and manage data movement efficiently on GPUs, and EnergAIzer captures GPU power usage patterns based on those software optimizations.

The initial estimation model did not account for fixed energy costs for program setup, per-operation energy costs, and variances caused by hardware fluctuations or data access conflicts. The researchers collected real measurements from GPUs to generate correction terms for the model. EnergAIzer outputs energy consumption estimates within seconds based on workload information such as AI model details, user input count and length, GPU configuration, and operating speed.

When tested on real AI workloads from actual GPUs, EnergAIzer estimated power consumption with about 8 percent error, comparable to traditional methods that take hours to produce results. The tool can be applied to a wide range of hardware configurations, including emerging designs that have not been deployed, provided the hardware does not change substantially in a short period.

Data centers are estimated to consume up to 12 percent of total U.S. electricity by 2028, according to Lawrence Berkeley National Laboratory. Data center operators could use the estimates to allocate resources across multiple AI models and processors to improve energy efficiency, and algorithm developers could assess potential energy consumption of a new model before deploying it.

"The AI sustainability challenge is a pressing question we have to answer. Because our estimation method is fast, convenient, and provides direct feedback, we hope it makes algorithm developers and data center operators more likely to think about reducing energy consumption," said Kyungmi Lee, an MIT postdoc and lead author of a paper on the technique.

The researchers plan to test EnergAIzer on the newest GPU configurations and scale the model to apply to multiple GPUs running collaboratively. "To really make an impact on sustainability, we need a tool that can provide a fast energy estimation solution across the stack, for hardware designers, data center operators, and algorithm developers, so they can all be more aware of power consumption. With this tool, we've taken one step toward that goal," Lee said. The research was funded in part by the MIT-IBM Watson AI Lab.