CAMBRIDGE, MASSACHUSETTS — Researchers at MIT's Computer Science and Artificial Intelligence Laboratory have developed a training method that reduces overconfidence in artificial intelligence reasoning models without sacrificing accuracy. The method, called Reinforcement Learning with Calibration Rewards, or RLCR, will be presented at the International Conference on Learning Representations later this month.

The team traced overconfidence in AI reasoning models to a flaw in how they are trained. Recent reinforcement learning methods behind AI reasoning breakthroughs reward models only for correct answers and penalize them only for incorrect answers, without intermediate feedback. Over time, this approach causes models to answer every question with high confidence regardless of the strength of their evidence. When deployed in fields such as medicine, law, and finance, AI systems that express uniformly high confidence become unreliable because users lack a signal to seek second opinions.

"The standard training approach is simple and powerful, but it gives the model no incentive to express uncertainty or say I don't know," said Mehul Damani, an MIT PhD student and co-lead author on the paper.

RLCR adds a single term—a Brier score—to the reward function to penalize the gap between a model's stated confidence and its actual accuracy. During training, models learn to reason about both the problem and their own uncertainty, producing an answer and a confidence estimate together. The research team proved formally that the reward structure guarantees models that are both accurate and well-calibrated.

The approach was tested on a 7-billion-parameter model across a range of question-answering and math benchmarks, including six datasets the model had never been trained on. Across the experiments, RLCR reduced calibration error by up to 90 percent while maintaining or improving accuracy on both trained tasks and entirely new ones. It also outperformed post-hoc approaches in which a separate classifier is trained to assign confidence scores after model training.

Standard reinforcement learning training degraded calibration compared to the base model, while RLCR reversed that degradation with no loss in accuracy.

"What's striking is that ordinary RL training doesn't just fail to help calibration. It actively hurts it," said Isha Puri, an MIT PhD student and co-lead author on the paper.

When models generate multiple candidate answers, selecting the one with the highest self-reported confidence or weighting votes by confidence in a majority-voting scheme improves both accuracy and calibration as compute scales. Training classifiers on model outputs that include the model's explicit uncertainty reasoning improved classifier performance, particularly for smaller models.

The paper, titled "Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty," lists Damani, Puri, Stewart Slocum, Idan Shenfeld, Leshem Choshen, Jacob Andreas, and Yoon Kim as authors.