<ul data-eligibleForWebStory="true"><li>Deep neural networks need reliable uncertainty calibration for safe deployment in critical applications.</li><li>Foundation models like ConvNeXt, EVA, and BEiT have improved predictive performance but their calibration properties are not well understood.</li><li>A study investigated the calibration behavior of foundation models, revealing insights that question existing beliefs.</li><li>Empirical analysis found that foundation models are often underconfident in in-distribution predictions, leading to higher calibration errors.</li><li>However, these models show improved calibration under distribution shifts.</li><li>Foundation models respond well to post-hoc calibration techniques in in-distribution scenarios, helping in mitigating underconfidence bias.</li><li>But the effectiveness of these techniques diminishes under severe distribution shifts and can sometimes yield counterproductive results.</li><li>The study highlights the intricate effects of architectural and training advancements on calibration, challenging the notion of continuous improvement.</li></ul>

Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks

Discover more