<ul data-eligibleForWebStory="true"><li>Large Language Models (LLMs) are surprisingly robust to structural interventions like deleting and swapping adjacent layers during inference.</li><li>LLMs retain 72-95% of their original top-1 prediction accuracy without any fine-tuning after interventions.</li><li>Performance degradation varies across layers: early and final layers see more degradation, while dropping middle layers has minimal impact.</li><li>Observation of four stages of inference in LLMs: detokenization, feature engineering, prediction ensembling, and residual sharpening.</li><li>These stages show depth-dependent computations in LLMs and are seen across different model families and sizes.</li></ul>

The Remarkable Robustness of LLMs: Stages of Inference?

Discover more