<ul><li>Post-training is essential for the success of large language models (LLMs), transforming pre-trained base models into more useful and aligned post-trained models.</li><li>This paper compares base and post-trained LLMs from four perspectives to understand how post-training affects LLMs internally.</li><li>Findings reveal that post-training does not change factual knowledge storage locations, adapts knowledge representations from the base model, and develops new knowledge representations.</li><li>Truthfulness can be effectively transferred for interventions, while refusal shows limited forward transferability between the base and post-trained models.</li></ul>

How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence

Discover more