<ul><li>Large Language Models (LLMs) generate content that frequently deviates from factual correctness or lacks logical reasoning.</li><li>A new decoding strategy called DeLTa aims to improve factual accuracy and inferential reasoning in LLMs without changing their architecture or pre-trained parameters.</li><li>DeLTa adjusts next-token probabilities by analyzing the trajectory of logits from lower to higher layers in Transformers and applying linear regression.</li><li>Experiments show that DeLTa achieves improvements of up to 4.9% over the baseline on TruthfulQA, 8.1% on StrategyQA, and 7.3% on GSM8K, which test reasoning abilities.</li></ul>

DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Ability

Discover more