<ul data-eligibleForWebStory="true"><li>Obtaining multiple diverse samples from Large Language Models for a prompt is a challenge.</li><li>Current methods focus on token-level diversity, leading to repetitive responses.</li><li>Intent Factored Generation (IFG) proposed to address diversity and engagement issues.</li><li>IFG involves sampling a semantic intent first and then generating a response based on the intent and prompt.</li><li>Higher temperature used for intent step to promote diversity, lower temperature for final generation for coherence.</li><li>Prompting the model to state its intent before generating enhances reasoning tasks.</li><li>IFG shows effectiveness in improving pass@k and Reinforcement Learning on math and code tasks.</li><li>IFG combined with Direct Preference Optimization enhances conversational diversity without loss in reward.</li><li>IFG maintains diversity and quality in general language modeling using reader comments and news articles dataset.</li><li>IFG is a simple method to boost diversity in Large Language Models while preserving performance.</li><li>The method is easy to integrate into various algorithms for improved performance across applications.</li></ul>

Intent Factored Generation: Unleashing the Diversity in Your Language Model

Discover more