<ul data-eligibleForWebStory="true"><li>Uncertainty quantification in long-form text generation is explored in a new study.</li><li>The study focuses on uncertainty within sub-groupings of data for large language model outputs.</li><li>Different methods are used to measure uncertainty at the level of individual claims and across the entire output.</li><li>Biography generation is used as a test case in this study.</li><li>Demographic attributes are considered to create subgroups of data.</li><li>Canonical methods for uncertainty quantification perform well for the entire dataset but struggle with subgroup analysis.</li><li>Group-conditional methods like multicalibration and multivalid conformal prediction are introduced to address subgroup uncertainties.</li><li>Additional subgroup information consistently enhances calibration and conformal prediction.</li><li>The study establishes benchmarks for calibration and conformal prediction in the context of long-form text generation.</li></ul>

Multi-group Uncertainty Quantification for Long-form Text Generation

Discover more