<ul><li>This paper explores the theoretical questions that arise when applying active learning of probabilistic deterministic finite automata (PDFA) to neural language models.</li><li>The paper defines a congruence that deals with null next-symbol probabilities in language models that arise when constraining the output of a language model by composing it with an automaton and/or a sampling strategy.</li><li>An algorithm is developed to efficiently learn the quotient PDFA created by the congruence, and case studies are conducted to analyze the statistical properties of large language models.</li><li>The experimental results demonstrate the relevance and effectiveness of the approach.</li></ul>

What Happens When Language Models Say 'Nothing'?

Discover more