<ul data-eligibleForWebStory="true"><li>Language models are being analyzed using Shapley Taylor interaction indices (STII) to understand how they represent internal structure.</li><li>Shapley interactions help measure how inputs in language and speech models work together to impact outputs beyond their independent influences.</li><li>The study looks into the relationship between models and underlying linguistic structures like syntactic structure, non-compositional semantics, and phonetic coarticulation.</li><li>Results indicate that autoregressive text models show interactions correlating with the syntactic proximity of inputs.</li><li>Both autoregressive and masked models encode nonlinear interactions in idiomatic phrases with non-compositional semantics.</li><li>In terms of speech results, models show the phonetic interaction necessary for extracting discrete phonemic representations.</li></ul>

Using Shapley interactions to understand how models use structure

Discover more