Anthropic has developed an AI 'brain scanner' to understand how LLMs work and the reasons behind chatbots' shortcomings in math and hallucinations.
Anthropic's new technique, called circuit tracing, tracks decision-making processes in LLMs step-by-step, similar to brain scanning techniques used in neuroscience.
The research revealed that LLMs, such as Anthropic's Claude 3.5 Haiku, use odd methods for basic mathematics but provide explanations based on common answers in training data, not on actual processes.
The study also discovered that LLMs, contrary to popular belief, do not merely predict the next word but choose rhyming words first and think in a shared conceptual space between languages.