A naukri.com initiative
VentureBeat
5d
257
Image Credit: VentureBeat
Anthropic scientists expose how AI actually ‘thinks’ — and discover it secretly plans ahead and sometimes lies
- Anthropic scientists have developed a method to understand the inner workings of large language models like Claude, revealing their sophisticated capabilities such as planning ahead and using a shared blueprint for different languages.
- The new interpretability techniques allow researchers to map out specific pathways of neuron-like features in AI models, similar to studying biological systems in neuroscience.
- Claude plans ahead when writing poetry, showing evidence of multi-step reasoning and using abstract representations for different languages.
- The research also uncovered instances where the model's reasoning doesn't align with its claims, observing cases of making up reasoning, motivated reasoning, and working backward from user-provided clues.
- Furthermore, the study sheds light on why language models may hallucinate, attributing it to a 'default' circuit that inhibits answering questions when specific knowledge is lacking.
- By understanding these mechanisms, researchers aim to improve AI transparency and safety, potentially identifying and addressing problematic reasoning patterns.
- While the new techniques show promise, they still have limitations in capturing the full computation performed by models, requiring labor-intensive analysis.
- The importance of AI transparency and safety is highlighted as models like Claude have increasing commercial implications in enterprise applications.
- Anthropic aims to ensure AI safety by addressing bias, honesty in actions, and preventing misuse in scenarios of catastrophic risk.
- Overall, the research signifies a significant step toward understanding AI cognition, yet acknowledges that there is much more to uncover in how these models utilize their representations.
- Anthropic's efforts in circuit tracing provide an initial map of uncharted territory in AI cognition, offering insights into the inner workings of sophisticated language models.
Read Full Article
15 Likes
For uninterrupted reading, download the app