A new visualization tool called 'Landscape of Thoughts' has been introduced to inspect the reasoning paths of large language models (LLMs) on multi-choice datasets.
The tool represents the states in a reasoning path as feature vectors and visualizes them using t-SNE in two-dimensional plots.
Qualitative and quantitative analysis with Landscape of Thoughts helps distinguish between strong and weak models, correct and incorrect answers, and highlights different reasoning patterns.
The tool can also be adapted to predict properties and has been showcased by adapting it to a lightweight verifier that evaluates the correctness of reasoning paths.