In this paper, we introduce the Tree-of-Thought (ToT) framework, a novel approach aimed at improving the problem-solving capabilities of auto-regressive large language models (LLMs). The ToT technique is inspired by the human mind’s approach for solving complex reasoning tasks through trial and error.
Here is the problem: LLMs do not know whether the answer they are currently generating is accurate or optimal. Once they start down a particular path, they are locked in, unable to reconsider unless they are later prompted to.
Language models do not explicitly perform logical correctness checks as it generates a new token based on the previous tokens. This limits the model’s capacity to rectify its own mistakes. A minor error could be amplified as the model generates more tokens
Tree-of-thought lets the model explore multiple solutions, backtracking when a particular solution is deemed to be suboptimal. Compared to previous “chain-of-thought” prompting techniques, tree-of-thought gives the LLM more computation time before arriving at a final conclusion.
As mentioned above, LLMs typically generate a token based on the preceding sequence of tokens without backward editing. On the contrary, when a human solver attempts to solve a problem, she might backtrack to previous steps if a derivation step is incorrect, or if she becomes stuck and is unable to make further progress towards arriving at the final answer.
[…]
[The tree-of-thought framework] incorporates several components which enhance the problem solving capability of the LLM, including a prompter agent, a checker module, a memory module, and a ToT controller.
It is fascinating to think about what studying language models can teach us about our own cognition.
Related: Loom—A “multiversal tree writing interface for human-AI collaboration”