Beyond Chain-of-Thought: Graph-of-Thought Reasoning in LLMs

Created in September 03, 2023

2023   ·   llm   reasoning   ·   paper-review

This paper argues that Chain-of-Thought prompting, while effective, is fundamentally limited by its linear structure. Real problem-solving often requires non-linear reasoning — revisiting earlier steps, exploring parallel branches, and merging independent inferences.

Graph-of-Thought represents reasoning as a directed acyclic graph where:

  • Nodes are intermediate thoughts or conclusions
  • Edges represent inference or dependency
  • Multiple paths can be evaluated simultaneously

Compared to CoT and Tree-of-Thought, GoT achieves better performance on tasks requiring complex compositional reasoning while using fewer tokens in some settings, since graph structure prevents redundant re-derivation of shared sub-conclusions.



Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • The Expressive Power of Transformers with Chain of Thought
  • Landscape of Thoughts — Visualizing Where LLM Reasoning Actually Goes
  • Magellan — Guided MCTS for Escaping the Gravity Wells of LLM Creativity
  • PriorZero — Injecting LLM Priors into MuZero-Style World Models at the MCTS Root
  • SuperThoughts — Reasoning Tokens in Superposition