Thinking in LLM¶

A first-principles guide to how LLMs "think," starting from next-token prediction and ending with practical system design.

Languages: 中文 | English Read online: yingwang.github.io/thinking-in-llm

Glossary: English terminology

For software engineers who can program and are building, or plan to build, products powered by LLMs. No prior background in machine learning is required.

The Architecture of This Book¶

flowchart LR
    subgraph P1["<b>Part I: What It Is</b>"]
        direction TB
        C1["1. Everything Is Continuation"]
        C2["2. Attention Is Information Routing"]
        C3["3. Scale Emerges"]
        C4["4. From Pretraining to Alignment"]
        C1 --> C2 --> C3 --> C4
    end

    subgraph P2["<b>Part II: Where the Boundaries Are</b>"]
        direction TB
        C5["5. What LLMs Are Truly Good At"]
        C6["6. Hard Limitations"]
        C7["7. The Nature of Hallucination"]
        C8["8. Reasoning or Imitation"]
        C5 --> C6 --> C7 --> C8
    end

    subgraph P3["<b>Part III: How to Build</b>"]
        direction TB
        C9["9. Prompt Is Programming"]
        C10["10. Knowledge Injection"]
        C11["11. Agents"]
        C12["12. Evaluation"]
        C9 --> C10 --> C11 --> C12
    end

    subgraph P4["<b>Part IV: The Frontier</b>"]
        direction TB
        C13["13. Interpretability"]
        C14["14. Multimodal"]
        C15["15. The Future"]
        C13 --> C14 --> C15
    end

    P1 --> P2 --> P3 --> P4

    classDef chapter fill:#ffffff,stroke:#555,color:#222
    class C1,C2,C3,C4,C5,C6,C7,C8,C9,C10,C11,C12,C13,C14,C15 chapter

    style P1 fill:#e8eaf6,stroke:#3949ab,color:#1a237e
    style P2 fill:#fff3e0,stroke:#e65100,color:#bf360c
    style P3 fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
    style P4 fill:#f3e5f5,stroke:#6a1b9a,color:#4a148c

Core trajectory: Understand how LLMs compute -> Map their fundamental boundaries -> Architect systems with the right mental model -> Explore the frontier.

Most available literature leans toward extremes: purely theoretical papers on one side, empirical prompt cookbooks on the other. This book bridges that divide by deriving how to build directly from how LLMs work. When you grasp the mechanics of attention, designing effective prompts becomes second nature. When you understand scaling laws, you can readily discern whether to refine a prompt or migrate to a larger model class.

Table of Contents¶

Part I: What an LLM Is (The Machine)¶

#	Chapter	Core Question
1	Everything Is Continuation	An LLM does only one thing: predict the next token
2	Attention Is Information Routing	Every token asks, "Where should I look?"
3	Emergence From Scale	Why do large models suddenly "get it"?
4	From Pretraining to Alignment	Alignment does not create capability; it shapes expression

Part II: The Capability Boundaries of LLMs (The Boundaries)¶

#	Chapter	Core Question
5	What LLMs Are Truly Good At	Pattern recognition, transformation, compression
6	The Hard Limitations of LLMs	Why counting, arithmetic, and long-range reasoning fail
7	The Nature of Hallucination	Hallucination is not a bug; it is an inherent feature of continuation
8	Reasoning or Imitation?	CoT, reasoning models, System 1 vs System 2

Part III: Building With LLMs (The Practice)¶

#	Chapter	Core Question
9	Prompt Is Programming	System prompt = class definition, few-shot = test cases
10	Three Paths for Knowledge Injection	RAG vs Fine-tuning vs Long Context
11	First Principles of Agents	Tool use extends token space into the external environment
12	Evaluation: The Most Underestimated Step	Write evals first, then tune the system

Part IV: Frontier and Future (The Frontier)¶

#	Chapter	Core Question
13	Interpretability: Opening the Black Box	What is happening inside the network?
14	Multimodal: Beyond Text	Images, audio, and video all become tokens
15	The Future of LLMs	Will scaling hit a wall?

Relationship to "The Complete Guide for LLM Training Engineers"¶

	Training Guide	Thinking in LLM
Perspective	How to train LLMs	How to understand and apply LLMs
Audience	Training engineers	Software engineers and system designers
Prerequisite	Machine learning background	Core programming proficiency
Objective	Train and optimize models	Architect robust LLM systems

The two books complement each other: the training guide demonstrates how to build the engine, whereas this book teaches you how to drive it; not through superficial recipes, but through a first-principles understanding of the mechanics under the hood.

Author¶

Ying Wang

License¶

CC BY-NC-SA 4.0

Last updated: 2026-04-26 (all 15 chapters completed)