A free and open book CC BY 4.0
From nothing
to the frontier
of language models.
Bits, vectors, probability, and gradients first. Then the transformer, piece by piece, and everything built on it: training at scale, reinforcement learning for reasoning, retrieval, agents, serving, evaluation, safety, and the field as it stood on 2 November 2026.
- Pages
- 421
- Words
- 98k
- Cited sources
- 422
- Runnable labs
- 12
01 / Contents
Eight parts, one connected map.
- IFoundationsComputation and number formats, linear algebra, probability, information theory, calculus, optimization.
- IINeural networks learn languageNeurons and training stability, embeddings, tokenization, logits and the next-token objective.
- IIIThe transformer, piece by pieceAttention, latent and sparse attention, long context, mixture of experts, hybrids, diffusion models.
- IVCreating and shaping a modelData, scaling, distributed and low-precision training, preference learning, RL with verifiable rewards, reasoning.
- VBuilding systems around modelsContext engineering, retrieval, tools and MCP, coding and computer-use agents, memory, multimodality.
- VIServing, measuring, governingInference systems, evaluation, security, safety and alignment, interpretability, law.
- VIIBuild it to understand itTwelve labs and a capstone, from a hand-made sampler to an agent under attack.
- VIIIThe frontier and beyondA dated landscape of hosted and open models, research directions, forecasts.
- A–EAppendicesFormula sheet, glossary, myths, checklists, a one-year curriculum, and the full bibliography.
02 / Labs
Build it to understand it.
Small programs you can run and break. The later labs use only the Python standard library and print the exact output shown in the book.
- 00Establish an experimental habit
- 01Train a tiny tokenizer
- 02Build scalar autodiff
- 03Implement one attention head
- 04Train a tiny transformer
- 05Fine-tune responsibly
- 06Build and evaluate RAG
- 07Build a safe tool loop
- 08Create an evaluation harness
- 09Budget the key–value cache
- 10Reinforce a verifiable answer
- 11Run an agent loop under attack
03 / How it was made
Written by AI.
Checked against 422 sources.
AI language models wrote the text under the direction of the Open LLM Project. Separate AI fact-checking passes then reopened the cited primary source behind each claim, recomputed every number, and ran every code example.
Automated checking reduces errors; it does not prove their absence. No human expert has reviewed the full text. Every claim carries a citation you can follow, and corrections are welcome.
04 / Formats
Read it anywhere.
v2.1.0 · Files come from the latest GitHub release · Verify with SHA256SUMS
05 / Questions
Before you start.
Who is it for?
Beginners who want the whole picture, engineers moving into AI, students, and practitioners who want their foundations and the 2024–2026 developments in one place.
Do I need mathematics or AI experience?
No. Every equation is explained in words before symbols, and each part builds on the one before it.
How current is it?
The technology snapshot is 2 November 2026. Concepts are meant to last; product names, laws, and capabilities are dated and will change. New releases appear on the releases page.
What may I do with it?
The text uses CC BY 4.0: share and adapt it for any purpose, including commercially, with attribution. Code uses the MIT License. Full terms are in the license file.
How do I cite it?
Open LLM Project. Open LLM — From Nothing to the Frontier of Language Models. Second edition, release 2.1, 2026. Machine-readable metadata: CITATION.cff.
How do I verify a download?
Download SHA256SUMS next to the file and run sha256sum -c SHA256SUMS --ignore-missing.
Something is inaccessible or wrong.
Open an issue on the tracker with the section and what you expected. The accessibility notes describe what each format provides.