⭐ If you find this repository helpful, please consider giving it a ⭐ here on GitHub (click the star button in the top right corner) It's a quick way to show support for this openly available code. ⭐

Transformers - The definitive Guide

This is the corresponding code for the book Transformers - The definitive Guide The book can be found here

TOC

• Chapter 1 From First Principles to State-of-the-Art Transformers
• Chapter 2 Transformers for Time Series
• Chapter 3 Transformers for Vision Tasks
• Chapter 4 Transformers for Image Generation
• Chapter 5 Transformers for Video Generation
• Chapter 6 Transformers for Audio Tasks
• Chapter 7 Reinforcement Learning Transformers
• Chapter 8 Transformers for Planing, Reasoning and Coding
• Chapter 9 AI Agents for Complex Tasks
• Chapter 10 Optimizing Transformer for Problem Solving
• Chapter 11 Deploying transformer models
• Chapter 12 Where to go next

Instructions and Navigation

All of the code is organized into folders. Each folder starts with CH followed by the chapter number. For example, CH01. The notebooks are then organized as follows: ch01_attention_mechanism_variations.ipynb, where ch01 indicates the chapter and attention_mechanism_variations what is done in the notebook.

Repo structure

├── LICENSE
├── README.md             <- The top-level README for developers using this project.
├── CH01                  <- Per chapter folder with Jupyter notebooks.
    ├── [name].ipynb      <- Jupyter notebooks with naming as mentioned above.
├── CH02                  <- Per chapter folder with Jupyter notebooks.
...                       <- Same structure for all chapters.
├── utils                 <- Custom classes and functions and utility functions.
├── resources             <- Some miscellaneous resources.

Virtual Envrionment

The provided bash script create_env.sh automates the process of creating a Python virtual environment using either conda or pipenv, installing the required packages from a requirements.txt file. To use the script run bash create_env.sh in your terminal on Microsoft Windows (with WSL), Apple macOS, or Linux operating systems.

NOTE: A virtual environment is not necessary for the notebooks in this repository, as they are designed to be run on a cloud service with GPU support. Therefore, the provided instructions for creating a virtual environment are more for reference and general guidance than a strict requirement.

Running the Notebooks

Every notebook can be opened and run on Google Colab directly from the links below. Just click the Open In Colab badge next to the notebook you want to run.

Chapter 1 — From First Principles to State-of-the-Art Transformers

Notebook	Colab
Attention Mechanism Variations
Embeddings
Perplexity

Chapter 2 — Transformers for Time Series

Notebook	Colab
Chronos
PatchTST Hyperparameters (IBM, 10 Days Ahead, 32 Context Window)
Time Series Fine-Tuning (PyTorch)

Chapter 3 — Transformers for Vision Tasks

Notebook	Colab
Fine-tune SAM with W&B + Optuna
ViT Embeddings & Tokens
Image Classification
Segment Videos with SAM 2

Chapter 4 — Transformers for Image Generation

Notebook	Colab
DiT
KV Compression
PixArt-Σ XL Inference
Quantize Text-to-Image Models

Chapter 5 — Transformers for Video Generation

Notebook	Colab
LTX
Latte
Tora

Chapter 6 — Transformers for Audio Tasks

Notebook	Colab
Kimi-Audio Meeting Transcription
Qwen2-Audio Audio Tasks
SAM Audio
Music Generation
Waveforms & Spectrogram Plots

Chapter 7 — Reinforcement Learning Transformers

Notebook	Colab
STORM
Decision Transformer
Replay Buffer (Decision Transformer)

Chapter 8 — Transformers for Planning, Reasoning and Coding

Notebook	Colab
Qwen3
Rethink MCTS
TreeQuest
Kimi K2 Instruct

Chapter 9 — AI Agents for Complex Tasks

Notebook	Colab
LangGraph Multi-turn Conversation
Market Research Team (LangGraph Multi-Agent)

Chapter 10 — Optimizing Transformers for Problem Solving

Notebook	Colab
AdaptThink
ART·E with LangGraph
rLLM

Chapter 11 — Deploying Transformer Models

Notebook	Colab
LLM Performance Evaluation
Advanced LoRA Fine-Tuning
LangGraph Code Interpreter

Chapter 12 — Where to Go Next

Notebook	Colab
SAM 3 Agent

Each notebook is connected with this Github repo, meaning by running a notebook, it will automatically clone the repo, so you can easily access all resources outside the notebook. Like customs functions and classes as well as utility functions to automatically install the requirements per chapter:

!git clone https://github.com/Nicolepcx/transformers-the-definitive-guide

current_path = %pwd
if '/transformers-the-definitive-guide' in current_path:
    new_path = current_path + '/utils'
else:
    new_path = current_path + '/transformers-the-definitive-guide/utils'
%cd $new_path

NOTE: You need to run the notebooks with a GPU.