How torch.compile Works Internally: Dynamo, AOT Autograd, and Inductor

Adding model = torch.compile may make a PyTorch model faster, but it does not guarantee a speedup. This explanation covers what actually happens behind that one line of code, why compilation helps some models more than others, and what to check when the results are disappointing.

■ The Three Stages of Compilation
- Dynamo: tracing Python execution, capturing tensor operations into an FX graph, and recording guards that decide when compiled code can be reused
- AOT Autograd: preparing the graph by decomposing higher-level operations, and capturing forward and backward graphs for training
- Inductor: generating code for the hardware target, including op fusion, scheduling, and reduced intermediate memory traffic (Triton kernels on GPU, C++ on CPU)

■ Graph Breaks and Recompilation Costs
- What triggers a graph break and how it reduces fusion opportunities and adds boundary overhead
- Using fullgraph=true to require a single capture, and how compilation time differs from steady-state execution time
- Recompilation when input shapes or assumptions change, plus measurement considerations for GPU benchmarks

■ Diagnosing When Compilation Does Not Help
- Four practical questions to ask about capture coverage, graph breaks, recompilation, and measurement method

This is aimed at PyTorch users who want to understand model optimization beyond simply calling compile. It gives a mental model for why speedups vary, and a concrete checklist for diagnosing and improving performance in practice.