Token Factory: Understanding the pipeline

Short summary

Interactive simulation teaches LLM serving optimization by mapping game mechanics to real infrastructure: prefill/decode separation, KV-cache paging (reducing GPU memory 4x), and speculative decoding. Progress through three difficulty levels to optimize throughput from 30 to 120+ tokens per second using concepts from vLLM, TensorRT-LLM, and Hugging Face TGI. A hands-on exploration of how production inference frameworks maximize performance.

• •Interactive game maps LLM serving optimization to real-world infrastructure patterns
• •Teaches prefill/decode phases, KV-cache paging, and speculative decoding through three difficulty levels
• •Progresses from 30 to 120+ tokens/sec throughput, demonstrating concrete optimization impact

Generated with AI, which can make mistakes.