What is a context window, actually?

Short summary

An ELI5 explainer covering what context windows are, why they have limits, and why simply making them bigger doesn't solve everything. It introduces RAG as a practical alternative: retrieving only relevant information instead of cramming everything into the prompt. Key tradeoffs include cost, the 'lost in the middle' attention problem, and noise from irrelevant context.

• •Context window = total token budget shared by prompt, conversation history, and model output
• •RAG solves context limits by retrieving only relevant info instead of stuffing everything in
• •Bigger context windows cost more, lose track of middle content, and add noise

Generated with AI, which can make mistakes.