In short
The context window is the maximum amount of text, measured in tokens, that a language model can process simultaneously in one request: system instruction, conversation history, supplied documents and the generated answer together. It is the model's working memory and determines how much information fits into a single task.
How the context window works
A language model does not read words but tokens: text units of around three to four characters in English, and somewhat fewer characters per token in German because of its long compound words. An A4 page of running text corresponds roughly to 500 to 700 tokens, a 300-page novel to around 150,000 tokens. The context window indicates how many of these tokens the model can "see" at once. With a window of 200,000 tokens, several hundred pages of text fit into a single request; with one million tokens, a medium-sized code base or a year of support tickets.
Technically, the limit results from the transformer architecture. Its core mechanism, self-attention, relates every token to every other token in the window. The computing and memory effort grows quadratically with the length: twice as much context means four times as much attention computation. That is why the windows of the first generations were small. In 2020, GPT-3 processed around 2,000 tokens; GPT-4 started in 2023 with 8,000 and 32,000 tokens. Only advances in positional encoding (such as RoPE with scaling), more efficient attention variants (FlashAttention, sliding window, grouped query attention) and targeted training on long sequences made windows of 128,000, 200,000 and finally one million tokens possible. With Gemini 1.5 in 2024, Google was the first provider to demonstrate a window of one million tokens in production use.
What counts towards the window
- System prompt: role description, rules, format requirements. In agent set-ups, often several thousand tokens.
- Tool definitions: for agents, the description of all available tools; with many integrations, this can quickly take up 10,000 tokens and more.
- Conversation history: all previous user and model messages, including tool calls and tool results.
- Attached content: documents, code files, search results, database extracts.
- The answer itself: the generated tokens also take up space. Many models also have their own, smaller upper limit for output length.
- Thinking tokens: in models with visible or hidden reasoning, this intermediate text also takes up context.
If the limit is exceeded, the API rejects the request, or the application has to cut off or summarise older parts of the history. Chat interfaces do this automatically in the background, which explains why an assistant "forgets" details from the beginning in long conversations.
Typical window sizes at a glance
Providers' figures have shifted by several orders of magnitude in just a few years. A rough classification, which moves further upwards with each model generation:
- Up to 8,000 tokens: the first generation of modern chat models (2020 to early 2023). Enough for a conversation or a short document, not for entire contracts.
- 32,000 to 128,000 tokens: standard from 2023 and 2024. A comprehensive contract, an annual report or a longer support history fits in completely.
- 200,000 tokens: for a long time the upper limit of the Claude models and many open models. Several hundred pages or a medium-sized software module in one request.
- 1 million tokens and more: standard since 2024 with Gemini and now with the top models from Anthropic and OpenAI as well as several open models from China. Entire code bases, book series or video analyses with a transcript become possible.
The figures apply to the API in each case. Chat interfaces and cheaper subscription tiers often offer a smaller window than the model's technical maximum, and some providers charge higher token prices for windows above 200,000 tokens.
Size is not the same as quality
A large context window does not mean that the model uses all the content in it equally well. Since 2023, research has shown an effect that became known as "lost in the middle": models find information at the beginning and end of the context more reliably than in the middle. Current models usually pass so-called needle-in-a-haystack tests, in which a single fact is hidden in a long text; however, more demanding tasks in which many facts spread across the context have to be combined still show falling hit rates as length increases.
Then there is the price. Input tokens are billed per million, and a fully used window of one million tokens costs several US dollars per individual request with a top model. In agentic workflows, where the same context is sent again at every step, this multiplies. Providers counter this with prompt caching: context beginnings that have already been processed are cached and reloaded at a fraction of the price on the next call. Latency also increases with length, because the model has to process the entire context before it generates the first answer token.
Relevance for companies and SMEs
In practice, the context window determines the architecture of an AI application. Three typical cases:
| Use case | Context requirement | Typical solution |
|---|---|---|
| Contract review, comparing quotes | One or a few long documents in full view | Load the entire document into the window; no problem with windows from 100,000 tokens |
| Knowledge assistant across thousands of documents | Far more content than fits into any window | Retrieval augmented generation: search for suitable extracts and put only these into the context |
| Coding agent in a large code base | Many files, long history, many tool results | Large window plus active context management: summarising, hiding old tool output, prompt caching |
The principle is: not everything that fits into the window belongs there. Every token costs money and the model's attention. A precisely assembled context of 20,000 tokens often delivers better results than an indiscriminately filled one of 500,000. Good context management is therefore a core competence in building AI systems and has acquired its own name in the professional discussion: context engineering.
For data protection and compliance, it is also relevant that everything in the context window is transmitted to the provider. Anyone who packs an entire customer folder into a request transmits it in full, including the parts that are irrelevant to the task. Data minimisation applies to the context just as it does to any other processing.
A real-world example: reading entire code bases
Coding tools provide a tangible example of the benefit of large windows. Before windows of 100,000 tokens and more, assistants such as the first versions of GitHub Copilot had to make do with a few lines of context around the cursor; they knew neither the architecture of the project nor the conventions of other files. With windows of 200,000 tokens and more, tools such as Claude Code, Cursor or Codex CLI can load entire modules, tests and documentation at the same time, implement a ticket across several files and keep track of the history of all previous changes.
The same example shows the limits. A medium-sized application with 500,000 lines of code comprises several million tokens and does not fit into any window. The tools solve this with targeted reloading: they search the repository, read only relevant files and summarise older work steps as soon as the context fills up. Providers such as Anthropic document in their developer documentation on the context window how input and output tokens, thinking tokens and caching interact. The large window is therefore a prerequisite, but context management makes the difference.
Distinction and misconceptions
The context window is not training knowledge. What the model learned during training is in its weights and is always available. The context window only contains what is sent with this request. When the request ends, the context is gone; a model does not "remember" anything between sessions unless the application itself builds in a memory (such as a database of earlier findings).
The context window is not the output limit. Most models can generate considerably fewer tokens than they can read. A window of one million with an output limit of 64,000 tokens is common.
A large window does not replace search. RAG and long contexts are not competitors but complements. Search filters out the relevant documents from millions, and the window processes this selection in context. Only when the entire knowledge base is small enough can the window replace search.
"1M context" is a technical upper limit, not a promise of quality. Providers state the maximum window accepted; the effectively usable length beyond which quality measurably declines is often smaller and varies depending on the task. Independent long-context benchmarks provide better guidance on this than the figure in the data sheet.
Outlook
Windows continue to grow, but the focus is shifting. Instead of sheer size, effective use and costs are coming to the fore: caching prices are falling, models are being trained specifically to combine many widely distributed facts, and agent frameworks are building in context management as a standard function, such as automatic compaction of the history or offloading intermediate results into files. At the same time, architectures beyond classic quadratic attention are emerging that handle very long sequences more efficiently. For users, this means: the context window will less often be the limiting factor, and the art of filling it correctly will matter all the more often.
Frequently asked questions
How many words fit into a context window of 200,000 tokens?
Roughly 130,000 to 150,000 German words, i.e. around 250 to 300 book pages. The exact value depends on language, technical vocabulary and formatting; code and tables use more tokens per word than running text.
What happens when the context window is full?
The API rejects the request with an error. Chat applications and agents catch this beforehand by shortening, summarising or removing older messages. As a result, details from the beginning of a long conversation are lost.
Is a larger context window always better?
No. It allows more content but costs more money and time per request, and quality can decline with very long inputs. A specifically assembled short context is often better than a full one.
When do I need RAG instead of a large context window?
As soon as the knowledge base is larger than the window or changes frequently, or when costs per request play a role. For a few stable documents, the window is enough; for a corporate wiki with thousands of pages, you need a search in front of it.
Do the model's answers count towards the context window?
Yes. Input and output share the window. Anyone who sends 900,000 tokens of input with a window of one million has only 100,000 tokens left for the answer, and that is additionally limited by the model's respective output limit.