Context Window

A context window is the maximum amount of text, measured in tokens, that a large language model can take into account in a single request. OpenAI defines it as "the maximum number of tokens that can be used in a single request", and that budget covers the input, the output and, for reasoning models, the reasoning tokens 1. Anthropic describes it as the model's "working memory": everything the model can reference while generating a response, including the response itself, as opposed to the much larger body of data it was trained on 2.

What counts toward the context window

Everything sent with a request uses up part of the window, not only the latest user message 2:

• System prompt: standing instructions, style rules, persona
• Conversation history: every earlier user message and model reply that the application sends back each turn
• Tool definitions and tool results: the name, description and parameter schema of every tool the model may call, plus whatever those tools return. OpenAI notes that function definitions "count against the model's context limit and are billed as input tokens" 5. See tool schema.
• Attached content: documents, images and retrieved passages
• The output: the tokens the model generates for this turn, including any reasoning or thinking tokens

Because the whole conversation is resent on every turn, a long chat or a long-running agent fills the window steadily even when each individual message is short.

Tokens, not words

Context windows are measured in tokens, the sub-word units a model's tokenizer splits text into. The ratio of tokens to words depends on the tokenizer and the language, so the same page of text can use a different number of tokens on different models. Providers publish tools for counting tokens before a request is sent: OpenAI points developers to its tokenizer 1, and Anthropic offers a token counting API 2. Window sizes differ widely between models and change with each release, so check the provider's model documentation rather than relying on a fixed number.

What happens when you hit the limit

The behavior depends on the provider and model. OpenAI's documentation warns that tokens generated beyond the context window limit may be truncated in API responses 1. Anthropic's API returns an error when the input alone is too long, and on newer models stops generation with a dedicated stop reason if the output runs into the limit 2. Chat apps often hide this by dropping or summarizing the oldest turns, which is why a long conversation can "forget" instructions given at the start.

Bigger is not automatically better

A larger window lets a model read longer documents or keep more history, but models do not use all of that space equally well. The 2023 study "Lost in the Middle" found that performance was often highest when relevant information sat at the beginning or end of the input, and dropped when the model had to use information placed in the middle of a long context 4. Anthropic calls the general effect context rot: as the number of tokens grows, accuracy and recall degrade, so curating what goes into context matters as much as how much fits 2 3. Anthropic's engineering team treats context as "a critical but finite resource" and recommends finding "the smallest possible set of high-signal tokens" for the task 3.

Managing context in practice

Common techniques for staying inside the window without losing quality 3:

• Retrieval instead of stuffing: fetch only the relevant passages at the moment they are needed, as in retrieval-augmented generation
• Compaction: summarize older parts of a conversation and continue from the summary
• External notes: let an agent write progress and decisions to files or a store it can read back later (see agent memory)
• Sub-agents: give a focused task to a separate agent with its own clean context and keep only its condensed result
• Lean tools: expose fewer, non-overlapping tools and keep tool results short, since both occupy the window

Why it matters for AI-assisted social media work

For marketing teams, the context window sets how much material an assistant can consider at once: a brand guide, a batch of past posts, a long comment thread or an analytics export. When a task goes beyond that, the practical fix is to split it up, summarize, or retrieve only the relevant pieces, rather than paste everything into one prompt. Agents that publish through an MCP server also spend part of the window on the server's tool definitions, so connecting only the servers a task needs leaves more room for the actual content. Placing the most important instructions near the start or end of a prompt, and repeating key constraints in long sessions, also helps given the position effects described above 4.