AI concepts
Context window (a model's context window)
The maximum number of tokens (fragments of vocabulary) that an LLM can take in and process at once in a single query. The context window covers both the user's prompt and the model's response. As of 2026: GPT-4o 128,000 tokens, Claude 3.7 Sonnet 200,000, Gemini 2.5 Pro 2,000,000.
Primary source: Anthropic Claude 3.7 Technical Report 2025, Google Gemini 2.5 Documentation 2026, OpenAI GPT-4o Reference 2024
The context window is the most visible technical LLM metric for boards, because it translates directly into "how much of a document you can show the model in a single query". In 2022, GPT-3 had 4,000 tokens. In 2024, GPT-4 Turbo had 128,000. In 2025, Gemini 1.5 Pro had 1 million. In 2026, Gemini 2.5 Pro has 2 million. The growth rate is roughly 5x per year.
What it means in plain terms
Anthropic Claude 3.7 Technical Report 2025 gives a conversion: 200,000 tokens is around 150,000 words of English text or 500 A4 pages of a book. Gemini's 2 million tokens are the equivalent of 5,000 pages, that is, an entire corporate library.
Practical consequences
Google Gemini 2.5 Documentation from 2026 lists three use cases that became possible with large context windows. First, analysing full annual reports without prior summarisation. Second, the source code of large applications can be analysed as a whole rather than in pieces. Third, 200-page legal contracts can be reviewed in a single pass instead of being chunked.
Where the context window does not help
Three myths. First, a larger window means better answers. OpenAI GPT-4o Reference from 2024 confirms the "lost in the middle" effect: information in the middle of a long context is remembered worse than at the beginning or end. A large window does not replace retrieval (RAG).
Second, costs rise linearly with context length. A query of 200,000 tokens to GPT-4o costs around 0.5 dollars per response. A hundred such queries a day is 50 dollars a day, 1,500 dollars a month, 18,000 dollars a year. For a single board-level use case.
Third, latency rises significantly. A response to a query of 200,000 tokens takes 30 to 90 seconds. For a real-time application that is a disqualifying limitation.
RAG vs a large context window
A practical rule from 2026: if the knowledge base is larger than 100 thousand tokens or updates more often than once a week, it is better to build RAG than to dump everything into the context. If the resource is one-off (a single large report, a single contract), a large context window is simpler.
An architecture that uses both the context window and RAG together is part of AI implementation into processes.