Why Your AI Chatbot Suddenly "Forgets": Context Windows, Explained With Real Numbers
Think of it less like a diary and more like a whiteboard of fixed size. Every new line you write can push an old one off the edge. Start a fresh chat and you get a blank whiteboard, not the old one carried over. From the model's side, it has never spoken to you before.
What a token actually is
A token is the unit AI models actually read and write, not quite a word and not quite a character. For English text, one token works out to roughly four characters, about 75% of a word, per OpenAI's own help center. "Apple" is a single token; a longer or less common word like "hamburger" splits into three ("ham", "bur", "ger"). Every message, every pasted document, every reply gets converted into tokens and added to the running total.
≈4 characters
what one token is worth in English text
roughly 75% of a word, per OpenAI's token guide.
1,000,000
tokens in Claude Sonnet 5's default context window
the default and the maximum, per Anthropic's own docs.
Why a chatbot seems to "forget"
Once a conversation's token count crosses the model's window, something has to give, and it's the oldest content, per Anthropic's own explanation of how context windows work. The model never actually "remembers" anything between messages the way a person does; it re-reads the entire conversation, up to the window's limit, from scratch every single time you hit send. That's also why starting a new chat wipes the slate clean: there's no separate memory being carried over, just whatever text is inside that one window.
| If this happens to you... | Here's why |
|---|---|
| The chatbot ignores something you told it 40 messages ago | That part of the conversation has aged out of the context window. |
| A brand-new chat has no memory of your last one | Each conversation starts with an empty window by design, not a shared memory. |
| Pasting a long document makes replies get noticeably worse | The document itself is eating most of the window, leaving less room for the model to work with. |
| A single huge prompt runs slower or costs more than a normal one | Some providers change how a request is billed or throttled once it crosses a size threshold inside the same model. |
The number on the box isn't the number you'll ever hit
At roughly 0.75 words per token, a 1-million-token window works out to about 750,000 words, close to a 1,500-page book. Almost nobody gets there in a normal conversation, even a long one is a few thousand words. The ceiling mostly matters for specific jobs: pasting in an entire codebase, a lengthy legal contract, or hours of transcript at once, not for everyday back-and-forth.
What actually helps once you're hitting the wall
- ✦Start a fresh chat when you switch topics. Carrying an old, unrelated conversation forward just spends tokens the new topic doesn't need.
- ✦Keep permanent information outside the chat, not inside it. A fact you'll need again shouldn't have to survive being re-read from scratch every message. Starlight's Notes autosaves locally in your browser, so you can park it somewhere that won't age out.
- ✦Transcribe a long recording once, then paste the text, not the audio. Re-uploading or re-describing a recording each time it comes up burns far more of the window than pasting a transcript once. Starlight's Audio Transcriber runs Whisper locally in your browser for exactly that.
- ✦Summarize before you paste a huge document in. A tight summary of the relevant section uses a fraction of the tokens a full document does, and leaves more window for the model to actually reason with.
What is a context window, in plain terms?
It's the maximum amount of text, measured in tokens, that an AI model can hold in one conversation at a time. Everything you've written and everything it's replied with counts toward that total.
Why does my chatbot forget something I said earlier?
Once a conversation grows past the model's context window, the oldest messages fall out of view to make room for new ones. It's a hard technical ceiling, not the AI losing interest.
Does starting a new chat carry over what I discussed before?
No. Each new conversation starts with an empty context window. The model has no memory of a previous chat unless you paste the relevant parts back in.
How big is a token, really?
About four characters of English text, or roughly three-quarters of a word, per OpenAI's own guidance. A 1-million-token window works out to roughly 750,000 words.
Which AI has the biggest context window right now?
As of August 2026, GPT-5.6, Claude Sonnet 5, and Gemini 3 all sit at roughly 1 million tokens per their own documentation, though real-world usable quality can vary well before you hit that ceiling.
Tools in this guide
Try the tools ✦
Free browser tools that never upload your files.