The Conversation That Was Never Paused
You close the laptop mid-project, grab dinner, come back the next morning, and type a quick follow-up. The assistant picks up mid-thread like it never stopped thinking. Uncanny, almost.
It wasn't waiting for you. The moment you closed that tab, it ceased to exist in any meaningful sense. What looks like continuity is reconstruction, done in milliseconds, from text alone.
That distinction matters more than it sounds.
What Actually Gets Fed Back In
Most AI assistants, including the big consumer ones you're probably using right now, are stateless. Every time you send a message, the system bundles the entire prior conversation, appends your new message at the end, and fires the whole thing to the model as one unbroken block of text. The model reads it top to bottom, as if for the first time, and replies.
This block is called the context window. Think of it less like memory and more like a stack of sticky notes handed to a very fast reader who has never met you before.
The context window has a hard size limit, measured in tokens (roughly three-quarters of a word each, on average). GPT-4 Turbo supports up to 128,000 tokens; Claude 3 Opus handles up to 200,000. Those numbers sound enormous until you paste in a few long documents and watch them evaporate. A sprawling three-hour work session with error logs and revised drafts can burn through 20,000 tokens without really trying.
The reconstruction isn't magical. It's mechanical: the app stores your prior messages, retrieves them, and stuffs them back into the model's view on every single turn.
The Crust That Builds Up Inside
Here's where most people's mental model quietly falls apart.
Short conversation? Full reconstruction is trivial. Every word goes back in the window, the model has complete information, no problem. But say you've been grinding through a complex coding project across multiple sessions, pasting error logs and revised functions, and the conversation is now 80,000 tokens deep.
Two things can happen, depending on the product.
The blunt solution is truncation: older messages get dropped from the front. The model simply loses access to early instructions, decisions, constraints you set up in hour one. You might not notice immediately, and then you ask something that depends on an early agreement and the assistant contradicts itself with complete confidence. Confidently wrong is its own special problem.
The more sophisticated solution is summarization. A background process compresses older sections into a shorter digest, which gets prepended to the active window. You keep the gist, lose the detail. Reasonable trade-off. Also lossy in ways you can't fully audit, which is the part nobody mentions in the product marketing.
Here's a worked example that makes this concrete. Two people use an AI assistant to draft a legal memo. One works in a single two-hour session. The other spreads the same task across four days, returning in short bursts. The first person's final draft was generated with every prior instruction visible. The second person's was generated with several early constraints quietly summarized away. Same assistant, same task, meaningfully different information available at generation time. Neither person knew.
What "Returning" Looks Like Under the Hood
Say you open a conversation from yesterday. You'd typed out a detailed brief for a product launch campaign: target audience, tone restrictions, three competitors to avoid mentioning by name. You got a solid outline back, made some notes, closed the laptop.
Today you type: "Can you tighten up section two?"
The app retrieves yesterday's full exchange from its database. It reconstructs the context window: your original brief, the assistant's outline, everything. It sends that entire block, plus your new message, to the model. The model reads the brief again, reads the outline again, then reads your follow-up and responds accordingly.
It didn't remember your brief. It re-read your brief. Subtle difference in outcome, significant difference in implication: if that database entry gets corrupted, the conversation gets deleted, or your brief falls outside the context window and gets truncated, the assistant has no fallback. No underlying memory to recover from. The information is simply gone.
What People Consistently Misread About This
The biggest misconception is that AI assistants are learning from your conversations over time, slowly building a model of you.
For the vast majority of consumer products, this is just not how it works. The model weights, the actual neural network parameters, don't change between your conversations. What changes is only the text in the context window. The assistant knows what you told it in this conversation. It does not know you, and I'd argue that conflating the two is how people end up over-trusting outputs they should be checking.
Some products have started adding explicit memory features, where the system extracts key facts and stores them in a separate database to inject into future sessions. That's a genuinely different mechanism. But it's curated, selective, often editable by the user. It is not the model developing an understanding of you. The assistant that seems to "remember" you actually retrieved a bullet point that some automated process wrote down on your behalf.
Not quite the same thing.
There's also a subtler error worth catching. Because reconstruction usually works smoothly, people assume the assistant is weighting the context the way they are. It isn't. It processes the full transcript fresh, with no accumulated sense of what mattered most. If you spent forty minutes on a nuance buried in message three, the model gives that message no more gravity than any other. You carry the significance. The model just reads the words, the way a substitute teacher reads a lesson plan, competent and completely without history.
The Practical Upside
None of this is a complaint, by the way.
The stateless-plus-context-window design is actually what makes these systems reliable and auditable. Because context is explicit text, you can see exactly what the model sees. Paste in a summary, correct a misconception, reset a constraint with a single message. That's more control than you'd have over a system with opaque, persistent memory you couldn't inspect or edit.
So: treat the context window like working memory you're actively managing, not a relationship that accrues over time. For long projects, open new sessions with a crisp recap of your constraints. Don't assume the assistant knows what you meant last Tuesday.
You're not returning to a conversation. You're handing someone a transcript and asking them to catch up fast. They're very good at it. But they're catching up, not remembering, and the sooner you internalize that, the better you'll be at using these tools for work that actually matters.