A new study found that when your AI forgets instructions in a long conversation, it is usually because the chatbot quietly summarized the chat behind your back and dropped your rules in the process. On average, only 17% of those personal instructions survive the cleanup, which explains a frustration almost every regular user has felt.
The Gist
- Chatbots compress long conversations into summaries to save room, a process called compaction.
- A Penn State study measured that only 17% of user instructions survive that compression on average.
- Even with the full conversation intact, the AI follows personal rules only 59% to 71% of the time.
- A simple fix exists: the team built a module that preserves over 90% of instructions.
Have ChatGPT Recap This Article
ChatGPTMeet compaction, the cleanup your chatbot runs mid-chat
Every chatbot has a limited working memory for your conversation. That space is measured in tokens, the small chunks of text the AI reads and writes, and once a chat gets long enough, the space fills up.
When that happens, the tool does not stop working. It quietly replaces the older parts of your conversation with a shorter summary of what was said, then keeps going from that summary. This cleanup is called compaction, and you never see it happen. If the word token sounds fuzzy, our plain-words guide to what a token is and why it limits your chats covers it in two minutes.
The reason your AI forgets instructions sits in what falls out of that summary. Your instructions, like asking for short answers or telling it to avoid a topic, are exactly the kind of detail a summary tends to skip.
How a Penn State team measured the memory loss
Researchers at Penn State University, led by Zhiqi Wang with Yichi Zhang, Dongwon Lee and Yuchen Yang, decided to measure the damage instead of guessing at it. They built a test suite called COMPINT and published their full paper describing how instructions get lost during compression at the end of July.
They planted personal rules into three kinds of sessions: regular back-and-forth chats, an AI agent working through a task, and a long research session. Then they let compaction run and checked what survived. The answer was fewer than one rule in five, 17% on average.
The study holds a second, humbling number. Even with the full conversation intact and nothing compressed, the models only followed the planted rules 59% to 71% of the time. Compression makes things much worse, but perfect memory was never on the table either.
Keep learning on AI Noobies:
- ChatGPT Trip Planning Works Until It Invents a Hotel
- AI Books Are 20% of Amazon’s Self-Published Catalog
- Claude Watermark Detection Opens to Everyone
What a side constraint actually means
The researchers call your personal rules side constraints. A side constraint is any standing instruction that should apply to everything that follows: answer in French, keep it under 100 words, never suggest peanuts in recipes.
These rules feel important to you, but to a summarizer they look like small talk, which is precisely why an AI forgets instructions while keeping the topic. A summary keeps the main topic and the conclusions, and your one-line allergy warning rarely makes the cut. That is also why the chatbot memory features you may have toggled on behave differently: stored memories live outside the conversation, as we showed in our walkthrough of seeing and erasing what ChatGPT remembers about you.
There is good news in the paper too. The team built a small plug-in module that spots side constraints and carries them across the compression step, and it preserved over 90% of them without changing the chatbot itself. Fixes like this one could reach your favorite everyday tools fairly quickly.
How to keep your rules alive this week
Until then, the practical move is simple: the moment your AI forgets instructions and the tone drifts, restate your rules in a fresh message. A repeated instruction lands at the front of the AI’s working memory, where it carries the most weight.
Starting a new conversation for a new task works even better, because a fresh chat means no compression at all. And when you feed the AI something big, like the giant files from our guide to reading a 100-page PDF with ChatGPT in minutes, put your instructions after the document, in their own short message.
The next time your chatbot suddenly ignores a rule it followed all afternoon, you will know what happened. It did not get lazy, and it did not break. It summarized you out of the conversation, and now you know how to write yourself back in.
Stay tuned on AI Noobies.
About the Author
