Adding "memory" to an agent usually starts as a small feature: save a few notes at the end of a session, load them at the start of the next. It works great in a demo. Months later, the memory store is a pile of stale facts, contradicting preferences, and details nobody remembers agreeing to keep. The fix is the same one we apply to any data store: decide up front what goes in, how long it lives, and who can see it.

Decide what is worth remembering

Not everything a conversation contains deserves persistence. A useful filter is to ask what a future session would do *differently* if it knew this:

  • Durable preferences and constraints ("always use metric units", "this repo uses tabs") change future behavior and rarely go stale quickly.
  • Task state ("migration is half done; step 4 failed") is valuable but short-lived. Give it an expiry or tie it explicitly to the task it belongs to.
  • Derived conclusions ("the flaky test is caused by shared fixtures") are the riskiest. They were true when written, and code changes. Store them with a date and a pointer to where they came from, and treat them as hints to re-verify, not facts.
  • Raw transcripts are rarely worth keeping verbatim. They are large, noisy, and the most likely to contain sensitive content.

Give every memory metadata

A memory that is only a string of text cannot be maintained. At minimum store: when it was written, which session or source produced it, its scope (user, project, or global), and a type (preference, task state, conclusion). With that, you can expire task state after a few days, flag conclusions older than some threshold for re-checking, and answer the question "why does the agent believe this?" when behavior looks wrong.

Plan for contradiction and deletion

Memories will conflict. A user says they prefer short answers in March and detailed ones in June. Newest-wins is a reasonable default, but make it a deliberate rule rather than an accident of retrieval order. Also provide a way for users to see what is stored and delete it. If you cannot enumerate and remove a user's memories on request, you have a compliance and trust problem waiting to happen.

Decide what is never stored

Write the exclusion list down: credentials, API keys, payment details, health or other sensitive categories your product has no business retaining. Enforce it in code with filters before writing, not just by instructing the model to be careful. Models are inconsistent at self-censoring; a deterministic check is not.

Treat retrieved memory as untrusted context

Memory written from one session is read as input in another. If an earlier session ingested a malicious web page or document, an instruction planted in it can be saved and replayed later. Keep recalled memories in a clearly labeled section of the prompt, and don't grant them the authority of system instructions.

The practical closing point: start with a small, typed, expiring memory store and a way to inspect it. It is much easier to loosen a conservative policy than to clean up a store that has been silently accumulating anything the model thought was interesting.

AI AgentsSoftware Engineering