Memory in AI Agents: Short-Term, Long-Term, Episodic and More

A home tutor in Bhawarkuan teaches about 30 students a week. She never forgets which of them struggles with trigonometry. How does she manage?

During a lesson she works on the whiteboard, which holds only today's problem and gets wiped after class. In a register she keeps facts about each student: class, school, weak topics, exam date.

In a diary she notes what happened in each session: "Tuesday, Rohan finally solved the ladder problem". And over the years she has built up habits about how to teach, such as always starting a weak student with a solved example.

Without the register and diary, every lesson would start from zero.

What is memory in AI agents?

The whiteboard is short-term memory: what the agent is working with right now, held in the model's context window. The register is semantic memory, stored facts. The diary is episodic memory, records of past events. The teaching habits are procedural memory, knowledge of how to do things. The last three together make up long-term memory.

Memory in AI agents is the system that stores information from past steps and conversations and brings the relevant parts back into the model's context, so the agent can stay consistent within a task and improve across sessions.

An LLM itself has no memory between calls. Each call sees only the text you send. So "memory" is always something your application builds around the model: lists, databases and retrieval logic. The building blocks article named memory as one of six design decisions; this article shows what that decision involves.

Short-term memory lives inside the prompt. Long-term memory lives in storage and is pulled in only when relevant.

Short-term memory: the context window

Short-term memory, also called working memory, is the conversation so far plus the agent's own thoughts, tool calls and tool results. It is sent to the model on every step, which is why the agent can say "as you mentioned earlier".

It has a hard limit: the context window, measured in tokens. Long tasks fill it quickly, because every tool result is appended. Long prompts also cost more, and details buried in the middle get less attention.

The common fix is a summary buffer. Keep the last few turns word for word, and compress older ones into a short running summary. The block below simulates it with a word budget instead of tokens and a fake summariser that keeps only sentences carrying facts.

def fake_summarise(turns):
    # a real agent would ask the LLM to summarise; we keep sentences with facts
    keep = [t for t in turns if any(ch.isdigit() for ch in t) or " my " in f" {t.lower()} "]
    return "Summary: " + " | ".join(keep)

def build_context(history, budget_words=30, keep_last=2):
    recent = history[-keep_last:]
    older = history[:-keep_last]
    context = ([fake_summarise(older)] if older else []) + recent
    while sum(len(t.split()) for t in context) > budget_words and len(context) > 1:
        context.pop(0)                     # drop the oldest piece if still too long
    return context

history = [
    "User: Hi, I want to learn data analytics.",
    "User: My exam is on 14 March.",
    "Agent: Great, let's plan around that date.",
    "User: I can study 2 hours a day.",
    "User: Which topic should I start with?",
]
for line in build_context(history):
    print(line)

The greeting is folded away, the exam date survives in the summary, and the last two turns stay word for word. The trade-off: summaries save space but can lose detail, so decide what must never be dropped. Choosing what enters the prompt is the core of context engineering.

Long-term memory: semantic, episodic and procedural

Long-term memory survives after the session ends. It is stored outside the model, in a database, a key-value store or a vector database, and retrieved when needed.

Semantic memory

Semantic memory holds facts: "Priya is in class 12", "the user prefers Hindi explanations", "the company's refund window is 7 days". Facts about specific people or things are sometimes called entity memory. These are the memories that make an agent feel personal.

Episodic memory

Episodic memory holds events with a time attached: "on 3 November the agent tried query A, it returned nothing, then query B worked". Episodes let an agent learn from experience, for example by recalling how a similar task was solved last week.

Procedural memory

Procedural memory holds how to do things: instructions, rules and learned workflows. In practice this is often the system prompt and a library of tested instructions that the agent, or its developer, updates over time.

How AI agents save and retrieve memories

Every turn, a memory-enabled agent does two extra things around the model call. Before it, the agent reads: it searches long-term memory for items relevant to the new message and adds them to the prompt. After it, the agent writes: it decides whether anything new is worth storing.

Reading happens before each model call, writing after it. Both are ordinary code you control.

Retrieving the right memories

Relevance alone is not enough. A memory from yesterday usually matters more than a similar one from last year. Many systems combine a similarity score with a recency score. Below, similarity is simple word overlap, standing in for embeddings.

memories = [
    {"text": "Student is weak in trigonometry", "age_days": 40},
    {"text": "Student scored 62 percent in trigonometry practice test", "age_days": 2},
    {"text": "Student likes cricket examples", "age_days": 90},
    {"text": "Student exam is on 14 March", "age_days": 10},
]

def similarity(a, b):
    a, b = set(a.lower().split()), set(b.lower().split())
    return len(a & b) / len(a | b)

def score(memory, query, recency_weight=0.3):
    recency = 0.97 ** memory["age_days"]           # 1.0 today, fading with age
    return (1 - recency_weight) * similarity(memory["text"], query) + recency_weight * recency

query = "How is the student doing in trigonometry"
ranked = sorted(memories, key=lambda m: score(m, query), reverse=True)
for m in ranked[:2]:
    print(f"{score(m, query):.2f}  {m['text']}")

The recent test score ranks first, ahead of the older but equally relevant "weak in trigonometry" note. Tune recency_weight to your domain: a support agent cares a lot about recency, a medical history agent much less.

Writing and updating facts

The harder half is writing. If the agent saves every message, retrieval drowns in noise. If facts change ("I moved from Indore to Pune"), the old fact must be replaced, not kept beside the new one. Keying facts by entity and attribute makes updates explicit.

facts, history = {}, []

def remember(entity, attribute, value, day):
    key = (entity, attribute)
    if key in facts and facts[key] != value:
        history.append((day, key, facts[key], value))   # keep an audit trail
    facts[key] = value

remember("Priya", "city", "Indore", day=1)
remember("Priya", "goal", "data analyst job", day=1)
remember("Priya", "city", "Pune", day=30)                 # she moved
remember("Priya", "goal", "data analyst job", day=31)     # same value, no change

for (entity, attr), value in facts.items():
    print(f"{entity}.{attr} = {value}")
print("Changes:", history)

In a real agent, an LLM call extracts candidate facts from the conversation, and code like this decides whether each one is new, an update or a duplicate.

Memory as a tool: letting the agent decide

There are two ways to wire memory. In the automatic style, your code retrieves and saves on every turn, as above. In the tool style, you give the agent tools such as save_memory and search_memory, and the model decides when to use them.

The tool style is flexible: the agent can search memory only when a question needs it, and save only what it judges important. The cost is reliability.

Models sometimes forget to save or search, so many production agents combine both: automatic retrieval of core profile facts, plus memory tools for everything else. Retrieval over documents in the same way is the idea behind agentic RAG.

Privacy, forgetting and cost

Memory raises questions that a stateless chatbot never faces. Who can see what the agent remembers? Can a user ask for their data to be deleted? Is a phone number or health detail being stored without need?

Design forgetting in from the start. Give memories an expiry where it makes sense, let users view and delete what is stored, and never save secrets such as passwords or OTPs. Keep memories per user, so one person's facts never appear in another person's prompt.

Cost matters too: every retrieved memory adds tokens, so fetch the top few items, not fifty.

Common mistakes with agent memory

• Treating the context window as memory. It is wiped when the session ends. Anything that should last must be written to storage.
• Saving everything. Raw transcripts make retrieval noisy. Store distilled facts and meaningful episodes.
• Never updating facts. Old and new values side by side confuse the model. Overwrite, and keep history separately.
• Ignoring recency. Pure similarity search can surface a year-old preference over yesterday's correction.
• Mixing users' memories. Always filter by user or session ID before retrieval.
• No way to forget. Users and regulations may require deletion. Build it in from day one.

Key takeaways

• LLMs are stateless; memory is storage and retrieval code around them.
• Short-term memory is the context window; manage it with summaries and budgets.
• Long-term memory is semantic (facts), episodic (events) or procedural (how-to).
• Retrieve before each call, write after it, and score by relevance and recency.
• Update changing facts, forget what's stale, and keep each user's memory private.