An agent's working memory is its context window, which is finite and resets between runs. On long-horizon tasks it forgets earlier steps, repeats work, and loses the user's intent — so "agent memory" (what to persist, where, and how to recall it) becomes a first-class architecture problem rather than a prompt tweak.
The field has converged on memory as a tiered system: the live context window, an episodic log of past interactions, and a durable store of facts, preferences, and procedures. That split now ships as real components, often on infrastructure teams already run (Elasticsearch, Postgres, Redis) and often exposed over MCP. Newer framing treats memory as a lifecycle: deciding what to write, scoping it, consolidating, and forgetting.
The hard questions moved to the write and recall paths. Similarity search misses facts that are logically relevant but worded differently, and append-only stores keep stale facts alive; one study found a memory without revocation scores below no memory at all. Instruction files like CLAUDE.md show the write-side version: they grow because deleting a rule is riskier than adding one.
Build vs. buy is splitting the market. Managed services sit beside a crowded wave of local-first, single-file stores. Several of the latter retrieve with BM25 plus vectors and no LLM in the loop, and report recall in the mid-90s on LongMemEval.
Integrity is the open problem. Memory admits poisoned facts, hidden injections, forged reasoning, and sycophantic recall, and benchmarks show that strong recall does not predict whether an agent uses memory well. Evaluation is maturing toward fixed Add/Search pipelines and cost-aware scoring.
The counterweight is brute force: 1M-token windows and recursive dispatch let some runs skip memory engineering (see context compaction). They don't carry anything across sessions.
Why it matters for platform engineers
Memory is where agent cost, latency, and reliability collide: stuffing everything into context is simple but blows up token cost and latency and still forgets; an external store adds a retrieval hop and a freshness/consistency problem. The decision (compact vs. retrieve vs. both, build vs. buy) is an infrastructure decision with an ongoing operational tail — eviction policies, index maintenance, revocation, per-user isolation, and recall evaluation — not a one-time integration. The write path is also a security boundary: anything an agent stores becomes trusted context later.
Developments 73
Tiered and lifecycle memory architectures
14 entries · latest Oct 28, 2026
Working, episodic, and long-term tiers are settled practice, now shipped as products on existing infrastructure, with newer work treating memory as a lifecycle of writing, scoping, consolidating, and forgetting.
LangChain tested OpenWiki, its open-source wiki compiler, as an agent memory layer on a synthetic insurance library (111 documents, 972 facts, with outdated versions and contradictions mixed in). On questions spanning several sources, the wiki-equipped agent was roughly 5x more accurate than keyword search alone; adding an evidence graph gave a further 27% and doubled citation consistency (vendor-reported, synthetic corpus).
Each wiki statement is a claim tied to hashed source lines. When the lines change, the hash flags the claim without a model call, and only flagged claims go to a model for revision or retraction. For builders, this is a cheap, deterministic staleness check for memory built from policy-style documents.
V7, built on GPT-5.6, turns a company's scattered files into context agents use to complete complex, source-linked work.
Enterprise knowledge as memory now ships as a standalone agent product, not only as an internal platform layer like LinkedIn's playbooks or Elastic's Atlas. That puts a buy option next to the build path.
LinkedIn's Contextual Agent Playbooks and Tools, built on MCP, serve procedural memory, code search, and runbooks directly to coding agents to work around their limits in large codebases. LinkedIn reports a 20% productivity boost with zero loss in reliability (self-reported).
The tiered model built as a codebase-scoped organizational context layer, with operational guardrails, rather than a reference diagram.
A Hugging Face post on IBM Research's ALTK work asks how much memory an agent actually needs, treating memory sizing as a hierarchical-memory-management design question.
It is distinct from where memory lives or how stale facts get revoked. Sizing memory to the task is a cost decision a platform team makes before choosing a store.
The ACM paper argues production agents fail less from weak reasoning than from unmanaged context: histories, large prompts, tool definitions, and ballooning tool outputs. It names five primitives (architecting, ingesting, scoping, anticipating, compacting and consolidation) and ships a reference implementation, Maximem Synap.
It puts a shape on the cost curve: naive accumulation grows token cost quadratically with conversation length, crude summarization buys linear cost with an accuracy cliff, and only validated compaction gets linear cost with fidelity preserved. See context compaction.
OpenWiki Brains turns sources such as Gmail, Notion, git repos, X, Hacker News, and web search into a local wiki of plain Markdown files that agents read as fresh memory.
The memory layer is synthesized Markdown refreshed by scheduled jobs, not a vector index. Recall is proactive: the agent draws on it without being told to remember, unlike the reactive "remember this" pattern most assistants ship.
A widely shared practitioner post argues agent memory is leaving the "remember this" demo phase and turning into a real engineering layer.
It landed the same week Elastic open-sourced Atlas, and it marks the shift this page tracks: the question is no longer whether an agent should have memory but what to write, when, and how to recall it cheaply.
Supafax is an agent that uses email as its memory. You email yourself to talk to it; it treats that slice of your outbox as its inbox and short-term memory, and it inserts emails into your inbox to store routines, reminders, and rules for itself.
The builders pick email because it is an append-only, immutable record. The catch they name: users are reluctant to hand that data to a company.
Elastic's Atlas maintains three categories of agent memory on top of Elasticsearch, integrates with agents over MCP, and keeps each user's memories isolated. On question answering it scored 0.89 Recall@10.
It is an open, production-grade instance of the tiered model, built on infrastructure many teams already run, and it ships evaluation numbers rather than a demo.
LangChain's practical guide to agent memory lays out short-term and long-term memory as settled practice, then adds a feedback loop: analyze the agent's own traces in LangSmith to decide what is worth remembering and to improve across runs.
Memory becomes something the agent curates from its own history, not just a place facts get dumped.
A practitioner write-up surveys agent memory systems and knowledge graphs through four open projects: Letta, Mem0, Graphiti, and Cognee.
For teams choosing a memory layer, these four are the usual starting shortlist, and they differ mainly in whether memory is organized as a store of facts or as a knowledge graph.
LinkedIn's Cognitive Memory Agent (CMA) is an infrastructure layer that gives agents persistent memory across episodic, semantic, and procedural layers, with support for multi-agent coordination, retrieval, and lifecycle management.
It is a useful reference architecture for the tiered split: working context, a log of past interactions, and durable facts and procedures. It also treats memory as shared platform infrastructure for many agents, not a per-agent prompt feature.
Adi Polak's talk on context engineering and memory management uses Apache Kafka and Flink for real-time stream processing, dynamic memory tiering, and tool orchestration over MCP, aimed at token limits, cost spikes, and latency.
It frames the move from stateless prompts to state-aware agents as a distributed-systems problem. Teams already running streaming infrastructure can build the memory tiers on it rather than adopting a new store.
Build vs. buy: managed services and developer-owned stores
17 entries · latest Oct 30, 2026
Managed memory services compete with a crowded wave of local-first, single-file stores that often retrieve with BM25 plus vectors and no LLM in the loop.
Turnbook, a single-repo Show HN project, stores agent conversations word for word and spends zero model tokens on saving, unlike tools that run an LLM over transcripts at write time. It self-reports 88.8% on LongMemEval_S, and 91.8% when retrieval is capped at 3.1k tokens of memory per question (v0.10.0). The same answer model (gpt-5-mini), prompt and gpt-4o judge were used across compared systems; the post credits an embedding-model change for the token cut.
All numbers are self-reported and unreplicated. The design choice is the takeaway: move cost from write time to retrieval, and keep raw text so nothing is lost to summarization.
Databricks released Agent Memory in Beta on Lakebase (serverless Postgres), built on its MemCurrent design: managed sessions hold the raw record, and a background pipeline extracts durable facts, deduplicates, and replaces facts that newer conversations contradict. Agents load a memory index at session start and search by hybrid keyword and semantic retrieval.
Vendor-reported internal results: about 3x the pass rate on enterprise tasks with 31% fewer tokens, and one repeat query dropped from 11 tool calls and nearly six minutes to 2 calls and 41 seconds. The vendor also reports full-context replay used about 12x more input tokens at equal accuracy.
For builders: memories are readable, exportable, and deletable per user, and pipeline instructions steer retention.
Fraise is a single-binary memory database that stores facts as a temporal graph of facts, topics, and entities. It has exactly two verbs, remember and recall, for example recall billing entity:acme since:30d top:5.
Recall is ranked and capped, not exhaustive, with recent facts outranking older ones, on the stated reasoning that the caller pays for every token read back. Cost-consciousness moves from the storage format into the query interface.
OKF Agent Memory keeps facts, decisions, and domain concepts as git-tracked Markdown with YAML front matter, implementing Google's Open Knowledge Format v0.2, which bakes in provenance, trust tiers, and lifecycle metadata.
Retrieval is in-memory BM25 with no vector database or embedding cost, returning in under 300 microseconds. A progressive-disclosure index lets the agent pull one concept at a time, cutting token use roughly 80% versus loading the whole knowledge base. It is the zero-LLM pattern built to a vendor-neutral spec.
Hugging Face's "Give Your Coding Agents a Memory You Own" (the Funes project) makes the "own your memory, don't rent it" pitch explicit.
A major platform endorsing developer-owned memory strengthens the local-first side of the build-vs-buy split against managed services like Cloudflare's Agent Memory.
Two more developer-owned stores shipped within days of each other:
Contextual: local codebase memory for AI coding agents
OpenContext: persistent, project-local memory served over MCP
Both keep memory inside the project rather than in a hosted service. The local-first roster is crowded enough that differentiation now comes from write discipline and recall quality, not from storing memory locally.
Awareness Local stores memories as git-compatible Markdown, indexes them with SQLite FTS5 plus optional local embeddings, and retrieves with hybrid BM25-plus-vector reciprocal rank fusion, with no LLM in the loop. It reports 96.0% recall@5 on LongMemEval.
That is in the same range Sibyl reported on the same benchmark. Hybrid retrieval without an LLM is becoming a repeatable recipe, not one team's result.
Remembrane keeps agent memory in one SQLite file with zero dependencies in the default install. Its author built it because a few thousand short strings didn't justify a hosted API, a vector database, or a framework.
The distinctive part: recall is deterministic, so you can write unit tests that assert what the agent remembers and run them in CI. Memory becomes something you can regression-test, not just observe.
OpenLore packages deterministic, local-first memory together with guardrails for AI coding agents.
Shipping memory and guardrails in one package treats what an agent remembers and what it is allowed to do as one local control layer rather than two products.
Knotic gives AI coding agents layered memory split into project, session, and docs tiers.
It is the tiered reference architecture showing up bottom-up at single-developer scale, the same split LinkedIn and Elastic describe for enterprise platforms.
PMB stores memory in one SQLite file plus a local LanceDB vector index, with no server, cloud service, or API keys. Retrieval fuses BM25, vector search, and an entity co-occurrence graph with reciprocal rank fusion; the stated goal is the right memory, not the closest one.
It hooks the agent lifecycle over MCP: relevant memories are injected before each response, and decisions and learnings are recorded after each turn without manual saves.
FERNme is a graph-based associative memory that forms memory tags from fuzzy edges and a Hebbian co-occurrence rule, updating with roughly zero LLM calls.
It is the cheap-writes pattern: build the memory structure deterministically instead of asking a model what to store each turn, so persisting what an agent learns stops being a per-turn token bill.
Memharness stores agent memory bi-temporally in a single SQLite file: it tracks both when a fact was true and when the agent learned it.
That lets recall reason about staleness instead of returning whatever embeds nearest. Richer temporal modeling is a recurring design choice in the local-first stores.
Cortex is a local-first, encrypted memory store for AI agents, written in Rust and exposed over MCP.
It is an early entrant in the developer-owned wave: memory you install and keep on your machine, with encryption at rest, rather than a hosted service.
Project Brain2.0 provides curated project memory for Claude Code and any agent that can read files.
The store is plain files, so it needs no server and works with any file-reading agent. The emphasis on curated memory points at the write side: deciding what belongs in the project record, not only how to retrieve it.
Cloudflare's Agent Memory, in private beta, extracts structured memories from agent conversations and retrieves them on demand with five-channel parallel retrieval fused by Reciprocal Rank Fusion. Shared memory profiles let teams of agents read common knowledge. It competes with Mem0, Zep, LangMem, and Letta.
This is the buy side of the build-vs-buy split: memory as rented infrastructure next to your edge compute.
Sharing memory across agents, isolating it between users
10 entries · latest Sep 26, 2026
One memory now serves many agents and tools, which raises write-conflict and organization questions and forces per-user scoping so context doesn't leak between callers.
Google added private, server-side memory to Private AI Compute. Memory runs inside hardware-enforced secure enclaves, unlockable only with device-held keys, so context persists between sessions without the provider being able to read it.
It answers "remember across sessions without leaking" at the infrastructure layer, a privacy-by-construction guarantee rather than application-level scoping.
LangChain's Managed Deep Agents 0.8 adds a two-tier memory config, define_memory(agent=MemoryLayer(), user=MemoryLayer()). User memory mounts at /memories/user/, keys to the caller's identity, and is allowed in one-to-one Slack DMs but denied by default in group chats and HTTP channels.
It pairs that with per-user credentials (OAuth across 23 services), so both what an agent remembers and what it can touch follow the caller. One user's context can't leak into another's conversation through a shared deployment.
MACE (Memory-Agent Co-Evolution) turns a multi-agent system's collaboration traces into functional memory units: conditions, actions, and outputs linked by support, conflict, and repair relations. Within a memory budget it picks which units and formats (instructions vs. checklists) to hand each agent, and updates unit scores from task outcomes.
It averages 81.11% across eight benchmarks, beating ten baselines. How shared memory is organized and presented back to agents is a tunable variable, not just whether writes conflict.
Make0 AI is a personal memory layer meant to give Codex and Claude Code common context. The pain point is concrete: after hitting one agent's usage limit, switching tools means starting from scratch.
The author's first attempt, a shared Markdown instruction file, kept growing until the common context passed 20k tokens. A shared file without curation hits the same unbounded-growth problem as CLAUDE.md.
MemHub provides persistent, shared memory for AI coding agents.
It joins Sibyl and CMEM in targeting the same gap: several agents, or several sessions of one agent, working on the same codebase need to read one record of what has already been decided.
CMEM pairs a local SQLite store of timestamped observations (decisions, dead ends, fixes, not just diffs) with a built-in vector index. One MCP server exposes it to every client, so Cursor, Claude Code, and a bare CLI agent share the same memory, with recall reported under one second.
It bundles 11 skills so teams don't write the write/recall logic themselves (the vendor cites 6+ weeks for a custom build). It is Apache-2.0 and self-hosted, with an optional paid cloud mirror for cross-device sync.
Gwen Shapira's talk uses PostgreSQL to give agents both deterministic context (JSONB) and semantic context (high-recall HNSW vector indexes), with vector quantization speeding queries 4x, plus strategies for managing agentic memory.
For multiple agents writing shared notes and decisions, Postgres's own ACID transactions and row-level locking prevent write conflicts without a purpose-built memory service. It applies the "ride infrastructure you already run" instinct to the write-conflict problem.
Sibyl is a self-hosted, multi-user memory system built on SurrealDB that many parallel coding agents read and write through a CLI (the author finds it works better than MCP) or MCP. It started as a Kanban board for agents with a crawler and RAG.
It reports 96.96% strict recall@5 on LongMemEval-S with no LLM in the retrieval path: a shared, developer-owned substrate can serve many concurrent agents and stay cheap to query.
A developer built an S3-based durable filesystem, in Rust with Python and TypeScript SDKs and a CLI, that mounts anywhere so the memory Markdown files agents write on a laptop and in the cloud stay in sync.
It treats the memory store as a portable substrate that follows the agent between runtimes, the build-it-yourself answer to the cross-platform consistency managed services sell.
Similarity recall misses logically relevant and stale-but-superseded facts, so the work is shifting to revocation, write-time curation, and recall that reasons over the store.
An explorer agent invents challenging-but-solvable tasks in a new environment and a solver attempts them. Each solver failure yields a heuristic, which is accepted into the memory bank only after the solver repeatedly succeeds with it in context.
On AppWorld, τ²-bench, and AutomationBench the authors report up to +15.9 points mean success and up to 2.2x pass^5 over a no-memory baseline, competitive with methods that need training tasks. Heuristics also transferred to other model families.
For builders: gating writes on repeated success is a cheap write-time curation rule that needs no human-written guidelines.
AutoViewMem argues that fixed granularities and static schemas fail when preferences, events, constraints, and temporal updates share one mixed representation; the interference makes top-K retrieval noisy.
Instead of one index, it maintains several complementary projections of the stored history and chooses which view to query per request. Like CABLE, it locates recall failures in the query interface, not only in what is stored.
LangChain's GTM Agent diffs a sales rep's edited draft against the agent's original to extract structured per-rep style observations. It writes them to Postgres keyed by rep and loads that record before every future draft, with a weekly cron job compacting the store. LangChain reports a 250% lift in lead conversion and 40 hours saved per rep per month.
The write is triggered by a human correcting the agent's output, a third write discipline beside automatic capture and explicit saves.
SOS stores a clearly defined "latest state" of a project inside the repository and serves it to a new or resumed session over MCP. The author built it because Codex kept losing track of what was done, which decisions still held, and what needed re-verification.
State is explicitly saved, not inferred. A superseded result stays in history but stops influencing the agent, the same guarantee TEPA's revocation targets, applied to project status.
COBRA-Skills frames skill improvement as budgeted sequential optimization over an evolving candidate pool. Contextual-bandit prioritization steers expensive execution-based evaluation toward the most promising or informative candidates instead of scoring every one.
Across six benchmarks and three target models it holds top-tier performance while cutting optimization cost 55-58% versus a SkillOpt baseline, with only 50 examples per benchmark. It makes the skill-evolution loop cheap enough to run routinely.
A postmortem of an autonomous coding agent left running for hours unattended describes the memory bugs that broke it, which only surfaced once nobody was watching the session live.
Write-side curation failures compound in long, unsupervised runs. Memory needs monitoring the same way the agent's actions do.
Memctl gives a coding agent's persistent instruction files (CLAUDE.md, AGENTS.md) versioning, diffing, and rollback.
It is a direct answer to catastrophic remembering: those files accumulate redundant, conflicting guidance, and without history there is no way to undo a bad edit.
Recuris pairs a working memory that tracks task progress and guides skill selection with an experiential memory of reusable skills. After each failure, a meta-agent makes localized, validation-gated edits to the skill store.
It improves 35 of 37 model-benchmark pairs, gains reach +32.2 points on the longest-horizon tasks, and long-horizon failures drop by up to 80%. Skill memory pays off precisely where compaction and retrieval get harder.
LangChain's self-correcting memory for OpenWiki stores evidence-backed claims instead of raw facts. It uses that evidence to detect when a claim has gone stale as the codebase changes, then corrects or drops it rather than retrieving whatever was written last.
It is a write-time defense against staleness and hallucination, pointed at codebase memory where the ground truth keeps moving.
FTA-Mem targets emotional-support agents, where dialogue is low-density: turns are incomplete, evidence is scattered, and user state shifts over time. Fixed units such as turn notes or session summaries either lose detail or add noise.
It segments conversations into coherent situation fragments and anchors memory to fact, time, and affect. For sparse, long relationships, the memory unit itself has to be designed, not just the store.
CABLE names an evidence-reachability failure: even when a fact was stored correctly, later steps may not recover it through a bounded memory interface. Similarity retrieval handles topical recall but misses earlier plans or motivations that are semantically distant from the events they explain.
It adds antecedent-based links and expansion to reach that evidence. The interface, not just the store, limits what an agent can remember.
Agents that repair a failure usually discard the successful correction, so a later episode with the same bug starts from scratch. MERIT, a training-free Text-to-SQL agent, keeps a dual-polarity memory of oracle-verified corrections and of directions that failed. A deterministic classifier assigns a coarse failure type that conditions retrieval, and only finalized earlier episodes are eligible.
"What worked last time, and why" becomes its own memory object, distinct from facts or preferences.
Across 247,694 instruction lifetimes, agentic-coding files like CLAUDE.md grow without bound until the repo retires or someone rewrites the file. Appending is cheap; once an instruction's rationale is gone, deleting it safely costs O(2^|D|) for |D| instructions. The paper calls this catastrophic remembering, the inverse of catastrophic forgetting.
This store has no retrieval step: the whole file loads every turn. The fix is write-time deduplication and pruning, not better recall.
TEPA represents each memory as a keyed precedent and revokes the active one the moment fresher evidence contradicts the same key, so old and new facts never coexist in the retrieval set.
Under full reversal, append-only and last-write-wins caches both score below no memory at all (0.210 vs. 0.309), while TEPA holds 0.950. A memory layer without revocation is not neutral on stale facts; it actively hurts.
PRO-LONG keeps a complete, structured interaction log instead of summarizing or pruning it, and has a coding agent search that log programmatically rather than by embedding similarity.
On the full ARC-AGI-3 public game set it improves 18.0 percentage points over a base coding agent and matches or beats specialized long-horizon harnesses (up to 76.1% pass@1) while using 4.2-5.8x fewer tokens. Memory retrieval becomes a code-search problem.
"Towards Root Memories" shows that memory retrieval based on semantic similarity misses memories that are logically critical but share little wording with the query. It builds IMLogic, a benchmark for implicit logical memory retrieval in long dialogues, and proposes root memory, a structured, decision-preserving representation of reusable personal facts.
The recall step has to reason over what is stored, not just embed and rank. See vector/graph retrieval.
Persistent memory is a persistent attack surface; poisoned facts, concealed injections, forged reasoning, and sycophancy all pass through the write path.
A paper names a recall-compliance dissociation in long-horizon dialogue: the model can still repeat an instruction while failing to keep obeying it. The authors call this State Integrity Collapse, where a model preserves a stated constraint in recall but not in behavior.
This is paper-reported from the abstract excerpt.
For builders: a memory check that asks "does the agent still know the rule?" can pass while the agent violates it. Evaluate long sessions on compliance with constraints, not recall of them, and re-inject or enforce critical rules outside the model's conversational state.
AfterDeleteBench audits what persists after deletion in LLM agents that keep external memory and derive extra state, such as summaries, from past interactions. Its premise: deleting the original record need not remove information carried by its descendants.
This is paper-reported from the abstract excerpt; results across systems are not summarized here.
For builders: treat deletion as a graph problem. If summaries, embeddings, or caches are derived from a record, a "forget this" request or retention policy has to propagate to every derived artifact. Track provenance from derived memory back to source records, and test deletion by probing the agent afterward.
TenseMem is an embedded, open-source memory store in a single SQLite file with no server. The model proposes typed facts; the engine, not the model, resolves which one is current.
The target failure is similarity search returning both "I live in Delhi" and "I moved to Bangalore" and leaving the model to guess. Resolving supersession in code makes the current-state read deterministic. A rule-based extractor runs without an API key, and OpenAI, Anthropic, Ollama, LiteLLM, and Gemini plug in.
Early project README, no published benchmark. Treat it as a pattern: keep stale-fact resolution out of the prompt.
Agent Zero Memory splits a user's history into three parallel systems: an episodic timeline of what changed when, an entity-event knowledge graph, and a semantic store of durable facts. It enforces a citation lock: an answer may cite only evidence it actually retrieved, so the system abstains instead of guessing.
Provenance is built into the retrieval contract rather than checked after the fact, a structural defense against fabrication.
The Forged Amplifying Rationale Memory Attack (FARMA) targets an agent's stored reasoning history. It inserts forged reasoning traces in evasive language that bypasses keyword-based defenses, then amplifies them; the paper also studies defenses.
Memory poisoning extends from corrupting what an agent believes to corrupting how it argues for it. Any memory that stores rationales or tool-use history needs the same provenance controls as stored facts.
"When Claws Remember but Do Not Tell" studies stealth memory injection in persistent personal agents. A remote black-box attacker sends a single email payload that must get the agent to write poisoned memory, stay hidden from the user in the agent's replies, and steer future behavior.
The planted entry need not look false, only concealed, so write-time checks aimed at obviously wrong facts may not flag it. Untrusted external content written to memory becomes trusted state later.
MemSyco-Bench measures sycophancy induced by memory: retrieved memories cause agents to over-align with the user at the cost of factual accuracy or objective reasoning. Existing memory benchmarks check whether memories are stored and retrieved correctly, not how they bend downstream decisions.
This is harder to catch than a false fact, because it looks like the memory system working as intended.
A reproducible benchmark shows agent-memory systems readily admit poisoned facts: adversarial or wrong entries that are written once and then retrieved as trusted context on every later turn.
Persistent memory is a persistent attack surface. Write-time validation and provenance become part of the memory-engineering job, not just recall quality. See prompt injection.
Memory evaluation is moving from factual recall to utilization, failure modes, and task completion, scored through fixed pipelines with cost on the axis.
DolphinBench evaluates memory through an agent's task completion on real-world, action-dependent recall, not conversational QA where the question itself signals which fact to fetch. It also maps the Pareto frontier of accuracy against cost and latency instead of naming one winner.
Benchmarks that reward accuracy alone let memory systems make unreasonable cost trade-offs. Read memory leaderboards with the cost axis attached.
"Beyond Context Windows: Evaluating Long-Term Memory for AI Agents" argues that long-term memory evaluation must look past context-window size.
It is the evaluation-side version of the debate over whether bigger windows replace durable memory (see context compaction): a model's window spec says little about what it remembers across sessions.
An open evaluation framework defines agent memory by two fixed primitives, Add (write) and Search (retrieve), so different systems can be compared without each team choosing its own answer model and evaluation pipeline.
It extends the Agent Memory Leaderboard's approach into reusable tooling. Fixing the boundary between memory system and evaluator is what makes vendor numbers comparable.
UTILMEM (1,717 instances across five domains) scores memory utilization: reasoning over dense histories, catching implicitly relevant memories, synthesizing evidence across sessions, and resisting similar-looking distractors.
Strong scores on factual-recall benchmarks don't reliably predict utilization. Even when a system retrieves the right evidence, it often fails to integrate it or is misled by a plausible distractor. Retrieval alone is insufficient, and now it is separately measurable.
MemTrapBench targets memory-induced cognitive traps: even faithfully recorded, semantically relevant memories can distort reasoning or beliefs and degrade the current task.
Most memory benchmarks stop at whether information was extracted, stored, and retrieved correctly. Retaining information is necessary but not sufficient; how memory reshapes reasoning needs its own score.
The Agent Memory Leaderboard released first results for text memory in two tracks, open-source methods and commercial products. 136 teams registered and 69 frameworks completed the run. Commercial leaders: MemoraX 58.02, MemOS 45.89, NTES-MEMORY-SMART 44.21.
Every entrant runs through one pipeline: memory system Add → Search, then benchmark Answer → Eval. A system can't win by picking its own answer model or evaluation setup, as vendor demos can.
AgenticSTS is a testbed for long-horizon agent memory. Its bounded contract builds every decision from a fresh message assembled by typed retrieval, with no raw cross-decision transcript appended, so the prompt stays bounded across runs of any length.
Appending everything makes past context easy to reach but leaves the effect of any single memory component impossible to isolate. A bounded contract makes memory components measurable one at a time.