LLM Digest
Subscribe

AI Storyline

3 items · 2 sources · 3 days

View as JSON

Operational story trace

Benchmarking Memory

Current stateDevelopingstatus changed Nov 6

Latest change

MemoryDojo (Nov 6) is a benchmark for poisoning attacks through LLM agent memory, where malicious instructions seen in one session can carry into later ones.

Earlier contextThe story so far

In October a paper benchmarked personal agent memory on interaction preferences, asking whether agents stop re-asking what users already told them. In early November, two more benchmarks moved the question from convenience to correctness and security.

editor-curated · source-linked

State over time

● preferences · Oct 21poisoning · Nov 6 → now ●
  • preferences · Oct 21
  • hallucination · Nov 5
  • poisoning · Nov 6 → now
PREFERENCES · Oct 21
Benchmarking whether agents remember how users want to interact
1 source · show source ▾
RELIABILITY · Nov 5
MCPHallu measures reasoning, execution and memory hallucinations in MCP agents
1 source · show source ▾
THE TURN · Nov 6
MemoryDojo treats memory as an attack surface
1 source · show source ▾

What to watch — open questions

  • Do any agent frameworks or memory products publish results on MemoryDojo?
  • Do the three benchmarks converge on shared metrics or test harnesses for memory?
  • Do vendors ship documented mitigations for cross-session memory poisoning?
How this thread was built
editor wrote the arc · 3 beatswatcher 1 status change

Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.