One Assistant, Many Memories: Benchmarking Real-World Group Assistance When Recall Is Not Enough
Opens the cluster: memory evaluated by the quality of assistance it enables across many users.
7 items · 4 sources · 7 days
Operational story trace
Follow in this browser to see new updates on your Live feed.
Latest change
The newest member (Nov 16) is a quantum recurrent-network benchmark for time-series prediction; it shares only the word "benchmarking", so PrivacyPR (Nov 13) remains the last agent-relevant entry.
This is a research trend, not one event: six unrelated arXiv papers in ten days each build an agent benchmark from real sessions, real tools or real fixes. It began with group-assistant memory (Nov 3), added MCP hallucinations and memory poisoning, then moved to long-horizon infra work, researcher workspaces and privacy-aware code review (Nov 13).
Arc
Opens the cluster: memory evaluated by the quality of assistance it enables across many users.
Shifts to failure modes: hallucinations in reasoning, execution and memory for agents using MCP tools across servers.
Adds a security angle: malicious instructions from one session persisting through agent memory into later ones.
Moves the real-world framing to infrastructure: long-horizon optimization tasks for agents.
Applies it to research agents, scoring delivery inside real researcher sessions rather than standalone tasks.
Extends it to code review with a dataset of real privacy fixes tied to their defects.
Keyword match only: a quantum ML time-series benchmark that adds nothing to the agent-evaluation thread.
Opens the cluster: memory evaluated by the quality of assistance it enables across many users.
Shifts to failure modes: hallucinations in reasoning, execution and memory for agents using MCP tools across servers.
Adds a security angle: malicious instructions from one session persisting through agent memory into later ones.
Moves the real-world framing to infrastructure: long-horizon optimization tasks for agents.
Applies it to research agents, scoring delivery inside real researcher sessions rather than standalone tasks.
Extends it to code review with a dataset of real privacy fixes tied to their defects.
Keyword match only: a quantum ML time-series benchmark that adds nothing to the agent-evaluation thread.
What to watch — open questions
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.