LLM Digest
Subscribe

AI Storyline

3 items · 2 sources · 2 days

View as JSON

Operational story trace

Attack Security

Latest change

A Nov 19 audit questions whether AI-agent security benchmarks score attacks correctly, asking if reported "attack succeeded" verdicts actually hold up.

Earlier contextThe story so far

This is a loose research thread on agent security, not one event. It opened with an arXiv paper arguing that coding harnesses wired to shells, filesystems, and sockets become Living-off-the-Land platforms once an injected instruction lands.

editor-curated · source-linked

Arc

Nov 10Nov 19 · now
THREAT MODEL · Nov 10
Coding harnesses framed as Living-off-the-Land attack platforms
1 source · show source ▾
NOW · Nov 19
Practitioner take on old failures, plus an audit of benchmark verdicts
2 sources · show sources ▾

What to watch — open questions

  • Which named agent-security benchmarks does the verdict audit find mis-scoring attacks, and in which direction?
  • Do benchmark maintainers publish corrected verdicts or revised scoring rules?
  • Does the compositional defense architecture get an implementation or evaluation against a real harness?
How this thread was built
editor wrote the arc · 2 beats

Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.