Your Agent's Memory Is a Dataset Nobody Is Curating

Short summary

Agent memory systems are uncurated datasets that agents write to themselves and then use as highest-authority context — functionally equivalent to fine-tuning on unreviewed self-generated text. Three critical failure modes emerge: consolidation manufactures false confidence by stripping hedges, repeated summarization causes semantic drift and self-reinforcing hallucinations, and the write path is a proven attack surface with AgentPoison achieving 80%+ success rates. Existing benchmarks like LoCoMo and LongMemEval only test the read path, leaving write-path vulnerabilities completely unmeasured.

• •Agent memory is an unreviewed self-generated dataset that silently becomes the agent's highest-authority context
• •Consolidation strips hedges and provenance, manufacturing false confidence that agents act on without verification
• •Memory poisoning attacks achieve 80%+ success with <0.1% poison rate, and corrections fail because they enter the same untrusted store

Generated with AI, which can make mistakes.