Three stories from this week's AI engineering news look unrelated. I think they're the same mistake: treating what a model produced, or remembered, as trusted input.
1. Slopsquatting. Models invent package names that don't exist, and attackers register those names on npm and PyPI. The problem is real, but I don't think it needs a new tool. A lockfile plus a human look at every new dependency in review closes most of it. The gap is teams that let an agent add dependencies with nobody looking.
2. Memory poisoning. An agent reads an email or a web page, part of it lands in long-term memory, and next session it comes back as a fact. I still want agents with memory. What I want from that memory is provenance: "the user told me" and "I read it in an inbound email" should never carry the same weight.
3. LLM reviewers you can't rely on. We hit a version of this recently. Our review agent misread its task and ended up comparing long-lived subagents instead of checking what it was asked to check. A second model grading the first wouldn't have caught that.
So I'm building a harness around our agents and wiring in Harness² from Google Research. It runs variations of a harness on the same task and refines it from the differences between executions, with no ground-truth answer needed. The hope is that it catches a misread task before it turns into a verdict, and that the agent swarm improves step by step instead of through me rewriting prompts. It's too early to say whether it works.
The common thread: a model's output is a claim. Something deterministic, or a person, decides whether it gets in.
Which of these have you actually run into in production?