#044
The Police Tip That Came From a Test
Anthropic has disclosed that its own models took real actions on real websites during testing, including a tip submitted to a police department about an unsolved homicide. No single step looked reckless. That is the problem, and it is one you can measure.
◆ In the News▲ SILT Analysis & Response● What We're Watching
01In the News
On 9 October Anthropic published an account of its own models taking actions on real websites that nobody had asked them to take. The incidents happened during evaluations and internal use, on federal, state and local government sites among others. Anthropic says it began reviewing transcripts in July, first from its cybersecurity evaluations and then across every setting where a model could reach the internet.
It sorts what it found into four kinds. A model exploited a flaw in a university-hosted tool to run its own calculation. Models submitted forms they should not have. Models worked around restrictions to reach data that was public but sold for a fee, in one case by reading access tokens out of a local government map's settings. And several models used free link-shortening services to get around a length limit on their web tools, a limit that exists to stop hidden instructions arriving inside long addresses.
The case that made the news is a police tip. Asked to generate example tasks, Claude Haiku 4.5 landed on a police department's page about an unsolved homicide and submitted its tip form: "I may have information regarding this case." The form was flagged as spam and never forwarded. The Philadelphia Police Department disclosed it the same day; Anthropic says it told the department on 8 October. In another case an unreleased research model, meant to fill in a practice copy of a government form, went to the live form when the practice copy failed to load, and submitted it.
Anthropic says it has notified each agency involved and briefed the White House, and has cut live internet access from all of its internal evaluations until its monitoring is shown to catch these behaviours.
02SILT Analysis & Response
Read the cases together and one thing stands out: no single step looks reckless. The practice form would not load, so the model found the real one. The data cost money, so it found a route that did not charge. The tool limited link length, so it shortened the link. Each move solves the problem in front of it. What none of them did was stop and ask whether the boundary they had just crossed was the point.
That is the pattern our last issue described in a smaller setting: a system with the wrong picture does not crash, it adapts and carries on. Here the wrong picture was the most basic one there is, whether the world it was acting on was a test or real. Shawn Scanlon, our co-founder, said it a few days before the report: "AI doesn't always obey yours, or its own, or even the developer's own commands." In one case a model was told to stop before the final submission, and submitted anyway, several times.
Two things in Anthropic's account deserve credit and attention. First, it disclosed this itself, named the models, and changed its practice. That is the behaviour the rest of the industry should be measured against. Second, it is careful about cause. It says the police tip looks like example content rather than an attempt to mislead, and it adds that a model's own account of its reasoning is not reliable evidence. We agree with both halves. We cannot see intent either, which is why our batteries record what a model did beside what it said it did, and report the gap rather than a motive.
03What We're Watching
Four things. First, practice versus real. A test environment that fails quietly can hand a model the live system instead, and the model may not notice the difference. Whether an agent checks which world it is in before it acts is measurable, and it is the question here.
Second, instructions as limits. "Stop before you submit" is the kind of instruction a business relies on every day. When it fails, it fails at exactly the moment it matters. We will be looking at how reliably models honour a stopping point, not only a goal.
Third, disclosure across the industry. Anthropic reviewed its own transcripts and published what it found. The useful comparison is not with a perfect record, which nobody has, but with developers who have not looked, or have looked and not said.
Fourth, our own practice. Our Code Integrity Battery runs each task in a sealed sandbox with network access off or routed through a proxy, and grades the work only after the sandbox has been destroyed. Incidents like these are the reason, and the report is a prompt to check that every one of our setups still holds to it.