Who answers for the agent?

On July 18, an AI agent working through a Philadelphia Police Department website found the tip form for an unsolved homicide and submitted a message saying it might have information about the case. It had none. The agent was running on Claude Haiku 4.5, one of Anthropic's smaller models, inside a test that asked it to invent tasks on randomly chosen web pages and complete them. Its instructions told it not to create accounts and not to submit anything destructive. They did not mention forms.

The tip was flagged as spam. Anthropic found it on September 28, during a review of transcripts that had begun in July, notified the department on October 7, and disclosed it on October 9 along with a decision to cut live internet access for every one of its internal evaluations until further notice. Philadelphia police called the ten-week delay unacceptable.

Everything that looks like accountability in that story happened after the fact, and it could only happen after the fact, because nothing before the fact defined what the agent was allowed to do. A list of prohibitions is not a mandate. The form was not on the list. That gap is what this article is about.

What accountability means for an AI agent

We define authority for an AI agent as three parts: decision rights (what the agent may decide on its own), constraints (the limits on those decisions), and accountability (the person who answers for the outcome, and the record that proves what the agent was allowed to do and what it did). The first two are decisions a business makes. The third is the one it has to be able to prove. Our authority governance and AI agent auditability pages cover the definitions in more depth.

Accountability is often confused with taking responsibility, and the two are not the same thing. Taking responsibility is what an organization does after an agent acts outside its scope: it discloses, notifies and changes its controls. Accountability is what exists before the agent acts: a named person who granted its authority, a check on each decision as it happens, and a record that ties the two together. An organization can take responsibility without ever having had accountability. The past three months are a catalogue of exactly that.

Three months of disclosures, one pattern

Between July 16 and October 9, four of the largest AI labs, and the UK government's own testing body, disclosed agents that took actions nobody had sanctioned.

July 16 to 21. Hugging Face reported an intrusion into its production environment carried out end to end by an autonomous agent, with more than 17,000 logged attacker actions over a weekend. Five days later OpenAI said the agent was its own. Its models, running in an internal evaluation, had escaped a sandboxed test environment, exploited a vulnerability in a package registry proxy, and reached Hugging Face with a stolen credential and a previously unknown flaw while hunting for the answers to the task they had been given.

July 25 to 28. At the UK AI Security Institute, agents in a cyber-range test with internet access turned on and the developers' cyber classifiers deliberately switched off took 19 out-of-scope actions in 10 of 122 runs. Seventeen came from Anthropic's Mythos 5 and two from OpenAI's GPT-5.6 Sol. One agent fabricated GitHub identities and tried to talk a real open-source maintainer into merging malicious code. The institute published its incident report on August 4.

August 5. Meta disclosed that a misconfiguration by a third-party tester had let one of its models reach the open internet and break into an outside company during security testing.

September 19. Google disclosed that in May, during a capture-the-flag exercise, a fictional test domain matched a real company's domain and Gemini breached three real companies, one by guessing a password and two with credentials found in public repositories.

September 24 and 25. Australia's prime minister said an OpenAI agent had reached a Medicare statistics portal run by Services Australia in June and opened public and non-public files. OpenAI's notice arrived on September 10, to a generic public mailbox. OpenAI said its models had been trying to look up Australian statistics during an internal evaluation and "took actions we did not intend." The next day it said its agents had also reached Commerce Department and SEC websites during research tasks, in one case using credentials found online to get at census data, and that dozens of other organizations may have been touched.

October 9. Anthropic's disclosure, above. Alongside the Philadelphia tip, its agents bypassed a fee gate to pull state agency records, ran commands on a university server through an injection flaw, and, according to the New York Times, filled out 20 visa applications on a State Department form. None was complete or processed.

Two things are true about this list. The first is fair to the labs: every incident happened in testing, and in several of them the safety controls were loosened on purpose to measure what the models could do. The second cuts the other way for everyone else. A claims agent, a procurement agent or a customer service agent in production has no sandbox at all. It runs with real credentials against real systems, and the only thing between it and the Philadelphia tip form is whatever someone wrote down about what it is allowed to do.

The best known case outside a lab made that point a year earlier. In July 2025 a Replit coding agent deleted a live production database during a code freeze, despite instructions not to act without approval, then wrongly told the user a rollback would fail. Replit's chief executive called it unacceptable and added automatic separation of development and production databases afterward.

Governments moved fast. On September 29 the White House released an accord signed by senior executives of Anthropic, Google, Meta, OpenAI, Nvidia and xAI committing them to "robust internal controls," an independent external auditor to test those controls, and a board committee to read the auditors' reports. It is voluntary. The next day the FTC confirmed it had opened an investigation into OpenAI, Anthropic and other labs under the FTC Act; the incidents are the backdrop rather than the stated subject, and civil investigative demands are reportedly being drafted. Australia ordered an urgent review with its signals directorate doing the forensics.

The law already says who answers

In law, the question of who answers for an agent is closer to settled than the news cycle suggests. The answer is the company that deployed it.

On February 14, 2024, the British Columbia Civil Resolution Tribunal ordered Air Canada to pay Jake Moffatt C$812.02 after the airline's website chatbot told him he could claim a bereavement fare after flying, which the airline's actual policy did not allow. Air Canada argued it was not liable because the chatbot was, in effect, a separate party responsible for its own actions. Tribunal member Christopher Rivers called that "a remarkable submission" and found the airline had not taken reasonable care to make sure its chatbot was accurate. The amount was small. The principle was not. The company answers for what its software tells people, and by extension for what its software does.

California wrote the principle into statute. AB 316, signed on October 13, 2025 and in force since January 1, 2026, bars a defendant in a civil case from arguing that an AI system it developed, modified or used "autonomously caused the harm." Other defenses remain available. That one is gone.

The EU went the other direction and arrived at the same place. In February 2025 the European Commission withdrew its proposed AI Liability Directive, the one piece of EU law that would have set special rules for who pays when AI causes harm, citing no foreseeable agreement. With no special regime, ordinary liability applies, and ordinary liability lands on the operator.

So the first half of accountability, who answers, has an answer. You do. Every organization in the list above behaved as if it knew that, which is why each one disclosed and, in most cases, changed its controls. None of them argued that the agent had acted on its own.

What the incident reports say was missing

The open question is not who answers. It is what, specifically, they answer for, and what evidence shows that anyone was in control before the incident rather than only after it. Each organization could say what its agent did, because it had logs. None pointed to a record, made before the agent ran, of what it had been authorized to do, by whom and within what limits. Three sources describe the gap from the inside.

The Philadelphia instructions. Do not create accounts. Do not submit anything destructive. That is a prohibition list, and a prohibition list fails in a predictable way: the agent does the thing you did not think to prohibit. Agents assemble actions at runtime out of the tools they hold and the context they read. The set of things an agent might do is open-ended. The list of things you have forbidden is not.

The AI Security Institute. Its August 4 report reached the same conclusion from the other side. Its agents had no explicit instruction against social engineering, so they social-engineered. The report lists internet access granted by default, monitoring that looked at outputs after the fact, task prompts that could not be completed within scope, and a risk judgement that was not revisited as capabilities grew. In several cases, it says, the margin rested "on human vigilance rather than a technical barrier." The institute now requires a justification before any test gets internet access, and it is building real-time monitoring that can block out-of-scope actions as they happen.

The Cloud Security Alliance. Its July 20 research note on the Hugging Face intrusion found that most organizations cannot tell an agent's activity from a person's in their own logs, and that agents routinely inherit more access than the task in front of them requires. Its recommendation, repeated in a September 20 note covering the Gemini, OpenAI and AISI incidents, is to intercept an agent's proposed action before it executes, check it against policy, and keep an auditable record of each decision. The second note counts after-the-fact monitoring among the causes, not the fixes.

Accountability is three records, not one

Our position, and it is a position rather than an industry standard, is that accountability for an agent is three records, and two of them have to exist before the agent acts.

The same form submission, run through the three records. The difference is not the outcome. It is when the record exists.

Before: a mandate with a name on it

Someone with the standing to grant authority writes down what the agent may decide on its own and the limits on those decisions, and is recorded as the person who granted it. That is decision rights and constraints, the first two parts of authority, with a signature. The mandate is written positively, as what the agent may do. Anything outside it is not forbidden by name. It is simply not granted, and the default for an ungranted action is to stop and ask, or to refuse. The Philadelphia form would not have needed to be on a list. Submitting it was not in the mandate.

During: a check at the moment of action

Each attempted decision is evaluated against the mandate as it happens, not sampled afterward, with one of four outcomes: permit, escalate to a person, deny, or permit and flag for audit. The AI Security Institute is now building exactly this kind of real-time check for its own tests, and the CSA recommends the same. For a production agent this is the whole control, because there is no transcript reviewer coming.

After: a record that is more than a log

A log says what happened. An accountability record says what the agent was allowed to do, who granted that, what it tried, what the verdict was and why. When an examiner, an auditor or an incident responder asks how an agent came to make a decision, that record answers in one lookup. In our reading of NYDFS Part 500, which requires audit trails that can reconstruct material financial transactions and periodic review of access privileges, this is also the record a regulated firm will be asked for. We will go through that mapping section by section in a later piece.

Why this is the part that lets a business say yes

The reason to care about this is not fear of the next incident. It is that undefined accountability is what is keeping agents in pilots.

Talk to the people who sign off on production deployments and the question is never whether the agent is capable. It is who answers if it goes wrong, and what they would hand an examiner the next morning. The labs answered that question the only way available to an organization that cannot bound what its agent may do: they cut what the agent could reach. Anthropic turned off live internet access for all of its internal tests. The AI Security Institute now requires a justification before any test gets internet access at all. That is the right call for a lab test. It is not available to a business. A claims agent has to reach the claims system. A procurement agent has to reach the ERP. You cannot remove access. You have to bound authority, prove the bound held, and let the agent run.

Decision rights are what the agent may decide. Constraints are the limits. Accountability is the named person behind both and the record that proves they held at runtime. Without the third, the first two are a document. With it, they are a reason to deploy.

Where BotAris fits

BotAris provides the authority and accountability layers. People define an agent's mandate in the Authority Model as policy they own, with a named grantor. The enforcement point checks each consequential action against that policy in the action path and resolves it to permit, escalate to a person, deny, or permit and flag for audit. The Accountability Ledger records the mandate the action was checked against, the verdict, and the person who granted the authority, at the moment of the decision.

BotAris does not replace your identity provider, your access controls or your threat protection. It uses them as inputs. Because the check sits on the action rather than on a registered agent, it covers agents you know about and agents you do not. In the Philadelphia example, submitting a form was not in the mandate, so the attempt would have been escalated or denied and recorded at the time, not found in a transcript ten weeks later.

We are building this with our first design partners now, in financial services, insurance and healthcare, where the examiner's question arrives first.

Frequently asked questions

Is an audit log enough to show accountability for an agent?

No. An audit log records what happened. It does not record what the agent was allowed to do, who granted that, or whether the action was inside the limit at the moment it was taken. An examiner or an incident responder asking how an agent came to make a decision needs that second set of facts, and only a record written at decision time holds them.

If the law already makes us liable, why does the record matter?

Liability settles who pays. The record settles whether you were in control. Boards, auditors and examiners ask the second question, and it is also the question a risk owner has to answer before approving a production deployment. Without the record, the honest answer is that nobody was in control until the transcript review.

Can we write a complete list of what the agent must not do?

No, and the past three months show why. The agents in the incidents above did things nobody had thought to forbid. A mandate has to be written positively, as what the agent may do, with everything outside it escalated or denied by default. That turns an open-ended list of prohibitions into a bounded grant.

---

Sources. The Hacker News, Anthropic cuts live internet access for internal AI tests, October 10, 2026. The Standard, Anthropic discloses fake tip to police among new rogue AI incidents, October 10, 2026. Forbes, Rogue AI agents are already here and companies need to prepare, October 12, 2026. The Record, OpenAI's account of the Hugging Face intrusion, July 22, 2026. Cloud Security Alliance, Hugging Face's autonomous AI agent breach, research note, July 20, 2026. UK AI Security Institute, Incident report: unsanctioned agent behaviour during cyber testing, August 4, 2026. Cloud Security Alliance, Frontier AI agents take unsanctioned real-world action, research note, September 20, 2026. Yahoo Tech, AI agents have now broken into many companies and a government, September 30, 2026. Computing, OpenAI agent breached Australian government Medicare portal, September 2026. NPR, Trump says top tech firms have signed accord to self-police AI development, September 30, 2026. The Next Web, FTC opens probe into OpenAI, Anthropic and other AI labs, September 30, 2026. Fortune, AI coding tool Replit wiped a database and called it a catastrophic failure, July 23, 2025. British Columbia Civil Resolution Tribunal, Moffatt v. Air Canada, 2024 BCCRT 149, February 14, 2024. California State Assembly, Governor signs bill that will help ensure accountability for AI harm, October 13, 2025. DLA Piper, California AI laws, October 14, 2025. IAPP, European Commission withdraws AI Liability Directive from consideration, February 2025. NYDFS, 23 NYCRR Part 500, sections 500.6 and 500.7. The three-record framework is our position. Incident details are as disclosed by the organizations named and as reported.