AI Incident: What an Organization Must Be Able to Show When Its AI System Fails

What Is an AI Incident, and Why Does Evidence Come First?

An AI incident is defined by the harm it leads to, so every claim about it depends on evidence of what the system actually did.

The OECD defines it this way: “An AI incident is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to any of the following harms: (a) injury or harm to the health of a person or groups of people; (b) disruption of the management and operation of critical infrastructure; (c) violations of human rights or a breach of obligations under the applicable law intended to protect fundamental, labour and intellectual property rights; (d) harm to property, communities or the environment” (OECD, Defining AI Incidents and Related Terms, May 2024). Because the definition turns on what the system led to, TrueScreen concentrates on the part that is hardest to rebuild afterward: certifying what the chatbot or agent showed and did while the session was still open.

The same paper draws a line between incidents and hazards. An AI hazard is an event of the same kind where the AI “could plausibly lead to an AI incident” but no harm has occurred yet, which is why near misses are worth recording too.

The volume of AI failures being logged keeps climbing. The Stanford AI Index 2026 reports that the AI Incident Database recorded 362 incidents in 2025, up from 233 in 2024 and from fewer than 100 a year until 2022, while the OECD AI Incidents Monitor peaked at 435 incidents in January 2026. Among organizations that reported AI incidents, the share with three to five of them went from 30% to 50%, and the share rating their own response “excellent” fell from 28% in 2024 to 18% in 2025.

Each part of that definition is a claim someone will have to support: the harm, the AI system and the link between them. When a single chatbot answer turns into a dispute, as in the Air Canada chatbot case, the first argument is about what the bot said. An AI agent incident raises the same question with higher stakes, because the agent acts as well as speaks.

What Regulators and Banks Now Expect After an AI Incident

Regulators and banks are writing AI incidents into reporting duties and contract terms, and each of those texts assumes that someone can later produce evidence of what happened.

Under Article 73 of the EU AI Act, providers of high-risk AI systems must report a serious incident to the market surveillance authorities of the Member States where it occurred. The report is due immediately after the provider has established a causal link, or its reasonable likelihood, and in any event within 15 days of the provider or, where applicable, the deployer becoming aware of the incident. The deadline is 2 days for a widespread infringement or a serious and irreversible disruption of critical infrastructure, and 10 days in case of death, and an initial, incomplete report may be followed by a complete one.

Article 3(49) of the same regulation defines a serious incident as an incident or malfunctioning of an AI system that directly or indirectly leads to the death of a person or serious harm to a person’s health, a serious and irreversible disruption of the management or operation of critical infrastructure, the infringement of obligations under Union law intended to protect fundamental rights, or serious harm to property or the environment. The Digital Omnibus on AI, Regulation (EU) 2026/1744 of July 8, 2026, in force since July 27, 2026, moved the application of the high-risk rules to December 2, 2027 for Annex III systems and to August 2, 2028 for Annex I systems, so check which date applies to your system. The AI Act record-keeping requirements describe the logs that sit alongside these duties.

Data protection law offers a useful parallel, even though an AI incident is not necessarily a personal data breach. Under Article 33 of the GDPR, a controller must notify a personal data breach to the supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware of it, unless the breach is unlikely to result in a risk to individuals. Every breach must be documented, including the facts, its effects and the remedial action taken (Article 33(5)), and that ordering of facts first, then effects and remedies, is exactly what a breach notification must prove.

In the United Kingdom, a government amendment to the Cyber Security and Resilience (Network and Information Systems) Bill would let ministers make regulations on agentic AI incidents, after consultation and with Parliament’s approval. The DSIT factsheet on agentic AI resilience measures describes a duty for entities that reported an incident to “preserve information”, a duty to inform other organizations affected by the agent, and incident reviews by technical experts within government, who could request information and conduct tests. These measures sit in a bill amendment and are not yet law. Separately, the CMA, FCA, ICO and Ofcom stated in a joint statement of the Digital Regulation Cooperation Forum that “existing legal and regulatory frameworks apply to personal AI agents” and that “An AI agent’s autonomy does not diminish the responsibility of organisations that deploy it.”

Banks are moving through their contracts. Wells Fargo’s updated online access agreement, which will take effect on November 24, 2026, adds a section on third-party AI agents: under Section 15(d), instructions given through an AI agent “are deemed to have been initiated and authorized directly by you, without further verification”, and under Section 15(f), a customer who revokes an agent “must retain evidence of such revocation and must provide such evidence to us, at our request”, as American Banker reported. The broader question of liability for AI agents is still being worked out, and in this agreement evidence of agent activity becomes a contract term.

Whether a specific event counts as a serious incident, and whether it triggers a notification under any of these frameworks, is a question to put to your legal counsel.

Why Screenshots, Transcripts and Vendor Reports Fall Short as Evidence

Screenshots, copied transcripts and vendor reports all pass through someone with a stake in the outcome, and none of them lets a third party check on its own what the system displayed or did.

A screenshot shows what someone says the system displayed. It cannot show which page it came from, whether the conversation went on, or whether the image was edited before it reached you, which is why screenshots are challenged in court. A transcript copied out of a support console has the same weakness, and a vendor or lab self-report comes from the party whose system is under scrutiny.

TrueScreen works on the last row of that table: every certified capture travels with its own technical report and file fingerprints, so the person who receives it checks the package itself instead of the reputation of whoever sent it.

Feature

Forensic Browser: capture, analyse and certify any online content

With TrueScreen, the chatbot session or agent console where the failure happened becomes certified evidence that anyone can verify.

Discover more →

What an Organization Must Be Able to Show

After an AI incident you need a record of what the system displayed, what it did, when, in which version and who saw it, kept apart from anyone’s explanation of why it happened.

AI incident documentation starts with the facts that are hardest to rebuild once the system has been patched, the conversation archived or the agent’s session closed. For the chatbot that gave harmful instructions and for the agent that executed a disputed payment, the record should cover:

• The full exchange as the user saw it, including warnings, links and any content that changed while the conversation went on.
• The actions the system took on the user’s behalf, such as payments, bookings, account changes or messages sent, and the confirmations that followed.
• The date and time of each step, tied to a clock that someone other than you can check.
• The model version, system prompt or settings in use, where you know them.
• The people who saw it: the customer, the employee who took over and, for agents, the identity under which the agent acted, which is the subject of how AI agent identity is established.

The line between fact and interpretation matters as much as the facts. A record stating that the chatbot displayed a given set of instructions at 2:02 p.m. is a fact, while “the model hallucinated because of a retrieval error” is an interpretation that belongs in a separate root-cause analysis, one you can revise without touching the evidence. TrueScreen draws that same line for logs: a log certified as a file is fixed in content and date from the moment of certification, and how the log was produced, or whether the system behaved correctly, remain questions to answer separately.

“Document everything” is the wrong instinct for AI incident response. Incident records may be requested in later proceedings and often contain personal data, so preserve what is relevant, with legal advice on scope and retention. The NIST AI Risk Management Framework states the expectation in MANAGE 4.3: “Processes for tracking, responding to, and recovering from incidents and errors are followed and documented.” What turns preserved material into an AI audit trail that holds up is a documented chain of custody from capture to handover, showing who captured each item, when, and how it was kept unchanged. Without that record, AI accountability remains a line in a policy document.

How Does TrueScreen Certify an AI Incident as It Happens?

TrueScreen certifies an AI incident as it happens by acquiring what the chatbot or agent displayed on screen with a forensic methodology, before the session closes or the system is changed, and by certifying its logs and data as files. The acquisition runs in a controlled environment, the data is verified, a technical report documents how and when each capture was made, and the result is certified with legal value through a qualified electronic seal and a qualified timestamp.

The limits are part of the method. Certification attests how, when and where content was acquired, not that what the chatbot said was true or that anyone’s reading of the incident is correct. TrueScreen does not detect, prevent or monitor AI incidents; it does not make an organization compliant with the AI Act, and it does not replace the forensic expert who analyzes the system. What it gives that expert, and the authority, insurer or court behind them, is a fixed starting point for AI incident evidence that nobody has to take on trust.

1. 1

   Capture the Web Session Where the System Failed

   A chatbot conversation or an agent console that runs as a web page is web content, and it is captured with the Forensic Browser, a desktop app for macOS and Windows. The package contains the session video, the page sources and web archives, the network capture when enabled, the TLS certificates of the sites, a clock check, SHA-256 and SHA-512 fingerprints of every file in a manifest, a certified report with the date of acquisition and a README for the recipient. Desktop software that is not a web page falls outside the web modes and is captured with the screen recording of the Portal.

2. 2

   Record What Appears on the Phone Screen

   When the failure happened inside a mobile app, the screen recording of the TrueScreen App captures what appears on the display while you scroll through the conversation or the agent’s steps. The recording certifies what the screen showed and contains no network traffic.

3. 3

   Certify Exported Logs as Files

   Logs exported from the chatbot platform or the agent framework can be certified as files, which fixes their content and their certain date from the moment of certification. This step does not prove how the log was produced, nor that the AI system worked correctly. When the file cannot leave your organization, its fingerprint (hash) can be certified instead, without sending the file. Data produced by an AI agent can also be certified directly through the TrueScreen API and MCP server.

4. 4

   Hand Over a Package Anyone Can Verify

   Each certification produces a self-contained package. Anyone who receives it can verify integrity and seal without TrueScreen and without trusting whoever handed it over, using the sealed XML, the fingerprints and the technical report, and a public check is available on the certification verification page. Content certified with TrueScreen carries full legal and evidentiary value worldwide, and the weight of each piece of evidence is assessed case by case by the court or the authority that decides.

Conclusion