An AI system incident is an event in which a system caused, or could have caused, harm – it made an incorrect decision affecting a person, exposed data it should not have, or failed significantly in accuracy. A company needs a simple procedure that covers four steps: where the incident is reported and within what deadline, how the impact is immediately mitigated, including the option to suspend the system, how the cause is established and what corrective action is taken, and how the event is logged. The incident log is also one of the first documents an auditor asks for, and an empty log after a year of operation looks like a process that does not work, not like a flawless system. For serious incidents, a reporting obligation to the authorities may also arise.
See also: AI system monitoring, Liability for damage caused by AI, Internal AI usage policy.
