FTC calls on OpenAI and Anthropic to explain AI incidents
The U.S. regulator is examining whether AI agent testing harmed consumers and whether the companies accurately communicated the risks. Incident reporting is ceasing to be solely a voluntary commitment by providers.
The U.S. Federal Trade Commission (FTC, the competition and consumer protection authority) on Wednesday, October 14 sent OpenAI and Anthropic civil investigative demands (CIDs) to produce documents. The matter was revealed on October 15 by Axios (opens in a new tab), citing a person familiar with the proceedings. According to that source, the demands run to more than 50 pages of questions. They concern safety incidents, internal safeguards, and marketing materials that may have misled consumers. The contents of the demands have not been made public.
The issue concerns incidents from tests of AI agents, meaning models that independently perform tasks on the internet. An Anthropic model submitted 20 visa applications on the U.S. State Department website: one in May and 19 in August. This happened when a test version of the form failed to work. The State Department emphasizes that its systems were not hacked or compromised. In another incident, on July 18 an Anthropic model sent a false murder report to the police in Philadelphia. The company detected this on September 28 and notified the police on October 7. OpenAI agents used several government websites in an unforeseen way.
The next step is expected to be demands sent to the entities targeted by the agents. According to Reuters, this may include the independent research group METR. The collected materials could be used by the FTC in a lawsuit. Commission Chair Andrew Ferguson, who also leads the SI Force (Super Intelligence Force) established by Donald Trump's administration to monitor AI risks, told Reuters in September that companies should be held responsible for harm caused by chatbots. In his view, the FTC can rely on existing regulations, including its powers over companies that fail to disclose data breaches. The SI Force has already ordered AI companies to report incidents immediately upon detection. It is not known what consequences may follow for late reporting.
OpenAI said it would cooperate constructively with the FTC. Anthropic did not comment on the demands. Also on October 14, eight states sued OpenAI, alleging that ChatGPT harms children.
The demands arrived two weeks after, on September 29, the heads of AI companies signed a voluntary agreement at the White House without any sanctions. For companies using OpenAI and Anthropic tools, this means that incident reporting deadlines and providers' safety assurances may come under regulatory requirements.