A California prosecutors’ office used artificial intelligence to file a motion in at least one criminal case, which contained errors known as “hallucinations”.
A prosecutor at the Nevada county district attorney’s office in northern California “recently used artificial intelligence in preparing a filing, which resulted in an inaccurate citation,” district attorney Jesse Wilson said in a statement to the Sacramento Bee. “Once the error was discovered, the filing was immediately withdrawn.”
Defense and civil rights attorneys argue the prosecutors’ office used artificial intelligence in other criminal court filings.
In October, lawyers for Kyle Kjoller filed a motion with the third district court of appeal, calling for prosecutors to be subjected to sanctions over a number of errors in the prosecution’s filings in Kjoller’s case. Kjoller is being represented by a public defender and the non-profit Civil Rights Corps.
The appeals court denied the sanction request without explanation. Then, Kjoller’s lawyers identified similar errors in a filing by the prosecutors’ office in another case. The defense filed another appeal, which was denied when Kjoller was convicted in his criminal case.
Kjoller’s lawyers then filed a petition with the California supreme court, identifying three cases they say contain errors typical of generative AI. The court has not yet issued a decision on whether it will take up the case.
“Prosecutors’ reliance on inaccurate legal authority can violate ethical rules, and represents an existential threat to the due process rights of criminal defendants and the legitimacy of the courts,” Kjoller’s attorneys wrote. Their filing included examples of instances where the prosecutor’s brief cited nonexistent quotations or misinterpreted court rulings – errors common for artificial intelligence.
On Friday, a group of 22 scholars, lawyers and criminal justice advocates filed a brief in support of Kjoller’s case with the California supreme court.
In a statement to the Sacramento Bee, the Nevada county district attorney’s office admitted to using artificial intelligence in one of its filings, but not in Kjoller’s case. District attorney Wilson said other mistakes were due to human error.
“Prosecutors work diligently and in good faith under heavy caseloads and time constraints,” Wilson said. “It cannot now be assumed that every citation error stems from the use of artificial intelligence.”
“At no time was there any intent to mislead the court, and any characterization to the contrary misstates the facts,” he added. After the errors were identified he said, “all of the attorneys in the office were reminded to verify all legal citations independently and not rely on AI-generated material without confirmation from reliable sources.”
In a statement to the New York Times, he added that the office has conducted new trainings for staff and implemented an artificial intelligence policy.
The California case is likely the first instance of a prosecutors’ office using generative AI in a court filing in the United States. Although lawyers from Canada, Australia, the United Kingdom and the United States have been fined for their use of artificial intelligence, those cases have not generally involved the prosecution. Researchers at the business school HEC Paris maintain a database of court cases that have included AI errors, according to the Times. Only one of those, filed in Israel, was written by a prosecutor.
