A Georgia Court of Appeals panel has vacated part of a Henry County Schools race discrimination ruling after finding the trial court relied on a proposed order containing fabricated legal citations, adding to a growing list of cases raising concerns about the use of artificial intelligence in Georgia courtrooms
The case stems from three related lawsuits filed by former Black Henry County Schools employees who alleged race discrimination
In a June 10 opinion, the Court of Appeals said the trial court’s order contained multiple legal errors, including what it described as “hallucinated” case law by generative AI
Legal experts say so-called AI “hallucinations” remain one of the biggest risks of generative AI
Large language models can produce text that appears authoritative but includes nonexistent cases, statutes or quotations unless <a href="https://todaytrendnews7.com/glp-1-users-should-mind-their-nutrition-not-just-weight/” title=”GLP-1 users should mind their nutrition, not just weight”>users independently verify every citation
“They call them hallucinations. Large language models are basically really good guessers. They guess the next word that someone wants. So when you’re talking with Chat-GPT, it’s kind of guessing what you want,” said AI legal expert Cat Casey.
“It’s also kind of a people pleaser. So sometimes the answer it gives you is not quite accurate, right?”
The appellate court ultimately reversed portions of the lower court’s decision, vacated the ruling on the federal Title VII claims and sent the case back for reconsideration under the proper legal standards.
The underlying lawsuits were brought by three former Henry County Schools employees who alleged employment discrimination and other claims. The school district appealed after a trial judge denied motions seeking dismissal of the lawsuits.
But rather than focusing solely on the legal dispute, the Court of Appeals devoted part of its opinion to a growing problem facing courts nationwide: attorneys using generative AI without verifying the information it produces
The judges wrote that the trial court’s order — drafted by the plaintiffs’ attorney before being signed by the judge — contained several mistakes, including reliance on at least one hallucinated case. The court called it “troubling” that such an order was submitted and “regrettable” that the trial court failed to identify the errors before issuing it.
The panel also reminded attorneys and judges of their professional responsibilities when using artificial intelligence in legal work, noting that courts cannot rely on fabricated authorities regardless of how they were created.
The decision is the latest in a string of Georgia cases involving AI-generated legal citations
Earlier this year, the Georgia Supreme Court addressed another case involving fabricated legal citations generated through artificial intelligence, prompting additional scrutiny of how attorneys use AI-assisted drafting tools.
The Court of Appeals also referenced that case in its Henry County opinion while encouraging trial courts to carefully review materials submitted for adoption.
According to French legal researcher Damien Charlotin, more than 1,600 AI hallucination incidents- with over 22 in Georgia – have been documented worldwide across courts and arbitration proceedings
CBS News Atlanta reached out to the Georgia Bar Association seeking comment on whether the Bar has established a formal disciplinary process for cases like the one in Henry County.
We didn’t receive a comment by the time of publication, but we will be sure to provide an update if one is given
