$2,000 per hour! Top law firm "admits fault": AI tool "fabricates statutes and cases"

$2,000 per hour! Top law firm "admits fault": AI tool "fabricates statutes and cases"

A top law firm where partners earn over $2,000 per hour was forced to apologize to a judge after AI-generated “hallucinations” led to false legal citations in court filings.

According to the Financial Times, the prestigious law firm Sullivan & Cromwell (S&C) admitted in a high-profile bankruptcy case that multiple errors in its court documents were generated by AI software, including incorrect legal references and case citations, igniting new concerns about the risks of using AI tools within the legal sector.

Andrew Dietderich, head of S&C’s restructuring practice, formally apologized to New York federal judge Martin Glenn last Saturday for errors in the firm’s April 9 filing, including misquoted U.S. Bankruptcy Code sections and improper case citations. “We deeply regret this,” he wrote in the letter. Dietderich also stated that the document in question was not prepared in accordance with the firm's internal policies regarding AI use, and said the firm is assessing whether “further improvements” to internal training and review processes are needed.

The AI errors were first discovered by opposing counsel Boies Schiller Flexner (BSF). In a filing submitted last week, BSF pointed out that the language cited in S&C’s motion “does not appear in Chapter 15 of the U.S. Bankruptcy Code,” and highlighted several instances of “incorrect citation or misidentification.” BSF also noted that one case cited by S&C “does not actually exist”, and the referenced citation was actually from a different circuit court ruling.

In response to the criticism, S&C told the court that the firm maintains “strict” standards for using AI tools, requiring lawyers to “trust nothing, verify everything”; failure to verify AI-generated content “constitutes a breach of company policy.” However, this incident reveals the internal policy was not effectively implemented.

This incident is hardly an isolated case. Last year, Latham & Watkins admitted that a lawyer used Anthropic’s Claude model to help draft documents, resulting in fabricated journal article titles and author information. A federal appeals court in New Orleans fined a lawyer $2,500 for submitting a brief containing 21 errors that were either generated or altered by AI.

Notably, BSF also has similar past experience. According to informed sources, former BSF partner John Kucera admitted in September last year that filings he prepared using AI tools in a case against Amazon contained “significant citation errors” due to “failure to verify” details, and said he was “ashamed and deeply regretful” about these mistakes.

Sources revealed that S&C holds an enterprise license for ChatGPT. S&C also told the judge that other “non-substantive and/or clerical errors” were found in other documents in the case during review, clarifying these were human errors, not caused by AI. WilmerHale attorney Jeffrey Dennhardt previously noted that there is an ongoing “arms race” in the legal industry, but “a degree of skepticism remains.” This incident may intensify scrutiny of the reliability of AI tools in the sector.

Risk warning and disclaimerThe market contains risks, and investments should be made with caution. This article does not constitute personal investment advice, nor does it take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, viewpoints, or conclusions in this article are suited to their particular circumstances. Investing based on this information is at your own risk.