FLRA: HHS Brief's AI Citations 83% Unreliable

• The FLRA rejected all of HHS's exceptions to an arbitration award concerning a return-to-office mandate.
• Over 80% of the citations in HHS's brief were found to be fictitious, erroneous, or irrelevant.
• The FLRA suggested that generative artificial intelligence might have been used, cautioning about its significant risks.
• This incident highlights the critical need for meticulous human verification of AI-generated legal content to ensure accuracy and compliance.
• Similar issues with non-existent or unsupportive citations have been noted in other HHS documents and grant solicitations.

The Federal Labor Relations Authority's recent rejection of the Department of Health and Human Services' brief due to pervasive fictitious and erroneous citations underscores the critical need for meticulous human verification of AI-generated content in legal filings.

The Federal Labor Relations Authority (FLRA) recently issued a decisive ruling against the Department of Health and Human Services (HHS), rejecting all of its exceptions to an earlier arbitration award. This award had mandated HHS to reverse its return-to-office directive, which had unilaterally rescinded existing flexible work agreements for its employees. The National Treasury Employees Union had challenged the department's move on behalf of its members, leading to the arbitration.

While the merits of the case were not close, the FLRA's decision gained significant attention for a different reason: the alarming inaccuracy of the citations presented in HHS's brief. The Authority found that a staggering 22 out of the brief's 29 footnotes, and 35 out of 42 legal or record citations, were either fictitious, erroneous, or irrelevant. Specifically, nine citations were entirely fabricated, eleven failed to support the propositions for which they were cited, and ten contained partially incorrect information or did not support the claims made.

This extensive pattern of inaccuracies led the FLRA to suggest that HHS might have utilized a generative artificial intelligence (AI) tool in drafting its submission. The brief was deemed approximately 83 percent unreliable by citation count. This incident far surpasses the notoriety of the infamous Mata v. Avianca filing, which involved six fake cases, whereas HHS's brief included at least ten flatly fictitious citations, alongside dozens of others that were either incorrect or misrepresented their purported support.

The underlying dispute centered on HHS's attempt to force employees back to their physical workplaces, a move that directly conflicted with existing flexible work agreements. HHS's legal theory posited that a presidential memorandum superseded a five-year collective bargaining agreement. A top labor relations official for HHS even testified that 'the president is our chief,' attempting to justify the department's actions.

However, the FLRA's ruling indicated that this argument lacked merit, particularly given that even the presidential memo from the Trump administration regarding return-to-office policies stipulated implementation 'consistent with applicable law,' implicitly acknowledging contractual obligations. The FLRA, while not prohibiting the use of AI tools in legal filings, issued a clear caution. It emphasized that AI presents significant risks of error and stressed the imperative for parties to meticulously verify the accuracy of any AI-generated material to ensure compliance with all regulatory requirements.

This cautionary stance highlights the ethical and professional obligations of legal practitioners to ensure the veracity of all information presented to adjudicative bodies. The widespread nature of the citation errors in the HHS brief underscores a critical failure in human oversight, regardless of whether AI was the direct cause or merely a contributing factor to the inaccuracies.

The FLRA's rejection of the Department of Health and Human Services' brief due to pervasive fictitious and erroneous citations underscores the critical need for meticulous human verification of AI-generated content in legal filings. This case serves as a stark reminder of the potential for AI hallucinations in legal documents, which can severely undermine credibility and lead to adverse legal outcomes. The incident highlights a broader concern within the legal community regarding citation accuracy and legal ethics in an era of increasing reliance on advanced technological tools.

This is not an isolated incident for HHS. Previous reports have documented similar issues, such as a 2023 report that cited non-existent studies, with tell-tale 'oaicite' tags appearing in URLs, which the White House initially dismissed as 'formatting issues.' More recently, Judge Christopher Cooper enjoined an HHS teen pregnancy grant overhaul after discovering that the solicitation relied on studies that 'appear either not to exist or not to support the propositions,' a characteristic he identified as a 'hallmark of AI-generated citations.'

These repeated instances suggest a systemic issue within HHS regarding the verification of research and legal citations, whether generated by AI or human error. The implications extend beyond the immediate case, signaling to all legal professionals and compliance officers the absolute necessity of rigorous fact-checking and human oversight when integrating AI into legal drafting processes to prevent severe reputational damage and potential sanctions.

Lawyers and compliance officers must meticulously verify all AI-generated content in legal filings to avoid severe credibility issues and potential sanctions, as demonstrated by the FLRA's rejection of HHS's brief due to pervasive fictitious and erroneous citations. This case underscores the critical need for human oversight in AI-assisted legal drafting to ensure accuracy and compliance with regulatory requirements.

Source: Original reporting via Above the Law