최근 뉴욕주 항소법원 영상에서 변호사가 생성형 AI가 만들어낸 존재하지 않는 허위 판례와 법 원칙을 소명 자료에 인용하여 판사들로부터 강도 높은 질책을 받는 현장이 포착되었습니다. 이는 법조계 전반에 AI 환각(Hallucination)으로 인한 허위 정보 유포 문제가 얼마나 심각한지를 보여주는 사례입니다. 이 사건은 AI를 업무에 활용하는 모든 전문가에게 정보의 사실관계 검증이 필수적임을 시사합니다.
번역된 본문
최근 몇 년 동안 변호사들이 생성형 AI를 사용하다가 소장에 허위 판례, 인용구 및 기타 중대한 오류를 포함시킨 채 적발된 사례가 잇따르고 있습니다. 이러한 상황은 대개 법원 기록에서 상대방이나 판사가 이를 발견하면서 빚어집니다. 법원 특유의 정중한 표현을 빌려 그들은 모두의 시간을 낭비하고 법조계의 수치를 가져온 변호사들을 꾸짖습니다. 이로 인해 심각한 제재를 받는 경우도 종종 있습니다. 하지만 늘 흥미로운 읽을거리가 되곤 하죠.
지난달 있었던 한 항소 심리에서 법원의 실시간 스트리밍 영상은 AI로 조작된 허위 판례를 인용한 혐의를 받는 변호사가 적발되는 생생한 현장을 고스란히 담아냈습니다. 5월 20일, 뉴욕주 대법원 항소부에서 발렌시아 브레이스웨이트 넬슨(Valerie Brathwaite Nelson) 판사와 헥터 라살레(Hector LaSalle) 판사는 20분이 넘는 시간 동안 해당 변호사와 상대방 변호사를 호되게 질책하며 이 상황을 "충격적이고, 우려스러우며, 실망스럽고, 슬프다"고 표현했습니다.
이 사건의 원고인 주디스 랜드버그(Judith Landberg)는 나무 뿌리로 인해 솟아오른 인도의 비뚤어진 벽돌에 걸려 넘어진 후 뉴욕시를 상대로 소송을 제기했습니다. 이 심리에서 그녀의 변호사인 마이클 샌더스(Michael Sanders)는 '인도'의 정의에 대해 논증을 펼치려 하고 있었습니다. (전체 영상은 링크에서 볼 수 있으며, 허위 판례에 대한 내용은 약 19분 경부터 시작됩니다.)
넬슨 판사는 다음과 같이 말했습니다. "이번 구두 변론을 준비하고 상고인의 요약서를 검토하면서, 법원은 원고가 제출한 요약서에 허위로 보이는 판례가 적어도 3건 인용되어 있다는 사실을 발견했습니다. 이 판례들과 인용된 문구는 어느 것도 실제로 존재하지 않습니다."
샌더스가 존재하지 않는 판례를 인용했을 뿐만 아니라, 넬슨 판사에 따르면 그는 법을 왜곡하는 것으로 보이는 다른 10건의 판례도 추가로 인용했습니다. 넬슨 판사는 "어떻게 설명하시겠습니까?"라고 물었습니다. 샌더스는 해당 인용 문헌들에 대해 발언할 준비가 되어 있지 않다고 말하며 즉시 스스로 구덩이를 파고 들어갔습니다. 그러자 넬슨 판사가 즉시 그의 말을 끊었습니다.
그녀는 이렇게 말했습니다. "말을 계속하기 전에, 변호사 직업윤리 규칙 제3.3조 A항을 지적해 드리겠습니다. 이 규칙에 따르면 변호사는 법정에 사실이나 법에 대한 허위 진술을 고의로 해서는 안 되며, 이전에 자신이 법정에 한 중대한 사실이나 법에 대한 허위 진술을 바로잡지 않아서도 안 됩니다."
그는 더듬거렸습니다. 그는 "잘못된 인용이 있다면 깊이 사과드린다"고 말했습니다. 라살레 판사가 "어디서 그 판례들을 가져온 겁니까?"라고 묻자, 샌더스는 "이 판례들이 구체적으로 무엇인지 모르겠습니다"라고 답했습니다. 라살레 판사와 넬슨 판사는 판례의 출처와 관련하여 몇 분 동안 샌더스를 추궁했습니다.
판사들은 생성형 AI를 직접적으로 언급하지는 않았지만, 변호사들이 소송과 항소 서면을 작성하기 위해 AI를 사용하면서 허위 판례를 포함시키는 일이 점점 전염병처럼 퍼지고 있다는 점을 고려할 때, 그들이 거의 확실하게 암시하고 있었던 것은 AI였습니다. 다른 사건들에서 AI를 사용한 적발된 변호사들은 심한 감기부터 촉박함, 보조사(Paralegal)의 탓까지 온갖 핑계를 댔습니다. 판사들은 대체로 이런 상황에 진절머리가 난 듯했습니다.
라살레 판사는 말했습니다. "참고로 말씀드리는데, 당신이 더 큰 구덩이를 파지 않았으면 해서 말합니다만, 당신은 존재하지 않는 법 원칙을 인용하고 있습니다. 한 가지 말씀드리죠. 우리는 지난주에도 이런 일을 봤습니다. 오늘 준비를 하면서 당신이 이 서면을 읽고 '아, 제가 실수를 했네요, 판사님'이라고 말할 것이라고 희망했습니다. 가끔 그런 실수는 일어나잖아요, 그렇죠? 저는 그것을 바랐습니다. 하지만 우리는 그런 대답을 듣지 못했습니다. 지금 당장 이 판례들을 찾아볼 시간을 줄까요?" 샌더스는 아마 15분보다 더 걸릴 것이라고 답했습니다.
이후 라살레 판사와 넬슨 판사가 번갈아 가며 이 일이 얼마나 매우, 매우 심각한 문제인지를 샌더스에게 각인시키며 공방이 오고 갔습니다. 인도 마주 보인 건물의 소유주를 대리하는 변호사인 로스 프리시아(Ross Friscia)가 판사들 앞에 섰습니다. 그가 말을 시작하려 하자, 라살레 판사는 질책을 아직 끝내지 않은 상태였습니다.
라살레 판사는 프리시아의 말을 가로막으며 말했습니다. "그는 존재하지 않는 항소법원의 기준을 끌어왔습니다. 그는 그것을 자신의 논증의 근거로 삼았고, 상대방인 당신은 우리에게 이 사실을 주지시켜야 한다고 생각하지 않았습니까?" 프리시아가 "저는 그가 인용한 법 원칙을 구체적으로 알아채지 못했습니다"라고 말하며 변명하려던 찰나였다.
In the last few years, we’ve heard case after case where attorneys used generative AI and were caught including fake citations, quotes, and other major errors in their filings. This generally plays out in dockets, where their opponents or judges spot them and, in the polite language of the courts, scold them for wasting everyone’s time and being a disgrace to the legal profession. Sometimes, this results in serious sanctions. But it's always entertaining to read. In an appeal hearing last month, a court’s live stream captured this happening on camera in real time, with an attorney caught for likely using AI-fabricated citations. On May 20, in the Supreme Court of the State of New York Appellate Division, Justices Valerie Brathwaite Nelson and Hector LaSalle reamed out that lawyer and his opposing counsel for more than 20 minutes, calling the situation “striking, concerning, disappointing, and saddening.” The plaintiff in the case, Judith Landberg, is suing the city of New York after she tripped on some askew bricks on the sidewalk that were pushed up by tree roots. In that hearing, her lawyer, Michael Sanders, was attempting to argue the definition of a sidewalk. The full video is here , and the portion about fake citations begins a little after the 19 minute mark. “In preparing for this oral argument and reviewing the brief of appellant, it came to the attention of the court that the brief submitted by plaintiffs cites at least three cases that appeared to be fictitious,” Nelson said. “None of these cases, nor the quoted language, appears to exist.” Not only did Sanders cite cases that don’t exist, Nelson said, he cited 10 other cases that appear to misrepresent the law. “How do you respond?” Nelson asked. Sanders instantly started digging a hole, saying that he wasn’t prepared to speak on those specific citations. Nelson promptly cut him off. “Before you go any further,” she said, “let me point out to you that Rule 3.3 A of the rules of professional conduct indicates that a lawyer shall not knowingly make a false statement of fact or law to a tribunal, or fail to correct a false statement of material fact or law previously made to the tribunal by the lawyer.” He stammered. “If there's any citations that are incorrect, my deepest apologies,” he said. “Where did you get them from?” LaSalle asked. “I don't know what these cases were specifically,” Sanders said. LaSalle and Nelson grilled Sanders for several more minutes about the citations and where he got them. The judges didn’t bring up generative AI specifically, but considering the growing epidemic of lawyers including fake citations while using AI to draft arguments and appeals, it’s almost certainly what they’re alluding to. Attorneys caught using AI in other cases have blamed everything from head colds to being in a rush, to paralegals. Judges, in general, seem sick of it. “Just so you know, because I don't want you to dig a bigger hole here, you're citing principles that don't exist,” LaSalle said. “Let me tell you something. We saw this last week. I was hopeful that, in preparation for today, that you were going to read this and say, 'Oops, we made a mistake, Judge.’ It happens sometimes, right? That's what I was hoping for. We didn't get that. Should we give you some time right now to go look these cases up?” Sanders replied that it would probably take longer than 15 minutes. They went back and forth, with LaSalle and Nelson taking turns trying to impress upon Sanders that this is very, very bad. Ross Friscia, the attorney representing the owner of the property that faces the sidewalk, stood up before the judges next. He started to speak, but LaSalle wasn’t finished with the dressing-down. “He’s raising a court of appeal standard that doesn’t exist,” LaSalle said, interrupting Friscia. “He was using it as a component of his argument, and you didn't think you should bring it to our attention?” “I didn't notice in particular that the principle of law that he was citing was incorrect,” Friscia said. “I'm sorry, I'm going to give you every opportunity to make your argument,” LaSalle said. “But I'm befuddled. I honestly am. I'm absolutely—and I'm not here to—lawyers make mistakes. It's not an easy profession. I don’t want to sit here beating up on lawyers, but we rely on the bar so much in what we do. So the first thing that I did, I don't want to speak for my colleagues, but after seeing what he wrote, when I went to your papers, I expected to see something referencing [...] It wasn't one case, counsel, it was several cases, and you didn't see fit to bring it to our attention either. It's just striking to me.” Friscia, now with the fear of the bar in him, apologized profusely. “Your honor, I apologize to the court. I will do further due diligence going forward from this point on.” “I hope so,” LaSalle said. “You should apologize to your client, not to me.” “Yes, I apologize for that,” Friscia said. “And I will, going forward, check every single case, even if it stands for, you know, general principles of law, like the construed liberally to effectuate remedial purpose, and things like that. I will bring them to the court’s attention.” At this, Nelson jumped in: “The misrepresentations here are of such a degree that they could not merely reflect a difference of opinion,” she said. “As an appellate court attorney, you would have to, if you were doing the work and reading the briefs and responding to the briefs, you would have to notice that something in the wording of the main brief for the appellant was wrong, if not many things being wrong. It's concerning because we are all officers of the court, and there is a responsibility that you also have to notify the court to do the work, notify the court when these types of misrepresentations and fictitious cases and fictitious citations and misrepresenting the holding of a court of appeals case. I could go on and on, but if you read the brief and looked at the cases, you would have realized it was your responsibility also to alert the court.” Friscia said he tailors briefs to respond to specific issues but didn’t keep explaining himself for long; he apologized again, repeated that he’d be more thorough next time, made his point about the city being responsible for the askew bricks, and sat down. Subscribe Join the newsletter to get the latest updates. Success Great! Check your inbox and click the link. Error Please enter a valid email address. Next up was Elizabeth Freedman, an attorney representing the City of New York. She got the same questioning from Nelson: “So, how do you explain your failure to bring to the attention of this court that a brief was filed with this court by appellant's counsel with apparent fabrications and misrepresentations?” Freedman tried to explain. “I certainly read the briefs,” she said. “I certainly read all of the briefs here, but I certainly didn't focus on it, because it was not our issue. And I do apologize to the court for not catching that, but I tended to focus more on the issue of prior written notice.” When Freedman finished, all of the attorneys stood up and attempted to leave quickly. “Don’t go anywhere yet,” LaSalle said. “Have a seat. I just want to say this to you all. This is a very distressing situation. I know this is an outlier. We're very fortunate, my colleagues and I, we have the privilege of working with what I think is one of the best benches in the state, the bars in the state. For me the appellate bar here in the city of New York and its surrounding suburbs, we see excellent work. For me personally, it's been a highlight of my career to have the opportunity to work with such outstanding judges, and to have the opportunity to work with such outstanding lawyers,” he said. “A part of this profession, a big component of it, is that there's an element of trust, and mistakes are made. We make mistakes as judges, we've made mistakes. I don't want to speak for my colleagues, but I dare say that we've al