구글 딥마인드의 수석 연구원은 최신 논문을 통해 AI와 같은 컴퓨팅 시스템은 물리적인 육체와 생존을 위한 내재적 동기가 없기 때문에 결코 의식을 가질 수 없다고 주장했습니다. 이는 AI 기업 CEO들이 설파하는 범용인공지능(AGI) 도래와 같은 장밋빛 전망과 대비되며, 사실상 AI의 상업적, 실용적 한계를 명확히 규정하는 철학적 주장입니다. 전문가들은 이 주장의 타당성에는 공감하면서도, 이미 수십 년 전부터 제기되어 온 학계의 오래된 논의를 재활용했을 뿐이라고 지적했습니다.
번역된 본문
구글의 인공지능 연구소인 딥마인드(DeepMind)의 수석 연구원인 알렉산더 러크너(Alexander Lerchner)는 새로운 논문에서 어떠한 AI나 다른 컴퓨팅 시스템도 결코 의식을 갖게 되지는 않을 것이라고 주장했습니다. 이러한 결론은 딥마인드의 데미스 하사비스(Demis Hassabis) 자신을 포함하여 범용인공지능(AGI)의 도래를 반복적으로 언급하며 부풀려 온 AI 기업 CEO들의 서사와 충돌하는 것처럼 보입니다. 하사비스는 최근 AGI가 "산업혁명의 10배에 달하는 영향을 미칠 것이며, 그 속도 역시 10배가 될 것"이라고 주장한 바 있습니다. 이 논문은 AI 기업들이 언론에서 홍보하는 자신들에게 유리한 서사가 엄밀한 검토 아래에서는 어떻게 무너지는지를 보여줍니다.
내가 대화한 다른 의식 철학자들과 연구자들은 러크너의 논문('추상화의 오류: AI가 의식을 시뮬레이션할 수는 있지만 실체화할 수 없는 이유')이 탄탄하며, 이런 주장이 거대 AI 기업의 내부에서 나왔다는 점에 환영의 뜻을 표했습니다. 하지만 이 분야의 다른 전문가들도 수십 년 동안 똑같은 주장을 펼쳐왔다고 덧붙였습니다. 진화 시스템 생물학자이자 철학자인 요하네스 예거(Johannes Jäger)는 나에게 이렇게 말했습니다. "제 생각에 그(러크너)는 스스로 이 결론에 도달했고 바퀴를 재발명한 것 같습니다. 그는 특히 철학 분야의 문헌은 물론이고 생물학 분야에서도 독서를 많이 하지 않은 것 같습니다."
러크너의 논문은 복잡하고 전문 용어로 가득하지만, 그 주장은 광범위하게 요약하자면 모든 AI 시스템은 궁극적으로 '지도 제작자에 의존적(mapmaker-dependent)'이라는 점으로 귀결됩니다. 즉, 연속적인 물리적 세계를 유한한 의미 있는 상태의 집합으로 알파벳화(분류)하기 위해서는 능동적으로 경험하는 인지적 주체, 즉 '인간'이 필요하다는 뜻입니다. 다시 말해, AI 시스템에 유용한 방식으로 세상을 먼저 조직화할 사람이 필요합니다. 예를 들어, 아프리카의 저임금 노동자 대군이 AI 학습 데이터를 만들기 위해 이미지에 라벨을 붙이는 방식처럼 말입니다. 소위 '추상화의 오류(abstraction fallacy)'란 우리가 AI가 감각이 있는 행동을 모방하는 방식으로 언어, 기호 및 이미지를 조작할 수 있도록 데이터를 조직화했다는 이유만으로, AI가 실제로 의식을 달성할 수 있다고 잘못 믿는 것을 말합니다. 그러나 러크너가 주장하듯, 이는 물리적 몸체 없이는 불가능합니다.
예거는 나에게 이렇게 말했습니다. "인간으로서 여러분에게는 많은 다른 동기가 있습니다. 상황은 조금 더 복잡하지만, 그 모든 동기는 먹고, 숨쉬고, 살아남기 위해 끊임없이 물리적인 노력을 투자해야 한다는 사실에서 비롯됩니다. 비생명 시스템은 그렇게 하지 않습니다. 대규모 언어 모델(LLM)은 그렇게 하지 않습니다. 그것은 하드 드라이브에 있는 패턴의 묶음일 뿐입니다. 그런 다음 프롬프트가 주어지면 작업이 완료될 때까지 실행되고 끝이 납니다. 따라서 그것은 어떤 내재적 의미도 갖지 못합니다. 그 의미는 외부의 인간 주체가 의미를 정의한 방식에서 비롯됩니다." 인간과 같은 물리적 욕구가 프로그래밍된 구현형(embodied) AI를 상상해 볼 수도 있으며, 예거는 그러한 시스템이 왜 의식을 달성할 수 없는지에 대해 이야기했지만, 이는 본 기사의 범위를 벗어납니다. 이러한 질문에 투입된 방대한 문헌과 수십 년의 연구가 있지만, 러크너의 논문에는 그 중 거의 인용되지 않았습니다.
런던 대학교 골드스미스의 인지 컴퓨팅 교수인 마크 비숍(Mark Bishop)은 나에게 이렇게 말했습니다. "저는 그(러크너)가 말한 모든 것의 99%에 동의합니다. 제 유일한 반박점은 이 모든 주장이 수년 전에 이미 제시되었다는 것입니다." 비숍과 예거는 모두 구글이 러크너가 이 논문을 출판하도록 허용한 것이 좋으면서도 이상한 일이라고 말했습니다. 두 사람은 러크너가 제기하고 그들이 동의하는 이 주장이 평범한 사용자와는 무관한 모호한 철학적 논점이 아니라고 말했습니다. 오히려 AI가 의식을 달성할 수 없다는 주장은 AI가 실질적이고 상업적으로 성취할 수 있는 것에 명확한 한계가 있음을 의미합니다. 예를 들어, 예거와 비숍은 이 관점에 따르면 딥마인드 CEO 하사비스가 예측하는 산업혁명의 10배 영향력을 미칠 AGI는 가능성이 없다고 말했습니다. "[일론] 머스크 스스로도 [자율주행차에서] 레벨 5 자율성을 얻으려면 범용 인공지능이 필요하다고 주장했습니다."
A senior staff scientist at Google’s artificial intelligence laboratory DeepMind, Alexander Lerchner, argues in a new paper that no AI or other computational system will ever become conscious. That conclusion appears to conflict with the narrative from AI company CEOs, including DeepMind’s own Demis Hassabis, who repeatedly talks about the advent of artificial general intelligence. Hassabis recently claimed AGI is “going to be something like 10 times the impact of the Industrial Revolution, but happening at 10 times the speed.” The paper shows the divergence between the self-serving narratives AI companies promote in the media and how they collapse under rigorous examination. Other philosophers and researchers of consciousness I talked to said Lerchner’s paper, titled “The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness,” is strong and that they’re glad to see the argument come from one of the big AI companies, but that other experts in the field have been making the exact same arguments for decades. “I think he [Lerchner] arrived at this conclusion on his own and he's reinvented the wheel and he's not well read, especially in philosophical areas and definitely not in biology,” Johannes Jäger, an evolutionary systems biologist and philosopher, told me. Lerchner’s paper is complicated and filled with jargon, but the argument broadly boils down to the point that any AI system is ultimately “mapmaker-dependent,” meaning it “requires an active, experiencing cognitive agent”—a human—to “alphabetize continuous physics into a finite set of meaningful states.” In other words, it needs a person to first organize the world in way that is useful to the AI system, like, for example, the way armies of low paid workers in Africa label images in order to create training data for AI. The so-called “abstraction fallacy” is the mistaken belief that because we’ve organized data in such a way that allows AI to manipulate language, symbols, and images in a way that mimics sentient behavior, that it could actually achieve consciousness. But, as Lerchner argues, this would be impossible without a physical body. “You have many other motivations as a human being. It's a bit more complicated than that, but all of those spring from the fact that you have to eat, breathe, and you have to constantly invest physical work just to stay alive, and no non-living system does that,” Jäger told me. “An LLM doesn't do that. It's just a bunch of patterns on a hard drive. Then it gets prompted and it runs until the task is finished and then it's done. So it doesn't have any intrinsic meaning. Its meaning comes from the way that some human agent externally has defined a meaning.” One could imagine an embodied AI programmed with human-like physical needs, and Jäger talked about why a system like that couldn’t achieve consciousness as well, but that’s beyond the scope of this article. There are mountains of literature and decades of research that have gone into these questions, and almost none of it is cited in Lerchner’s paper. “I'm in sympathy with 99 percent of everything that he [Lerchner] says,” Mark Bishop, a professor of cognitive computing at Goldsmiths, University of London, told me. “My only point of contention is that all these arguments have been presented years and years ago.” Both Bishop and Jäger said that it was good, but odd, that Google allowed Lerchner to publish the paper. Both said the argument Lerchner makes, and that they agree with, is not an obscure philosophical point irrelevant to the average user, but that the claim that AI can’t achieve consciousness means that there’s a hard cap on what AI could accomplish practically and commercially. For example, Jäger and Bishop said AGI, and the impact 10 times the Industrial Revolution that DeepMind CEO Hassabis predicts, is not likely according to this perspective. “[Elon] Musk himself has argued that to get level five autonomy [in self-driving cars] you need generalized autonomy” which is Musk’s term for AGI, Bishop said. Lerchner’s paper argues that AGI without sentience is possible, saying that “the development of highly capable Artificial General Intelligence (AGI) does not inherently lead to the creation of a novel moral patient, but rather to the refinement of a highly sophisticated, non-sentient tool.” DeepMind is also actively operating as if AGI is coming. As I reported last year, for example, it was hiring for a “ post-AGI ” research scientist. Lerchner’s paper includes a disclaimer at the bottom that says “The theoretical framework and proofs detailed herein represent the author’s own research and conclusions. They do not necessarily reflect the official stance, views, or strategic policies of his employer.” The paper was originally published on March 10 and is still featured on Google DeepMind’s site . The PDF of the paper itself, hosted on philpapers.org , originally included Google DeepMind letterhead, but appears to have been replaced with a new PDF that removes Google’s branding from the paper, and moved the same disclaimer to the top of the paper, after I reached out for comment on April 20. Google did not respond to that request for comment. “We can imagine many financial and legislative reasons why Google would be sanguine with a conclusion that says computations can't be consciousness,” Bishop told me. “Because if the converse was true, and bizarrely enough here in Europe, we had some nutters who tried to get legislation through the European Parliament to give computational systems rights just a few years ago, which seems to be just utterly stupid. But you can imagine that Google will be quite happy for people to not think their systems are conscious. That means they might be less subject to legislation either in the US or anywhere in the world.” Jäger said that he’s happy to see a Google DeepMind scientist publish this research, but said that AI companies could learn a lot by talking to the researchers and educating themselves with the work Lerchner failed to cite in his paper, or simply didn’t know existed. “The AI research community is extremely insular in a lot of ways,” Jager said. “For example, none of these guys know anything about the biological origins of words like ‘agency’ and ‘intelligence’ that they use all the time. They have absolutely frighteningly no clue. And I'm talking about Jeffrey Hinton and top people, Turing Prize winners and Nobel Prize winners that are absolutely marvelously clueless about both the conceptual history of these terms, where they came from in their own history of AI, and that they're used in a very weird way right now. And I'm always very surprised that there is so little interest. I guess it's just a high pressure environment and they go ahead developing things they don't have time to read.” Emily Bender, a Professor of Linguistics at the University of Washington and co-author of The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want , told me that Lerchner might have been told that he’s replicating old work, or that he should at least cite it, if he had gone through a normal peer-review process. “Much of what's happening in this research space right now is you get these paper-shaped objects coming out of the corporate labs,” but that do not go through a proper scientific paper publishing process. Bender also told me that the field of computer science and humanity more broadly “if computer science could understand itself as one discipline among peers instead of the way that it sees itself, especially in these AGI labs, as the pinnacle of human achievement, and everybody else is just domain experts [...] it would be a better world if we didn't have that setup.” About the author Emanuel Maiberg is interested in little known communities and processes that shape technology, troublemakers, and petty beefs. Email him at emanuel@404media.co More from Emanuel Maiberg