휴깅페이스 사고 이후 촉발된 AI 안전 논쟁에서 다리오 아모데이 앤스로픽 CEO는 정부-기업 협력을 통한 개발 속도 조절을 주장했지만, 마크 저커버그 메타 CEO 등은 시장 자율 규제로 충분하다는 입장이다. 트럼프 행정부도 규제를 기업 자율에 맡기는 방침이라, 업계 자율 표준 기구 설립 논의가 진행 중이다.
번역된 본문
테크 업계는 AI 안전이라는 주제에 어떻게 접근해야 할까? 오픈AI의 에이전트가 여러 기업을 해킹한 휴깅페이스(Hugging Face) 사건과 같은 사례가 AI가 세계를 혼돈에 빠뜨릴 위기의 징후인지를 두고 논쟁이 뜨거워지면서, 점점 더 많은 고위 임원들이 이 문제에 목소리를 내고 있다. 최근 몇 주간 주요 AI 연구소들은 AI 발전 속도를 늦추자고 촉구했다. 가장 극적으로는 다리오 아모데이 앤스로픽 CEO가 약 4,000 단어에 달하는 에세이를 발표하며, 충분한 안전장치를 마련할 수 있도록 AI 개발 속도를 늦춰야 하는 이유를 제시했다. 이 계획에는 안전한 배포를 위한 기업과 정부 간 국제 협력 전략 등이 포함되어 있다. 경쟁사 임원들인 오픈AI의 샘 알트먼과 xAI의 일론 머스크도 아모데이의 계획을 지지했다. 이 에세이는 테크 업계 전반에서 추가 논의를 촉발했지만, 모두가 글로벌 차원의 조율된 행동이라는 아모데이의 주장에 동의한 것은 아니다. 실제로 많은 이들은 정부의 감독이 꼭 필요한 것은 아니라고 생각하는 듯하다. 이 논쟁에 최근 가세한 인물은 메타 CEO 마크 저커버그로, 이번 주 X(구 트위터)에 자신의 견해를 공유했다. 저커버그는 "나의 견해는 신뢰(trust)와 정렬(alignment)이 에이전트와 모델을 차별화하는 가장 중요한 역량으로 빠르게 부상하고 있다는 것"이라고 밝혔다. 그는 안전 우려로 인해 자사 AI 모델의 출시를 연기했다고 공개했다. 저커버그는 "메타는 안전과 보안에 집중하기 위해 Muse의 출시를 수개월간 연기했다. 우리는 다른 회사들에게 먼저 그렇게 하라고 요구하지 않았다. 사람들과 우리에게 분명히 옳은 일이었기에 일상 업무의 일환으로 그냥 실행했을 뿐"이라고 말했다. 아모데이 계획의 일부를 지지한 저커버그는 정부의 개입이 불필요하며, 업계를 구조화하는 자연스러운 인센티브가 기업들로 하여금 보안을 제대로 갖추도록 이끌 것임을 시사하는 듯하다. 시장은 궁극적으로 혼돈을 일으키지 않는 모델을 만들기 위해 노력하는 기업에게 성공을 안겨줄 것이라는 것이다. 이는 저커버그가 지난달 발표한 대규모 에세이와도 맞닿아 있다. 그 에세이에서 그는 필요한 경우에 한한 정부와의 제한적 협력을 주장하면서도, 번영하는 기술 미래의 핵심으로 미국의 경쟁력을 크게 강조했다. 전반적으로 이것이 다른 많은 테크 임원들의 입장인 듯하다. 발언에 나선 또 다른 고위 임원은 레딧 공동 창업자 알렉시스 오하니언으로, 그는 수요일 CNBC와의 인터뷰에서 테크 업계가 AI의 위험을 대중에게 설명하는 데 있어 대체로 "무감각(tone deaf)"했다고 느꼈다고 밝혔다. 그러나 저커버그와 마찬가지로 오하니언 역시 기업들이 스스로 문제를 해결할 수 있으리라는 희망을 표현한 듯했다. 구글 딥마인드 공동 창업자 셰인 레그도 자신의 생각을 공유했다. 레그는 "지금 우리는 역량(capabilities)이 매우, 매우 빠르게 발전하는 시기를 살고 있다. 하지만 역량이 안전을 앞질러서는 안 된다"고 말했다. 그는 덧붙였다. "우리는 그 세부 사항과 실제로 어떻게 작동할지에 대해 정말로 깊이 고민해야 한다." 대형 연구소들의 공공-민간 협력 촉구에도 불구하고, 규제 찬성 측 인사들 중 일부도 기업들이 스스로 규칙을 만들도록 맡기는 것에 상대적으로 만족하는 듯하다. 실제로 이번 주 덜 주목받았지만 결코 덜 흥미롭지 않은 이슈는, 더 인포메이션(The Information)의 보도였다. 이 보도에 따르면 오픈AI, 앤스로픽 등 주요 AI 기업들이 AI 표준 기구를 만들기 위해 협력하고 있으며, 한 언론사는 이를 민간 '자율 규제 기구'로 묘사했다. 유사한 연방 주관 기구 설립에 대한 논의는 이전에도 있었다. 그러나 목소리 높은 AI 지지자들을 포함하고 있으며 대체로 규제 없는 테크 비즈니스를 추구해온 트럼프 백악(행정부는 주정부가 자체 AI 법을 도입하는 것을 강력히 저지하려까지 했다)은 아무런 조치도 취하지 않았다. 행정부의 'AI 차르(AI 국장)'인 테크 업계 베테랑 데이비드 색스는 AI 규제는 이를 개발하는 기업들에 맡겨야 한다고 말했다. 백악관처럼 저명한
How should the tech industry approach the topic of AI safety? Increasingly, high-level executives are weighing in on this question, as a debate rages over whether incidents like the Hugging Face incident — in which an OpenAI agent hacked several different companies — suggest that AI is on the verge of throwing our world into chaos. In recent weeks, prominent labs have called for a slowing of the AI advancement. Most dramatically, CEO Dario Amodei penned a nearly 4,000 word essay in which he laid out the case for why AI development should be decelerated so that adequate guardrails can be deployed. That plan involves — among other things — an international strategy of collaboration between companies and governments on safe deployment. Rival AI executives — like OpenAI's Sam Altman and xAI's Elon Musk — have endorsed Amodei's plan. The essay has spurred further conversation throughout the tech industry, although not everyone agreed with Amodei's call for globally coordinated action. Indeed, many seem to think that government oversight need not necessarily play a role at all. The latest entrants to this debate include Meta CEO Mark Zuckerberg, who took to X this week to share his thoughts on the matter. "My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models." Zuckerberg revealed that his company had pushed back the release of its AI model over concerns for safety. "Meta delayed shipping Muse for several months to focus on safety and security," Zuckerberg said. "We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us." Zuckerberg — who endorsed parts of Amodei's plan — seems to imply that government action is unnecessary, and that the organic incentives structuring the industry will compel companies to get security right. The market should ultimately bring success to companies that take care to create models that don't cause chaos. This seems to align with Zuckerberg's own massive essay — penned last month — which argued for limited collaboration with government, where necessary, but largely touted American competitiveness as the key to a prosperous technological future. In general, this seems to be the position of many other tech executives. Another high-level executive to chime in was Reddit co-founder Alexis Ohanian, who told CNBC on Wednesday that he felt the tech industry had been largely "tone deaf" when it came to explaining the risks of AI to the public. Yet like Zuckerberg, Ohanian also seemed to express hope that companies would be able to figure things out on their own. Shane Legg, Google DeepMind co-founder, also shared his thoughts . “We’re living in a period now where capabilities are advancing very, very quickly,” said Legg. “But we can’t let capabilities get ahead of safety.” He added: “We need to really work through the details of that and how that would work in practice." Despite calls from big labs for public-private collaboration, some of those pro-regulation voices also seem relatively content with letting businesses write their own rules. Indeed, a slightly less visible but no less interesting story this week was a report from The Information that said OpenAI, Anthropic, and other major AI companies were working together to create an AI standards organization – described by one outlet as a private "self-regulatory body." Discussions have previously been held about creating a similar, federally-run organization. However, the Trump White House — which includes vocal AI proponents and has largely pushed for an unregulated tech business (the administration has even sought to forcefully stop state governments from introducing their own AI laws) — hasn't pursued anything. The administration's "AI czar," tech veteran David Sacks, has said that AI regulation should be left to the companies developing it. Like the White House, prominent members of Congress seems equally disinterested in taking Amodei up on the offer to regulate his company. Not long after the Anthropic CEO's essay was published, Speaker of the House Mike Johnson addressed concerns over catastrophic AI advancements thusly: "You’re not all going to be dead in 10 years,” he said . A private organization would keep the industry unregulated except by voluntary commitments and would naturally bestow a certain amount of authority and influence to the companies responsible for creating it. This leads to another ongoing claim, which is that the industry's calls for regulation are actually a strategy of "regulatory capture." According to this argument , powerful companies that already enjoy a position of prominence in the industry may use regulatory stratagems to ice out or disadvantage smaller, less resourced companies — thereby stifling their competition. This need not involve actual regulations but could involve voluntary industry standards that nevertheless put pressure on less powerful firms. Many prominent voices in the Chinese government have echoed that position. After Amodei called for "pacing the frontier" this week, China's Ministry of Foreign Affairs, Guo Jiakun, accused Americans of using "fear mongering" to disrupt the development of global AI governance, while an op-ed in China's state run newspaper called Amodei's rhetoric straight out of a "Cold War playbook." Restructuring the way companies do business with China — one of the things Amodei calls for in his essay — could naturally be an opportunity for American companies to sway things in their own favor. Amodei even admits this is the goal. “If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important,” he wrote in his essay. It's not just China, however, that sees big American AI labs as seeking to throw the match in their own favor. Aidan Gomez, the CEO of Canadian AI firm Cohere, recently accused those companies of forming a "cartel," writing, in a blog post : "AI needs guardrails. That is not the dispute and never has been. The dispute is over who writes them, who gets to participate and whose interests the rules are protecting." As Gomez notes, the field of AI safety is a deeply political one, wherein the terms by which safety are defined — and by whom — will ultimately help determine who has a head start and who gets left behind in the next stage of AI competition. Topics AI , ai safety , Anthropic , TC When you purchase through links in our articles, we may earn a small commission . This doesn’t affect our editorial independence. Lucas Ropek Senior Writer, TechCrunch Lucas is a senior writer at TechCrunch, where he covers artificial intelligence, consumer tech, and startups. He previously covered AI and cybersecurity at Gizmodo. You can contact Lucas by emailing lucas.ropek@techcrunch.com. View Bio October 13 - 15 San Francisco Last day to book an exhibit table is September 18. Don’t miss out on high-impact leads, investor access, and a brand spotlight in Disrupt’s Expo Hall. BOOK NOW Most Popular Clean tech startup Fluxnium found a way to tap 50,000 years' worth of nuclear fuel Tim De Chant Salesforce and Nvidia's new reasoning model is everything the AI labs should fear Julie Bort Jensen Huang took a call from Trump, and showed off something else, too Connie Loizos The 9 buzziest startups from Y Combinator’s latest Demo Day, according to VCs Marina Temkin Dominic-Madori Davis Tesla says it will finally unveil the second-generation Roadster on October 1 Anthony Ha Revolut confirms customer data breach through fake government requests Jagmeet Singh Matt Mullenweg tells (trolls?) Automattic staff, saying he's back in control after CEO ouster Sarah Perez