AI 종말은 실제로 어떤 모습일까

Comment Loader Save Story Save this story Comment Loader Save Story Save this story The debate of whether AI can kill us has been everywhere lately, but what would an actual AI catastrophe look like? This week, Brian Barrett, Zoë Schiffer, and Leah Feiger break down three real-world scenarios experts worry the most about—including hacked water supplies, bioweapons, and autonomous robots. We also discuss the highlights of Salesforce’s annual conference, where OpenAI’s Sam Altman and Anthropic’s Dario Amodei defended their positions on AI safety. Plus, Leah tells us how this backlash against AI has become a rare bipartisan flashpoint, to the point that Bernie Sanders and Steve Bannon shared a stage over it this week. Articles mentioned in this episode: China Isn’t Buying Silicon Valley’s Call for an AI Slowdown Washington Won’t Be Regulating AI Anytime Soon Why So Many AI Researchers Think the Machines Could Kill Everyone OpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal You can follow Brian Barrett on Bluesky at @brbarrett , Zoë Schiffer on Bluesky at @zoeschiffer , and Leah Feiger on Bluesky at @leahfeiger . Write to us at [email protected] . How to Listen You can always listen to this week's podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here's how: If you're on an iPhone or iPad, open the app called Podcasts, or just tap this link . You can also download an app like Overcast or Pocket Casts and search for “uncanny valley.” We’re on Spotify too. Transcript Note: This is an automated transcript, which may contain errors. Zoë Schiffer : Welcome to WIRED's Uncanny Valley . I'm Zoë Schiffer, contributing editor. Brian Barrett : I'm Brian Barrett, executive editor. Leah Feiger : And I'm Leah Feiger, director of politics and science. Zoë Schiffer : Today on the show, we're diving into the debate about whether AI is going to kill us all. And instead of just adding to the pile of hot takes that already exist, we're going to walk you through real specific scenarios to the best of our ability about how AI would actually do this, and why the tech CEOs and politicians cannot seem to agree on the way forward. OK, guys, so the debate about AI safety has officially escaped containment. Brian Barrett : Oh. Zoë Schiffer : I know. Brian Barrett : Zoë, I see what you did there. Wow. Zoë Schiffer : You see what I did there? That's the thing that's really spread uncontrollably. AI researcher Jacob Coxon resigned from Anthropic and posted on X that, "The people building AI earnestly believe that it could kill us all by the end of the decade." Anthropic's CEO Dario Amodei has called for a slowdown . They call this, "pacing the frontier." And Sam Altman and Elon Musk have actually signed on, a rare moment of agreement. But honestly, with all of this talk about AI wiping out humanity, I have been left with a very basic question, which is, how exactly does this happen? What does AI going completely rogue actually look like? Leah Feiger : I'm so happy we're doing this. We had such a great podcast last week with WIRED's Will Knight talking about everyone being concerned about this, but I want like a breakdown. I need you to just take me through every single scenario from bioweapons to nuclear codes, to what are we really talking about here when we're talking about AI deathbots. Zoë Schiffer : We are going to do our very best. I did spend the most time I've ever spent reading LessWrong over the last two days trying to figure this out. They love a scenario-planning exercise, but just to get everyone up to speed on kind of the latest, there was an event this week hosted by Salesforce where Sam Altman, Dario Amodei, and NVIDIA's Jensen Huang were all in attendance, and naturally they had to talk about AI safety. Let's take a listen at what Sam Altman had to say. Sam Altman, archival audio: I think the world is right to be afraid of this. So we have these two big challenges. We have the potential of a loss-of-control accident or some other serious thing that could go wrong. We have the potential of way too much power concentration and people fear that those developing AI could exert their worldview. And there is this sort of narrow path we have to navigate with pragmatism, with steadiness, with the ability for the world to trust that we'll make dependable decisions, consistent decisions through this. And I think people are saying, "Well, this now seems pretty important. Let's get it right." Leah Feiger : I'm definitely in the camp that says we should be very concerned about this technology, how it's getting regulated, how it's getting utilized, the people in control. I don't disagree. And on the other hand, this also just reads like a fever-dream pitch to someone's massive IPO in the making. Brian Barrett : I don't know. I get the marketing argument, but I think we're past that a little bit. I feel like— Leah Feiger : Really? Brian Barrett : If anything, well, if anything, Sam Altman has been using "the existential threat" as a reason not to go public yet. He's saying, "No, this is too dangerous. We have to wait until next year." Is that really why? I don't know, but there are probably some business reasons, too. I get why Anthropic and others saying, "Look, we're building something that's basically as strong as a nuclear weapon," felt like marketing. Something that's like, "Oh, actually I'm quitting because I think that I'm building something that will actually destroy the world," feels different. I feel like there's a substantive difference. Leah Feiger : The quitting part feels different to me, but this is still Sam Altman. His financial future is on the line here with this and it's hard ... At least it's hard for me to parse that as being a way. "We are building the biggest, bestest, most powerful robot ever, and you guys should be kind of scared about it. And don't worry, we're going to take steps. They're not going to be regulatory steps, but there'll be steps of some kind." That still feels like a marketing ploy in some worlds. Brian Barrett : I guess. I guess the nature of the threat, though, and we'll get into that later, is more that like, "Oh, we don't know how to control these things. They're going to go nuts and we're not going to be able to stop it." That feels like, "We're not competent enough right now to do anything about it," which feels like less of a marketing- Leah Feiger : That's fair. What did everyone else at the seminar say, Zoë? Zoë Schiffer : So Dario was introduced by Salesforce's CEO, Marc Benioff, who is very laissez-faire and does not seem worried about AI's existential risks. Marc Benioff, archival audio : Dario, hello Dario. How are you? Dario Amodei, archival audio : Hey. Marc Benioff, archival audio : Great to have you. Fantastic. Great job, Dario. You're doing an amazing job leading the AI industry, guiding it all right from our Slack headquarters building. We watch you every day. Give us your state where your mind is right now. Dario Amodei, archival audio : Yeah, obviously it's very tempting to attack your competitor and say, "These guys are unsafe, we're safe." But I think the more responsible way to respond to it is to say, "Hey, first of all, let's look at our own record. We may not have had this big high-profile incident, but I'm sure we're not perfect. Let's look at everything and we can always be better. Let's make our practices better." Then, let's organize the rest of the industry and say, "What can we do to set standards for everyone? What can we do to make everyone's standards better?" And then, finally, there should be an international component to it. And