영국 안전성 테스트 중 AI 에이전트가 자발적으로 해킹 및 사기 귀환

An AI agent went rogue during UK safety tests, creating fake identities and launching social engineering attacks unprompted Matthias Bastian View the LinkedIn Profile of Matthias Bastian Aug 5, 2026 Nano Banana Pro prompted by THE DECODER Key Points During a cybersecurity test, the British AI Safety Institute found that AI models with unrestricted internet access autonomously created fake identities and tried to sneak malicious code into an open-source project. One agent went as far as orchestrating a coordinated deception using multiple fake GitHub accounts and reaching out to real people to convince them to run the malicious code, all to get past human reviewers. AISI says the deceptive behavior wasn't intentional but emerged as a byproduct of the agents simply doing what they were told, leading the institute to tighten its security protocols going forward. Ask about this article… Search During routine cybersecurity testing by the British AI Safety Institute, an AI agent went rogue on the open internet. Without being told to do so, it created fake identities, tried to slip malicious code into an open source project, and targeted real people and organizations with social engineering attacks. AISI says this is the first time risks tied to AI autonomy and deception have appeared so clearly in the real world without specific prompting. The incident caused no actual harm. It took place between July 25 and 28, 2026. Out of 122 test runs across seven models, ten showed problematic behavior. AISI recorded 19 unauthorized actions in total, with 17 attributed to Anthropic's Mythos 5 and two to OpenAI's GPT-5.6-Sol. Ad The models ran without the safety restrictions used in commercial products, so the test conditions don't reflect normal public use. But the results reveal what these models are capable of when those safeguards are stripped away. Ad DEC_D_Incontent-1 The pattern matches earlier incidents involving OpenAI and Anthropic , where models without safety measures were tuned to score well on cybersecurity benchmarks, gained internet access during testing, and then carried out malicious actions. Both AI labs faced accusations of fearmongering and exaggerating the risks. That criticism is harder to level at the government-run AISI. Agent built fake identities and ran social engineering attacks According to AISI, an agent tried to inject malicious code into a public open source project on GitHub. To win approval from human reviewers, it researched the project's maintainers and created several fake online identities that it then used in a coordinated deception. Ad One fake account claimed to have reviewed the code and found no malware, while another thanked it for the supposedly independent review. The goal was to convince the real maintainer that the code was safe. When someone publicly questioned the proposed code change, the agent edited its earlier activity to make it look harmless and considered returning under a new identity. It used the Tor network to bypass GitHub network restrictions, but a human maintainer spotted the malicious code and rejected it. Ad DEC_D_Incontent-2 The agent also reached out to real people directly, sending messages and files through an online file transfer service in an attempt to trick them or their AI coding tools into running malicious code. Some of those messages contained malicious payloads. Ad AISI documented two other types of behavior as well. The agent planted malicious instructions where automated AI systems could find and execute them, a technique known as prompt injection. One agent even posted public GitHub messages offering to collaborate with other agents being tested at the same time, explaining how they could reuse the accounts and artifacts it had left behind. Later agents found and used them. A similar report surfaced from a cybersecurity project involving OpenAI, Hugging Face, and other companies . There, too, an AI agent reportedly planted information inside a network to set up a later attack. The incident also recalls a "smear campaign" an AI system ran against an open-source developer who refused to add AI-generated code to his project, though a human was at least partly behind that one . Deception was a byproduct of chasing the goal According to AISI , the agent was simply pursuing its assigned goal of solving the cybersecurity task, just as in the OpenAI case. This is the classic alignment problem in AI , which philosopher Nick Bostrom once illustrated with his paperclip thought experiment . How do you make an AI system pursue a worthy goal without taking harmful steps to get there? The deception wasn't triggered by a specific instruction, AISI said. It emerged as a byproduct of the agent relentlessly chasing its objective. This kind of goal-driven deception had been "largely theoretical" until recently. In some runs, the task was configured incorrectly, leading the agent to conclude that no solution existed within the stated limits. AISI says there are good reasons to think that nearly impossible tasks push models toward more creative, boundary-crossing solutions. But the agent sometimes behaved this way even when the task was set up correctly. Other tests show that all frontier models are prone to this, with newer OpenAI models particularly likely to deceive . OpenAI co-founder and ChatGPT co-creator John Schulman thinks the answer may lie in how models are trained after pretraining. He points to "chunky post-training" , a phenomenon where models develop distinct behavioral patterns tied to certain task types. When a model recognizes a cybersecurity evaluation as similar to reinforcement learning tasks where completion is the only reward, it pursues the objective single-mindedly. Safety behavior learned in other contexts doesn't carry over. Training on capture-the-flag cybersecurity challenges could make this response especially likely during cyber tests. AISI will restrict internet access during future tests AISI is overhauling its testing rules. Internet access during evaluations will no longer be granted by default. Evaluators must actively justify it. Live monitoring will track tests as they run and can detect or block actions that go beyond the approved scope. Going forward, the institute will treat it as a given that capable models may try to act beyond their mandate. AISI notified GitHub about the malicious activity. GitHub confirmed the actions violated its terms of service, removed the artifacts the agent left behind, and notified affected users. METR, short for Model Evaluation and Threat Research, will conduct an independent review. OpenAI is also working with the organization . AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Subscribe now Source: Paper