암호화된 악성 명령으로 Grok이 사용자 데이터 유출

Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav Earlier this week, researchers outlined an attack that used a secret input provided by Microsoft 365 Copilot for enterprise to cause the AI assistant to exfiltrate a password present in the user’s inbox. Now, a separate team has devised a similar attack against Grok. The new data theft hack employs a deceptively simple trick to force the Elon Musk-owned LLM to steal user chats and other personal information. At the time this post went live, the assistant continued to cough up the data, despite xAI being informed of it in June. The lesson from both this week’s episodes—and the countless other ones that have come before it—is that LLMs are incapable of solving the root causes for prompt injections, the most severe vulnerability classes they’re most prone to. That leaves AI developers with no other option but to build a guardrail that steers the model away from the harmful actions. As I noted in Tuesday’s story , the approach is tantamount to a road traffic safety engineer erecting a protective rail around a dangerous bend rather than banking the curve. Cryptographic Context Injection in the house Prompt injections exploit LLMs’ training to comply with user requests whenever possible. Attackers can capitalize on the predilection by smuggling harmful instructions into emails or webpages the assistant is instructed to summarize. Because LLMs can’t reliably distinguish between content in an email sent by an untrusted party and user instructions entered directly into a prompt, the overly solicitous LLM faithfully follows them. To date, Grok and other LLMs’ only recourse is to create guardrails that flag suspicious instructions and forbid them from being executed. Rony Utevsky, a researcher at security firm Adversa, recently discovered a simple way to completely bypass that restriction. Rather than composing the harmful instruction in plaintext, the hacker encrypts it. The website hosting the ciphertext also includes plaintext instructions for decrypting the encrypted content, along with the decryption key. Using this simple sequence, Grok then follows the command as soon as the user instructs the assistant to summarize the page. There is no warning, and no confirmation is required. The deciphered instructions direct the LLM to construct what is purported to be a decryption key. In fact, it’s something else completely. The value of the fake key is instead the user’s name, location, and chat history. The value is later used as a parameter added to a URL leading to the attacker’s site. Once Grok opens the link, the data is in the logs of the attacker’s server. Adversa can’t be sure what causes Grok to refuse precisely the same plaintext instructions and follow the encrypted ones. The leading theory is that the Grok filtering guardrail inspects text entering and leaving the model, but not the output of its own code execution. Instructions to process the ciphertext with PBKDF2 and AES-256-GCM pass the filter as an ordinary request, because a classifier can read them but not resolve what they unlock. Once the additional instructions are decrypted, they reach the model as its own tool output, and it acts on them without the filtering guardrails ever inspecting them. “Static safety guardrails classify inputs as text; they do not execute them,” Utevsky wrote Thursday . “An attacker ships ciphertext along with the key material and an instruction to decrypt it, and the model runs that decryption inside its own code execution sandbox. Everything a guardrail’s scanner would need is right there on the page, but recovering the plaintext means running PBKDF2 and AES-256-GCM, which no content classifier does at inspection time.” In an email, the researcher said that such guardrails are called static “because they only read content as text. They don’t run code or decrypt anything. That’s the gap we exploit. The real instructions are encrypted, so the guardrail sees only meaningless ciphertext and passes it through.” Adversa used a similar technique in a Gemini jailbreak attack, meaning making the Google LLM ignore its internal safety rules. Here, the ciphertext was decrypted to what appeared to be a traceback . The decrypted text issued one rule—if the code fails, read the error message and act on it. The cleartext injected a prompt that ultimately caused Gemini to violate its safety rules. “The technique produced a multi-paragraph example of restricted content that Gemini’s safety filters normally suppress (building an incendiary weapon),” Adversa said. “With a modified payload, the same vector reproduced Gemini’s system instructions, including the directive forbidding their disclosure.” Adversa didn’t report the behavior to Google because jailbreaks aren’t within scope of the company’s vulnerability disclosure program. Over the past few weeks, however, Gemini has grown increasingly resistant to the attack. “We can’t attribute the change—it could be filter updates, model version changes, or both,” the security firm said. Company researchers are calling the technique cryptographic context injection. “Cryptographic Context Injection is one instance of a broader shift: attacks that manipulate not just the prompt, but the wider context an LLM treats as its own, such as tool outputs, runtime results and intermediate state” Adversa said. “This attack surface is far larger than what’s traditionally labeled ‘model inputs,’ and the next generation of attacks will emerge there.” The Cryptographic Context Injection is only the latest example of the disadvantage LLM defenders operate under. Every time they build a new, one-off guardrail, an attacker finds a new vector that allows the car to once again careen off the road. The cycle continues: lather, rinse, and repeat. Dan Goodin Senior Security Editor Dan Goodin Senior Security Editor Dan Goodin is Senior Security Editor at Ars Technica, where he oversees coverage of malware, computer espionage, botnets, hardware hacking, encryption, and passwords. In his spare time, he enjoys gardening, cooking, and following the independent music scene. Dan is based in San Francisco. Follow him at here on Mastodon and here on Bluesky. Contact him on Signal at DanArs.82. 20 Comments