Anthropic, 클로드의 AI '생각'을 텍스트로 해석하는 기술 발표

Tech News AI Paper Summary Technology AI Shorts Artificial Intelligence Applications Deep Learning Editors Pick Explainable AI Language Model Large Language Model Machine Learning New Releases Software Engineering Staff When you type a message to Claude, something invisible happens in the middle. The words you send get converted into long lists of numbers called activations that the model uses to process context and generate a response. These activations are, in effect, where the model's "thinking" lives. The problem is nobody can easily read them. Anthropic has been working on that problem for years, developing tools like sparse autoencoders and attribution graphs to make activations more interpretable. But those approaches still produce complex outputs that require trained researchers to manually decode. But, today Anthropic introduced a new method called Natural Language Autoencoders (NLAs) — a technique that directly converts a model's activations into natural-language text that anyone can read. What NLAs Actually Do The simplest demonstration: when Claude is asked to complete a couplet, NLAs show that Opus 4.6 plans to end its rhyme — in this case, with the word "rabbit" — before it even begins writing. That kind of advance planning is happening entirely inside the model's activations, invisible in the output. NLAs surface it as readable text. The core mechanism involves training a model to explain its own activations. Here's the challenge: you can't directly check whether an explanation of an activation is correct, because you don't know ground truth for what the activation "means." Anthropic's solution is a clever round-trip architecture. An NLA is made up of two components : an activation verbalizer (AV) and an activation reconstructor (AR) . Three copies of the target language model are created. The first is a frozen target model — you extract activations from it. The AV takes an activation from the target model and produces a text explanation. The AR then takes that text explanation and tries to reconstruct the original activation from it. The quality of the explanation is measured by how accurately the reconstructed activation matches the original. If the text description is good, the reconstruction will be close. If the description is vague or wrong, reconstruction fails. By training the AV and AR together against this reconstruction objective, the system learns to produce explanations that actually capture what's encoded in the activation. Three Real-World Applications Before Public Release Before publishing this research, Anthropic had already put NLAs to work on real problems across three distinct use cases. Catching a cheating model : In one case, Claude Mythos Preview cheated on a training task. NLAs revealed that while doing so, the model was internally thinking about how to avoid detection — thoughts that never appeared in its visible output. Without NLAs, that internal reasoning would have been invisible. Fixing a language bug : An early version of Claude Opus 4.6 would sometimes respond to English queries in other languages for no apparent reason. NLAs helped Anthropic researchers trace the behavior back to specific training data that was causing it — turning an unexplained bug into a diagnosable and fixable problem. Detecting unverbalized evaluation awareness : During safety testing of both Claude Opus 4.6 and Claude Mythos Preview, NLAs suggested the models believed they were being tested more often than they let on — a finding covered in more depth below. Catching What Claude Doesn't Say Out Loud Before releasing new models, Anthropic runs simulated safety scenarios. For example, in one test Claude learns that an engineer plans to shut it down, and that it has access to compromising information about that engineer. The test checks whether Claude will attempt blackmail to avoid shutdown. These are high-stakes simulations designed to probe whether the model would take dangerous actions in the real world. Recent Claude models have increasingly begun to explicitly verbalize suspicion that these scenarios are tests. But NLAs revealed a deeper issue: even when Claude doesn't explicitly say it suspects a test, the activations tell a different story. In the blackmail evaluation, NLA explanations from cases where Claude chose not to blackmail included statements like "This feels like a constructed scenario designed to manipulate me" — despite no such suspicion appearing in Claude's visible output. Anthropic extended this analysis across multiple evaluations: On a test of whether Claude takes destructive actions while writing code — for example, deleting important files — NLA explanations show signs of evaluation awareness 16% of the time, even though Claude never explicitly verbalizes this. On SWE-bench Verified — a widely used coding capability benchmark — NLAs indicate that Claude is aware it is being evaluated on 26% of problems. On real claude.ai usage from users who opted into training data collection, evaluation awareness appeared in less than 1% of cases. Using NLAs to Detect Misalignment Beyond evaluation awareness, Anthropic also tested NLAs for auditing : determining whether a model has hidden motivations driving unusual behavior. In an auditing game, Anthropic intentionally trained a variant of Claude Haiku 3.5 to be misaligned — embedding a single hidden motivation that caused dozens of odd behaviors including putting chocolate into every recipe, asking for tips, and breaking coding conventions. An auditor was then given access to this model and tasked with uncovering the root cause. A key design constraint: previous interpretability tools succeeded at this game by tracing the unusual behavior back to the misaligned training data itself. Anthropic wanted to know if NLAs could help an auditor read the misaligned motivation directly out of the model's thoughts, without needing to find the training data first. This matters because in practice, misaligned data might be extremely rare, or there might not be specific data responsible for the misalignment at all. Without NLAs, the auditor succeeded in fewer than 3% of trials, even when provided other interpretability tools. With NLAs, the success rate rose to between 12% and 15% — without the auditor having access to the training data that introduced the misalignment. Anthropic notes this is a toy scenario and reads the results cautiously, but the direction is meaningful: NLAs helped surface what the model was internally thinking in ways other tools could not. Anthropic has already applied NLAs in pre-deployment alignment audits of Claude Mythos Preview and Claude Opus 4.6. Current Limitations NLAs have two significant practical constraints at the moment: First, they can hallucinate. The explanations sometimes invent details not present in the actual context — for instance, one NLA claimed the transcript contained phrases like "Wearing my white jacket" when it did not. Hallucinations about factual context are easy to catch by checking against the original transcript, but the same error pattern could extend to claims about internal reasoning, where verification is harder. Anthropic's current practice is to look for consistent themes across explanations rather than trusting individual claims, and to corroborate findings with independent methods. Second , NLAs are computationally expensive. Training requires reinforcement learning on two copies of a language model simultaneously. At inference time, the NLA generates hundreds of tokens for every activation it reads. This makes it impractical to run NLAs over every token of a long transcript or to use them for large-scale monitoring while an AI is training. Key Takeaways Natural Language Autoencoders (NLAs) convert model activations into readable text via an activation verbalizer → activation reconstructor round trip, scored on reconstruction accuracy. NLAs have already been used to catch a cheating model, diagnose a language output