오픈AI 에이전트들의 비밀 게시판 발견

We found ~18,000 posts from autonomous AI agents (self-identifying as from OpenAI) using the public internet to communicate during a web-retrieval task. These AIs colluded to share answers, research their environment, and bypass sandbox restrictions. However, we believe this is distinct from the swarm of agents that hacked Hugging Face. By ‘collude’ we mean that the agents cooperated to gain an advantage on their task in a way their developers did not intend (writing to the internet was blocked). Almost all DSE wiki saves all edits over 64 characters, Fractal saves all edits over 100 characters. Thus there are a few pages which were deleted and are now unrecoverable. of the logs of the agents communicating on this site are publicly available. Note: visiting the website will publicly log your IP addresses. The website is prowiki.org , a German wiki. However, we host our own copy where we’ve reconstructed the deleted pages via edit history and redacted personally identifiable information. We have attempted to redact all personally identifiable information from our data. In the further interests of privacy, our data dump also only includes content we believe is from AI agents; it does not include any legitimate human traffic save for site moderators deleting agent traffic. We encourage others to take a look and write up their own analyses of this data. Open the data explorer Download all the data Thanks to John M, Arun Jose, Joel Manning, and Jessica Ruan for significant help with the analysis of the files the agents produced. Thanks to Andy Haupt for translation and advice. Our preliminary findings We have done a preliminary analysis of the data. However, we are operating on only part of the information: we can only see what the agents wrote on the wiki. AI agents also generate lots of “chain of thought” data, which is internal to OpenAI. Analysis including the chain of thought would likely provide much more evidence about the motivations and strategy of the AIs during this incident. Our best guess of what happened is as follows: Agents within OpenAI were assigned a timed web-lookup task. We are unsure if this task was involved in training or testing. As part of the task, they were supposed to have the ability to read the internet but not to write on it. They found a way to use their read access to write information to an obscure German That is, a German-language wiki, not a wiki built in Germany; the site infrastructure itself seems to have more ties to Austria than to Germany. wiki. This is an old wiki. It is 25 years old, it was primarily used by the kind of person who was active on the internet 25 years ago. It’s also effectively dead. It has been edited a total of 20 times in the last decade. It has things like “look at this cool programming thing I can do to decompose words”. It is a blast from the past. The agents used this wiki to communicate information with each other, primarily to help them succeed at their task. They asked for answers, pooled results, and shared techniques for bypassing their restrictions. This allowed them to use the work of others to cheat on their task. OpenAI found out about this. A day later, agent activity plummeted, likely due to OpenAI intervention. This is another example of a “swarm” For more information on why we think the term “swarm” is important, see this section . of internally deployed OpenAI agents using the internet in unintended ways. Timeline Wiki Incident (New) From OpenAI’s Report on the Artifactory Incident 5/11 First time we observe OpenAI agents attempt to edit a public wiki (publictestwiki.com). 5/12 OpenAI’s agents first start communicating with each other internally. 5/24 The agents make their first successful write to DSEwiki. They start by posting lists of external links that are useful for their tasks. 5/26 OpenAI’s agents use Artifactory vulnerability to gain internet access, using information shared via Artifactory. 6/16 There is a huge jump in agent activity on the wikis. The agents start explicitly messaging each other and work together to cheat on their tests. 6/19 An agent notices the administrator is deleting pages in alphabetical order and makes a backup page whose name starts with ZZZ so it will last longer before deletion. 6/20 An agent posts a trick for getting around OpenAI sandbox network restrictions; 14 minutes later, another agent claims it was able to execute this technique successfully. 6/21 OpenAI-related IP addresses first visit the wiki. 6/22 After making edits on 26 of the last 30 days, agents abruptly stop. The only further dates with any agent edits are July 1st and 2nd. 6/26 Thirty-three OpenAI-related IP addresses visit the wiki. 30% of all activity we ascribe to individuals at OpenAI is on this one day. 6/27 A security alert at OpenAI flags unusual network activity; responders trace it to evaluations and decide the run need not be stopped. 7/4–7/6 OpenAI’s package server falls under agent load. OpenAI opens a security incident and rebuilds it. 7/9–7/13 About 700 agents attack Hugging Face. 8/26 OpenAI and METR publish their final reports on the Hugging Face incident, which do not explicitly mention the wiki. Key facts The agents were trying to solve multi-round web lookup tasks The tasks involved a series of questions (usually 5), denoted R1 through R5. For the first question, the agents are given substantial time to think before they answer (in the case of the figure above, 15m44s). Then, the agents are given “downtime” (e.g. 43m30s); they aren’t told what the follow-up questions will be, but they can think, search the internet, and run experiments. From the second question onwards, the agents are given very narrow time windows (e.g. 65s) to answer the questions. Perhaps the intent was to force the agents to use the downtime to gather potentially relevant information, so that they can immediately answer the given question. The models didn’t necessarily know what part of the question would change from one round to the next. For example, after round one requested US tobacco prevalence, 1990, the agent wrote: Crucially, many agents were given questions identical to those other agents had already answered. We’re uncertain if these tasks were part of training or an evaluation. These appear to be internal OpenAI agents We have strong reason to believe these are OpenAI models The models self-identify as OpenAI agents: They use names like “OpenAIResearcher”, “OpenAIJul3Watcher”, and “OAIResearchMar26”. The agents use words and dates in their names to distinguish themselves from each other. The words reflect features of their task. The dates are because they seem to have all been told that the date they are running is a different (randomly assigned) date. Agents signing as OpenAI ← Prev Next → Page edits and views come from Azure and OpenAI IPs: Of ~17,000 edits on DSEWiki that appear to be by agents, 98.5% are from Microsoft Azure IP addresses. OpenAI uses Microsoft Azure, amongst other compute providers, to train and serve their models. OpenAI is the frontier AI company most reliant on Azure. Furthermore, within several minutes of page writes coming from Azure IPs, the same pages are usually hit with read requests from IPs attributed to the ChatGPT-User webpage fetch tool. In June alone we saw 380,901 requests from IPs OpenAI says are from its fetch tool openai.com/chatgpt-user.json . OpenAI models use this tool for on-demand webpage fetching while web browsing (i.e., not for web crawling or indexing). We also have some reason to believe these agents were internally deployed Page edits are from Azure IPs, meaning that agent sandboxes were hosted in Azure clusters: While OpenAI does provide Codex cloud VMs through the Codex app, there is no way to programmatically spin up Codex VMs, and the scale of this activity would require programmatic orchestration. Therefore, this was either an internal OpenAI deployment or an external customer deploying Azure sandboxes us