에이전트 AI 담론에 결여된 것

Mark Nottingham recent entries all entries feed Hi, I'm Mark Nottingham. I write about the Web, protocol design, HTTP, Internet governance, and more. This is a personal blog, it does not represent anyone else. Find out more . Comments? Let's talk on Mastodon. @mnot@techpolicy.social other Internet and Web posts The Nature of Internet Standards (series) No One Should Have That Much Power Monday, 29 April 2024 RFC 9518 - What Can Internet Standards Do About Centralisation? Tuesday, 19 December 2023 Moving Control to the Endpoints Tuesday, 11 June 2019 What is the Web? Thursday, 4 December 2014 Five Favourite Protocol Design Papers Thursday, 15 April 2004 What's Missing in the ‘Agentic’ Story Friday, 24 April 2026 Internet and Web For much of the history of computing, it was reasonably safe to assume that a machine was doing what you told it to do (and what its creators promised it would do), because its operations were local. You bought a laptop or desktop with an operating system, and it did what it said on the tin: it ran programs and stored files. You bought a spreadsheet and a word processor, and those programs performed those tasks and didn’t do anything else. Software that didn’t do this was in a separate bucket called ‘malware’ and we had ways of dealing with it. That assumption has a more general precedent in tools – whether they be staplers, screwdrivers, or telescopes. When you buy a screwdriver, it turns screws; it has no agency of its own. It might do other things, but that’s because you’re misusing the tool, not because it decided to do something else. Most things that people use unambiguously follow this pattern: for example, my mechanical wristwatch can’t do anything but tell me the time. 1 That pattern is perpetuated in most 2 depictions of computers in fiction (especially sci-fi), which work for people diligently and always on their behalf, usually with minimal intrusion. They unambiguously act in the interest of their users, following in the footsteps of technological optimism which informs much of fiction and influenced a generation of nerds who tried to build it. All of these experiences combine to lead people to trust computers fairly unquestioningly; they don’t give much thought to the other purposes that might be served. When I use my phone, it’s my phone, and so it’s working for me, right? This is perpetuated in the press: recently, I saw an article in a major newspaper about how to talk to “your” AI agent. If you scratch the surface just a bit, however, none of this is true when applied to modern technologies, and these assumptions are not safe. The State of Trust on the Internet Every time you use an Internet-connected computer, you’re trusting someone (and most likely, a multitude) to act on your behalf. From an application’s code all the way down to the silicon, software and hardware and the network services they use reliably embed the interests of those that create them – and they may or may not be aligned with yours. Critically, those layers are usually – but not always – arranged in such a way that the interests of their producers and users are aligned. People creating computer chips are competing with other people creating chips, and so they focus on that; if they try to abuse their position by (say) exfiltrating your passwords in a side channel, the market (and possibly a legal regulator) will punish them. However, modern businesses have become adept at exploiting the gaps in this arrangement. Now, if you use a ‘smart’ watch or your phone to check the time, it’s likely more accurate but you have to contend with the possibility that it’s reporting your location, activities, and who knows what else back to its creator – and that they might be sharing that information with others. And that’s also the case for every other application running. Those abuses aren’t obvious, and it’s very easy for people to look at an Internet-connected device and fail to recognise that even though it’s “theirs” and that the data it processes is also “theirs”, they’re placing an inordinate amount of trust into a galaxy of faceless parties – trust that may not be deserved or protected. For example: TVs are widely known to spy on their users’ activities without consent . Meta decided to decrypt private traffic from ‘research’ users’ phones to competing services and store it on their own servers. Predictably, once the users found out, it ended up in court . At the same time, Facebook also let Netflix have access to users’ private Direct Messages , creating yet another lawsuit. Microsoft quietly changed the model of their ‘new Outlook’ e-mail client to surreptitiously send passwords for third-party e-mail servers to their cloud , so that they can share it with more than 700 of their closest friends (i.e., data brokers and advertisers). Various automakers collect detailed information and share it with other parties, including data brokers and insurance companies – to the point where it’s difficult to find a car that doesn’t violate your trust. Ring (i.e., Amazon) was so sloppy with their security practices that ‘rogue insiders’ as well as hackers exploited their access to people’s video cameras . Grindr shared highly sensitive health information with third parties without permission. Photobucket aggressively changed terms of service to allow AI use of people’s photos, but failed in court. This is just a small selection; there are many more. All of these are stunning violations of trust. And, it’s becoming normal . How did we get here? If I were to speculate on the reasons for that, I’d say it’s a combination of the normalisation of cloud computing (because everything is now running on or connected to computers you don’t control), the expectations of higher and higher growth and returns by investors, putting pressure on companies for new and recurring revenue, and – more than anything – the weakness of any regulating forces on these actors. User Agents are a Form of Collective Bargaining Although it’s difficult to trust anyone on the Internet given the examples above, it could be much, much worse. Imagine if you had to install a program on your computer from every company, government body, and other entity that you interact with, and those programs had full access to do what they like on your system. In other words, every online interaction becomes an opportunity to install malware that can extract your personal information, delete files or hold them ransom, profile and monitor your behaviour, and generally ignore your interests in favour of theirs. What prevents that on the modern Internet? In many cases, it’s the humble Web browser, which selectively exposes capabilities to Web sites without offering full access to your computer. This is called a User Agent – software that acts on your behalf, representing your interests in your interactions with other parties. And while the Web browser is representing your interests, it’s also balancing them with the interests of the sites that you visit – it’s an agent for them too . They want the page to render in a predictable way, but some users want to use accessibility tools. People don’t want to be tracked, but sites need some indication of how their pages are consumed. For the Web, all of these delicate tradeoffs are made within a framework of shared principles and values and decided in transparent fora using consensus processes – namely, the relevant standards bodies (usually, the W3C or IETF). There’s also more than one Web browser, so you can choose the agent that best represents your interests – thereby creating market pressure to do so. Importantly, this is done in a way that results in the same deal for everyone . If you had to negotiate what Web sites are allowed to do on your computer on a case-by-case basis, you’d quickly give up out of exhaustion (and indeed, we see this in cookie banners, a notable failure). In the bargain between big sites and individual users, the sites have more bargaining