우리가 챗GPT에 비밀을 털어놓는 이유: 엘리자(ELIZA)가 남긴 예언

Comment Loader Save Story Save this story Comment Loader Save Story Save this story In the 60 years that ELIZA has been influencing computation and culture, conventional accounts portray it as the earliest example of what we now call chatbots , one that could converse as an automated psychologist. The deceptively simple program is known for “fooling” even the secretary who watched MIT professor Joseph Weizenbaum create it. That’s how the story goes. However, in all those accounts—even after all its adaptations across programming languages and research fields, in classrooms and popular culture—one essential piece of the story has been missing: the source code for the ELIZA program itself. Our new book, Inventing ELIZA , recovers this source code from the MIT Archives, offering for the first time a close reading and discussion of that code along with newly uncovered dialogs for ELIZA scripts beyond its popular “DOCTOR” persona. This investigation revealed many ELIZAs: in its different program versions, designed to run a wide variety of scripts or personas, built using a series of technical innovations. Inventing ELIZA seeks to correct and to complicate ELIZA’s history and influence by exploring the misconceptions, multiple versions, and missing code of ELIZA. In this excerpt from the book, we examine one of ELIZA's earliest interactions, how it laid the groundwork for human relationships with computers for decades to come, and how the complex program continues to speak to the unrestrained drive of today's AI industry. Men are all alike. IN WHAT WAY They’re always bugging us about something or other. CAN YOU THINK OF A SPECIFIC EXAMPLE Well, my boyfriend made me come here. YOUR BOYFRIEND MADE YOU COME HERE He says I’m depressed much of the time. I AM SORRY TO HEAR YOU ARE DEPRESSED That dialog has been reprinted countless times and has inspired programmers and writers to dream up many of the chatbots that followed. Yet the closer one inspects that dialog, the more questions arise: Who was this young woman? Was she a real person, or is she the invention of ELIZA creator Joseph Weizenbaum? How exactly did the ELIZA system generate its responses, and how much were they edited? Why did the system work so well to draw people in? ELIZA, and her “DOCTOR” persona, helped catalyze a mode of thought and an anxiety about people’s relationships with computers. Weizenbaum explored this in his 1976 book Computer Power and Human Reason , invoking philosophical, social, and political critiques. The unique machine interaction presented by his program revealed how new forms of human-computer relation would have profound effects that he attempted to explore and to contest. After seeing its public reception, Weizenbaum was startled by the quick and often emotional attachments people would form with ELIZA, which he saw as “clear evidence that people were conversing with the computer as if it were a person who could be appropriately and usefully addressed in intimate terms.” The tendency to attribute empathy and invest private feelings into a computer puzzled Weizenbaum. He was concerned by the extent to which people associated rationality with computation, and ascribed understanding and intelligence to computer systems where none existed. This tendency became known as the “ ELIZA effect .” By 1991 the term was appearing in online forums, but its use predated that appearance by decades. Sociologist Sherry Turkle defines “the ELIZA effect” as “our more general tendency to treat responsive computer programs as more intelligent than they really are. Very small amounts of interactivity cause us to project our own complexity onto the undeserving object.” Cognitive and computer scientist Douglas Hofstadter describes it as “the susceptibility of people to read far more understanding than is warranted into strings of symbols—especially words—strung together by computers,” which applies easily to generative AI systems today. To understand the power and provocation of ELIZA, we can look to the infamous challenge formulated by computer scientist Alan Turing in the essay “ Computing Machinery and Intelligence ,” in which Turing posed the question “Can Machines Think?” Turing premised his thought experiment on a parlor game—not about technology but about gender: A man and a woman are hidden in a separate room and an interrogator tries to identify who is which gender by asking a series of questions. The man tries to mislead the interrogator, pretending to be a woman, while the woman tries to convince the interrogator of the “correct” answer. That is, both of them claim they are the “real” woman, a challenge to essentialist notions of gender. In his revision of this game, Turing replaces the original gender question with what is now called the Turing test. In this challenge a machine pretends to be a man, rather than a man pretending to be a woman. More than a mere opening gambit, Turing’s choice of the initial gender imitation game has ensured that artificial intelligence would remain intertwined with questions of gender and identity. In this sense imitation, drag, and gender deconstruction laid the groundwork for AI and the performance of intellect. Weizenbaum’s ELIZA picks up where Turing left off, not least with the opening line of the dialog: “Men are all alike.” Nonetheless, even though Weizenbaum references Turing’s imitation game in his 1966 paper introducing ELIZA, he explicitly distances his creation from any claims of intelligence: The crucial test of understanding … is not the subject’s ability to continue a conversation, but to draw valid conclusions … In order for a computer program to be able to do that, it must at least have the capacity to store selected parts of its inputs. ELIZA throws away [most] of its inputs … ELIZA in its use so far has had as one of its principal objectives the concealment of its lack of understanding. This assessment shows that ELIZA was never intended to pass the Turing test, but rather to explore the psychological factors that might lead humans to misinterpret its capabilities. In that tradition, ELIZA continues to play with the notion of performative identity, as a vehicle for personas made out of responses that stick to the script, literally and figuratively. By naming the system after Eliza Doolittle—a working-class female character who is taught to pass as an upper-class white woman—Weizenbaum continues Turing’s provocation about the performance of identity. “I chose the name ‘Eliza,’” Weizenbaum says, “because, like G.B. Shaw’s Eliza Doolittle of Pygmalion fame, the program could be taught to ‘speak’ increasingly well, although, also like Miss Doolittle, it was never quite clear whether or not it became smarter.” As Shaw’s Eliza performs race and ethnicity, class, sexuality, and gender through linguistic transformation, the ELIZA system performs an assumed persona through scripted and repetitive linguistic patterns and transformations, but without possessing humanlike understanding. Feminist philosopher Judith Butler’s theories of gender performativity offer a possible framework to think about this. Butler argues that gender and sexuality are not innate but are performed through repeated acts. They are iterated. Just as Eliza Doolittle challenges assumptions of class by performing speech that help her pass as among the upper classes, Weizenbaum’s system performs personas like DOCTOR through speech acts, or code acts, and sample dialogs that perform gendered, classed, and racialized identities. In this way, ELIZA both reinforces and challenges assumptions about gendered and embodied communication. It is interesting to note that in the published dialogs and popular stories about ELIZA, the women who appear alongside the program remain unnamed. They are conversing with a therapist called DOCTOR, a name that, though it does not carry gender markers today, would have sounded like a man’s title in the 1960s. Therefore, stories that sug