플록의 경찰용 AI 검색 도구

Comment Loader Save Story Save this story Comment Loader Save Story Save this story Flock Safety’s latest search tools give police several ways to find people on camera , not just the license plates on their cars . The capabilities sit inside a suite of software now at the center of dozens of police-stalking cases . WIRED collected and analyzed that software, and created a mockup of what officers see when interacting with the system. Among the tools is an AI-powered watchlist. An officer can cordon off an area of a city, and make the cameras inside it run a continuous, automated search for anyone fitting a written description. Over the past year, Flock has faced a backlash to its technology. Police departments across the country have canceled contracts, and city councils that once approved Flock have quietly voted the cameras back out. Two members of Congress wrote to CEO Garrett Langley after a Texas deputy searched more than 83,000 cameras for a woman who had an abortion. And Illinois found the company had let federal immigration agents reach state camera data in violation of state law. Flock answered in August with a package of changes that shortens the default period over which data are retained, requires case codes for every search, and adds automated auditing to catch officers misusing the tool—all of which Flock says will be mandatory by the end of the year. WIRED examined the code those promises run on, but much of what governs a search happens on Flock’s servers. Crucially, the analysis found that Flock’s guardrails may discourage some abusive uses of its tools, and will record them, but will not stop them. Experts who study automated screening and police technology say Flock has given its AI a job no technology does reliably. Flock’s system decides, out of sight of the department using it, which descriptions of a person an officer may search for—and no screen operating at that scale is accurate, the experts say. Further, nobody outside the company can measure how often it is wrong, or in which direction. This puts users in a bind: The company itself warns officers that results may be inaccurate or incomplete and should not be relied on alone, but puts the “risk of any inaccuracies” on the officer running the search. Some of those decisions the experts find harder to defend than others. Flock’s system scores an officer’s written description against several categories of sensitive content, including race and religion, and “political, social, and cultural expression” is the only one that will not stop a search. Tom Bowman, policy counsel with the Center for Democracy and Technology's security surveillance project, says the one sensitive category Flock guards with a speed bump instead of a stop is the one courts have guarded most closely. Political and cultural expression is “one of the most protective areas” under the First Amendment, he says, and it “seems like that actually has the least concern from a policing perspective.” WIRED sent Flock a detailed description of each of its findings along with 16 questions. The company responded with a written statement saying its search tools are built to help police find relevant information “while maintaining clear safeguards around how the technology can be used.” Queries are evaluated against its content policies, it says. The statement does not describe how its AI model was built, what instructions it works from, or what separates one verdict from another. Data flows in both directions. When the software flags a search, it sends Flock the language the officer used, the category and confidence score the model assigned it, and whatever the officer did next, whether continuing, dismissing, or disputing the warning. Flock’s statement does not address how long those records are kept, who inside the company can see them, and whether they are used to evaluate or improve the moderation system. Every Vote Counts Flock’s interface holds several tools, and each one works differently. A search that uses text to find images, for example, produces a statistical ranking of which footage best fits the words. Every image a camera captures is stored as a set of numbers describing what it shows, and any description typed into the search box is converted into numbers the same way. A model then compares the two, ranking the nearest footage at the top. Anything that might otherwise influence the rankings—additional filters, thresholds, or instructions fed to Flock’s model—remain out of view on the company’s servers. A feature called Smart Sort lets the officer approve or reject each image that comes back. A few votes rearrange the ranking, moving footage that resembles the approved images to the top. “Too many results?” a tooltip asks. “Vote on a few results to quickly refine.” An officer looking for a vehicle picks from menus of colors, body types, makes, models, and features like roof racks and bumper stickers; a person search, by contrast, includes no filtering and accepts only a written description. An example offered by the system: “person wearing scrubs.” Flock calls this plain-language search FreeForm, and says it has sold it for years. 404 Media reported in July that officers were using FreeForm to search for people using descriptions of clothing and tattoos. Standard vehicle search and FreeForm are separate capabilities, Flock says, with standard search running on defined vehicle attributes and FreeForm on natural-language descriptions. On video cameras, the company says, FreeForm searches “clothing, colors, objects, or other case-relevant details.” On license plate cameras, it is limited to vehicles, and person attributes cannot be searched there at all. The code, which WIRED collected using custom software that gathers files Flock sends to a user’s browser before they log in, sets out which searches officers can run, which controls and warnings they see, and how one result becomes the basis for the next. Out of reach on company servers are any actual search results, camera feeds, or department settings, along with the AI model that answers police prompts. The code WIRED collected holds Flock’s user interface and shows both the network calls it makes and the answers it expects in return. It shows that departments can, for now, configure the tool for some searches without a reason or a case number linked to a real investigation, and can leave off the automated system that flags suspicious use; Flock says the controls will be mandatory by year’s end. Search activity is recorded in audit logs, Flock says, and required search restrictions apply across its search tools. The company did not say whether anything on its servers would stop a search submitted without a reason or a case number. Fired, Suspended, Resigned Police officers have already used Flock’s software to track people for reasons unrelated to police work. At least 50 officers in the US have recently been charged with, or accused of, misusing license plate readers , according to The Washington Post. Forty-six of those cases involve Flock. In over half, the alleged target was a wife, a girlfriend, an ex-partner, an ex-partner’s new boyfriend, or a woman the officer wanted to meet. Details of how police databases can be abused appear repeatedly in the reports of officer misconduct: A Milwaukee officer searched for a woman he was dating 124 times and her former partner 55 times, and logged each search as an “investigation.” A Georgia police chief ran his ex-girlfriend and her daughter 600 times, was charged, and took his own life five months later. A Kansas chief, who was never charged, ran his ex 164 times and her boyfriend 64 times, and lost his badge. A North Carolina officer ran her boyfriend's ex-wife 31 times and filed 29 of them as traffic infractions. Police abusing investigative tools is not new. A 2016 Associated Press investigation found officers running database searches on exes and people they found attractive, with police employe