AI 붐 야기한 하드드라이브 대란, 인터넷 아카이브 위기

Skyrocketing hard drive and storage costs caused by the AI data center boom are making it more expensive and more difficult for digital archivists, academics, Wikipedia, and hobby data hoarders to save data and archive the internet. Specific drives favored by some high profile organizations like the Internet Archive have become far more expensive or are difficult to find at all, archivists said. Over the last several months, prices for both consumer level and enterprise solid state drives, hard drives, and other types of storage have skyrocketed. As an example, a 2TB external Samsung SSD I purchased last fall for $159 now costs $575. PC Part Picker, a website that tracks the average price of different types of drives, shows a universal increase in storage prices starting in about October of last year. Prices of many of the drives it tracks have doubled or increased by more than 150 percent, and at some stores SSDs and hard drives are simply sold out . There is now even a secondary market for some SSDs, with people scalping them on eBay and elsewhere. Brewster Kahle, founder of the Internet Archive and the Wayback Machine, the most important archiving projects in the history of the internet, told 404 Media that the skyrocketing costs of storage is “a very real issue costing us time and money.” “We have found that the preferred 28-30TB drives are just not available or at very high price,” Kahle said. “We gather over 100 terabytes of new materials each day, and we have over 210 Petabytes of materials already archived on machines that need continuous upgrades and maintenance, so we need to constantly get new hard drives.” “We are fortunate to have an active community that donates to the Archive, and we are also looking for help from hard drive manufacturers in these difficult times. We are always looking for more help,” he added. “So far we have ways to work around these shortages, but it is a very real issue causing us time and money.” The Wikimedia Foundation, which runs Wikipedia and various other projects, including Wikimedia Commons, an open repository of royalty free media, told 404 Media that the cost of storage has become a concern for the foundation’s projects as well. “With over 65 million articles on Wikipedia alone, access to server and storage capacity is vital to us. We’ve certainly seen price increases since the end of 2025.These price increases are of concern to us, as with every other player in the industry. We see the primary impact in the purchase of memory and hard drives but also in terms of lead times on server deliveries and our capacity to place future orders,” a Wikimedia Foundation spokesperson told us. “The Wikimedia Foundation is a non-profit, and as such how we allocate budget is very carefully considered . We maintain our own data centers to serve our users from all over the world. We’re putting workarounds in place where we can, mainly involving being smart with how we prioritize investment in hardware, building in flexibility as well as extending the life of existing hardware where possible.” 💡 Have you been affected by skyrocketing SSD or RAM prices? I would love to hear from you. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co. Western Digital , one of the largest manufacturers of hard drives and other storage systems, said that it has essentially sold out of its 2026 inventory to enterprise clients, many of which run data centers. Micron, which made RAM and SSDs under the brand name Crucial , has exited the consumer market altogether because “AI-driven growth in the data center has led to a surge in demand for memory and storage. Micron has made the difficult decision to exit the Crucial consumer business in order to improve supply and support for our larger, strategic customers in faster-growing segments.” The AI boom is thus harming critical archiving projects in multiple ways. As a reaction to AI companies indiscriminately scraping the entire internet to train their large language models, website owners have increasingly put up registration walls, blocked web scrapers by changing their robots.txt to disallow bots, and have otherwise attempted to stop bots from accessing their websites. Many of these websites have either accidentally or purposefully ended up blocking bots from the Internet Archive and other archiving projects. The Electronic Frontier Foundation suggested “blocking the Internet Archive won’t stop AI, but it will erase the web’s historical record.” Beyond that logistical challenge, archivists are now needing to make difficult decisions about how and what to archive because they are, in some cases, simply running out of storage. Mark Phillips, a University of North Texas professor who helps runs the End of Term Archive, which archives government websites between changes in presidential administrations, told 404 Media that he has had to consider the price of infrastructure recently: “When we went to refresh some of our servers, the costs of the RAM and SSDs for those machines were a dramatic increase and made us rethink some of the capacity we were hoping to go with,” he said. “We have not had to do any major storage purchases in the past six months, and I hope that by the time we do the market will have leveled out a bit.” The cost of storage has become a constant topic of discussion on Reddit’s r/DataHoarder community, where digital librarians and hobby archivists discuss different archiving setups; many posts are from people who say they have simply had to stop buying drives, have had to put their archiving plans on hold, or are looking to vent about the price of drives. Occasionally, there are posts from people who managed to find a large drive for a decent price on clearance or at a thrift store. Many of these posts are from people who say that they have essentially given up on archiving new content until prices go down: “I've decided to just call it quits for now. I don't really download much anymore. I just maintain my current data.” “Slim pickings currently. Check Facebook marketplace as occasionally a deal can be had there especially from people who accidentally bought a sas drive and can't use it.” “I'm looking for efficient ways to use older smaller drives that I have laying around doing nothing, because I need more space for backups. I can't see buying a 28tb drive right now. I've started adjusting my backup retentions to stretch the space I have.” “Bust out your wallet is the only way or try to ride this out and hope prices come down.” “You don't [buy new drives] right now. Better pray we actually get drives going forward.” “Every vendor i worked with offered me a dinner and told me wait when i asked for a rather large quote.” “Bwwaahahahahahahahahhahaha.....not until 2029...MAYBE. All the AI/datacenters have prepurchased hard drives.” The question that seems to be on everyone's mind is how long will this shortage last, and will the price of storage ever go down again? About the author Jason is a cofounder of 404 Media. He was previously the editor-in-chief of Motherboard. He loves the Freedom of Information Act and surfing. More from Jason Koebler