AI 코딩 에이전트, 연구 소프트웨어 혁신 이끌지만 '과학적 검증'엔 한계

AI coding agents can modernize research software but can't judge if the science is right Jonathan Kemper View the LinkedIn Profile of Jonathan Kemper Aug 1, 2026 Nano Banana Pro prompted by THE DECODER A field report from OpenAI and academic partners shows that coding agents can update and speed up aging research software. Much of the work, however, shifts from writing code to verifying the results. Many widely used research tools began as supporting code for a single paper. Small academic teams often wrote them without the time or resources for proper testing, maintenance, or optimization. The result is fragile software that remains critical to entire fields but requires constant repairs. A field report from OpenAI and academic partners suggests AI coding agents could help close that gap. The report documents eight case studies, mostly in biology, in which research groups used coding agents such as Codex and Claude Code. The projects range from basic maintenance and targeted optimization to full rewrites in modern programming languages. Coding agents delivered speedups of more than 60 times One of the simpler projects involved modernizing cyvcf2, a Python library for reading genetic data. GPT-5.5 replaced its outdated build and installation process with a modern one. The MHCflurry migration was far more involved. MHCflurry is an immunology model that predicts which targets immune cells will recognize. Claude Code and Codex alternated between the roles of developer and reviewer while porting about 10,000 lines of code from TensorFlow to PyTorch. The rustar-aligner project was more ambitious. It rebuilt STAR from scratch in Rust. STAR maps sequencing reads from cells to the corresponding locations in a genome. The original contains more than 20,000 lines of C and C++ and is no longer actively maintained, even though it remains part of many research pipelines. To check whether the rewrite behaved like the original, the team tested both tools on 10,000 short sequencing reads from yeast cells. For single-end reads, rustar-aligner produced the same result as STAR in 99.815 percent of cases. For paired-end reads, the agreement rate was 99.883 percent. The comparison covered more than the mapped location in the genome. It also included several other key fields that both programs produce for each read. Neither tool mapped any reads that the other failed to map. RustQC delivered the largest speedup by combining 15 separate quality-control tools in a single program. On a large dataset, runtime fell from 15 hours and 34 minutes to 14 minutes and 54 seconds, a speedup of more than 60 times. Another project, HelixForge, replaced a tool for generating synthetic genomic data with a version that runs on GPUs. In a test using data from one donor and a ten-million-base-pair section of the genome, HelixForge completed the full pipeline 59.6 times faster than BamSurgeon. The main compute step alone ran 98.6 times faster. Fast code can still produce bad science Across the case studies, agents completed well-defined tasks quickly but couldn't reliably judge whether their work was scientifically correct. Even when their code contained errors, the systems often presented it with full confidence. "With coding agents, it's quite easy to go fast; for now, to go far in science, there's still a need for expert guidance, understanding, taste, and care," writes cyvcf2 developer Brent Pedersen. Philip Ewels, who led RustQC, describes agents as "eloquent, convincing, and confidently wrong in ways that are easy to miss." He never allowed the models to judge the accuracy of their own work and instead built an independent test harness. The bayesm case study shows how difficult these errors can be to catch. Its Rust rewrite ran between two and twenty times faster than the original, but the first versions of two advanced methods contained errors that were hard to spot from the output alone. In one method, the agent inverted a key control parameter, causing the program to use the reciprocal of the intended values. A separate bug affected the calculation itself. Researchers only found it after running a detailed calibration test against thousands of synthetic datasets with known results. Another method, called HART, produced plausible results overall but still contained several flaws. These included needlessly expensive calculations and an incorrectly scaled correction factor. Plausible test results alone couldn't establish that the code was correct. An earlier attempt to port MHCflurry to PyTorch had failed in early 2025. Developer Sergey Feldman now attributes the failure to the models available at the time rather than the coding tools themselves. In his view, only newer model generations became reliable enough to handle much of this work on their own. Humans define the tests while agents write the code The projects followed a consistent division of labor. Humans defined the goals, success criteria, and validation methods, while agents handled implementation. The hifiasm project shows how this worked in practice. Hifiasm assembles a complete genome from many short fragments. Before asking GPT-5.5 to optimize it, the researcher built a test setup with separate training and validation datasets. The model then found changes that cut runtime on real human genome data by nearly 15 percent. HI.SIM, a library for simulating genetic data, required even less human involvement. GPT-5.2 found ways to optimize individual parts of the program in a single pass. A second pass with a newer model found more improvements. Together, the changes reduced runtime by about 31 percent without changing the output. Cheap rewrites create a maintenance problem The authors also provide rough estimates of the potential savings. If agents could resolve between one-quarter and one-half of all installation problems affecting research software, the saved research time across 100 packages would be worth between $600,000 and nearly $5 million. For NumPy alone, the report estimates that agents could save about 650 hours of maintenance work each year. Long-term maintenance remains a major open problem alongside validation and scientific accuracy. Low-cost rewrites could fragment user communities and spread experienced maintainers' already limited time even thinner. The teams took different approaches to ownership and maintenance. Some changes went directly into the original projects. Because STAR was no longer maintained, rustar-aligner moved to the scverse research consortium. The author of FastQC declined to replace the original tool with its Rust rewrite. The team instead added the improvements it had found to the original Java version, which achieved the same threefold speedup. The field report looks back at completed projects and relies on accounts from the people involved. Its authors stress that the findings don't come from a representative study. They still see the main bottleneck moving away from coding itself and toward validation, scientific review, and clear responsibility for maintenance and future development. The same pattern appears in software development outside research. A study by METR found that actual project maintainers would reject about half of the solutions that the widely used SWE-bench Verified benchmark rates as passing. A study of developer frustration with AI-generated code found a similar tradeoff. Time saved while generating code may instead be spent reviewing it. The curl project shut down its bug bounty program after AI-generated vulnerability reports consumed maintainers' time without producing useful results. The field report is part of OpenAI's broader push into science. The company has created a dedicated science team led by Kevin Weil, who expects 2026 to be for science what 2025 was for software engineering . In April, OpenAI introduced GPT-Rosalind, a model for life sciences research and released a freely available life sciences plugin for Codex that connects models to more than 50