코딩 에이전트로 인해 코드 품질이 저하된다는 우려는 무작정 PR을 병합하는 경우에만 해당된다는 글입니다. 스펙 기반 개발과 AI 리뷰, 95% 이상의 단위 테스트 커버리지, 수동 테스트, 광범위한 E2E 테스트라는 다층적 품질 관리를 적용하면 버그를 오히려 줄이면서 생산성을 2-3배 높일 수 있다고 설명합니다.
번역된 본문
AI 코딩이 코드 품질을 떨어뜨린다면, 품질 관리 방식이 잘못된 것입니다
Iouri Khramtsov, 2026년 9월 14일
코딩 에이전트에 대해 자주 보는 견해 중 하나는 이렇습니다: "AI가 코드 생산량을 늘려주긴 하지만, 품질이 떨어지지 않을까?" 물론 PR을 무작정 병합해서 프로덕션에 배포한다면 틀림없이 그렇게 될 것입니다. 하지만 신중하고 다층적인 접근으로 품질을 관리한다면, 버그 수를 유지하는 것뿐 아니라 실제로 줄이면서도 생산량을 2배 이상 늘리는 것이 가능하다는 것을 저는 경험했습니다. 이러한 방어 계층 중 상당수는 Claude/Copilot/Codex 등이 등장하기 전과 거의 동일하지만(지금은 AI 덕분에 더 쉬워졌습니다), 일부는 새로운 것입니다. 다음은 제 팀과 다른 곳에서 실제로 성공적으로 사용된 것을 본 방어 체계입니다.
계층 1: 요구사항 제대로 잡기
스펙 기반 개발(spec-driven development)을 시작한 후 가장 큰 놀라움 중 하나는 새로 작성된 코드의 버그가 급감한 것이었습니다. 스펙 기반 개발 이전에는 새로운 기능을 만들 때 팀이 전체 노력의 최대 3분의 1을 개발 후 '다듬기', 즉 각종 버그를 발견하고 수정하는 데 소비했습니다. 이러한 버그 상당수는 예상치 못한 상호작용과 엣지 케이스를 간과했거나, 개발자가 그날 피곤해서 충분히 생각하지 못했거나, 디자이너나 PM이 특정 시나리오를 제대로 고려하지 않았기 때문에 발생했습니다. 일부 버그는 놓쳐져 프로덕션까지 가기도 했습니다. 스펙 기반 개발을 시작한 후 제 코드에서 이런 버그가 급격히 줄었고, 팀원 중 일부(전원은 아님)에게서도 같은 감소를 확인했습니다. 제가 보기에 이 감소의 주된 원인은 프로세스의 특정 단계, 즉 AI에게 요구사항이나 기술 설계를 검토시켜서 빠진 부분, 엣지 케이스, 기존 코드와의 예상치 못한 상호작용 등의 문제를 찾게 하는 것입니다. AI는 피로해지지 않고, 프롬프트를 잘 주면 잠재적 문제를 찾는 것을 포기할 가능성이 훨씬 적습니다. 오히려 때로는 과도하게 열심이라서, 요구사항에 대한 수정 제안에 존재하지 않는 문제를 지어내지 않는지 제가 꼼꼼히 검토해야 할 정도입니다.
계층 2: 95% 이상 커버리지의 단위 테스트
이제 코딩 에이전트 덕분에 테스트 주도 개발(TDD)이 너무 쉬워져서 하지 않을 이유가 없습니다. 다만 올바르게 해야 합니다. 에이전트가 자신이 방금 코드에 넣은 버그에 대해 무작정 통과하는 테스트를 작성하게 해서는 안 됩니다. 제가 본 가장 우수한 계획 및 구현 방식은 대개 이런 패턴을 따릅니다: 에이전트에게 요구사항 기반으로 테스트 시나리오와 테스트 케이스를 생각하게 하고, 테스트 케이스를 작성하고, 구현을 작성하고, 테스트 케이스에 대해 구현을 검증하며 문제를 수정하게 하고, 남은 커버리지 공백을 요구사항을 염두에 두면서 보완하게 합니다. 또한 에이전트가 테스트를 작성해주는 지금은, 거의 전면적인 커버리지를 목표로 하지 않거나 빠진 단위 테스트를 보완하지 않을 변명이 없습니다.
계층 3: 수동 테스트
여전히 인간(본인, QA, PM 등)이 직접 기능을 사용해보고 모든 엣지 케이스를 살펴보며 예상대로 작동하는지, 변경이 필요한지 확인하는 것을 대체할 수 없습니다. 이러한 수동 테스트는 시간이 걸릴 수 있으며, 특히 테스트 시나리오를 설정하는 데 공이 들면 더 그렇습니다. 이는 지금까지 생산성 향상이 미미했던 단계 중 하나이며, 제 생산량이 10배가 아닌 2-3배만 늘어난 주된 이유입니다. 그런데 생각해보면 여기에 제가 놓친 자동화 기회가 몇 가지 있을지도 모르겠습니다.
계층 4: 광범위한 자동화 엔드투엔드(E2E) 테스트
엔드투엔드(E2E) 테스트는 새로운 변경이 최종 사용자가 경험하는 기존 기능을 망가뜨리지 않았는지 검증하기 때문에, 논쟁의 여지 없이 코드베이스에서 가장 중요한 테스트입니다. 이상적으로는 PR에서, 테스트/스테이징 환경에서, 그리고 매 배포 후 프로덕션에서 실행되어야 합니다.
If AI coding is lowering your code quality, you’re not managing quality right Iouri Khramtsov Sep 14, 2026 Share One common take on the coding agents that I see goes something like this: “Sure, AI helps you output more code, but won’t the quality suffer?” It certainly will if you just blindly merge the PRs and send them off to prod. But if you take a thoughtful, layered approach to managing quality, I find that it’s possible to not just keep the number of bugs stable but actually reduce it—while still increasing the output by 2-2x. Many of these defensive layers are pretty much the same as before Claude/Copilot/Codex/etc. (though they’re made easier now by AI), while others are new. Here’s a defensive setup that I’ve seen successfully used in practice, both on my team and elsewhere. Layer 1: Getting the requirements right One of the biggest surprises after I started using spec-driven development was the drop in bugs in the freshly written code. Before spec-driven development, when building, e.g., a new feature, the teams I was on often spent up to a third of the total effort on the post-development “polishing,” i.e., discovering and fixing various bugs. Many of these bugs occurred either because we didn’t foresee certain interactions and edge cases, or because the developer was tired that day and didn’t put in enough thought, or because the designer or PM didn’t think through certain scenarios. Some of these bugs were missed and ended up in production. After I started using spec-driven development, the number of these bugs in my code sharply dropped, and I’ve seen the same drop for some (but not all) of my teammates. As far as I can tell, the main cause of this drop is one specific step in the process: having the AI review the requirements or the tech design and find any gaps, edge cases, unexpected interactions with the existing code, or other similar problems. The AI doesn’t get tired and, when prompted right, is a lot less likely to give up hunting for potential issues. If anything, it can sometimes be overzealous, and I have to carefully review its proposed edits to the requirements to make sure that it doesn’t invent any issues that aren’t there. Layer 2: Unit tests at >95% coverage Coding agents now make test-driven development (TDD) trivial to the point where there’s no reason not to do it. However, it needs to be done right: you don’t want the agent to blindly write passing tests for any bugs it just added to the code. So the best planning and implementation skills I’ve seen usually follow this pattern: Instruct the agent to think through the test scenarios and test cases based on the requirements, Write the test cases, Write the implementation, Test the implementation against the test cases and fix any issues that come up, Maybe backfill any remaining coverage gaps—but again, keeping the requirements in mind. Also, with the agents writing the tests, there’s no excuse not to shoot for near-universal coverage or to wait on backfilling any missing unit tests. Layer 3: Manual testing There’s still no substitute for a human (you, QA, PM, or someone else) actually trying out the feature, going through all the edge cases, and seeing whether everything works as expected or whether you need to make changes. These manual tests can take a while, especially if the test scenarios take some effort to set up. This is one of the steps that so far has seen only modest gains in productivity, and it’s the main reason that my output has increased only 2-3x instead of something like 10x. Though now that I think about it, there may be a few opportunities for automation here that I’ve missed. Layer 4: Extensive automated end-to-end tests End-to-end (E2E) tests are arguably the most important tests in the codebase because they verify that new changes haven’t broken any existing functionality as experienced by the end user. Ideally, they’d run on the PRs, in the test/stage environments, and in production after every deployment. Ideally, they’d also be maintained by the same developers who write regular code, but I understand that some organizations aren’t really set up for that. AI does make it easier to write E2E tests, but to do that effectively, it needs access to the tools or MCP servers that let it debug test failures—e.g., a browser tool or MCP access to the logs. However, it’s important to keep in mind that E2E tests aren’t a substitute for manual testing because they’re just a rough, incomplete check that nothing important broke. Layer 5: Code quality passes by AI I find that coding agents aren’t great at following complex instructions in AGENTS.md or CLAUDE.md . But they do pretty well if you add a separate pass to find and fix specific issues. These can be: Security issues, Finding overcomplicated or duplicated code, Compliance with naming, file organization, or formatting rules, A general code review pass to find any issues with the logic, Overly long comments written in AI-ese instead of regular English, Any other specific things that you’d like to find and fix. If added to the planning or implementation skills, these can be pretty much “free” additions, adding maybe 5-15 min to the implementation time with no additional attention required. They can be also added to the PR reviews if you prefer to take a look at the comments before applying any fixes. Layer 6: PR reviews by humans and AI I think I’m becoming convinced that for minor tweaks and simple bug fixes, human reviews can become optional. Provided that other defensive layers are still in place. But for complex changes, I find that it’s still necessary to review AI-written code. I still regularly find big-picture mistakes, missed adverse interactions with other features, overcomplicated or suboptimal implementations, and other problems. Not to mention weird word choices like “mint” instead of “generate” or “stamp” instead of “set.” AI code reviews have also been a really great addition. On my current team, we run both Claude and Cursor reviews on the PRs, and surprisingly, each of them finds different problems. You can also add other custom reviews from various angles, like security, efficiency, interactions with other repos, and so on, though be aware that AI can be overly nitpicky in its reviews, so it’s important to also have a pass where another agent prunes the proposed AI-generated PR comments that aren’t actually meaningful. Layer 7: Monitoring and alerting Once the code is in production, at a minimum, it’s good to have someone periodically scroll through the logs or watch any user recordings in something like Fullstory, or review various dashboards that track error rates, latencies, and other issues. Even better would be an error tracking service like Sentry or GCP’s Error Reporting that detects and deduplicates errors. The best approach, however, would be to then have Claude/Cursor/whatever auto-diagnose these errors, figure out the root cause, and make PRs with the proposed fix. Like what you see? Subscribe for more: Subscribe Conclusions I’m sure I’ve missed other important components of maintaining high quality, but the main idea is that with the right set of defensive layers, the increased output doesn’t have to come at the cost of reliability. If anything, coding agents now make it cheaper to add more and deeper checks than before: more tests, more review passes, faster diagnosis of production issues. So if you’re sufficiently focused on quality, I think it’s entirely possible to double the delivery speed while keeping the bugs under control. Or maybe even reducing them. Share