Skip to content

created: 2026-04-24 updated: 2026-04-24 tags: [source, video, youtube, research, maintainability, study] type: source url: https://www.youtube.com/watch?v=b9EbCb5A408 author: "Dave Farley" (modern-software-engineering) published: 2026-01-28

We Studied 150 Developers Using AI (Here's What's Actually Changed...)

Summary

Dave Farley (Modern Software Engineering) discusses a published pre-registered controlled experiment with 151 participants (95% professional developers) measuring whether AI-assisted code is harder to maintain than human-written code.

Key Takeaways

Research Design

Phase 1: Developers add a feature to buggy code — some with AI assistants, some without.

Phase 2: A different set of developers are randomly assigned code produced in Phase 1 and asked to evolve it without knowing whether it was AI-assisted or not. No AI allowed in Phase 2.

This measures how easy code is for someone else to change later — a much better proxy for maintainability than "did the developer finish faster."

Key Findings

  1. No significant difference in maintenance cost between AI and human-generated code. AI code was no harder to change, no easier, no worse quality, no better quality. From a downstream perspective, AI didn't break anything.

  2. AI users were ~30% faster in initial development (Phase 1). Habitual AI users were ~55% faster.

  3. Experienced developers using AI habitually showed a small but measurable improvement in downstream maintainability. Not huge, but consistent. One explanation: AI tends to produce "boring, idiomatic, unsurprising code" — and boring code is good for maintainability.

  4. Developer skill matters more than AI usage. AI acts as an amplifier — if you're doing the right things, AI amplifies them. If you're doing the wrong things, AI helps you dig a deeper hole faster.

What "Doing the Right Things" Means (Jason Gorman)

  • Working in small batches
  • Solving one problem at a time
  • Iterating rapidly with continuous testing
  • Highly modular designs that localize the blast radius for changes
  • Organizing around end-to-end outcomes instead of role/technology specialisms
  • Working with high autonomy
  • Making timely decisions on the ground

Two Slippery Slopes Toward Disaster

  1. Code bloat — When generating code becomes almost free, the temptation is to generate far too much. Volume alone is a massive driver of complexity. AI makes it easier to drown in your own codebase.

  2. Cognitive debt — If developers stop really thinking about the code they create, understanding erodes, skills atrophy, and innovation slows over time. This doesn't show up in sprint metrics.

Conclusion

  • AI assistants improve short-term productivity
  • They do not appear to damage maintainability (might even slightly improve it when used well)
  • They don't remove the need for good engineering, good design, or thinking hard about problems
  • Compartmentalization through decomposition is the central technical skill — not typing speed
  • This is what differentiates good software development from slop, whether AI-generated or not