Why AI Coding Agents Still Reward Developer Expertise

Research from 400,000 Claude Code sessions shows that coding agents can execute more work, but knowledgeable users still define better goals and recover from failures.

AI coding agents are rapidly moving beyond autocomplete.

Modern agents can inspect repositories, edit several files, run commands, execute tests and continue working across multiple steps. GitHub has introduced an agent-focused interface in Visual Studio Code, remote agent sessions and infrastructure for synchronizing agent state across clients. Researchers have also detected hundreds of thousands of agent-attributed open-source commits per month.

It is tempting to interpret this progress as evidence that software knowledge is becoming less important. Recent research suggests a more nuanced conclusion: agents can perform more execution, but knowledgeable users still produce better outcomes because they understand what should be built, what can go wrong and what evidence counts as success.

The emerging skill is not simply writing code manually or producing clever prompts. It is directing, constraining and verifying an increasingly capable software agent.

What researchers found in 400,000 Claude Code sessions

In June 2026, Anthropic published a privacy-preserving analysis of approximately 400,000 Claude Code sessions involving around 235,000 people. The sessions occurred between October 2025 and April 2026.

The study examined the work users attempted, how decisions were divided between the user and the agent, and whether sessions produced evidence of success.

Approximately 56% of the analyzed sessions involved writing, fixing, testing or orchestrating code. Operating software accounted for another 17%. Planning and exploring represented 14%, while analysis and document work represented 13%.

These figures show that a coding agent is no longer used exclusively to generate functions. It increasingly participates in deployment, configuration, investigation and other work surrounding production software.

The most interesting finding concerned the division of responsibility.

On average, people made approximately 70% of planning decisions. Claude made about 80% of execution decisions.

In practical terms, users usually decided:

What problem to solve
Which approach was appropriate
What constraints mattered
What the finished result should accomplish

The agent more often decided:

Which files to inspect
What code to write
Which commands to execute

How to implement the requested change

This looks less like replacing a developer and more like changing the developer’s position in the workflow. The human moves toward specification, architecture and verification while the agent absorbs more implementation work.

Expertise makes each instruction more productive

The study categorized task-specific user expertise using signals such as the precision of instructions, the verification users requested and whether users corrected the agent’s misunderstandings.

This distinction matters because expertise was specific to the task—not merely a person’s job title.

A senior JavaScript engineer using Rust for the first time may be a novice for that task. Meanwhile, an accountant with limited programming experience may be the expert when defining reconciliation rules, financial edge cases and the conditions a program must satisfy.

In sessions classified as novice-level, one user instruction triggered approximately five agent actions and around 600 words of output. In expert-rated sessions, an instruction triggered approximately 12 actions and around 3,200 words.

The important lesson is not that more output is automatically better. It is that knowledgeable users gave the agent enough direction and context to continue productively for longer.

A vague instruction forces an agent to guess:

Fix the payment bug.

A domain-informed instruction establishes a testable boundary:

Investigate duplicate payment records created when the provider retries
a successful callback.

Do not change the provider verification logic.

Before editing:
1. Trace the callback from the route to the database write.
2. Identify whether the transaction reference has a unique constraint.
3. Add a regression test that sends the same verified callback twice.

The second request must return a successful idempotent response without
creating another payment record.

The second instruction does not contain the solution. It supplies context, constraints and a definition of success. That lets the agent explore implementation details without inventing the product requirement.

Successful sessions contain verifiable evidence

The researchers did not treat a confident final response from the agent as proof that a task succeeded.

Their strictest success measure looked for evidence such as:

Passing tests
Commits or pull requests matching the intended work
Code changes supported by the session
Explicit confirmation from the user

Under this stricter definition, novice-rated sessions reached verified success approximately 15% of the time. Sessions rated intermediate or above reached it approximately 28–33% of the time.

The difference became particularly visible when sessions encountered trouble. Among troubled sessions, verified success increased from around 4% for novice-rated users to 15% for expert-rated users.

This suggests that expertise is especially valuable after the first approach fails.

An experienced developer can recognize that:

A passing unit test does not cover the production failure
A database migration may lock a large table
A frontend fix may introduce a hydration mismatch
A retry can create duplicate side effects
A security check is occurring after sensitive data is returned
A local success depends on an uncommitted environment file

Agents can run tests, but people still need to decide whether those tests represent the real risk.

More merged code is not necessarily more valuable code

A separate July 2026 study examined the early rollout of Claude Code and GitHub Copilot CLI across tens of thousands of Microsoft engineers.

The researchers estimated that adopters merged approximately 24% more pull requests than they otherwise would have during the four-month study window.

That is meaningful evidence that command-line coding agents can increase measurable engineering output. However, the authors explicitly acknowledged an important limitation: a merged pull request is not the same as delivered value.

A team can merge more changes while also creating:

Larger review queues
More fragmented pull requests
Additional maintenance work
Subtle security regressions
Increased architectural inconsistency
Tests that validate implementation rather than requirements

Throughput should therefore be evaluated alongside defect rates, rollback frequency, review time, production incidents and customer outcomes.

An agent that generates ten pull requests is not necessarily more useful than one that helps resolve a single costly reliability problem.

Coding agents are already part of real software development

An open-source study submitted in June 2026 examined 180 million repositories and detected more than 320,000 agent-attributed commits per month across its observed snapshots.

The researchers reported 886,122 Claude Code-attributed commits across 17,295 projects. They also found that different detection methods revealed different kinds of agent activity: pull-request-based analysis often captured feature work from cloud agents, while commit-based analysis found more maintenance activity from editor and command-line tools.

These results should not be interpreted as a complete measurement of all AI-generated code. Agent attribution can be removed, changed or omitted, and different tools leave different evidence.

Nevertheless, the scale demonstrates that agentic coding is no longer a small laboratory experiment. AI agents are contributing to real repositories, which makes responsible supervision an immediate engineering concern.

If you are new to the concept, Cognora’s guide to agentic AI explains how agents differ from ordinary chat interfaces.

A practical workflow for directing coding agents

Developers do not need to supervise every generated line manually. They need a workflow that prevents the agent from silently changing the problem.

1. Define the outcome before requesting implementation

Start with observable behavior.

Weak:

Improve authentication.

Better:

After five failed sign-in attempts from the same account within 15 minutes,
temporarily block additional attempts for 20 minutes.

Existing authenticated sessions must remain valid.

Add tests for the attempt limit, expiry and successful login after expiry.
Do not reveal whether an email address exists.

The improved request establishes the security behavior without dictating every implementation decision.

2. Ask the agent to investigate before editing

For unfamiliar or high-risk code, separate diagnosis from modification:

Do not change files yet.

Trace how authentication failures are recorded, identify the relevant models,
routes and middleware, and list the security assumptions you find.

Then propose the smallest safe implementation.

This produces an intermediate artifact that can be reviewed before the agent changes the repository.

It also reduces the chance that the agent confidently patches the first suspicious function without understanding the wider flow.

3. State what must remain unchanged

Agents optimize around the instructions they receive. Important invariants should be explicit:

Preserve public API behavior
Do not change database schemas
Maintain backward compatibility
Do not add dependencies
Keep authentication cookies secure
Preserve existing deployment configuration
Do not modify unrelated files

Constraints are not unnecessary prompt detail. They describe the boundaries of an acceptable solution.

4. Require proportionate verification

“Make sure it works” is not a verification plan.

A stronger instruction names the required evidence:

After implementing the change:

1. Run the affected unit tests.
2. Run the complete test suite.
3. Run lint and the production build.
4. Show the exact files changed.
5. Explain any warnings.
6. Identify one remaining risk not covered by tests.

For a database or deployment change, verification may also require a migration rehearsal, staging deployment or rollback test.

5. Review the diff, not only the agent’s summary

An agent-generated summary is useful for orientation, but it is not an independent review of the change.

Inspect:

Unexpected file modifications
Removed validation
Broad exception handling
Hard-coded secrets or URLs
Disabled tests
New dependencies
Changes outside the requested scope
Generated code that duplicates existing utilities

When a change affects authentication, payments, authorization, encryption or destructive operations, require a second review by a person with relevant expertise.

6. Keep changes small enough to understand

Large autonomous tasks can hide incorrect assumptions inside hundreds of changed lines.

Prefer a sequence such as:

Investigate the current behavior
Add a failing regression test
Implement the smallest fix
Run verification
Review the diff
Refactor only if necessary

This structure produces checkpoints where incorrect direction can be detected early.

It also helps prevent context loss, a problem discussed in Cognora’s article about Claude Code context eviction.

Where beginners can still benefit

The research does not say that only senior developers should use coding agents.

The largest improvement in success appeared between novice and intermediate task expertise. The gap between intermediate and expert users was smaller.

That suggests developers do not need complete mastery before an agent becomes useful. They need enough understanding to:

Describe the desired outcome
Recognize important constraints
Read the resulting changes
Interpret test failures
Detect obviously unsafe behavior
Ask for evidence instead of accepting confidence

This creates a practical learning path. A beginner can ask the agent to explain the existing system, identify unfamiliar concepts and propose tests before writing code.

Used this way, an agent can support learning rather than bypass it.

The dangerous pattern is accepting a solution that the user cannot explain, test or maintain.

What changes for software engineering careers?

Coding agents are likely to reduce the time spent on some implementation-heavy tasks. They can already perform repository searches, repetitive refactors, test generation and routine debugging with increasing autonomy.

But the current evidence does not show that understanding the problem has become optional.

Instead, valuable skills shift toward:

Translating business needs into precise system behavior
Evaluating architectural trade-offs
Identifying security and reliability risks
Designing meaningful tests
Reviewing generated changes
Recovering when automation fails
Deciding what should not be automated

This aligns with Cognora’s broader analysis of jobs affected by AI: tasks change before entire professions disappear.

The developer who only converts detailed tickets into predictable code faces more automation pressure. The developer who understands systems, users and failure modes becomes more capable because an agent can execute more of that person’s decisions.

Limitations of the evidence

These findings should be interpreted carefully.

Anthropic studied usage of its own product and used classifiers to estimate expertise, decisions and success. Although the analysis included telemetry checks and statistical controls, it did not directly observe every real-world outcome.

The Microsoft study used merged pull requests as a productivity proxy. A merged pull request measures activity, but not necessarily code quality, customer value or long-term maintainability.

The open-source census depended on detectable agent signatures. It may miss unattributed usage and should not be read as a complete count of AI-written code.

Together, the studies provide useful evidence of adoption and changing workflows. They do not prove that every team, tool or task will experience the same result.

The real advantage is controlled delegation

The central lesson from current coding-agent research is not that agents replace expertise. It is that expertise becomes leverage.

A knowledgeable user can give an agent clearer objectives, allow it to perform more execution per instruction and recognize when the result is incomplete. A less experienced user may still work faster, but is more vulnerable to plausible-looking mistakes and unsuccessful recovery attempts.

The safest operating model is controlled delegation:

Humans own goals and constraints
Agents perform bounded execution
Tests provide evidence
Review checks the reasoning and diff
Production outcomes determine whether the work was valuable

AI coding agents are becoming more autonomous, but autonomy does not remove responsibility. It increases the importance of deciding what the agent may do and how its work will be verified.