Nobody Opened a .py File: Agentic Coding at Production Grade

Can an AI coding agent ship production code that nobody reads?

Engineering organisations that adopt coding agents typically adopt them under one condition: a senior engineer reads what the agent wrote before it ships. The agent drafts, a human reviews, and the review is where responsibility lives. That arrangement feels prudent. It is where most teams start.

We run a customer support automation solution at VIS Solutions that does not work that way. It handles first line for real client tickets in production. We specified it, tuned it and took it to production capability, and I never opened a single Python file in it. Not as a stunt. That was the working method. The approach reflects our broader thinking on agentic AI: when you design the specification and evaluation rigorously, the implementation becomes trustworthy without line-by-line review.

So here is what that actually took. Where my hours went, which parts still get a human review, how we train people for this, who to hire once the code stops being the hard part, and what your development team should expect in its first month.

What does "the Python was never read" actually mean on a system this size?

The solution is not a script. Its root folder holds 24 items, more than half of them folders, and each of those folders has several items of its own. Not all of them are code: many are configuration files or Markdown documents. The Python itself runs to 12,791 lines. I counted that for the first time while talking this system through — it had never been measured before, because the line count was never something anybody needed to know.

The exclusion is specific. It covers implementation code, not documentation.

“Having said I never read the Python does not mean I don't read the Markdown. Those documents are the solution's context, so they not only help me point the agent in the right direction — they teach me more about agentic AI in the process.

That distinction is the whole shape of the working method. What I own and maintain is the context: the specifications, the instructions, the documentation the agent reads before it writes. The implementation sits downstream of that. Treating it as downstream is exactly what makes it possible to leave it unread.

If the agent writes every line, where do the human hours go?

It began as my own personal project — my agentic coding "PhD thesis", if you like. It ran over months, mostly in my free time, sometimes taking up most of the day on vacation. Hundreds of my own hours went into it. And if the agent writes every line, the obvious question is what all those hours were spent on.

“Now that is the typical superficial assumption — that coding is most of software engineering work. It never was, and it never will be.

Three categories account for the first part of it: the domain know-how you need just to specify a solution like this, the architecture design, and the steering — keeping the agent pointed in the right direction. But most of the time went somewhere less glamorous. It went into tuning the solution against historical tickets, iteration after iteration, raising the percentage of tickets solved. And doing it while making sure the instructions I gave the AI did not quietly become hard-coded to the specific historical tickets I was testing against.

How do you evaluate whether an AI agent's answers are good enough to send a customer?

AI agent evaluation needs a harness, so I built an evaluation engine — with Claude Code, of course. It ran against a sample of roughly 50 historical tickets from the first client we were tuning for production. All of them were mail-initiated, so the agent received exactly the same trigger that human support had historically received. And each ticket came with its "golden" solution: the true resolution as recorded in the CRM. This disciplined evaluation approach is foundational to how we build agentic AI at VIS — we treat the harness as critical infrastructure, not an afterthought.

Grading was done by a second, independent model. The support automation itself runs on OpenAI; the cross-check was performed through the Anthropic API, so the judge was not the same system being judged. On top of that I double-checked the results ticket by ticket by hand. The judge agent's classification was usually correct.

This is also where my personal time went, in the least automatable part of the whole exercise: reading the reports after every iteration. Which tickets the agent solved, which it did not, and why. And most important of all, where it would embarrass us with a misfit message to an end user. That last category is the one that can kill an entire automation programme, and no harness catches it for you.

How resolution went from thirty-something percent to over three quarters

The first meaningful run was really disappointing — my word, and I stand by it. Something in the region of 30-something percent of tickets fully solved. A team that stops there concludes the technology is not ready. Instead I went through every ticket individually, and the diagnosis was the opposite of what the score suggested. Some genuinely difficult tickets had been solved impressively well.

“So the problem was never ability. It was the agent missing a piece of documentation, a tool, a proper instruction.

We repeated the process over many iterations until the goal was met: more than 75% of tickets fully solved. My intuition is that over 90% is reachable with further iterations, and we will probably get there soon. I am equally direct about the cost. This is the point where humans still have to invest a lot of time, or the project will fail. The agent's ceiling was never the constraint. The completeness of what it was given was.

What did the automation actually deliver once it was live?

The solution has been in production for weeks, at most months — a direct consequence of the enabling technology itself being new. For this particular client, throughput was never the point, and the value did not arrive as a volume number. It arrived as coverage.

The agent took first line during the vacation period, when we were short on human personnel. That was the original design intent: absorb the first shock. Open tickets are monitored on a dashboard anyway, so a human can step in and help the agent when needed. What the automation removed was not the work of answering. It was the interrupts and the demand for instant availability — especially the telephone calls, where somebody had to drop all other work at a moment's notice and take over a client's problem.

Who fixes the bugs when an AI coding agent wrote the code?

The claim that makes senior engineers uncomfortable is not that the agent writes the code. It is that the agent fixes the code. In practice this is easier than it sounds, because we do detailed testing before production, not after.

During development, the agent is started via Claude. So when something breaks, Claude is immediately aware of it and already working on it. In production the path is more formal: when the first-line agent authorises a user and classifies an incoming request as a bug, the request goes straight to the cloud version of Claude that works on the codebase.

This is the one place where we still have a human read the pull requests, and the reason is precise. A bug fix normally touches one line or a few lines. If the agent instead hallucinates and decides to re-engineer the whole solution, somebody has to be there to stop it. The review exists to catch scope, not syntax.

Agents that go in circles are rare in this setup. When something suspicious happens, we stop it and analyse what happened — and even that analysis can now be done with AI assistance.

What working method keeps an AI coding agent productive instead of expensive?

Why every agent session gets exactly one achievable goal

Before every major change, I make the agent plan first. The plan I hand it is a human-optimised functional specification — the document I would want to receive if I were the developer. Non-functional requirements go in at the same step, along with anything else the agent needs to be aware of.

“The planning process is what turns text optimised for humans into a task list optimised for the machine.

After that come the iterations, because things almost never work properly on the first attempt — exactly as when humans did the coding. I give the feedback in those iterations in plain natural language, with no further planning sessions. The session ends only when the result is satisfactory.

Every session gets a single achievable goal: implement and test a new feature, or run a new testing iteration and improve the score. That constraint is what keeps the work moving forward instead of dissolving into endless conversations with the AI.

Why "slow is fast" keeps daily agentic development inside one subscription

The habits are narrow and deliberate. When a session has to be interrupted mid-flow, I continue that same session rather than starting a new one — and a continued conversation is always compacted first. I compact only when continuing, and I continue only when I was forced to stop in the middle. Bigger upgrades get a planning step; smaller ones get direct instructions. We do no multi-agent development, and nothing else that would inflate consumption for its own sake. Between them, these habits keep us inside a Claude Pro subscription, which we rarely exceed despite using it daily.

“We like to plan systematically and review meticulously, so the human is the bottleneck — not agentic speed, and not the number of tokens.

The underlying principle is that starting with superficial instructions, seeing what comes back and then iterating endlessly is what eats tokens. It is the same principle that applied when humans wrote the code themselves.

“Think before doing something rather than after, and you will save a lot of time, a lot of nerves and, nowadays, a lot of tokens.

How do you train a development team on agentic coding?

At VIS we train all of our employees on agentic coding using Claude's official materials, and our company library holds several books on the topic. The Claude Certified Architect materials are the ones I recommend personally, because they worked well for me.

There is no formal examination. All of our technical employees are mid to senior software developers, so instead of testing them we encourage them to learn and embrace the agentic approach while the field is still fresh. Procedures and benchmarks are still being worked on. As far as I can see, other software companies are in exactly the same position.

Who should you hire once writing code is no longer the hard part?

Ask me what I would hire for today and I will not answer with seniority. In the old world, software development was like solving assignments in university maths — difficult — so everybody was fixated on abstract thinking and coding ability.

The qualities that matter more than ever now are different ones: taking full responsibility for what you do, being able to communicate clearly to humans and machines alike, and business domain knowledge as a plus. In my view these fundamental qualities are more individually determined than related to someone's age. Which is why whether to hire juniors is the wrong question.

What should a development team do in its first month with a coding agent?

My recommended order is counter-intuitive, so let me state it plainly: start with more formal training, based on a book or the coding agent vendor's official course. Learning by doing will turn out to be more costly, in every sense.

And that is precisely the mistake most teams will make in their first month — open Claude and start doing it straight away. Understanding the theoretical principles first pays back heavily once you get to the practical work.

Line-by-line code review versus the specify-and-evaluate method

What happens to a ten-person development team that adopts agentic coding?

I will not soften the consequence. Productivity per person rises, and the need for hand-written code falls with it. My expectation is that a ten-person team with genuine agentic know-how will win more work, and that the additional work will compensate for the higher productivity.

“And what will people do instead? Less coding, more specifying. Some will find it hard to accept, but for most people it is actually a relief.

The reason it is a relief is that the attention moves to where the value always was. You can focus on functionality rather than on some function's or some API's bizarre syntax. And functionality has always been what the client is paying for in the first place.

None of this is a claim that the review question is settled everywhere. We still read the pull requests for bug fixes, deliberately. What this case establishes is narrower and harder to argue with: a production-grade support solution, tuned against real historical tickets, graded by an independent model, resolving more than three quarters of them — with no human ever reading its implementation code. This represents a shift in how we think about code ownership in agentic systems. We proved the pattern on our own solution before we offered it to anybody else.

If your development team currently relies on code review as the guardrail for shipping, you almost certainly have one of these conversations brewing — between the speed agentic coding delivers and the review overhead it should eliminate. The next instalment is about building evaluation harnesses that you trust enough to let the agent loop without human inspection; what happens to quality metrics when you move from reviewing syntax to grading outcomes. Subscribe to the series to see it when we publish.