The COBOL nobody wants to touch is exactly where AI agents earn their keep

For thirty years, the most important software in a bank has also been the least understood.

I have seen this up close, again and again. Modernizing legacy systems has been a thread running through most of my career — including migrating a private-banking ERP off the mainframe, a system that had been growing, layer over layer, for more than twenty years. The code worked. It cleared trades, calculated positions, closed the books. But the people who wrote the original logic were long gone, the documentation described a system that no longer existed, and every change carried the quiet fear that you were about to break something nobody could fully explain.

That fear is the real cost of legacy. Not the language. COBOL is not the problem — COBOL is fine. The problem is that the business knowledge is locked inside the code, and the only way to retrieve it used to be a slow, expensive archaeology dig performed by a shrinking number of experts.

This is exactly where the new generation of AI agents changes the economics.

From "read the code" to "interrogate the system"

Reverse engineering used to mean one analyst, one screen, and a lot of patience: tracing a variable across thousands of lines, reconstructing a calculation by hand, guessing at intent from naming conventions written in 1998.

Agent mode flips the workflow. Instead of reading code linearly, you give an agent a goal — "explain how interest is accrued on this account type," "map every program that touches the settlement table," "find the business rules behind this batch job" — and it works the codebase the way a senior engineer would: searching, following call chains, cross-referencing copybooks, and assembling an explanation grounded in the actual source.

The valuable output is not "AI that writes new code." It is AI that recovers lost knowledge:

• Plain-language explanations of what a program does and why, traceable back to specific lines
• Dependency maps showing what breaks if you touch a given module
• Reconstructed business rules, extracted from the logic instead of from a wishlist
• A migration scaffold — the same rules expressed in a modern stack, with the COBOL kept as the reference

Done well, this turns a multi-month expert-only effort into something a broader team can do in weeks. The expert is still essential — but now they are reviewing and steering, not excavating by hand. For an organization carrying decades of legacy, that is not a productivity tweak. It is the difference between modernization being feasible and being permanently postponed.

So why isn't everyone doing it already?

Because three hard problems sit between the demo and production. And in a regulated bank, they are non-negotiable.

Challenge 1: The code is the crown jewels — it cannot leak

Here is the uncomfortable truth about pointing an AI agent at legacy banking code: that code is some of the most sensitive material the institution owns. It encodes proprietary business logic, risk models, account structures, and — often, embedded carelessly across decades — real data, credentials, and connection strings.

And here is the second uncomfortable truth, the one the "just run a local model" crowd glosses over: for the core of this work, the frontier tools — Codex, Claude, and their peers — are genuinely better. Reverse engineering tangled legacy logic is exactly the kind of deep-reasoning task where the gap between a top-tier model and a small self-hosted one is large and very visible in the output. Pretending otherwise leads to disappointing results and a stalled programme.

So the real question is not "local or cloud?" It is a trade-off you have to make deliberately: the most capable models give the best reverse engineering, but using them on your crown-jewel code means deciding exactly what leaves your perimeter, to whom, and under what guarantees. That tension doesn't disappear; it has to be managed.

In practice that means treating the work as tiered, not all-or-nothing:

• Match the model to the sensitivity of the task. The deep reasoning over genuinely sensitive logic is where you want the strongest model and the strongest guarantees — an enterprise agreement with no training on your data, no retention, isolated deployment, or the model brought as close to your perimeter as the provider allows. The bulk of lower-sensitivity work — search, summarization, navigation — can run on smaller or self-hosted models, which is also where our own GPU environment pulls its weight.
• Sanitize before anything leaves. Whatever the model, source code and embedded secrets shouldn't flow raw into a prompt. Detecting and stripping credentials, and scrubbing what you can, is the price of using a powerful external model safely.
• Know your provider's posture. Data residency, retention, sub-processors, jurisdiction — these are not procurement footnotes here. They are the difference between a tool you can use on this code and one you can't, regardless of how good it is.

Challenge 2: The translation problem is human, not technical

The second challenge no amount of GPU or clever architecture solves, and in my experience it's the one that quietly decides whether a modernization actually lands: people.

A legacy estate is held up by two groups who barely speak the same language. On one side, young technical experts who are fluent in the new stack — agents, pipelines, modern architecture — but have little idea why a given calculation exists or what regulation it satisfies. On the other, seasoned functional experts who hold the business meaning of every rule in their heads, but for whom the new tooling feels like a foreign country.

AI agents can extract the what from the code. They cannot, on their own, supply the why — the regulatory intent, the historical exception, the "we do it this way because of an audit finding in 2009." That still lives with people. Which means the agent doesn't remove the need for these two groups to work together; it raises the stakes on it.

So the real enablement work runs in both directions:

• Get the young technical experts speaking functional. They need enough domain literacy to ask the agent the right questions, to recognize when an extracted rule is plausible versus dangerously wrong, and to sit with a functional expert without the conversation collapsing into jargon.
• Get the seasoned functional experts onboarded onto the new technology. Not to make them engineers, but so they can validate what the agent surfaces, steer it, and trust it — instead of treating AI as a black box that threatens the expertise they spent a career building.

This is where most of my energy actually goes. The agent is a translator between old code and new systems. But someone still has to translate between the people who own the technology and the people who own the meaning — and that is a training, trust, and culture problem, not a model problem. Get it right and the two groups amplify each other. Get it wrong and you ship a beautifully reverse-engineered system that no one is confident is correct.

Challenge 3: Agents are hungry — and it's capacity, not just cost

The last problem is less dramatic but just as capable of killing a programme: agents consume an enormous amount of compute, and that bites in two ways — cost and capacity.

Legacy estates are enormous — millions of lines, thousands of programs, deep call graphs. Agentic reverse engineering is iterative by nature: the agent reads, follows a reference, reads more, reasons, retries. A single non-trivial task burns through tens of millions of tokens without much effort. That's the cost side. But now multiply it: dozens or hundreds of developers running agents in parallel, every day. The token volume becomes staggering, and the binding constraint stops being "what does this cost?" and becomes "can my provider actually serve this throughput, reliably, without throttling me at the worst moment?"

That reframes the whole sourcing decision. The mix you land on for security reasons changes the shape of the spend — frontier APIs bill per token, your own GPUs bill as capacity you've already paid for — but neither makes the problem disappear. Self-hosting means you own the capacity ceiling: your GPUs are your throughput, full stop. Frontier providers can offer far more headroom, but only the ones genuinely built to sustain that scale — which is itself a selection criterion. Either way the discipline is the same:

• Scope before you summon. Don't unleash an agent on the whole estate. Target a module, a transaction flow, a single batch job. Narrow goals produce better answers, lower bills, and less load.
• Right-size the model. Most reverse-engineering steps — searching, extracting, summarizing — don't need your largest model. Route the heavy reasoning to the big model only when it's warranted, and let smaller models do the legwork.
• Cache the expensive thinking. The same copybooks and core programs get analyzed again and again. Reuse prior analysis instead of re-deriving it every run.
• Cap the loops. Agents can spiral — re-reading, re-trying, chasing dead ends. Hard limits and good stopping conditions are what separate a tool from a money pit.
• Measure cost per outcome, not per call. The metric that matters is what it costs to document one program reliably, not what a single query costs.

Treat cost and capacity as architectural constraints from day one and the programme scales. Treat them as something to look at later, and the pilot looks brilliant right up until the first full-month invoice — or the first time the whole team gets throttled mid-sprint.

The bottom line

The promise is real: AI agents can finally make legacy reverse engineering something a whole team can do, instead of a dark art practiced by a vanishing few. For institutions sitting on decades of COBOL, that unlocks modernization that has been stuck for years.

But the capability is the easy part. What determines whether it works in a bank is everything around the model — a pipeline secure enough that crown-jewel code never leaks, a provider with the capacity to sustain a whole team's token-hungry agents without buckling, the cost discipline to keep it sane across millions of lines, and a team where technical and functional experts can finally understand each other.

The next phase of legacy modernization won't be won by whoever has the cleverest model. It will be won by whoever builds the safest, most scalable system around it — and brings their people with them.

If you're modernizing a legacy estate: what's holding you back more right now — the security and data-residency questions, the cost and capacity of doing it at scale, or getting your technical and functional experts to truly work as one team?