Fill the gaps with AI before you replace the core

Every business has a list of things it would do if only there were time. Ours included reporting dashboards that stayed current, faster root cause analysis after incidents and a closer eye on the logs across our client sites. Over the last year, AI has quietly worked its way down that list, and that's where our biggest gains have come from.

Most businesses start at the other end. The natural instinct is to point AI at something you already do and ask it to do it cheaper. The ONS figures published this summer suggest that's what most firms are doing. Improving existing operations is the most common use of AI among UK businesses, cited by close to 60% of those using it.

I think it's the harder place to begin.

This is the second of three posts on what AI has changed at TSD. The first covered the development team, and this one is about operations.

Why replacing a process is harder than it looks

Big organisations write their processes down. Somebody owns each one, and the edge cases have usually been argued over in a meeting and written into a manual. When you hand a process like that to an AI agent, most of the hard thinking has already been done.

SMEs mostly run on processes that live in people's heads. They work, often very well, because the person running them carries the exceptions around. The customer who always pays late but always pays. The supplier whose delivery notes never match the invoice.

Handing that kind of process to an agent is like passing on your gran's recipe. "A handful of flour, bake until it looks right." The agent will follow it to the letter and you'll get a cake. Ask anyone who ate gran's whether it's hers.

The ONS has spotted the same pattern from the other direction. Firms with stronger management practices are more likely to adopt AI and other advanced technologies. It's one of the few areas where a big organisation's paperwork gives it a head start over a smaller, nimbler competitor.

So if you want to start by replacing something, write it down first.

AI is surprisingly good at helping with that part. Have it interview the person who runs the process, read back what it thinks happens, and let them correct it. The exceptions surface much faster that way, and you end up with the documentation whether or not the automation ever follows.

Start with the gaps

New processes are easier for a simple reason. There's no legacy routine to unpick and no workaround that somebody depends on. You can design the whole thing around the agent from day one. Three of ours show the pattern.

Watching the logs

Our client sites produce a constant stream of logs. Reviewing them used to take one of our application engineers several hours a day, and it's the sort of work that matters most on the days it feels least necessary.

Strictly speaking, this one did replace something. The new process started from a blank page, designed around what an agent does well. It reads everything and groups the repeats, so a person only sees what's new or unusual. Several hours of reading became a short review of what it surfaced.

I've written before about why logging matters. This is the bit where somebody, or something, finally reads them.

Root cause analysis

When something breaks on a client site, the evidence is scattered. Some of it sits in Azure, some in AWS, some in application telemetry and some in our structured Serilog logs, each with its own console and its own idea of what time it is. Lining it all up used to be the slow part of any root cause analysis.

An agent can now pull from all of those sources at once and look for correlations across them. An error rate might climb in one place just as response times drift in another. It surfaces in minutes what used to take an engineer hours to piece together.

We still restore first and investigate second during an incident. The investigation is where this has changed things.

Dashboards

Reporting dashboards sat on our wish list for years, and the trouble was always keeping them current once they existed. Now they're up to date whenever anyone opens them.

The catch: confident answers from half the evidence

Correlation is the thing AI does best in operations, which makes the next part uncomfortable.

An agent will find a pattern in whatever data you give it, and it will explain that pattern with complete confidence. If the data is incomplete, the explanation can be convincing and wrong.

Say errors spike at 14:05, and a deployment went out at 14:00. The agent connects the two and recommends a rollback. The real cause was a payment provider slowing down at the same moment, and its status page wasn't among the sources the agent could see. Nothing in the data contradicted the deployment theory, so it had no reason to doubt it.

Someone who has looked after that client for years might have asked about the payment provider straight away. The agent didn't know what it couldn't see.

So we treat every root cause an agent proposes as a hypothesis until a person has checked it. Two questions help. Which sources did you have? And what evidence would prove you wrong?

The verification loop

I value this part most. Checking a theory used to be expensive. You'd pull the logs, line up the timestamps and rule things out one at a time, and a single wrong guess could cost an afternoon. When checking is slow, you test fewer ideas and lean harder on instinct.

Now a theory can be tested in minutes. Being wrong is cheap, so we can afford to be wrong more often on the way to being right.

Large organisations with dedicated operations teams have always had that fast loop from hypothesis to verdict. A business our size can now run it too. I'd rank that above anything the agent has told us directly.

The long game

At the UK Umbraco partner summit, Mats quoted Amara's law, coined by the futurist Roy Amara in 1978.

"We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run."

It sits neatly alongside the Hype Cycle I leaned on last time, and it matches what we've seen in operations.

The ONS figures show the short-run half playing out. Adoption among UK businesses has nearly tripled since late 2023, yet the average number of AI technologies each business uses has only crept from 1.4 to 1.6. The ONS reads that as limited transformative impact so far.

Each of our gaps is small on its own. A log review here, a dashboard there. If Amara is right, the bigger effect comes later, as small processes like these pile up and start feeding each other.

Where I'd start

Pick something you've never had time for, and leave your most important process alone for now. A gap gives you room to get through the trough I described last time while the day job carries on as normal.

Give the agent every data source it needs and make it tell you which ones it hasn't got. Until it has a track record on that particular job, a person signs off the conclusions. And if you want to replace an existing process, start with the write-up.