The 85% that isn't AI

Yogi Berra said baseball is ninety percent mental and the other half physical. That’s roughly my accounting for AI systems: ninety percent is plumbing, and the other half is the model.

Call it eighty-five percent if you want a number that adds up. I don’t have a study. This is a judgment from building the things, and I’d rather say that plainly than dress it up as research — which, as it happens, is most of what this piece is about.

The model is the part everyone talks about, and it’s genuinely the easiest part to get. You can have an excellent one running before lunch. So can the company across the street, for the same price, with the same three lines of setup. Whatever advantage exists is not in there.

It’s in the other four-fifths, and that part has no launch event.

What the other half is

Intake — where the work comes from. Almost every AI demo starts with the task already handed over: here is the document, summarize it. Real work doesn’t arrive that way. It arrives as a message someone half-read, a commitment made verbally in a meeting, a report that quietly stopped refreshing three weeks ago. A system that only acts on what you remember to hand it is not reducing your load — it’s adding a second place you have to remember things. The hard, boring, decisive question is what gets caught before anyone thinks to ask for it.

Definitions — what the numbers mean. This is where most analytics dies, long before anyone mentions AI. Two reports disagree, both are “right,” and the difference is that one counts a category the other excludes. Nobody wrote it down, so the argument repeats every month. Fixing this is not clever work. It’s writing down what a field means, once, in code, and making every figure derive from that — which is exactly what rebuilding a company’s revenue reporting turned out to be: not a better report, but definitions that stop moving, and a standing check that every derived number still reconciles to the source.

Boundaries — what it may decide alone. The instinct is to ask what a system can do. The useful question is what it may do without asking. Those are different, and conflating them is how people end up with something technically impressive that nobody dares leave running. Authority is not capability. A system that can send email and a system that may send email should be separated by an explicit decision you made on purpose. When I built a platform to run my own operations, the load-bearing half of every role’s definition turned out to be the list of things it may never do without me — and the rule that when a check can’t evaluate, the answer is no. Defaults tend to run the other way: when something errors, it lets the action through.

Verification — how you know it worked. Not whether it ran. Whether it was right, checked by something that didn’t produce the answer. This is the part that gets cut first and matters most, because without it every other component is a claim.

It’s also the part that saves you from yourself. The first time I measured how non-billable time had changed after an operating handoff, my framing was wrong — it compared two periods that weren’t comparable, because the underlying data had changed meaning partway through. The number it produced looked good. It was worth nothing. Catching that before it reached anyone was not a matter of being careful; it was a matter of the check existing at all, and the measurement that eventually shipped is narrower and less flattering than the one I started with. That’s usually what verification does. It makes your claims smaller and true.

What it looks like when it’s missing

The failures are more instructive than the successes, and the ones I know best are my own. All three have the same shape: something read green while nothing worked.

I once wrote a rule that a subsystem had earned trust after seven consecutive green days. It sounds reasonable. It measures nothing — a week can span five tasks or five thousand, and calendar time is not evidence of anything except patience. I struck it and replaced it with verified units of work, because the question was never how long something had been running. It was how much it had actually done correctly.

Then a batch of overnight jobs sat wedged for nine hours while every health check reported fine. Two causes, both mine. A concurrency limit had been written to a config file the dispatcher never read, so the real limit was no limit. And the per-task heartbeats kept firing after the underlying processes were dead — the monitoring wasn’t watching the work, it was watching a signal that had come loose from the work. I hadn’t built a monitor. I’d built something that produced reassurance.

The third one is the one I think about. A background service on my own infrastructure had been restarting every ten seconds for twenty-two and a half hours — 7,689 times, no alert, no progress. Automatic restart on failure, without a budget for how many times it may fail, isn’t resilience. It’s a silent-failure generator. But here’s the part worth sitting with: I’d been carrying an unexamined assumption that automated restarts were quietly saving me all over the place. So I checked all thirty-nine services for evidence of successful self-healing.

There wasn’t any. Not less than I expected. None.

That’s the whole argument in one incident. The technology worked exactly as documented. The belief I’d built on top of it was false, and only checking found that out.

What to do with this

If you’re deciding where to spend on this stuff, the uncomfortable answer is that the model is rarely the constraint, and swapping it for a better one rarely fixes what’s actually broken. The constraint is usually that work arrives in a form nobody captured, described in terms nobody agreed on, handled by something whose authority nobody bounded, producing results nobody independently checked.

None of that requires AI to fix. All of it has to be fixed before AI helps.

The practical test I’d offer: for anything automated in your business, ask how you would find out if it were quietly wrong. If the answer is “someone would probably notice,” you don’t have verification — you have a service that hasn’t embarrassed you yet.

I know the difference because I spent twenty-two and a half hours on the wrong side of it.