A 99% Accurate Model Can Still Be a Bad Decision

Prediction answers "what is likely." Optimization answers "what should we do."

2026-05-22

A fraud model can be 99% accurate and still be a bad decision.

If the 1% it flags happens to be your most loyal customers, you didn't catch fraud. You taught good people to leave.

Prediction answers "what is likely?" Optimization answers "what should we do?" Those are different problems, and we keep confusing them.

Accuracy was never the objective. The objective is economic. Every score cutoff, every policy rule, every override is a bet about which tradeoffs you are willing to accept. That judgment layer, not the model underneath it, is where the real decisions live.

Now LLMs are making prediction abundant. Anyone can generate a competent predictive model in an afternoon. Andrej Karpathy has made this point better than I can.

So the scarce skill isn't building the model anymore. It's knowing what to optimize, which constraints are real, and what not to do.

The math is becoming the commodity. The judgment is becoming the moat.

First published on LinkedIn, 22 May 2026. Lightly revised. Original post.