As I write this, the most capable AI model on the market just disappeared. Anthropic released Fable 5 on a Tuesday. By Friday it was gone, pulled after the US government raised national security concerns about what it could do. I'll come back to what that means for the rest of us in a later essay. For now, notice the shape of the conversation. It's about how capable these models are, how fast they're improving, what they might be able to do next.

I want to make a different argument. How powerful the model is matters far less than most people think. What decides whether AI helps you or hurts you is something more nuanced: whether you can tell good output from bad.

For some time I was chasing an electrical fault in our camper. I've picked up a bit about these systems over the last year, not enough to call myself competent. So I uploaded the wiring diagram to Claude and asked where a problem was likely to be. The answer came back fast, specific, and confident, pointing straight at one part of the system and explaining why.

I had a different idea. Reading the diagram, I thought the fault could sit somewhere else, and I said so. The model steered me away, confidently. I pushed back again. It didn't fold. It restated its answer with more conviction, explaining its rationale step by step so I could follow the logic. And the logic was sound, as long as every component behaved the way it should. What I didn't know was how important the assumption hiding inside that "as long as" really was.

It turned out the model was wrong. And my dad, who spent his career in electrical engineering, found the real fault almost immediately, right where I'd suspected it to be. A component that was supposed to cap the current had failed, resulting in a blown fuse some critical components depend on. The model's explanation didn't consider every potential failure mode of every part in the chain. I'd had the right instinct. What I couldn't do was explain why, or hold my ground when the model insisted I was wrong.

This wasn't a sloppy mistake, or due to AI hallucination. The reasoning was clean. It's also quite impressive it can read and reason across a complex electrical wiring diagram. The logic it came back with fit the diagram perfectly, and every step held. The flaw wasn't in the steps. It was in an assumption underneath them, and catching that takes someone who knows how these parts fail, not just how they work. To me, an answer that looked right and an answer that was right were the same thing.

There's a name for this. Researchers call it automation bias: the pull to trust a confident, automated answer over your own judgment, especially when you're unsure. My whole career revolves around data, data science and AI. I'm very familiar with automation bias. I deferred anyway.

This is why I think the popular story about AI is backwards. We're told that the more capable it gets, the more it levels the field, giving everyone access to knowledge and expertise they never had. I argue that in the cases that matter most, it does the opposite. It widens the gap between people who can judge the output and people who can't. My dad gets a faster, sharper version of what he already knows and understands. I got a very convincing way to be wrong and no way to tell. And the most exposed people aren't the ones who know nothing about a subject; they at least know they can't judge it. It's the people who know just enough to engage, follow the reasoning, and form a view, but not enough to catch what it's missing. Or just lack the experience of working in a particular field. That was me with the electrical system. It's a lot of us at work.

The dividing line is the question in the title. Do you know what good looks like? For any task where you can check the answer yourself, AI is a gift, expert or not. The danger is the other kind of task, where checking the answer needs the very expertise you're missing. There, the better the model gets, the more convincing it is when it's wrong, and the fewer people can tell.

I've argued before that leaders need their own judgment to separate what AI can really do from what they're being sold. This is the same problem one level down, at the desk of whoever is using it. Across your organization, people are leaning on AI for analyses, code, and documents they couldn't have produced on their own. The work comes back polished and certain. The honest question isn't how good the model is. It's whether the person using it would know if it were wrong. If the answer is no, that's not a productivity gain. It's a risk that looks like one.

So the next time AI does something you couldn't have done yourself, pause before you're impressed. That feeling, "I could never have done this," is the thing everyone celebrates. It's also the precise moment you have no way of telling whether it got it right.