Can AI Fix Its Own Mistakes? I Checked It Is Not I Fixed It

When you ask an AI to check a piece of text or code, you usually get back “checked it, fixed it” — and the same mistake is still sitting there. The reason isn’t carelessness: the model reads its own paper. If it misread the question the first time, it misreads it the same way on the second pass, and the blind spot stays exactly where it was. Correction only begins when a measurement from outside points at the mistake. That is why an agent’s ceiling is set by the quality of that measurement, not by the intelligence of the model. For the groundwork, see What Is an AI Agent and AI Agent vs LLM; the full map is in the Agentic AI Guide.

Reading your own paper

Tell a student leaving an exam to look again. They look, and they say it’s right. They approve question four too, because they understand it on the second reading exactly as they understood it on the first.

When an AI says “I checked it,” this is what happened. A review took place — but the reviewer and the reviewed are the same mind.

Why looking harder doesn’t help

The obvious fix is to say “look more carefully.” It doesn’t work, because the problem isn’t attention. It’s comprehension.

If the question was misread from the start, a second reading doesn’t repair that misreading — it adds a layer of confidence on top of it. A blind spot is by definition the place you cannot see; looking twice with the same eye doesn’t make it visible.

What the answer key does

When does the student actually see the mistake? When they look at the answer key. What the key does is specific: it doesn’t ask what you thought. It says question four is wrong, and nothing else.

Correction begins right there. Everything before it isn’t correction — it’s self-approval.

What the answer key is in an AI system

On the AI side the equivalent is a measurement from outside: a test turning red, a compiler objecting, a query coming back empty.

What these three have in common: none of them is the model’s opinion. All three are the world’s answer. No matter how confident the model is, if the test is red, it’s red.

How it looks inside a coding agent

The clearest example is a coding agent. It writes the code, runs the test, reads the error, fixes it, runs again. It loops until the test turns green.

The value of that loop isn’t its speed — it’s that at no step does it say “that probably worked.” Every turn it gets an answer from outside.

Why thinking longer isn’t enough

“Couldn’t the model just reason about it for longer?” Sometimes it can, but you can’t rely on it.

With no key in hand, thinking longer doesn’t produce a more correct answer — it produces a more convincing wrong one. The reasoning gets longer, the language gets better, the confidence rises; the accuracy stays put. This is the most insidious side effect of long-reasoning models.

Separating opinion from measurement

“That part was weak, I fixed it” is an opinion. It sounds like a review, but it isn’t evidence.

If you weigh opinion and measurement on the same scale, what you’re left with is a machine that approves itself. The system looks like it’s working, the reports come back clean, and the errors accumulate.

Having a key doesn’t end the job

Don’t relax once you’ve set up a measurement. Because you only fix what the key measures.

A key aimed at the wrong thing hands you a spotless report while the bug stays exactly where it was. What matters isn’t that a measurement exists, but what it measures.

A case from this channel

This happened in my own production pipeline. I had built a check that transcribed the video’s audio and compared it against the script. The check came back near perfect — yet eight lines had been read in the wrong voice.

The measurement had worked; it had measured the wrong thing. It was counting words, not who was speaking. The text was right, the voice was wrong, and the comparison couldn’t see it.

What I added that day wasn’t a new model — it was a new measurement: a separate gate that checks whose voice the audio belongs to. The problem was solved by a better measure, not a smarter model.

Where the analogy breaks

The exam analogy carries you only so far. In an exam the teacher writes the answer key; here, you write the key yourself.

That’s why the real craft isn’t getting the work done — it’s building the measure for it. A system’s quality cannot exceed the quality of the measure you set for it.

The ceiling is set by measurement, not the model

A good model will fool itself with a bad measurement. An ordinary model will pull itself together with a good one. Model choice matters, but measurement sets the ceiling — and this is the most commonly skipped point in agent design.

Not every job has a key

Here’s the hard part. Code has tests. Numbers have arithmetic. But “is this writing persuasive” has no answer key.

There, the agent grades its own homework — and you’re back at the problem this video opened with. In unmeasurable work, confidence doesn’t substitute for measurement; it only makes the risk invisible.

The question to ask from now on

Don’t ask an AI “are you sure.” Ask “how do you know.” If the answer doesn’t rest on a measurement, there is no correction — only confidence. That single question separates a system that approves itself from one that is genuinely checked.

In the next video we’ll look at the harder question: who checks the work you can’t measure?