The confident wrong answer

There is a failure mode in human-machine teams that has nothing to do with the machine's accuracy and everything to do with its confidence. It is called automation complacency, and it is the reason a more accurate model can be more dangerous than a less accurate one.

Picture the doctor at the end of a ninety-patient day. The model has been right ninety times. Each correct answer, delivered fluently and fast, has taught him a little more to trust it, until checking the output feels like wasted motion on a tool that is always right. Then comes the answer that is wrong, in the same calm confident voice as the ninety before it, and he is exactly as primed to accept it as he was the others. The ninety right answers did not make the system safer. They lowered his guard for the wrong one.

This is why accuracy is the wrong thing to optimize for past a point, and why Glyph treats every output as a draft no matter how good the model gets. A model that earns trust by being right is, by the same token, eroding the very checking that catches it when it is wrong. The only durable answer is structural: keep the human in a position where reviewing is required, not optional, so complacency cannot quietly remove the last line of defense.

Practically, that means Glyph never presents an output as settled. It shows its sources so the doctor can check rather than trust. It flags its own uncertainty instead of smoothing it into fluent prose. It is built to keep the doctor deciding, because the moment he stops deciding and starts rubber-stamping, the model's confidence becomes the system's blind spot. The goal is not a model so good it is never doubted. It is a workflow where doubt never switches off.