# It looked right

> The most expensive failures I have had were the ones that looked right: the code read well and the model said it was done, but nothing had run.

Page: Methodology · Verify
Canonical: https://banes-lab.com/disciplined-methodology/verify#it-looked-right

This section is stop 40 of 102 in the learning route. Previous: [25 - Placement is a grammar](https://banes-lab.com/disciplined-methodology/build/placement-is-a-grammar.md). Next: [27 - Verify the verifier](https://banes-lab.com/disciplined-methodology/verify/verify-the-verifier.md). It builds on [03 - The stance](https://banes-lab.com/disciplined-methodology/start/the-stance.md).

The most expensive failures I have had were the ones that looked right: the code read well and the model said it was done, but nothing had run. Everything in this chapter follows from one distinction. A claim is a sentence that the developer or the model produced, and until something grounds it, it is [ungrounded content](https://banes-lab.com/records/arch/ungrounded-content.md). [Evidence](https://banes-lab.com/records/reason/node-ver-evidence.md) is an observation that a mechanism produced. Only evidence decides anything, as shown in [A1·a a claim's source](https://banes-lab.com/disciplined-methodology/verify#it-looked-right-panel-a), and [A1·b report and prose](https://banes-lab.com/disciplined-methodology/verify#it-looked-right-panel-b) shows how to tell the two apart. The [verify](https://banes-lab.com/records/stage/verify.md) node of [the loop](https://banes-lab.com/disciplined-methodology/start/the-loop.md) asks exactly this question, and the ontology's [verification](https://banes-lab.com/records/arch/verification.md) axis names its parts: [ground truth](https://banes-lab.com/records/reason/node-ver-ground-truth.md), [falsification](https://banes-lab.com/records/reason/node-ver-falsification.md), [confidence](https://banes-lab.com/records/reason/node-ver-confidence.md) and [refutation](https://banes-lab.com/records/reason/node-ver-refutation.md).

### Looks right is not runs right

A model judges its own edit as clean by reading it, and reading is not running. The model claims [compliance](https://banes-lab.com/records/arch/compliance.md) after one edit and never checks it again, and the next run of the tools finds the same fault plus a new one. A model produces text that resembles a finished result, because that is what finished results look like in its training.

For this reason the verdict comes from a machine, and the model's judgement is never the signal that the work is done. The report on disk decides when the work is done, rather than the reply that describes it. In practice, the exit code is the answer and the parsed findings are the answer, while the model's summary is treated as a story about the answer.

To check this, find the verifier output behind every claim of done in a session. A done with no output behind it is a claim, and a claim is unverified. A mechanism's output is evidence about the mechanism itself and only prose about anything else. The sentence most likely to stop a reader looking further is one whose subject is not the thing that printed it, because it reads as if the looking has already been done.

A completion claim reads *all done, the change is clean and everything passes*, while a completion signal reads *gate: every step passed, exit zero, report written to the run's path*. The first is a sentence a model produced, and the second is a number a machine produced.

Evidence comes in tiers, and the tiers are ranked: a measurement taken locally outranks a vendor's documentation, which outranks a community source, which outranks inference. A record carries its confidence and its derivation as two separate axes, because material adapted from elsewhere can be strong or weak evidence regardless of having been adapted, and folding a word about provenance into the vocabulary of strength would make the whole set impossible to rank. When sources conflict, both are recorded and the conflict is stated. A claim that is unverified or rests on a single source is labelled as such wherever it drives a decision.

A1·a a claim's source

```mermaid
flowchart TB
edit["An edit lands"]
read["The model reads it back"]
story["A story about the result"]
run["The verifier runs"]
exit["An exit code and parsed findings"]
done{"Done?"}
edit --> read --> story
edit --> run --> exit --> done
story -. never the signal .-> done
```

A1·b report and prose

```mermaid
flowchart TB
output["A message printed by a run"]
subject{"Is its subject the mechanism that printed it?"}
report["A report · the authority of an execution"]
prose["Prose · opened at the mechanism it names, never trusted for arriving as output"]
output --> subject
subject -- yes --> report
subject -- no --> prose
```

## Links to

- [Ungrounded Content](https://banes-lab.com/records/arch/ungrounded-content.md)
- [Ver Evidence](https://banes-lab.com/records/reason/node-ver-evidence.md)
- [Verify](https://banes-lab.com/records/stage/verify.md)
- [The loop](https://banes-lab.com/disciplined-methodology/start/the-loop.md)
- [Verification](https://banes-lab.com/records/arch/verification.md)
- [Ver Ground Truth](https://banes-lab.com/records/reason/node-ver-ground-truth.md)
- [Ver Falsification](https://banes-lab.com/records/reason/node-ver-falsification.md)
- [Ver Confidence](https://banes-lab.com/records/reason/node-ver-confidence.md)
- [Ver Refutation](https://banes-lab.com/records/reason/node-ver-refutation.md)
- [Compliance](https://banes-lab.com/records/arch/compliance.md)

## Linked from

- [PAG and the method](https://banes-lab.com/pag/introduction/pag-and-the-method.md)
- [Who does what](https://banes-lab.com/disciplined-methodology/start/who-does-what.md)
- [The stance](https://banes-lab.com/disciplined-methodology/start/the-stance.md)
- [Tools live in the tree](https://banes-lab.com/disciplined-methodology/build/tools-live-in-the-tree.md)