You have just had an idea. The next step is not to verify it but to try to destroy it. The difference is not attitude, it is the default: verification assumes "sound unless a problem appears", adversarial verification assumes "unsound unless it survives an attack".
| You are doing | What follows |
|---|---|
| looking for supporting evidence | you will find some — with enough data there always is |
| finding it, believing more | confidence grows independently of truth |
| meeting a counter-example, explaining it | an explanation always exists too |
The problem: "supporting evidence can be found" is true of sound and unsound ideas alike, so it carries no information. Exactly one thing distinguishes them: you attacked it seriously and it did not die.
| Rule | Note |
|---|---|
| Assume it is wrong | the burden is on the proposer, not the reviewer |
| "Cannot be shown" counts as overturned | you need not prove it false, only that it was not shown true |
| The reviewer does not fix it | raising the problem is the whole job |
| Attack from separate angles | correctness, cost, execution, basis — each sees different faults |
The second row is the core and the hardest to accept: presumed guilty is correct here, because when money is involved an unproven idea and a wrong idea have identical consequences.
One real-money line went through an adversarial review before launch: 13 defects found, all fixed before it ran. Two were fatal:
| Defect | Consequence | |
|---|---|---|
| ① | the cancel method called did not exist in the SDK, and the error was swallowed by a broad exception handler | the wallet's only cancellation defence would never run, silently |
| ② | the processed-set lived only in memory | a restart would re-order the same window; a crash loop compounds it |
Both share a property: invisible in normal operation. The first surfaces only when you actually need to cancel; the second only on restart — and those are precisely the moments you least want a surprise.
① Any process with automatic restart: for every piece of state ask "is it still there after a restart?" In-memory state resets and the process continues as though nothing happened.
② A broad exception handler turns a hard error like "method does not exist" into silent failure. A defence that never raises is identical to no defence — except you believe you have one.
③ A capital floor's "unknown" must not be wrong in either direction. Unknown treated as zero halts falsely; unknown treated as fine runs blind. In-flight funds must be modelled explicitly.
One of my own trading proposals was overturned by adversarial verification and archived without a cent being spent.
The questions used are plain and hard to talk past:
| Question | Blocks |
|---|---|
| What is the denominator? | quoting a subset rate as a whole-sample rate |
| Is that causal link in the source, or did you join it? | presenting a plausible guess as a record |
| How many subsets did you cut before this one? | picking the largest number in the noise |
| What is this approximation's bias direction? | treating an optimistic bound as the value |
| Does it survive a different basis? | a conclusion that came from the ruler, not the data |
The last one caught me hardest: a beautiful pattern that "survived every cut" turned out to be entirely a label artefact. Anything that "holds under every condition" should first be suspected of living in the instrument rather than in the data — full account in the broken ruler.
| Approach | Why it works |
|---|---|
| Sleep on it | time partially simulates "a different person" |
| Write "how it will die" first | see pre-claimed failure modes |
| Explain it to someone unfamiliar | the parts you cannot explain are the parts you have not thought through |
| Force yourself to write three objections | failing to reach three means you have not seriously considered being wrong |
| Cost first, upside second | a great many proposals die at the cost calculation |
"Supporting evidence can be found" is true of sound and unsound ideas alike, so it carries no information. The only thing that does: you attacked it seriously and it survived. Where money is concerned, an unproven idea costs the same as a wrong one — so the default has to be "unsound".