How an experiment is tested
Every record in the archive went through the same steps. This page describes them once.
What gets tested
Each experiment starts from a belief traders actually hold, stated the way people state it: large exchange deposits mean someone is about to sell, long shots are overpriced, prices bounce after a panic. The belief is tied to a mechanism, a reason someone would be forced, slow or uninformed, and to the argument for why nobody has already traded it away.
How a test is set up
The rule, the pass and kill criteria and the expected size of the effect are written down before the test data is examined. Each record says whether this was the case. The test uses public market data, an explicit cost model with fees, spreads and funding, and a control: the same rule at a time when the supposed cause was absent, or a placebo built the same way. Results are checked on data the rule was not designed on, and corrected for the number of variants tried.
How a verdict is reached
The result is judged against the criteria set in advance, not against what would look good afterwards. Only experiments that fail are published here, and each record names the reason it failed.
How records are checked
Every record is checked three times before release. The test runs its own controls. A validator checks that every number in the text matches the evidence and that no wording suggests a strategy works. A separate AI fact-checker, which did not write the record, recomputes the figures from the source results.
What a record contains
The belief, the method in plain words and in full technical detail, the numbers, two or three charts with their data, an interpretation marked as such, the scope of what was and was not covered, and the steps needed to reproduce the test. Records hold aggregates only, never third-party raw data. Trading venues are described by type, not by name, and dates are given to the month.
Limits
The research is AI-generated and not peer-reviewed. A failed test covers the data, costs and period it used, nothing more. It is historical research, not investment advice.
Why experiments fail
Each record names one main reason from this list.
No signal
The signal behaves exactly like its control.
Below costs
The effect is real but smaller than the cost of trading it.
Not replicated
It did not hold on data the test had not seen.
Decayed
It existed once and has faded.
Placebo equal
A placebo produces the same effect.
Underpowered
Too few events to reach a decision.
Tail risk
A few large losses outweigh the typical gain.
Concentrated
The average comes from a handful of days.