Method

Five rules. They exist because each one was written after being violated at least once.

1. Every claim leads to an artefact

A repository, a report, a benchmark, a graph, an experiment log. If there is no artefact, there is no claim — only an opinion, and opinions are labelled as such.

2. Validate the instrument before trusting the number

An A/A run (same code measured twice) is mandatory. If A/A falls outside 1.0000 ± 0.005, the instrument is unusable and the numbers it produced are discarded — not reinterpreted.

3. Results are dated snapshots

Leaderboards move. Every competitive result on this site carries a date and links to the original board. A rank is a position on a day, not a standing property.

4. Negative results stay visible

A hypothesis that failed is evidence. The kernel experiment that produced no speedup is published next to the ones that produced a gain, because a lab that only shows wins is a marketing page.

5. Facts and interpretations are separated

"125.0 → 135.9 tok/s between two configurations" is a fact. "Configuration matters more than the kernel" is a conclusion from one experiment, and it is written as such, with its scope.

On independence

Lobster Swamp is not a model vendor, not a cloud provider, not a regulator, and not affiliated with any of the systems it measures. When a measurement involves someone else's platform, the numbers come from that platform's own public output wherever possible.