Benchmark governance

Data-error playbook

Detection, triage, quarantine — and the commitment to disclose errors nobody noticed.

Detection

  • Automated guards run before every settlement. Quote-quality gates, MAD outlier trimming, staleness limits and the thin-book anomaly check (lib/indices/consolidated.ts) exclude bad inputs rather than averaging them in.
  • Engine fixtures run in CI. npm run index:verify asserts the engine still reproduces the published formulas to 1e-9. Drift fails the build.
  • The checksum chain makes any post-hoc alteration of history detectable by anyone holding an earlier checksum.

Triage

Severity Definition Response
Blocking The published value is materially wrong Settlement halted; notice published same day; correction under restatement.md
Contained A single constituent is wrong; the value is within tolerance Constituent quarantined; value stands; disclosed on the next settlement
Cosmetic Metadata, labels, display Fixed; logged; no restatement

Quarantine

A constituent whose input is suspect is excluded from settlement pending review rather than included with a caveat. The index is computed over inputs it trusts; a footnote is not a substitute for exclusion.

The reporting standard

An error that reached publication is disclosed even when nobody noticed. The value of a self-audited record is that it self-audits when it is inconvenient; disclosing only what has already been spotted is not disclosure.

Logged errors

Entries are appended, never edited. A Cosmetic finding is logged here rather than in restatement.md, because no published value changed and inventing a publishedValue → correctedValue pair for an event that had none would misdescribe the record.

2026-08-31 — WETGRI intraday indicatives mixed price movement with missing coverage

Severity: Blocking for indicative distribution. No governed WET Close, settlement row, checksum, or constituent record changed, so this is not a benchmark-ledger restatement.

What happened. The WETGRI indicative priced the live marked subset, but divided that result by a close-side gauge computed over the full settled roster. When only 78 of 91 constituents were marked, the other 13 therefore appeared as a market move rather than as missing coverage. The same path also retained the retired cap-before-severity calculation after the governed engine moved the 15% ceiling into effective liquidity × severity weight space.

Observed impact. On 2026-08-31 the two public surfaces read the same 78/91 observations within seconds of each other. The benchmark basket produced 114.07; the public indicative produced 82.99. Reproduction showed the old path comparing a 31.1446 live gauge with a 42.4990 full-roster close gauge, while the governed like-for-like path compared 31.6801 with 31.4557. The error predates the public v1 API, but that API would have distributed it as a canonical value.

The committed ungoverned tape contains 283 WETGRI indicative readings from 2026-08-13 through 2026-08-25; every one has partial coverage and is affected. Those points and the stale wetgri.indicative.json snapshot must not be interpreted or cited as price-only WETGRI moves. They remain in git as the record of what was published. They are not silently replaced with reconstructed numbers: the historical live books needed for an honest recomputation were not retained at the required observation instants.

Correction and guard. lib/indices/indicative.ts now reprices the same marked cohort at live and close probabilities, delegates both gauges to the governed riskGauge, and applies the unrounded gauge ratio to the last WET Close. benchmark-basket-verify.ts exercises partial coverage, concentrated liquidity, mixed severities, and an actual price move, then requires the public indicative and benchmark basket to agree exactly. Hourly artifact copy now states that missing coverage reduces the quality claim but never becomes a move. This correction is prospective; a separate historical-data product decision is required before any corrected intraday series can be published.

Exact-preview acceptance also found the client-refreshed 78/91 live header followed by a server-seeded 0/91 statistic and refusal that could never refresh. The settled statistic is now named WET Close, while the live marked count and any live refusal have one owner: the client snapshot above. A deterministic adapter fixture prevents the static seed from reasserting live coverage below a successful refresh.

2026-08-20 — three missing tape days made 21 daily returns describe three days each

Severity: Cosmetic (labels). No published level changed, on any index, on any date.

What happened. content/indices/marks/ was missing 2026-08-15, -08-16 and -08-19. Those sessions were recorded normally, and the commits carrying them were then destroyed unread: berry's runner had no retry on a rejected push, so when another runner reached master first during the ~40-minute publish window, the work was discarded by the next run's git reset --hard. The tapes were recovered from berry's reflog on 2026-08-20 and restored in b02e312b.

What that did to the record. The basket total return chains daily: Lₜ = Lₜ₋₁·(1 + DTRₜ). With 08-15 and 08-16 absent, the 08-17 point chained straight off 08-14, so its dtr was the compounded three-day move — published under a field that means one day. 21 points across 26 baskets carried such a figure, and every one of them spanned exactly 3 days.

Why no value was restated. The level is the published value, and the level was never wrong: the three-day ratio equals the product of the three daily ratios it was standing in for. Replaying the restored tape splits each figure into its true daily parts and reproduces the same level to the published precision. Measured over all 107 books: 656 already-published points, 0 level changes, 0 reconcile flips, and the compounding identity Π(1 + dtrₙₑᵥᵥ) = 1 + dtrₒₗd holds on 20 of the 21 to 1e-6, the 21st (flashpoint-watch 2026-08-17) residual 1.17e-6 — four-decimal storage rounding, not a discrepancy.

Worked example, ai-race 2026-08-17: level 1076.874 → 1076.874, dtr 0.041816 → 0.009734, with 08-15 (+1.519%) and 08-16 (+1.634%) now carrying the rest.

Disclosed although nobody reported it. Anyone who cached a dtr for one of those 21 points will see a different number and is entitled to know why it moved and that the level behind it did not.

Guard. scripts/btr-replay-verify.ts now runs in npm run index:ci:suite and refuses any replay that changes an already-published level, so the next tape recovery cannot quietly become a restatement. The push defect itself is fixed in berry's runner (rebase-and-retry, plus a real exit code); see content/indices/cadence.json.

The rest of the set
Methodology change & consultation
What counts as a material change, the 14-day consultation window, and why changes are prospective.
Data licence & rights
What the free tier grants, what attribution it requires, and what commercial use needs.
The loop architecture
The nine scheduled agents that cultivate indices, which one may publish, and the six mechanisms that assure accuracy.
The confirm gate
What human confirmation covers — merges only — and how to read the provisional and single-venue counts.
Restatement policy
The three narrow conditions under which a published value may be corrected — and the many under which it may not.
Cessation & fallback
What happens when a benchmark cannot be computed, and why the history stays published forever.
Conflicts & the conflict wall
The wall between index operations and the trading circuit, plus standing disclosures.
Complaints & challenges
How to challenge a value, a rule, or a constituent. No account required.
Settlement-reference licensing
What licensing a benchmark to settle against requires — and why publication alone does not grant it. Reading a value is free; referencing it in something that settles is not the same act.
Determination, disruption & cessation
Written for contracts rather than charts: who determines a final value, the disruption fallback ladder, and why a refusal is not a cessation.
The independent committee seat
The seven-part independence test, published before a candidate exists; the power to block rather than comment; and what the administrator may do while the seat is vacant.
Incident process
Who is paged, what they do, and what gets published when the machinery fails — as distinct from a wrong value, which is the data-error playbook.
Finding — the WETX coverage breach, August 2026
Why WETX refused for five consecutive days, the change that caused it, and what is still not established. Published because an availability figure without the incidents behind it is a number nobody can check.