Benchmark governance

Finding — WETX refusals, October–November 2026

Why WETX refused on data availability from 2026-10-02: Kalshi books lost to HTTP 429 at the close, measured, and the fix to the fetch.

Status: finding. Written 2026-11-19 under oracle build item O0.1, as wetx-coverage-2026-08.md was. Under incident-process.md §3 a refusal is not an incident: the methodology worked. This note exists because O0.1 requires the cause of a coverage breach to be named in writing, and an availability figure without its cause is a number nobody can check. Scope: WETX weight-basis coverage, and the settle-time book fetch in scripts/index-settle.ts (enrichBandDepth, attachAdmittedBooks) and lib/indices/marks.ts (fetchMarks, fetchKalshi). Measured from content/indices/benchmarks/wetx.json, its composition archive, berry's ~/wet-cron-logs/index-settle.log, the 2026-11-19 mark tape, three live reproductions, and one probe run from berry itself.

No published value changes as a result of this note, and none is restated. Every refusal below published what the methodology told it to publish. The floor (minWeightCoverage 0.95) and the way coverage is measured are untouched. This note is about why the books did not arrive.


1 · What happened

WETX has 96 rows, from 2026-08-06 to 2026-11-19: 46 printed and 50 refused. All 50 refusals are data-availability / min-weight-coverage. The first was 2026-08-14 (the August note). The current run began 2026-10-02, and since then 3 of 42 closes have printed (11-15, 11-16, 11-18).

Window Closes Coverage κ Kalshi unobserved per close
10-02 → 11-02 32, all refused 79.73% to 91.13% (median 82.04%) 422 to 1,049
11-03 refused 77.21% 736
11-04, 11-05, 11-13, 11-14 refused 91.32%, 91.56%, 91.73%, 90.26% 221 to 264
11-15, 11-16 printed 98.58%, 95.07% 37, 130
11-17 refused 94.91% 133
11-18 printed 98.15% 47
11-19 refused 93.99% 156

Missing rows. No WETX row exists for 08-20, 08-31, 09-01, or 11-06 to 11-12. Those are absences, not refusals. For 11-06 to 11-12 none of the four benchmarks has a row, and berry's log holds no settle run between 11-05 16:06 EST and 11-12 20:52 EST. That absence is unexplained in the governance record and is raised in §5.

Both daily fires, from berry's log. In EST, 20:05Z is 15:05 ET and 21:05Z (16:05 ET) is the on-time close. The later fire's row replaces the earlier one's (§5).

Date 15:05 ET fire 16:05 ET fire
11-15 94.63%, refused prints
11-16 94.46%, refused prints
11-17 prints (44.01) 94.91%, refused
11-18 94.97%, refused prints
11-19 prints (46.94, at 98.89%) 93.99%, refused

2 · Which constituents had no book

By venue, it is Kalshi. Over the 82 rows that record unobservedByVenue:

Venue Constituent-days observed Unobserved Book delivered
Kalshi 5,931 39,007 13.2% (median row 13.3%)
Polymarket 306,592 42 99.99%

Kalshi averages 99.7% of WETX's unobserved set (minimum 87.1%). Every horizon: family's legs are Polymarket listings, so counting by prefix hides no Kalshi constituents. On the 11-19 on-time close, 159 Kalshi constituents were admitted, 3 carried a book, and the 156 without one were the entire unobserved set.

These figures describe the rows that stand, and those mostly come from the later of the two daily fires (§5).

By market state, they are open markets. Admission requires a two-sided quote within the spread band (August note §2.1), so every unobserved constituent was a live, quoted market. In all three live reproductions (§3, §6), 0 missing refs were "answered without a book" (settled, absent ladder, failed side check). Every one drew an HTTP 429.

They are not always the same ones. The ledger does not name the unobserved, but it does name the observed, and the observed Kalshi set changes from close to close. On 11-15 it was 81 constituents across 10 series, on 11-18 it was 10, and on 11-19 it was 3, in series not observed on 11-18.


3 · Why the books did not arrive

Kalshi rate-limits berry's anonymous reads, and the settle's book fetch made one attempt.

The book fetch worked like this:

  • enrichBandDepth fetched every admitted ref's book through fetchMarks: eight at a time, unpaced, one attempt each.
  • A Kalshi contract takes two reads, the market document and then the orderbook. A 429 on either recorded depth: null, and the book was gone for that close.
  • The crawl that nominates the same contracts minutes earlier (kalshiGet, lib/indices/pool.ts) paces at 120 ms and makes four attempts at a throttled read. The fetch that weights them had neither.
  • Kalshi refs sit in a cluster in the fetch order (in the reproduction, all 85 within positions 2,233 to 2,409 of 2,780), so they reach the venue as one unpaced burst right after the crawl has drained the bucket.

Every Kalshi read on berry is anonymous. W.E.T.'s servers never sign a venue request: EN-NEUTRAL-002, as amended by END-044, allows exactly one venue credential, a member's read-only Kalshi portfolio grant. scripts/doctrine-verify.ts has kept the Kalshi key names out of production code since PR #59 (2026-08-31). So every Kalshi reader on the box shares one bucket.

The evidence, with n:

  • Berry, 11-19 morning (the paced recorder, fetchMarksScaled):
    • Of 516 Kalshi legs, 149 had a book.
    • 226 lost the market document.
    • 111 lost the orderbook, and all 111 were HTTP 429.
    • 30 were settled.
    • That run's own crawl lost 227 Kalshi series "after 4 attempts".
  • Berry, 11-16 to 11-19: WETFED refused empty-roster on 6 of the 8 fires, and 5 of them logged "Kalshi request failed with 429 … this is our own request volume". Same box, same bucket, a different reader.
  • Local reproduction, 11-19 22:10 EST (the settle's exact pool, probe and fetchMarks, from a different IP):
    • Polymarket got 2,695 of 2,695 books. Kalshi got 7 of 85.
    • All 78 failures were 429s (59 on the market document, 19 on the orderbook). No other non-200 status appeared.
  • Berry, 11-19 22:49 EST (a dependency-free probe of the same 85 tickers, under the cron lock): the 8-wide first read got 12 of 85, with 73 HTTP 429s, in 1 second.

Candidates ruled out:

Candidate Finding
A fetch budget or truncation No. Depth is requested for every admitted ref (the log's denominator equals the admitted count), and there is no cap on this path.
A venue sentinel read as no data No. The failures are HTTP 429s, not in-band values.
Settled or halted markets still admitted No. 0 missing refs were answered without a book.
A change merged around 11-16/17 No. Between 09-25 and 11-19 the only commit to pool.ts, eligibility.ts, benchmarks.ts, universe.ts, marks.ts or index-settle.ts is f99e4d457 (10-03, the recorder's rate guard), which landed after the 10-02 onset and does not touch the settle's fetch.

Why it swings around the floor. The admitted Kalshi count follows the venue's listings and the crawl's own 429 losses: 860 on 11-03 (US election day), about 340 on 11-04/05, about 250 on 11-13/14, then 57 to 159. At a ~13% delivery rate, about 160 Kalshi constituents on a ~2,600 roster put coverage at the floor. The three prints since 11-15 came from either a good delivery day (11-15: 68.6% of 118 Kalshi constituents) or a small Kalshi count (11-18: 57 admitted, 17.5% delivered).


4 · The remedy

This is remedy 2 in the August note §7.2: broader book coverage at settle time, fixing the fetch rather than the admission. It is a collection change, not a methodology change. In the words of consultations/2026-08-20-benchmark-settlement-window.md, a change like this is not itself a methodology change, because it "changes what is observed, not how" a published number is computed from observations.

  1. The first read is unchanged, except that fetchMarks now reports each HTTP status with the ref that drew it. A ref with no book is then either unanswered (it saw a 429, a 5xx or a transport failure) or answered (settled, absent ladder, failed side check).
  2. Only unanswered refs get a second read (rereadUnansweredBooks):
    • It uses the same fetchMemberMark, so books are parsed, banded and side-checked exactly as before.
    • It makes one read at a time, 1.5 s apart whatever the last answer was.
    • It works in rounds: every candidate once, then every still-unanswered one again, up to 6 reads per ref and 8 minutes in total.
    • A re-read is adopted only when it carries a book. Everything else stays depth: null, and the floor refuses exactly as before.
  3. Only for WETGRI and WETX, the two benchmarks whose number reads the depth basis. Every call site states the choice explicitly. WETFRAG passes false: its computation never reads bandDepth, so spending the bucket on its books would only starve the next reader.
  4. Every close now logs the first-read misses by venue and class, and the re-read's outcome (recovered, answered without a book, still unanswered, unreached).

Why a throttle does not lengthen the wait. This was measured, per read, on WETX's Kalshi refs the same evening:

Pacing Books per read n reads
The settle's 8-wide first read, unpaced 8% 85
One in flight, 1.5 s apart, nothing added after a throttle 40% 78
One in flight, 1 s apart plus a fixed 2 s pause after a throttle 33% 250
One in flight, waits doubling 5 s → 60 s after a throttle 33% 18

One read in flight is what matters. Waiting longer after a 429 bought nothing and only spent the deadline. The doubling cut managed 18 reads in 342 s and never reached 67 of its 74 candidates.

Why 8 minutes. The budget is set by the lock, not by the venue. Measured from each row's settledAt to its commit (11-13 to 11-19), the 20:05 UTC fire took 15m50s to 33m45s. standing-close starts waiting on the same lock at 20:25 UTC, for at most 1,800 s. At the slow end, one 8-minute pass (plus up to 30 s for a read already in flight) still leaves standing-close about 8 minutes. Ten minutes left about 5. WETGRI's pass has had no candidates on any close since 10-28.

Known limits, stated:

  • The cause of a refusal is in the log, not on the row. The first-read classes and the re-read report go to berry's log only. A refusal row still cannot say "throttled through six paced reads" as distinct from "answered empty". Recording them on the row as additive disclosure is the follow-up §3.1 of the August note asked for.
  • A re-read book is read later. It is taken up to ~8.5 minutes after the first read. Neither the row nor the tape records when a book was read. No published rule pins depth time, but WET-LOGIC §5.1 asks that every observation keep its timestamp.
  • All venues share one lane. On a day when Polymarket throttles, its refs would take turns in the same 1.5 s lane ahead of Kalshi's. This is latent: Polymarket delivered 2,695 of 2,695 books in the reproduction.
  • --compare-quote-basis gets noisier. It now re-reads in both of its arms, so its basis delta also carries which books came back. It is a diagnostic mode and publishes nothing.

5 · Also found, for the owner, not changed here

  • The later daily fire replaces the earlier one, in both seasons. Re-running a close is what incident-process §3.3 calls a silent restatement, and docs/BERRY_CRON.md calls the second fire "a no-op". This was raised in chat on 2026-11-15 and has no ruling. It is governed settle behaviour, so it is left as is.
    • In EST, the 20:05Z fire runs at 15:05 ET, before the 16:00 ET close. It commits a full row, and the on-time fire replaces it an hour later. On 11-17 and 11-19 the early fire printed WETX (44.01, then 46.94) and the on-time fire replaced each print with a refusal.
    • In EDT, the 21:05Z fire runs an hour after the close and replaces the on-time row. 82 of the 85 EDT-season WETX rows carry a 21:0xZ settledAt. On 11-14 a re-fire moved WETGRI from 109.55 to 109.62.
  • No row exists for any benchmark on 11-06 to 11-12. No settle ran on berry in that window (§1). That is an absence, and incident-process §6 asks for it to be named.
  • The DB projection has not run since September. no DATABASE_URL — DB projection skipped on berry is correct under R12, because the projection belongs to .github/workflows/settlement-project.yml on Actions. That workflow last ran on 2026-09-30, and it and the five runs before it failed in 3 to 5 s, consistent with the Actions billing block. So index_settlements has not been projected for about seven weeks.
  • WETFED's refusals have the same cause. computeImpliedFedPath reads Kalshi with no re-read.
  • The published commit windows are out of date. ops/berry/wet-index-store-load.timer and content/indices/cadence.json say the close commits at 20:27–20:37 and 21:27–21:35 UTC. Measured commits already run to about 20:39, and the re-read can add up to 8.5 minutes. A commit after :32 reaches the store on the next hourly load.
  • A batch reader would remove most of the exposure. Kalshi serves GET /markets/orderbooks?tickers=A&tickers=B, which returned 200 with one orderbook_fp per ticker (verified 2026-11-19). That would make WETX's ~160 Kalshi books a couple of calls instead of ~320. It is a new book reader, so it needs its own review against the one-fetcher rule.

6 · Measured result

All runs used WETX's real pool and admission, read live on 2026-11-19 between 22:10 and 23:05 EST. Coverage here is counted over refs, which is close to the row's constituent κ but not identical to it, since horizon families collapse legs.

Run Pacing Re-read candidates → recovered Kalshi books, first read → after Book coverage over refs
Local, first cut waits doubling to 60 s, 6 min 74 → 6 (67 unreached) 6 → 10 of 78 —
Local 1 s + 2 s pause, 10 min 165 → 82 (0 unreached) 11 → 93 of 176 94.27% → 97.12%
Local, shipped pacing 1.5 s, 10 min 200 → 116 (0 unreached; 387 reads) 5 → 121 of 205 93.13% → 97.12%
Berry's own IP, under the cron lock 1 s + 2 s pause, 10 min 73 → 65 (474 s) 12 → 77 of 85 —

At the shipped 8 minutes a pass makes about 310 reads. At the shipped pacing's measured 30% per read, that is about 90 books. Applied to the 11-19 on-time close (156 Kalshi constituents without a book, of 2,611), it would leave about 66 unobserved, roughly 97.5%. That is a projection, not a measurement. The measurement is the next closes on berry, and the settle log now prints it on every run. O0.1's fourteen-close clock (at ≥3 pp of headroom) has not started.

The rest of the set
Methodology change & consultation
What counts as a material change, the 14-day consultation window, and why changes are prospective.
Data licence & rights
What the free tier grants, what attribution it requires, and what commercial use needs.
The loop architecture
The nine scheduled agents that cultivate indices, which one may publish, and the six mechanisms that assure accuracy.
The confirm gate
What human confirmation covers — merges only — and how to read the provisional and single-venue counts.
Restatement policy
The three narrow conditions under which a published value may be corrected — and the many under which it may not.
Cessation & fallback
What happens when a benchmark cannot be computed, and why the history stays published forever.
Conflicts & the conflict wall
The wall between index operations and the trading circuit, the field crew rule (no trading, no relaying, latency never sold), and standing disclosures.
Data-error playbook
Detection, triage, quarantine — and the commitment to disclose errors nobody noticed.
Complaints & challenges
How to challenge a value, a rule, or a constituent. No account required.
Settlement-reference licensing
What licensing a benchmark to settle against requires — and why publication alone does not grant it. Reading a value is free; referencing it in something that settles is not the same act.
Determination, disruption & cessation
Written for contracts rather than charts: who determines a final value, the disruption fallback ladder, and why a refusal is not a cessation.
The independent committee seat
The seven-part independence test, published before a candidate exists; the power to block rather than comment; and what the administrator may do while the seat is vacant.
Incident process
Who is paged, what they do, and what gets published when the machinery fails — as distinct from a wrong value, which is the data-error playbook.
Finding — the WETX coverage breach, August 2026
Why WETX refused for five consecutive days, the change that caused it, and what is still not established. Published because an availability figure without the incidents behind it is a number nobody can check.