Skip to content

Proof

29 side projects, each one built to find out whether an AI system's output can be made to prove itself. Below, every card names what was found, the number behind it, and the file in the project's own repository that number is read from.

All 29 are public and honestly graded.

The projects

sluice

Exhibit
Screenshot of the sluice landing page showing its idempotent-execution comparison: 666 duplicate charges under naive retry versus 0 with sluice, next to a results table breaking down intent success rate and duplicate side effects.

Under fault injection, naive retry produced 666 duplicate side effects and sluice produced none, while refusing one call in five as indeterminate rather than guessing; the landing page serves those figures from the committed results file.

666 duplicate side effects with naive retry, 0 with sluice

chaos/results/latest.json · baseline.naive.duplicateEffects and baseline.sluice.duplicateEffects; pnpm chaos regenerates the file and CI fails on drift

Verified 2026-08-28

windlass

Exhibit
Screenshot of the windlass page headed "Three things this pipeline runner refuses to do, and the test that proves each", with the first refusal, that it will not run a model inside the runner, opened above the code that enforces it.

A privacy law that is not tested is a promise. Planting a token-shaped string in a gate answer proved it never reaches events.jsonl, and removing the redaction proved the test fails when it should.

1 planted token, 0 copies in the log, 0 model calls in the runner

tests/redaction.test.mjs · npm test runs the selftest and the redaction test end to end against the real runner; examples/demo-replay.html is the rendered replay

Verified 2026-09-06

driftwatch

Exhibit

Its first real run across seven repositories reported ten failures and every one was a false positive; after the four fixes it reports zero failures on the same repositories, and the unverifiable counts stayed high because most of what a document claims cannot be checked offline.

10 of 10 first-run failures were false positives, 0 after the fix

src/lib/finding.mjs · line 11 is the whole outcome contract, STATUSES = new Set(['verified', 'failed', 'unverifiable']); the run is git commit 5ada0bf; npm run selfcheck regenerates examples/self-check.json

Verified 2026-09-06

proofpage

Demo
Screenshot of the proofpage project page describing a real bug it caught in its own README: the first documented command failed with ENOENT because no examples directory existed, despite 51 passing tests.

proofpage's own README opened with a command that fails on a fresh install, and 51 passing tests never noticed, because every one of them ran inside the repo where the missing directory exists.

1 README command failing on a fresh install, missed by 51 green tests

README.md · git show cf40975 is the fix; the check that caught it packs the tarball, installs it in a temp directory and runs the README's first example (quality-harness cli-library pack, row CL6)

Verified 2026-09-06

claude-code-boundary-guard

Flagship

The guard failed open on Linux until its self-test ran on a second OS: unconditional case-folding and Windows-only fixtures let every cross-boundary write through, and a two-OS CI matrix now catches exactly that.

38 of 38 self-test cases on Windows, re-run on Linux in CI

selftest.mjs · node selftest.mjs prints passed/total; the matrix is .github/workflows/ci.yml

Verified 2026-09-06

chaff

Flagship
Screenshot of the chaff site showing its resident, on-demand and dropped token counts for a real CLAUDE.md profile: 5,904 resident, 184 on-demand, 109 dropped.

Run on its own author's context file, chaff reports zero enforced findings, and that run is a committed eval rather than a demo; it ships zero measured rules today and says so in every line of output, because the evals that would admit one need paid model calls.

44 golden cases at 100% exact match, 0 enforced findings on its own CLAUDE.md, 0 measured rules shipped

evals/cases/ · 44 entries scored by evals/golden.eval.test.ts; the self-run is evals/dogfood.eval.test.ts; pnpm test

Verified 2026-08-28

kedge

Exhibit
Screenshot of the kedge Raft simulator mid-run, showing a red "not linearizable" verdict with the witness operation that could not be placed anywhere in a consistent order, above the client history strip it was derived from.

kedge's linearizability checker agrees with Porcupine's published verdict on every one of the 102 vendored Jepsen etcd histories that carry one, and it says so from a corpus command anyone can re-run rather than from a README sentence.

102 of 102 Jepsen etcd verdicts matched

vendor/porcupine/jepsen/ (103 histories, MIT) replayed by src/cli.js corpus · node src/cli.js corpus runs every vendored history through the checker and compares each verdict with the one Porcupine published; etcd_095.log is zero bytes and is refused, not counted

Verified 2026-09-06

flume

Exhibit
Screenshot of the flume page explaining that every event carries both an event time and an arrival time, above its stream controls and a running tally of complete, revised and uncertain windows.

A watermark checker raised 124 findings against a correct engine on real streams and was wrong 124 times, and the project keeps that count reconstructible under test instead of quietly zeroing it.

124 false positives retired, 0 findings after, on three real streams

test/streams.test.js and vendor/gharchive, vendor/wikimedia, vendor/usgs (three 5,000-event slices, each with a SOURCE.md licence note) · node --test test/streams.test.js reconstructs the three retired conditions from the shipped results and asserts they total 124; node src/cli.js streams re-measures the three streams from the vendored slices

Verified 2026-09-06

cofferdam

Exhibit
Screenshot of the cofferdam crash harness showing a CORRUPT verdict at a single crash point, with the acknowledged record the log lost and a byte-level diff of the expected and recovered state beside it.

Real process kills on NTFS found zero corruptions in a store that the modeled device shows corrupting at 32 crash points, which is the measured reason crash-consistency testing needs a device model and not a kill loop.

12,600 crash points, 0 corrupt, 0 unverifiable on the correct build

crashes.json (the committed report) and src/cli.js control · node src/cli.js control enumerates 200 workloads of the correct build; cofferdam verify crashes.json re-runs the committed report and compares verdicts

Verified 2026-09-06

dipmeter

Flagship
Screenshot of dipmeter showing 230,059 earthquake hypocentres glowing inside a transparent Earth, coloured by depth from orange at the surface to blue at 700 km, with the Tonga and South American slabs standing out as dipping sheets beside the depth, magnitude and time controls.

106,108 of the 230,059 events sit at exactly 10, 33 or 35 km because a locator assigned the depth instead of solving for one, and every one of them falls in the shallow band, none at 70 km or deeper. They are drawn as their own class rather than mixed into the picture.

230,059 located hypocentres against 27 modelled slabs

public/data/manifest.json · counts.events, regenerated by npm run build:data and independently re-derived from the raw ComCat CSV and from the shipped binaries by npm run verify

Verified 2026-09-07

orrery

Exhibit
Screenshot of orrery showing the asteroid belt as a blue haze of 120,000 sampled bodies around the inner planets, with the population counts, the find-a-body index and an instrument readout of drawn count, frame time and solver beside it.

A blind local-minimum search over the semi-major-axis histogram, given no knowledge of resonances, located five gaps whose worst disagreement with independently computed Jupiter resonance positions is 0.0149 AU, about three histogram bins.

0.0149 AU worst blind-to-predicted gap disagreement

data/audit.json · the maximum of kirkwood.blindMatchToPredictedResonance[].nearestBlindMinimum.distanceAU across the four resonances; npm run audit regenerates it and npm run check re-asserts it

Verified 2026-09-07

snapgauge

Flagship
Screenshot of the snapgauge tool displaying a real MCP schema diff between two server versions, showing a breaking removed tool and a fail-on-risky gate that would exit CI with code 1.

A tool description edit is schema-valid, passes every validator, and still changes which tool a model picks; snapgauge grades it risky rather than cosmetic, and the golden set pins that grading at exact match.

37 golden and compat cases at 100% exact match

evals/cases/ · 30 golden plus 7 compat fixtures, scored by evals/golden.eval.test.ts and evals/compat.eval.test.ts; pnpm eval

Verified 2026-08-28

provenote

Flagship
Screenshot of the provenote page explaining its C2PA provenance-chain checker, above the drag-and-drop panel used to inspect an image's signed provenance chain entirely in the browser.

A valid C2PA signature on an obviously fabricated scene validates cleanly, because a signature certifies who signed which bytes and says nothing about whether the scene happened; the fixture that proves it is committed with its hash.

7 hash-verified fixtures, 1 valid signature on a fabricated scene

fixtures/hashes.json · pnpm run fixtures:verify; the fabricated scene is Fixture 2 in fixtures/README.md, f2-fabricated-signed.jpg

Verified 2026-08-28

dogwatch

Frozen
Screenshot of the dogwatch status page reporting a stalled watch, with its run tally of 78 checks, 2 findings, 0 gates and $0.0000 cost above a diagram of its approval-gate flow.

Its scheduled workflow had failed silently since birth; since the 2026-08-15 fix the nightly watch has published 13 scheduled run records to the repository as committed JSON, next to the two manual runs that came before.

13 scheduled nightly records, 2026-08-15 to 2026-08-28

runs/index.json · entries with kind scheduled, each naming its record file under runs/2026/; no record carries the date 2026-08-27

Verified 2026-08-28

tiltmeter

Frozen
Screenshot of the tiltmeter page showing its calibration numbers, a 0.0% false-positive rate and 95.0% detection power from 200 seeded pairs, above a diagram of how two harness runs are compared.

The instrument was calibrated before any real reading: 0 false positives in 200 null trials and 190 of 200 planted regressions caught, from a seeded simulation; the three scheduled runs since have each published a skipped record because no API key was ever set.

0.0% false-positive rate, 95.0% detection power

evals/calibration/RESULTS.md · pnpm calibration regenerates it byte-identical; the three skipped runs are observatory/readings/index.json

Verified 2026-08-28

shipgauge

Study
Screenshot of the shipgauge page's dark in-browser benchmark interface, with model and device selectors and a “run cold test” control for measuring shippability on the visitor's own machine.

The bytes a browser model actually transfers were 48 to 74 percent smaller than the size its README advertises, across 19 of 20 measured rows on one machine, with the execution provider read back from real GPU submits rather than trusted from config.

48% to 74% below advertised, 19 of 20 rows

results/results.json · rows[].bytes.deltaVsAdvertised.deltaPercent, generated 2026-08-15; the failed row is the twentieth

Verified 2026-08-28

assay

Frozen
Screenshot of the assay project page describing its deterministic brand-kit generator, showing the three CLI commands (gen, proof, diff) it runs to produce byte-identical brand assets from one JSON config.

Fed four shipped sibling brands, assay regenerated three byte-identical and refused the fourth: shipgauge's live accent-on-text contrast is 1.85:1, under the 3:1 gate, so no artifacts were written, and a test asserts that exact refusal on every run.

3 of 4 brands byte-identical, 1 refused at 1.85:1

case-study/shipgauge/DIFF.md · npm test runs test/case-study/drift.test.ts, which re-asserts the refusal and its exact error

Verified 2026-08-28

galley

Frozen
Screenshot of the galley landing page, its title rendered upside down, describing the Remotion-rendered release-video CLI built from a repo's real commits and CI data.

One real release video is committed: dogwatch's own five-commit CI-fix range rendered to 36 seconds of h264 with every on-screen number labeled by its source, from a build that fails on any number whose source it cannot name.

36.0 s, 913,793 bytes, 5 commits

examples/dogwatch-ci-fix.mp4 · ffprobe reports h264 1280x720 at 36.000 s; git rev-list --count 478fcbb..5143aeb in the dogwatch repository prints 5

Verified 2026-08-28

halo-halo

Frozen
Screenshot of the halo-halo Taglish code-switch segmenter mid-analysis, showing a sample sentence tokenized and colour-tagged by language: Tagalog, English, mixed intra-word, and named entity.

Unicode's word segmenter splits nag-book into three pieces, so intra-word Taglish switches never reach a labeler by default; the boundary score here is a consistency check against machine-drafted labels, flagged draft-automated, with no human annotation pass yet.

boundary F1 98.9% on 95 switch points, draft-automated

eval/results/v0-eval-report.json · boundaryF1.f1 and provenance.annotator; pnpm gen:eval regenerates it byte-identical

Verified 2026-08-28

swage

Frozen
Screenshot of the swage ASL fingerspelling classifier page, showing its seven-step on-device prediction pipeline and a boxed disclosure that its accuracy figure is provisional, not from a held-out signer.

95.7% top-1 on 281 held-out samples, and the report marks itself provisional because the split is file-level from one source; the project's own ship bar, 70% on a signer it never saw, is not met and cannot be without volunteers.

95.7% top-1 on 281 samples, provisional

model/eval-report.json · overallAccuracy, testCount, provisional and shipBarMet fields

Verified 2026-08-28

graticule

Demo
Screenshot of the graticule semantic-similarity map mid-render, showing scattered plotted notes beneath its on-device model status and a running count of zero network requests.

The default embedding model scores a flat contradiction (0.8645) as more similar than a genuine paraphrase (0.7481) of the same sentence; the page shows both numbers, and CI re-measures every pinned pair against live inference.

contradiction 0.86 vs paraphrase 0.75 cosine

fixtures/linguistic/negation-pairs.json · input.defaultPair; input.pairs holds 15 contradiction pairs, mean 0.7748, min 0.5768

Verified 2026-08-28

clarifier

Demo
Screenshot of the clarifier tool with a real penguin-measurement dataset plotted as a force-settling point cloud next to its column-mapping table assigning each CSV field a physical role.

On its own showcase dataset the physics layout scores 0.726 silhouette against 0.672 for the best two-axis view, and the tool reports that as no meaningful gain rather than a win; both figures are pinned in the golden set.

0.726 vs 0.672 silhouette, verdict: no meaningful gain

evals/fixtures.eval.test.ts · the blobs-3-known block pins pca2dSilhouette 0.6716 and physicsSilhouette 0.726; pnpm eval

Verified 2026-08-28

pitman

Demo
Screenshot of the pitman page's file-drop interface for recording or uploading a WAV or MP3 clip to see its on-device transcript with per-word confidence.

11 of 18 Filipino-accented clips had at least one Whisper error, but a same-day US-accented control group failed the same proper nouns the same way, so only one pattern survived (track heard as truck), and a test fails the build if the app ever attributes a mismatch to accent.

11 of 18 clips (61%) with an error, 1 pattern survived the control

docs/batch2-asr-probe.md · sections 5 and 8; the framing rule is packages/core/test/evidence-classes.test.ts

Verified 2026-08-28

orphanage

Demo
Screenshot of the orphanage crawl report explaining how a re-run against agentjames.vercel.app reduced 40 reported findings down to the one that actually mattered, a /games page missing from the sitemap.

On its first real site it reported 40 unlisted pages and every one was a false positive: 35 canonicalised duplicates and 5 machine endpoints. A tool that buries one real finding under 40 false ones is worse than no tool.

40 findings to 0 unlisted, 1 real orphan surfaced

site/report.html · node bin/orphanage.mjs <url> --max-pages 60 --delay-ms 900

Verified 2026-09-06

aeo-lab

Demo
Screenshot of the aeo-lab page showing a real logged scan against notion.com, listing the HTTP status each crawler, including Googlebot, GPTBot and ClaudeBot, received that day.

Its first real scan produced a block that the scan itself had caused: one host returned 429 at a 250ms delay and was completely clean at 2000ms. Rate limiting is now reported as rate limiting, never as a block.

1 self-caused block reclassified: 429 at 250 ms, clean at 2000 ms

examples/notion.com-2026-09-05.json · node bin/scan.mjs <url> --delay-ms 2000

Verified 2026-09-06

polis

Demo
Screenshot of the polis map mid-pan, showing agent guild districts as isometric buildings and a banner noting that 8 of 59 agent citizens answer to no charter.

Reading the ecosystem's own reciprocity rule against its own charters found eight members that no other charter names in either direction, which the constitution calls a violation and its groomer had not caught; reading the dispatch log then found that 33 of the 59 agents have never been dispatched at all.

8 unreachable members and 33 never-dispatched agents, out of 59

data/ecosystem.json · stats.unreachable here and stats.neverDispatched in data/workforce.json; npm run snapshot and npm run workforce regenerate both

Verified 2026-09-06

carillon

Demo
Screenshot of the carillon page's dark keystroke-to-sound interface, with its scale selector, typing textarea, and live counters for notes played and average interval.

The privacy claim is enforced by a checker that scans the shipped site, and every one of its rules has a fixture proving the rule fails when violated.

6 rules, each with a fixture that fails it, 0 network calls in the shipped site

verify.mjs · npm test runs tests/verify.test.mjs against the fixtures and then verify.mjs against the built site

Verified 2026-09-06

tally

Exhibit

Its first real target was itself, and two rules fired on non-defects: docs/SPEC.md was reported as a leaked test because spec is two words, and a private package was told to ship a LICENSE npm would never publish. Both were narrowed, with a test holding each direction so the rule did not simply switch off.

2 false positives removed after the first real run, 7 planted defect fixtures

tests/fixtures/ · npm test; the two narrowings are git commits 74f77cb and 4535aa3, each with a test for both directions; npm run selftest regenerates examples/self-audit.html

Verified 2026-09-06

ClubScope Insight Engine

Frozen
Screenshot of the ClubScope Insight Engine dashboard in replay mode, showing a country club's detected findings, active members, dues at risk, 90-day revenue and a churn-risk breakdown, with a declining-dining-revenue finding expanded below.

The model never produces a number: typed tools compute each figure, every figure cites its evidence, and a verifier recomputes it before it renders; the eval run recomputed 51 of 51 cited figures and caught the one fabricated figure planted to test the gate.

51 of 51 figures recomputed, 1 fabricated figure caught

packages/core/src/evals/cases.ts · pnpm evals; the three model-dependent cases report skipped, never passed

Verified 2026-08-28

Flagship · 5Exhibit · 8Study · 1Demo · 8Frozen · 7
  • Flagship · 5Positioned for other people to use, and the install path is being made real.
  • Exhibit · 8The strongest engineering here, shown for the work rather than seeking users. Not on npm.
  • Study · 1A dated measurement with n printed. A method, not a tool.
  • Demo · 8A browser demo you can try in one click. A shelf item, not a storefront.
  • Frozen · 7Complete as built, no further work planned, and honest about where it stopped.

26 of the 29 are deployed and live; the remaining 3 ship as repositories on purpose and have no site. The tier on each card is the same honest grading these projects got in their own review, not a marketing label.

How these numbers were checked: every figure above is read from the file named beneath it, inside that project's own public repository. Each was re-read from the committed file on 2026-08-28, and the “how” line under each number is the command that reproduces it from a clean checkout. Where a project has no such file, no number is shown here. The finding sentence says what was found instead. See also the full project record, which lists client work and demonstration builds alongside these 29.