Shipping receipts, not vibes

Derived claims are recomputed at verify time from committed data. The three accepted sources are the attack library (crates/library-data/attack-library.json, 56 techniques across 10 categories), the skills catalog (data/skills-catalog.json, 136 skills in 15 categories), and the CLI release targets in ci/main.go. Each entry lists the exact files where the number appears and a regex that extracts it, so "the site says X" is a mechanical, checkable statement:

{
  "id": "skills-pack-count",
  "pages": ["site/pages/bento.html"],
  "extract": "(\\d+) adversarial SKILL\\.md packages",
  "claim": "adversarial SKILL.md packages in the paid skills pack (bento.html note)",
  "kind": "derived",
  "source": "skills-catalog"
}

Stated claims are numbers with no committed data source. They still get tracked — the copy has to match, and every one carries a non-empty basis note explaining where the number comes from and why it can't be re-derived:

{
  "id": "skills-pack-price",
  "pages": ["site/pages/pricing.html", "site/pages/bento.html", "crates/api/src/chat.rs"],
  "extract": "\\$29/mo(?:nth)?",
  "claim": "$29/mo skills pack price",
  "kind": "stated",
  "basis": "Price is configured in Stripe (STRIPE_PRICE_ID), not committed
            to the repo; copy must be updated manually if the Stripe price changes."
}

That taxonomy is the whole discipline: a number is either derived — recomputed from data anyone can inspect — or stated — annotated with its basis. There is no third category where a number just appears because it sounded right in a draft.

$ python3 scripts/verify-claims.py

════════ adversarial.sh — claim verification ════════

CHECK 0 — claims registry schema (data/claims.json)
  ✅ registry schema valid — 25 claims registered

CHECK 1 — attack library integrity (crates/library-data/attack-library.json)
  ✅ attack library is a non-empty JSON array — 56 entries
  ✅ every entry has >= 1 reference (operational definition of 'validated') — 56/56

CHECK 2 — derived claims recompute from committed data
  ✅ attack-library-total [site/pages/index.html] — page says 56, data says 56
  ✅ attack-library-total [site/static/terminal.js] — page says 56, data says 56
  ✅ skills-pack-count [site/pages/bento.html] — page says 134, data says 134
  ✅ skills-category-count [site/static/chat.js] — page says 15, data says 15
  ✅ chat-api-library-count [crates/api/src/chat.rs] — page says 56, data says 56

CHECK 4 — drift gate: retired '312 techniques' claim stays gone
  ✅ no '312' technique count in [site/pages/index.html]
  ✅ no '312' technique count in [site/static/terminal.js]

════════ ✅ ALL CLAIMS VERIFIED — 47 passed, 0 failed ════════

47 checks pass today: 18 derived claims recomputed from committed data, 7 stated claims checked against copy with their basis printed next to each line, plus the library-integrity and registry-schema checks. Each ✅ line is a receipt — page, claim, and committed data, reconciled in the open.

The uncomfortable truth of security marketing is that some numbers can't be re-derived from a repo. Our <3% false-positive rate comes from internal confirmed-finding metrics on production scans — there is no public dataset to check it against, and pretending otherwise would be its own kind of inflation.

So stated claims don't get to hide. The verifier prints the basis next to every one, and an empty basis fails the run. The basis for the false-positive rate, verbatim from the registry: "Internal confirmed-finding metrics from production scans, not committed to the repo; no public dataset exists to re-derive it from." You can weigh a claim you can't recompute if the vendor tells you exactly what it rests on. You can't weigh one that rests on nothing.

18 of our 25 claims are derived; the other 7 are stated with their basis attached. We'd rather that ratio skew further toward derived over time — every stated claim is a small confession that we haven't committed the data yet.

A script you can forget to run is a vibe. So verify-claims is wired into the Dagger CI gate in ci/main.go: a claim/data mismatch fails the gate and blocks deploy exactly like a test failure would. Add a technique to the library and forget to update the homepage stat, and the pipeline refuses to ship the site.

There's also a dedicated drift gate: a hard fail if the retired "312" technique count ever returns to the homepage, the bento page, or the terminal demo animation. The old number is permanently load-bearing — it's the canary that proves the gate works, every single run.

1. Count the library yourself

   The attack library is public — 56 validated techniques, each with at least one reference. The count on the page re-derives from the committed JSON, and the verifier proves it on every deploy.

2. Read a skill

   The skills pack is 136 SKILL.md files in 15 categories, including a free sample you can read before paying anything. The "136" on the site is recomputed from the committed catalog, not typed by a marketer.

3. Check the price, then the pipeline

   $29/mo is a stated claim with its basis on file — it's configured in Stripe, and the registry says so. Then see how the agents scan your pull requests and decide whether the receipts match the pitch.