Evidence is part of the program

A profile is a claim about how an array behaves on a process. The evidence block is where the author shows the claim was tested: a held-out report, the measured false-alarm rate, latency, minimum detectable effect, exposure, and at least one deterministic fixture. The registry review checks the claim; the host still verifies the bytes.

The evidence block

{
  "evidence": {
    "grade": "B",
    "reportUrl": "https://…/EVIDENCE.md",
    "measuredAt": "2026-10-25",
    "dataset": "Held-out batch on the six-channel rig, one preparation, two sessions.",
    "metrics": {
      "farPerMonth": 0.5,
      "latencyS": 42,
      "mde": 0.12,
      "nSamples": 1800,
      "nAlarms": 6
    },
    "fixtures": [
      { "id": "step", "file": "fixtures/step.json", "expect": "warning" },
      { "id": "return", "file": "fixtures/return.json", "expect": "normal" }
    ]
  }
}

What the grades mean

Fixtures: the deterministic proof

A fixture is a JSON file beside the manifest. It is a sequence of samples and the state transitions that must occur. noses test reads every fixture declared in evidence.fixtures, simulates the profile's state machine, and fails the command on the first mismatch.

{
  "schema": "osmograph.vial-fixture/1",
  "description": "Synthetic step: baseline at 1.00, then 1.20 on both channels.",
  "dtS": 1.0,
  "samples": [
    { "t": 0, "channels": { "ch1": 1.00, "ch2": 1.00 } },
    { "t": 1, "channels": { "ch1": 1.00, "ch2": 1.00 } },
    { "t": 2, "channels": { "ch1": 1.00, "ch2": 1.00 } },
    { "t": 3, "channels": { "ch1": 1.20, "ch2": 1.20 } },
    { "t": 4, "channels": { "ch1": 1.20, "ch2": 1.20 } },
    { "t": 5, "channels": { "ch1": 1.20, "ch2": 1.20 } }
  ],
  "expected": {
    "final": "warning",
    "alerts": [{ "t": 3, "level": 1 }],
    "toleranceS": 1.0
  }
}

What the simulator models

noses test is a profile state-machine test, not the live engine. It reads the declared fields and replays the fixture:

• Baseline. It averages the samples inside anomaly.baseline.warmupS. The first sample after that window is the first scored sample.
• Response. It applies transforms.response (delta-over-r0, relative, or delta) to the declared baseline. A channel with no perChannelLimits entry cannot trigger.
• Corroboration. At least detection.minChannels channels must clear their limit on the same sample.
• Escalation. Continuous anomalous seconds drive warningS → criticalS → emergencyS. Continuous normal seconds plus the hysteresis band drive resetNormalS.
• Hysteresis. The strongest declared-channel response is latched. The level cannot clear until the response falls to (1 − hysteresis) × peak, even after the reset timer.

Running the tests

noses lint
noses test

Profile fixtures (deterministic, no hardware)
  ✓ step                     fixtures/step.json → warning (1 alert(s))
  ✓ return                   fixtures/return.json → normal (0 alert(s))

A missing fixture is a warning: noses test says the static surface is all it can check. A failing fixture exits non-zero. A fixture that references a path outside fixtures/ is refused.

What review checks

1. The raw manifest URL is HTTPS and returns the exact bytes that were submitted.
2. The manifest is osmograph.domain-pack/2. New listings require v2; existing legacy entries remain installable as Ungated until re-authored.
3. The profile ranges, escalation ordering, and admissibility are valid.
4. The scope, operating envelope, and citations are real and specific.
5. The evidence report exists, the dataset names the rig, and the metrics are present.
6. At least one fixture exists and its declared expect matches the simulated final state.
7. Every UI asset is self-contained, and the manifest and asset sha256 values are recorded.
8. The submitter has not changed the bytes since the issue was opened.

Approval re-fetches the manifest, every UI asset, and every declared fixture, re-runs the same checks, and refuses if a hash differs from the submission. The registry commit is the only write. There is no hand-edited registry entry in the normal path.

Hashes are the durable evidence

This is what makes an upstream edit visible. An author who changes `ui.html` without bumping the version does not silently change what a user runs; the install refuses and the mismatch is a reportable event. Fixtures are pinned at review so a changed evidence claim is visible too; the host does not execute them at runtime.

The antifragile loop

a user reports a wrong state
        │
        ├─ the report carries id, version, manifest hash, asset hashes
        ├─ the author reproduces it as fixtures/<case>.json
        ├─ the fixture is added to evidence.fixtures
        ├─ noses test fails before the fix and passes after it
        └─ the next version is reviewed with the new fixture pinned

A Vial without a fixture cannot regress safely, because there is nothing to replay. A bug fix that does not add a fixture is a claim. The review checks for both.

Red flags in an evidence block

• A false-alarm rate with no exposure time or sample count.
• An accuracy number measured on the training data, presented as held-out.
• A dataset named only by a commercial product, with no unit, session, or preparation.
• A latency that is shorter than the persistence timer.
• A fixture whose expected state is normal while its samples contain a sustained step.
• A UI screenshot with a number that does not appear in the report.
• A version bump that changes the profile but not the fixture.