Skip to content

Reproducibility

Current frozen system · HydroCore-v5 · v0.2.1 · source snapshot 4bbf6fa3ff9f

HydroSwarm separates reproducible software/artifact verification from reopening the final held-out test. The final M11.6 lock should not be rerun.

Reproducibility target

A reviewer should be able to verify:

  1. the exact frozen finalist identity;
  2. the calibration identity and applicability policy;
  3. the serving factory/allowlists;
  4. the generated final-test design/materialization provenance;
  5. the one authorized/opened execution;
  6. the immutable metrics/gate/safety/closure chain;
  7. the current source's V5 self-test and non-locked test behavior.

Frozen identity checklist

Artifact Expected identity
V5 checkpoint de2b3f56243a1933d1d7c5957cd74a29fade119f7d104ce7f1500b3dd7b6d2a5
release manifest f3fb08642738128f020c50e20e6b68c417bf80703f7ef6bc8f42db2aa41f8d34
calibration file 8f77f06b72316455e1f8040dbeb5907503e4eb623dd527d9ea809a56e96c046d
calibration artifact hash f2503e856c467eb38c6c7f6dbde679527c1921925941ec52809bd6e8e6dd16dd
selected seed 20260814
parameters 4,182,612
release schema hydroswarm-v5-release-v1
trained task sentinel
learned runtime outputs 5 frozen Sentinel outputs

Compare runtime manifest with M11.2 finalist freeze.

Verify the current runtime

From the current source checkout:

hydroswarm self-test --strict

The machine-readable trained-assets block should report V5 release identity and the frozen checkpoint hash above. The self-test also performs bounded learned inference, a real WNTR smoke run, SQLite/resource/frontend/reference checks.

Current V5 container reproduction

docker compose build
docker compose up

The current Dockerfile includes the V5 bundle and runs the strict self-test during image build.

docker-compose.release.yml is pinned to the published ghcr.io/insightlabs38-pixel/hydroswarm:v0.2.1 image and is also a valid way to reproduce the current V5 system without a local build:

docker compose -f docker-compose.release.yml up

That image was promoted only after the same strict amd64/arm64 self-test the local Docker build runs, so both paths verify the same V5 identity.

Verify the one-time lock without reopening it

The final evidence chain is:

  1. M11.2 finalist freeze
  2. M11.6 materialization manifest
  3. M11.6 opened record
  4. raw incident evidence
  5. metrics
  6. gate
  7. safety counters
  8. post-run governance
  9. closure

The opened record binds the run to checkpoint, calibration, release manifest, design freeze, evaluator, code-under-test, and materialization manifest identities.

The post-run governance confirms:

  • 125 rows complete;
  • exactly one opening;
  • authorization consumed;
  • no retry/resume;
  • no locked rerun;
  • no post-lock tuning;
  • no manifest/dataset changes;
  • no code/evaluator changes.

Pre-lock evidence hashes bind a historical commit, not the live tree

M11.6A's pre-lock safety evidence (m11_6a_design.PRE_LOCK_SAFETY_EVIDENCE) freezes the SHA-256 of a specific historical state of tests/integration/test_api.py and a bound PASS artifact, as they existed at the design-freeze commit recorded in data/locked/m11-6/m11-6-materialization-manifest.json's design_freeze_commit_sha -- proof that the evidence M11.6's pre-open verification actually relied on has not been silently altered since. It is not a promise that the current, evolving source tree keeps that shared, multi-purpose test file byte-frozen forever after M11.6 closed; that test file legitimately continues to change for unrelated reasons (e.g. a later product-version bump touching an unrelated assertion in the same file). tests/scientific/test_m11_6a_design_freeze.py verifies this binding against the immutable git blob at that recorded commit (git show <design_freeze_commit_sha>:<path>), not against the current working tree, while a separate, independent test in the same file re-runs the actual safety regression (tests/integration/test_api.py::test_new_sample_invalidates_prior_verification_and_reverify_restores_approvability) against current source to confirm the behavior itself still holds. Neither m11_6a_design.py nor run_m11_6_locked_evaluation.py was changed to make this correction -- only the test file's own verification target.

Why rerunning M11.6 would be the wrong reproduction

A one-time final holdout is evidence precisely because it is not repeatedly observed. Running it again after results are known would not “make the result more reproducible”; it would consume the same held-out data again and weaken the governance story.

To reproduce behavior:

  • rerun ordinary unit/integration/scientific tests;
  • run self-test;
  • use development/golden fixtures;
  • inspect/recompute statistics from already-recorded final evidence where allowed;
  • verify hashes and artifact relationships.

Do not reopen or regenerate the final locked population as a fresh performance attempt.

Non-locked checks

From an installed current checkout:

python -m pytest
python -m pyright
python -m ruff check src tests scripts

Historical artifacts record their own exact full-suite counts at each milestone; current counts can naturally change with later documentation/test-only work. Passing current tests does not rewrite the frozen M11.6 scientific result.

Training provenance

The selected training record is ARM_B_M9_6 seed 20260814. It records:

  • 600 physical train scenarios;
  • three equal-weight trained topology families;
  • 20 epochs;
  • 1,350 actual optimizer steps;
  • FINAL_STEP_1350 checkpoint policy;
  • canonical checkpoint hash matching the final release.

The M9.6 manifest preserves the other seeds and calibration/development evaluation row identities.

Known reproducibility caveats

  • Early V5 M1 corpus generation records cross-environment WNTR/NumPy RNG divergence relative to historical cycle-b2 replay; the generated V5 corpus is therefore identified by its own manifests rather than assumed to reproduce old scenario IDs.
  • The final M9.6 train/serve unobserved-age semantic deviation is frozen and documented.
  • Release packaging is aligned as of v0.2.1: the published docker-compose.release.yml image and the current V5 source resolve to the same frozen bundle identity.

These caveats should be fixed only in a separate authorized code/packaging pass, not silently rewritten into the frozen evidence.

Evidence integrity principle

Reproducibility here means: same artifacts, same hashes, same protocol identities, same recorded complete one-time execution, independently inspectable code and non-locked checks. It does not mean treating a final holdout as a benchmark that can be rerun until convenient.