New technical note AGE-PN-001: reproducible benchmark for WDM stability See evidence
Open protocol · Third-party verifiable

Falsify AGE: sealed forecasts. The forecast is fixed before it is measured.

Every AGE benchmark ships with a forecast produced before the held-out segment is revealed or scored. The SHA-256 hash of that forecast is published with the run manifest, so anyone can check that the forecast was not changed afterwards. If a forecast does not hold, falsify it: we publish the result either way.

What sealing is

A prior commitment, not a promise.

Sealing turns a claim about the future into a verifiable commitment: the forecast is recorded with its hash before the reference is known, and the run manifest reproduces it bit by bit.

01

Sealed forecast

The forecast for the held-out segment is generated and its SHA-256 hash is recorded together with the run manifest: code version, seed, input data and date. The hash changes if the file changes, so there is no room for retouching afterwards.

02

Independent scoring

The reference is scored afterwards, with the metric and failure conditions agreed before sealing. Anyone can repeat the scoring with the same files: the result does not depend on AGE computing it.

What sealing guarantees

  • That the forecast existed before the reference was known.
  • That it was not retouched after scoring: the hash gives it away.
  • That the manifest reproduces the very same forecast.

What it does not guarantee

  • It is not an independent audit: the hash is generated by AGE itself.
  • It does not validate the physics or the model quality: a sealed forecast can be a bad one, and that is how it gets published.
  • It does not replace peer review or a complete cryptographic guarantee of the process.
Evaluation protocol

The prediction is fixed before the held-out segment is scored.

The same flow is used in the AGE-PN-001 photonics benchmark and will be adapted to pulse traces in AGE-PN-002. Failure conditions are fixed before running: if the estimator does not pass them, that is what gets published.

01agree

Fix

Schema, metric, baseline, temporal split and failure conditions.

02identify

Fit

Estimate the model using only the allowed identification window.

03commit

Seal

Generate the prediction and record its hash with the run manifest.

04score

Evaluate

Compare with held-out truth and report metrics, failures and sensitivity.

For labs

Falsify our forecasts with your data.

Download the reproducible package, regenerate the forecast from the manifest and score it against your own reference. If your measurement contradicts the sealed forecast, you have a publishable result, and we link it from Evidence, favourable or not.

How to falsify a forecastreproducible
  • Download the package and verify the published hash.
  • Regenerate the forecast from the run manifest (version and seed).
  • Score it against your reference with the metric agreed before sealing.
  • Publish your result and send it to us: we link it from Evidence.
What the package includesage-photonic-stability
  • Python code with automated tests.
  • SHA-256 sealed-forecast flow.
  • Run manifest per execution.
  • Synthetic sample data and a nominal benchmark.