How it works — 6 of 7

Evidence & honesty

Every analysis this platform produces — a scenario run, a validation check — is tagged with a claim about what kind of evidence it actually is. The tag is not decoration: it comes from the same code that decides whether a run counts as one status or another, and it does not move unless that code moves with it.

The honest question about any check is not just "did it pass" but "what is it actually claiming". A committee behaving consistently on an invented scenario is a real result, but it is not the same kind of result as a government's actual growth rate being called correctly before anyone knew it. Collapsing both into a single pass/fail badge would flatten that difference and quietly blur unlike claims — so the platform keeps five separate labels instead of one, attached at the level of what a check measures, not chosen per run by whoever happened to run it.

The distinction that matters most among the five is whether a run's own parameters were tuned on the same evidence it is later checked against. A check that never touches real-world data makes a synthetic claim. One whose parameters were fit to a historical window is calibrated and in-sample. A retrodictive check holds out a historical episode, while a forecast is registered before its future outcome is known. A scenario explores a counterfactual instead. These are different questions and time relationships, not rungs on a single quality ladder; any result must earn and display the type it actually supports.

flowchart TD
    accTitle: Five evidence claim types
    accDescr: Synthetic, calibrated, retrodictive, forecast and scenario claims branch from one taxonomy and are not ordered as a quality ladder.
    T[Evidence claim type]
    T --- A[Synthetic]
    T --- B[Calibrated]
    T --- C[Retrodictive]
    T --- D[Forecast]
    T --- E[Scenario]
  

Five different claim types, not a quality scale.

The five statuses, exactly as the code defines them right now:

• Synthetic — Tests the mechanism on a made-up trajectory. No claim about the real world.
• Calibrated — Parameters tuned on a historical window (in-sample). Direction/sign/lag, not point prediction.
• Retrodictive — Reproduced outside the training window; the outcome was not used to fit parameters.
• Forecast — Probability was registered before the outcome was known.
• Scenario — Conditional counterfactual: 'if these assumptions hold, the model gives...'. Not a prediction.

This page explains the taxonomy; it does not restate the platform's full boundaries. For what the system does not claim, prohibited uses, and the one published bias declaration, see Scope & limitations.