Counterfactual assessments

A counterfactual assessment predicts the difference between two registered business choices at a fixed outcome day. Each choice schedules named actions, with the same costs, staffing requirements, prerequisites and delays as a normal plan. The prediction is intervention minus baseline, in the registered metric’s units.

Both branches start after ordinary operations close interveneOnDay, before any player decisions that day. Previously accepted commitments continue. Neither branch accepts later player decisions. Exogenous draws keep the same private seed and semantic addresses, while the two branch episode IDs differ. The submission deadline must be no later than the intervention day; the normal exclusive assessment cutoff applies. Within a choice, actions are scheduled in lexicographic action-ID order, preserving one fixed priority when they compete for capacity. The branch verifier checks this order as well as its registered action terms.

Players submit {kind: "counterfactual", value: ...} through submit_assessment. The dashboard presents named choices, timing and the predicted difference. Public snapshots expose the question contract, never a simulated answer, private seed or branch controls. HTTP, MCP and agent runtimes have no fork, rewind or counterfactual oracle operation. Factual verification leaves the optional answer pending; it does not treat a missing offline grade as a failed business objective. This is distinct from a question with an explicit hard requirement.

Required causal conclusions

A task can declare contract_component for a registered action’s financing_net_assets or irreversible_work, or supported_effects for that action’s declared effect set. The player submits a typed conclusion alongside the prediction. Trusted finalization enforces the required conclusion and branch evidence; an ordinary factual seal alone cannot award that requirement. The numerical forecast remains distinct from the contractual conclusion.

A participant’s selected action can fail a branch precondition even when the factual run is valid. That is a participant outcome; missing artifacts, broken execution or unverifiable branches are infrastructure or integrity failures. Neither is replaced by a fabricated counterfactual value.

Offline inspection

After completing a local development evaluation:

bun run worldseq validation counterfactual --results runs/example --out checks/counterfactual

For a private registered campaign, use its independently stored control file and registered agent ID:

bun run worldseq validation counterfactual runs/campaign \
  --control private/campaign.json --agent agent-id --out checks/counterfactual

The campaign form checks the private registration and execution identity. Omitting the control file cannot substitute public development seeds for that campaign. No seed override is accepted. The evaluator first verifies the factual command journal and seal, executes both choices offline, then independently reconstructs the outcome from each branch ledger. Both branch statuses are retained, including rejected or missing branches. Missing outcomes remain unresolved, without an assumed zero effect.

The proper loss is ((prediction - (intervention - baseline)) / scale)²; score is its negative. The latest eligible submitted answer is used. A skipped question remains unsubmitted. The command exits nonzero when any requested question is invalid, unresolved or unsubmitted, or when the result contains no counterfactual questions. Factual task success remains separate.

The new output directory contains report.json, compressed branch seals and checksums. It binds the registration, factual artifacts, question/choice contracts, execution, company/task identities and seed commitment. It does not change the original result directory. The directory is private because branch seals contain authored world conditions; do not publish it as confidential held-out evaluation material. Questions are executed and retained one pair at a time. An interrupted process or storage failure can leave partial artifacts without report.json; those artifacts are not a completed evaluation. Retry into a new output directory and retain the failed attempt for diagnosis.

These are effects of the registered simulator choices under a paired realization. They are not identified real-world treatment effects, a model identity attestation, or evidence that every possible plan has been explored. Tasks advertising this capability need separate passing offline controls before admission.