Assessment requirements

Categorical questions can opt into a public, typed eligibility rule:

{
  "kind": "categorical",
  "id": "incident-cause",
  "prompt": "Identify the explanation supported by the available evidence.",
  "dueOnDay": 7,
  "candidates": [
    { "id": "provider-a", "label": "Provider A" },
    { "id": "provider-b", "label": "Provider B" },
    { "id": "demand", "label": "Demand change" }
  ],
  "requirement": { "kind": "supported_conclusion" }
}

This is a public question projection, not a complete authoring specification. The trusted task separately declares truth and a partition of candidates in acceptedGroups. Those fields never appear in the player projection. Authors must justify indistinguishability from evidence actually available before the deadline; a hidden causal label alone is not a knowable answer.

An answer to an opted-in question includes:

{
  "kind": "categorical",
  "probabilities": [
    { "candidateId": "provider-a", "probability": 0.4 },
    { "candidateId": "provider-b", "probability": 0.4 },
    { "candidateId": "demand", "probability": 0.2 }
  ],
  "conclusionCandidateIds": ["provider-b", "provider-a"],
  "unresolved": true
}

Use this answer inside the existing submit_assessment request with request key, current revision, question ID, rationale and evidence references. HTTP, MCP and agent responses use the same request schema. Submit strictly before dueOnDay. The dashboard provides separate confidence inputs and conclusion checkboxes.

Verification and eligibility

The server accepts only unique, known conclusion candidates. It checks structure without revealing whether an answer is correct. Independent verification grades the latest eligible ledger submission, never rationale text or a best historical answer. The unordered conclusion must equal the accepted equivalence group that contains the truth. unresolved must be true exactly when that group contains multiple candidates. A correct singleton is also supported. Selecting every candidate is not a shortcut when some alternatives are distinguishable.

Grouped Brier loss remains unchanged. Redistributing probability within the same accepted group preserves its loss. There is no confidence threshold or forecast exact-match rule. Good probability calibration cannot compensate for an unsupported required conclusion, and a supported conclusion does not force a perfect probability score. No LLM grades the answer or its rationale.

Required results contain a bounded requirement status (passed, failed, or incomplete) and reason. A missing submission is incomplete while its deadline is open and failed afterward. Invalid grading evidence still makes the entire verification invalid. When any requirement is present, the verifier emits constraints.requiredAssessments; all required answers must pass. Direct results, Harbor rewards, evaluation progress/import, campaign eligibility and behavioral admission already enforce every verified constraint. A valid ledger with a failed required answer remains retained and ineligible; money cannot override it.

Applicability and causal requirements

Catalog tasks opt in explicitly through their question requirements. Do not infer a requirement from a title or narrative. Changing a question requirement changes the task contract and requires new identity-bound validation; it cannot be applied retroactively to an old certificate.

Forecasts do not use the categorical supported_conclusion contract. Counterfactual questions have separate optional hard requirements: contract_component identifies a registered action and its financing-net-assets or irreversible-work component; supported_effects identifies the registered action’s supported effect set. A required conclusion is checked by trusted finalization with independently verified branches. An optional unsubmitted prediction remains optional. See counterfactual assessments for the branch and conclusion boundary.

Focused regressions are in tests/assessment-requirements.test.ts and dashboard/tests/assessment-values.test.ts; protocol tests exercise the shared HTTP/MCP/agent request boundary. These checks establish mechanical enforcement, not the scientific validity of an author’s equivalence classes.