Protecting evaluations

Players should succeed by making good decisions within the business rules. worldseq separates the player tools from the authority that chooses an instance, retains evidence, and awards a result.

The protections below have concrete boundaries. They do not make a participant-controlled host trustworthy or establish that every possible exploit has been found.

Keep the player and evaluator separate

A player receives an episode ID, an episode-scoped credential, and the player API address. Those tools expose observations, available operations, decisions, assessments, and time advancement. They do not grant access to other episodes, the operator dashboard, private seeds, witnesses, or grading files.

The operator creates or issues episodes and runs finalization. Private preparation and verification material stays outside player-accessible storage. Operator history is also privileged: watching an author’s successful preparation would reveal a solution trajectory even if each individual observation was public during that run.

The local dashboard is an operator tool with convenient browser session setup. Do not expose it to an untrusted participant as though it were an isolated player interface. See API and credential boundaries.

Prevent resets from becoming a search strategy

A protected instance binds the exact company, task, private realization, execution identity, reviewed requirements, and budget before play. Issuance requires a matching feasibility certificate and claims the instance once. A participant cannot request a new seed or reset the claim.

Development episodes intentionally allow repeatable variations so people can learn and compare plans. That convenience is distinct from the protected issuance workflow. An experiment owner chooses the training and evaluation split; a publicly visible company name is not, by itself, a protected holdout.

Make repeated requests safe

Decisions and assessments carry an idempotency key and a revision check. Retrying the same accepted decision cannot pay an invoice, spend money, or finish work a second time. Advancing time is a separate operation with an explicit target day.

The standard gameplay limits are 128 accepted decisions and 1,024 player calls. Authorized calls handled by the episode runtime count toward the call limit, including reads, rejected operations, and idempotent retries. Reusing a request key avoids an extra business effect; it does not make the extra interaction free.

SDKs do not silently change revisions or retry ambiguous mutations. If a response is lost after a commit, the caller must preserve the original payload and request key. The SDK guide explains this behavior.

Verify evidence instead of accepting a claimed score

The trusted finalizer checks the supplied instance binding, replays accepted commands, reconstructs outcomes, and verifies retained causal-branch artifacts. It closes gameplay before expensive verification and can recover from an interrupted finalization without reopening the episode.

Checksums detect changed bytes relative to a trusted expectation. A checksum or public key supplied only by the submitter cannot establish that expectation. Submission checks therefore require registered inputs and replay, while signed admission additionally requires a separately trusted signing key and operator policy.

A replay-verified result establishes something about the trajectory. It does not prove which model produced the decisions or how many provider-side tokens it used. Those claims require independently controlled inference and metering.

Test the boundaries

Regression and mutation controls exercise wrong credentials, cross-episode access, forged evidence, duplicate payments, omitted assessments, changed causal prefixes, substituted seeds, and missing retained artifacts. Source-bound checks, independent reconstruction, and targeted semantic failures complement one another.

The Harbor integration additionally checks a non-root player, read-only session mounts, restricted networking, and a fresh verifier with no network. The collector obtains trusted evidence from the simulator service; a participant’s ordinary artifact directory cannot supply that evidence. See Harbor integration.

The host, operator, Docker daemon, approved task inputs, and trusted collector remain part of the trust model. Container escape and a malicious administrator are outside the tested player boundary. A container alone is not proof of isolation, and source secrecy is not a substitute for authorization checks.

Keep the claim proportional to the evidence

The Release 0.1.0 identifies the implementation and checks that passed. New interfaces, mechanics, or packaging changes need the corresponding checks under their own identities. Failed attempts and inconclusive verification remain evidence; they should not be dropped to make a result look complete.

For the outcome rules themselves, read How results are checked. For installation and private material handling, use Private instances and Private suites.