Structural Admission: A deterministic harness for preregistered admission of dependency structure in staged experimental tasks
Abstract
A staged experimental task in machine learning research carries an implicit claim: that the task contains the dependency structure the study intends to test. When that claim goes unverified, learning results are interpreted against a structure that may not exist in the environment as built. Structural Admission tests the claim before any learning result is interpreted. The harness runs a configured task through a preregistered sequence of operational checks. Calibration seeds are disjoint from task-rollout seeds. The conditional mutual information (CMI) threshold (Cover and Thomas 2006) is fixed from synthetic data before any candidate rollout is generated. Both a random policy and a scripted oracle are required, and the oracle receives only the observations and legal actions available to a learner. Environment noise and random-policy draws are paired across conditions. Pre-disclosure marginal leakage is checked on preregistered field groups, and oracle return, resolution accuracy, underdetermination, and constant-action insufficiency are checked automatically. A run produces one of three decisions. Admitted means every calibrated structural gate passed. Rejected means calibration was valid but a substantive gate failed. Inconclusive means calibration power or calibration/task matching prevented a valid decision. Rejected reports are successful experimental outputs; the harness never deletes or silently reclassifies them. Every run writes raw transitions before aggregation, records strict SHA-256 hashes of all deterministic artifacts in a manifest, and refuses to overwrite formal output directories. A reproduce command replays a run in another environment and compares every deterministic artifact byte for byte. Environment metadata is recorded descriptively but excluded from the byte-identity gate. Deterministic here describes the artifact generation and the decision procedure, not the structural conclusion. The verdict rests on statistical calibration, CMI estimation, fixed thresholds, and rollout sampling; what the harness guarantees is that the same configuration and seeds produce the same artifacts and the same decision, byte for byte. Admission is an operational verdict, not a scientific one. An admitted task passed its preregistered checks. The harness can not determine whether a task operationalization has external, psychological, or philosophical validity.
// Source
Authors: Bruce Tisler