Assessment Is Not Assurance: Recommendation Homogeneity, the Execution Gap, and the Limits of the Security Assessment Market in IT and OT

Abstract

Organizations spend billions per year on recurring security assessments: compliance audits, risk and maturity assessments, penetration tests, and advisory engagements. This paper tests the premise that assessment produces protection against three bodies of evidence. First, an original analysis of 122 CISA joint cybersecurity advisories (2020 to 2026) shows mitigation guidance is close to invariant with respect to the adversary: 92.6 percent recommend logging and monitoring, 91.0 percent credential controls, 86.9 percent patching, with ransomware and nation-state advisories prescribing nearly identical lists. Second, the assessment industry's own remediation telemetry shows fewer than half of pentest findings are ever fixed and median resolution times run multiples of contracted SLAs. Third, longitudinal compliance data show the assessed state decays within months (27.9 percent sustained PCI DSS compliance), federal agencies show two decades of recycled FISMA findings, and certified-then-breached firms are retroactively decertified, rendering the attestation unfalsifiable. The paper examines the transposition of this model to OT, where several standard-list items are infeasible as written, proposes a structural mechanism, and states what an assessment with execution accountability would require. The Section 3 measurement scripts and raw data are published with this record. Project-scoped consulting is explicitly distinguished and is not the subject of the critique.