For safety-critical systems, certify the controls around AI—not the AI itself

The review says existing safeguards reduce language-model errors but do not provide the bounded, verifiable behavior required by major safety standards.

Editorial illustration — not from the study.

A review of language models and safety certification finds a structural mismatch between the two. Certification regimes require behavior that can be bounded, traced and supported by evidence, while language models retain a nonzero error floor and can produce plausible but incorrect outputs.

The authors argue that common safeguards—including retrieval tools, guardrails, formal checks, uncertainty estimates and hybrid systems—can reduce risks but do not eliminate the gap. They identify two possible paths: set statistical acceptance criteria for narrowly bounded tasks, or place the language model in an untrusted proposer role inside a deterministic, independently verifiable control system.

What the review found

The review identifies a structural mismatch between large language models and certification regimes used in areas such as aviation, road vehicles, medicine and critical infrastructure. Standards including IEC 61508, DO-178C and ISO 26262 require system-level risk targets, traceability, controlled configurations and evidence of bounded behavior.

The authors report three main findings. First, under stated conditions, mathematical results imply a nonzero error floor for calibrated probabilistic generators. Second, benchmark results for legal, medical and agentic tasks cannot be directly converted into certification targets, and the reviewed deployments do not provide the hazard and exposure models needed for a compliance demonstration. Third, retrieval augmentation, guardrails, formal verification, uncertainty quantification and neurosymbolic hybrids are documented to narrow the gap but not close it.

The review also formalizes how errors can compound over an execution horizon. It identifies two possible ways forward: standards could define statistical acceptance criteria for bounded tasks, or system designers could confine the language model to an untrusted proposer role within a deterministic, independently verifiable execution envelope. On the evidence reviewed, the authors describe certifying that envelope rather than the model as the most defensible current approach.

Why the controls matter

A language model may produce convincing language without providing the predictable, bounded behavior that safety certification requires. For systems where errors can create serious hazards, the review shifts the focus from trying to certify the model as a whole to checking the controls that limit what it can do and independently verify its outputs. That approach is presented as consistent with both the mathematics discussed in the review and the obligations of existing safety standards.

Evidence and limits

This is a review article, not a new deployment study or certification demonstration. It brings together results from statistical learning theory, measurements of LLM errors in high-stakes tasks, and safety standards and regulatory documents. The review reports that no published deployment examined in its analysis provides the system-level hazard and exposure model needed for a compliance demonstration. Its conclusions concern the current evidence and existing certification frameworks; the abstract does not establish how future standards or architectures might change the assessment.

Reliable AI partnerships depend on people, not just stronger models

A survey of human-AI collaboration argues that more capable AI systems do not automatically produce reliable partnerships. It identifies human-centered design, governance and attention to high-stakes uses as central to making these systems trustworthy and beneficial.

9 hours ago·6 views

AI & Computing

RNA splicing tools can follow irrelevant clues and miss RNA’s shape

Deep-learning tools for predicting RNA splicing can rely on genomic clues unrelated to splicing and miss important effects of RNA structure, a study finds. These weaknesses were linked to systematic errors, especially when the tools evaluated sequences that differed from those used for training.

20 hours ago·5 views

AI & Computing

Better-trained AI finds more flaws in unfamiliar software projects

AI vulnerability detectors often perform less well on software projects outside their training data. A study finds that cleaner, more varied training data and an encoder-based design improved detection, including a reported 6.8% gain in recall on the BigVul benchmark.

2 days ago·34 views