Haitam Kadri

When Tests Need an Answer: Oracle Underspecification Suppresses Diagnostic Exploration in LLM-Generated Verification

Executable tests generated from natural-language requirements require definite expected outputs. When a requirement leaves an oracle unresolved, a generator can guess, abstain, or avoid inputs on which the unresolved choice matters. This controlled study tests whether requiring a...

When Tests Need an Answer: Oracle Underspecification Suppresses Diagnostic Exploration in LLM-Generated Verification

Executable tests generated from natural-language requirements require definite expected outputs. When a requirement leaves an oracle unresolved, a generator can guess, abstain, or avoid inputs on which the unresolved choice matters. This controlled study tests whether requiring a...

Separating Verification-Regime Effects from Verifier Stochasticity in Coding-Agent Evaluation

LLM-generated tests and behavioral probes are increasingly used as measurement instruments in coding-agent evaluation. This preprint studies whether candidate-patch visibility changes verification outcomes beyond the verifier's own stochastic self-disagreement. In a prospectively...

When Verification Explores Too Far: Semantic Coverage and Validity in LLM-Generated Code Checks

Large language models are increasingly used not only to generate code, but also to generate tests and other evidence intended to verify that code. This creates a methodological problem: a verifier may appear stronger when it explores behaviors beyond the public examples, while so...

When Verification Explores Too Far: Semantic Coverage and Validity in LLM-Generated Code Checks

Large language models are increasingly used not only to generate code, but also to generate tests and other evidence intended to verify that code. This creates a methodological problem: a verifier may appear stronger when it explores behaviors beyond the public examples, while so...