Daishiro Hirashima

When Is an LLM Code Audit Done? Estimating Audit Completeness with Capture-Recapture over Diverse LLM Lenses

Reproduction artefacts for the paper "When Is an LLM Code Audit Done? Estimating Audit Completeness with Capture-Recapture over Diverse LLM Lenses" (submitted to the Journal of Systems and Software). VSAT runs several deliberately diverse LLM audit lenses over one codebase, treat...

When Is an LLM Code Audit Done? Estimating Audit Completeness with Capture-Recapture over Diverse LLM Lenses

Reproduction artefacts for the paper "When Is an LLM Code Audit Done? Estimating Audit Completeness with Capture-Recapture over Diverse LLM Lenses" (submitted to the Journal of Systems and Software). VSAT runs several deliberately diverse LLM audit lenses over one codebase, treat...

VSAT: Estimating the Completeness of Multi-Lens LLM Security Audits of Web Applications via Capture-Recapture

VSAT (Vulnerability Saturation Auditing) makes the completeness of an LLM security audit a measurable, calibrated quantity. It runs several deliberately diverse LLM audit "lenses" over the same codebase and treats each lens as a capture occasion, so the overlap structure yields (...

Measuring Reverse-Specification Quality Without Ground Truth: A Three-Layer Framework and a Study of Scale-Induced Failure Modes in Autonomous Coding Agents

Tool-using LLM coding agents can reverse-generate specifications from source, but the quality of those specifications is hard to assess when no reference specification exists. We present a ground-truth-free, deterministic-first framework that measures three layers - inventory cov...

Measuring Reverse-Specification Quality Without Ground Truth: A Three-Layer Framework and a Pre-Registered Study of Scale-Induced Failure Modes in Autonomous Coding Agents

Tool-using LLM coding agents can reverse-generate specifications from source, but the quality of those specifications is hard to assess when no reference specification exists. We present a ground-truth-free, deterministic-first framework that measures three layers - inventory cov...