Heng Li

Rows Are Not Samples: Duplicate Records, Evaluation Bias, and Ranking Distortion in Standard Tabular Benchmarks

Standard tabular benchmarks are routinely evaluated with random stratified K-fold cross-validation, a protocol that treats every row as an independent sample. When a dataset contains duplicate records, that assumption fails: the same record can appear in both the training and the...

Extracting Reproducibility Metadata from Machine-Learning Papers: a Taxonomy of Silent Errors and the Cost of Human Verification

Reproducing a machine-learning result requires knowing how the experiment was run: the data, the model and its size, the optimiser, the learning rate, the batch size, the number of steps, the hardware and the seeds. This information is scattered through a paper---often only in a...

Repair Economics for Tool-Calling LLM Agents: the Gain, the Side Effects and the Cost of Failure Recovery

Agents built on large language models (LLMs) do their work by calling tools, and tools fail. The default engineering answer — “wrap it in a retry” — is never costed. We study repair economics: what a failure-recovery strategy buys in task success, what it costs in API calls, and...

Extracting Reproducibility Metadata from Machine-Learning Papers: a Taxonomy of Silent Errors and the Cost of Human Verification

Reproducing a machine-learning result requires knowing how the experiment was run: the data, the model and its size, the optimiser, the learning rate, the batch size, the number of steps, the hardware and the seeds. This information is scattered through a paper---often only in a...

Rows Are Not Samples: Duplicate Records, Evaluation Bias, and Ranking Distortion in Standard Tabular Benchmarks

Standard tabular benchmarks are routinely evaluated with random stratified K-fold cross-validation, a protocol that treats every row as an independent sample. When a dataset contains duplicate records, that assumption fails: the same record can appear in both the training and the...

Write-Time Contracts for Self-Evolving LLM Agents: a Three-Way Ledger of Gain, Regression, and Cost

Agents that edit their own prompt, skills and memory promise continuous improvement after deployment, but a self-editing agent can also damage behaviour that already worked, and it can modify the very artefacts used to measure it. We study write-time contracts: a kernel/layer sep...