Mohammed Sameer Syed

Agents Don't Game the Linter: Measuring Whether LLM Repair Satisfies the Analyser or Fixes the Defect

Four code-specialised LLMs were shown static-analysis findings on their own solutions and asked to fix them. The study was designed to catch Goodharting: satisfying the analyser that was shown while the defect stayed put. It did not happen. Across 4,439 repair steps the models wr...

Agents Don't Game the Linter: Measuring Whether LLM Repair Satisfies the Analyser or Fixes the Defect

Four code-specialised LLMs were shown static-analysis findings on their own solutions and asked to fix them. The study was designed to catch Goodharting: satisfying the analyser that was shown while the defect stayed put. It did not happen. Across 4,439 repair steps the models wr...