Computer Science · Information Systems
An exploratory study compared four ways to develop software with and without conversational AI systems. Human–AI collaboration produced higher-quality, better-organized test suites, while autonomous AI agents generated functionally correct code more quickly but added complexity and untested decisions.
A systematic survey examines how large language model-based AI agents are being developed to resolve software issues described in everyday language. It maps the field’s benchmarks, techniques and studies, and identifies reinforcement learning as an increasingly important training approach.
A system called ARC converted large requirement documents containing hundreds of scenarios into runnable web and Android software. Across two benchmarks, the systems passed more automated interface tests than other tested AI-based programming systems, while also producing designs, tests and links between requirements and code.
A system called IntentionTest generates software tests from a developer’s description of what a function should do in a particular situation. Across 3,680 test cases from 12 open-source projects, it outperformed four other test-generation methods on measures of relevance, coverage and successful execution.
Researchers introduced a benchmark and an AI-based framework for testing and improving the security of generated Python code. Across multiple language-model systems, the framework improved both first-attempt functional success and resistance to targeted security tests compared with existing approaches.
Researchers tested three Java code-checking tools and identified 14 previously unreported false negatives or false positives. Developers confirmed 11 of the issues, and nine had been fixed at the time of the study.
AI vulnerability detectors often perform less well on software projects outside their training data. A study finds that cleaner, more varied training data and an encoder-based design improved detection, including a reported 6.8% gain in recall on the BigVul benchmark.
