Researchers used a system that converts large, image-rich requirement documents into code, tests and software that can run.
The researchers designed ARC to read requirements written in a structured format and break them into user-interface, programming-interface and database components. It then creates tests for those components and generates code that must pass them, while keeping links between each requirement, the design and the resulting code.
In tests covering six web systems and 101 Android app-generation tasks, ARC performed better than the other AI-based systems used for comparison. A separate study found that 21 people with limited programming experience could write documents describing 50 to 174 scenarios and use them to generate runnable software, including a ticket-booking system of about 10,000 lines of code.
What the system produced
Across three independent trials, the web systems produced by ARC passed an average of 50.6% more automated graphical-interface tests than the tested AI-based baselines. On the AppForge Android benchmark, ARC achieved a 100% compilation success rate and a 68.3% test-case pass rate.
ARC also generated modular designs covering the user interface, programming interfaces and database, along with unit, modular and integration tests. In a study of 21 participants, people with limited programming experience completed structured requirement documents containing 50 to 174 scenarios in an average of 5.6 hours and used them to generate runnable software.
Why structured requirements matter
The results suggest that large, multi-part requirement documents can serve as inputs to an AI programming process that produces more than isolated code. By generating tests and maintaining links between requirements, designs and code, the system is intended to support checking, reuse and later maintenance as well as initial software creation.
This could make it easier for people who are not experienced programmers to describe a substantial software system in a structured document and obtain a runnable version. The study does not establish how well this approach would work for requirements or applications beyond the tested benchmarks.
Evidence and caveats
The evidence comes from two benchmark evaluations and a 21-person user study. The benchmark evaluation covered six runnable web systems with 50 to 200 scenarios each, plus 101 Android app-generation tasks, and used three independent trials. ARC was compared with other state-of-the-art AI-based programming systems, according to the abstract.
The abstract does not report the full benchmark results, the sizes of the comparison systems’ outputs, or a control group for the user study. It also gives no evidence beyond the tested web and Android tasks, so the results may not represent all kinds of software or requirement documents.
