Teaching AI models helped them write code for unfamiliar private libraries
A training approach improved how language models used private software APIs, rather than merely showing them documentation.
Editorial illustration — not from the study.
The study examined code generation involving private libraries, which are often absent from the public material used to train language models. The researchers found that supplying API documentation alone did not reliably teach models how to call those APIs correctly.
They developed PriCoder, which automatically creates training examples for a target library and then fine-tunes the model on them. Its data-generation process builds from basic API knowledge toward more varied examples and filters the resulting data for quality. On two benchmarks using libraries unfamiliar to the tested models, PriCoder improved performance across all evaluated models and generally outperformed retrieval-based approaches on standard code-generation tests.
How the training performed
Across two private-library benchmarks and all evaluated models, PriCoder improved pass@k by 13.53 to 31.66 percentage points compared with the unadapted, or “Vanilla,” models. Pass@k measures whether at least one of k generated answers passes the test. PriCoder also generally performed better than retrieval-based methods on the HumanEval and MBPP code-generation benchmarks.
The researchers’ earlier tests found that giving models complete information about the required private APIs was not enough. For example, with that information, LLaMA3.1-8B-Instruct’s pass@1 rose from 8.13% to 13.10%, while the larger LLaMA3.1-70B-Instruct improved by 8.32 percentage points. The study’s conclusion says PriCoder had little effect on general code-generation ability, and additional tests supported the contribution of both of its data-generation components.
Why API knowledge is not enough
The results suggest that access to API documentation is not the same as knowing how to use a private library in working code. For organizations that rely on software libraries not found in public training data, adapting a model with automatically created examples could be a more effective approach than only retrieving documentation during generation.
PriCoder still requires both data creation and model fine-tuning. The authors report that, for each tested target model, these steps together took no more than 50 hours, which they describe as a potentially acceptable one-time cost for long-term enterprise use.
Evidence and caveats
This is an experimental study using two newly constructed benchmarks based on recently released libraries that were unfamiliar to the tested models. The evidence comes from comparisons with unadapted models and retrieval-based baselines, along with ablation tests of PriCoder’s components; the excerpts do not identify the full set of evaluated models or provide all benchmark details.
The approach adds computational cost because it must synthesize training data and fine-tune a model. The tests also focus on two private-library benchmarks, so the results do not by themselves establish how PriCoder would perform across all private libraries, organizations or development tasks.
// Source
ACM Transactions on Software Engineering and Methodology · 2026 · DOI: 10.1145/3862481
Researchers developed two classical methods for choosing the starting parameters of quantum circuits used in chemistry calculations. The methods improved simulated results, while hardware tests showed that carefully chosen parameters could outperform random starting points without a later quantum optimization step.
Researchers tested multilingual AI models that label web writing—such as news reports and discussion forums—in 16 languages. The best system reached 79% F1 on average, while removing documents with uncertain labels raised performance above 90%, pointing to ambiguity in the text itself.
A study of 26 people using ChatGPT for a complex web development task identified nine types of failure, including wrong or incomplete answers, too much mental effort and lost context. Participants who received unhelpful responses were 11 times more likely to abandon ChatGPT, while additional prompts were linked to a lower chance of giving up.