Rodrigo Pato Nogueira

PROBE: Benchmarking code generation in large language models

Abstract Large Language Models (LLMs) are increasingly being used in everyday software engineering tasks, particularly in automated code generation. Despite their widespread adoption, these models remain far from perfect, making systematic and fair evaluation essential to underst...