TP-Agent: A LLM Agent for Frontier Theoretical Physics Research

Abstract

We present TP-Agent, an autonomous LLM-based agent utilizing a Plan–Execute–Reflect architecture and external computational tools (Python, Mathematica) to assist in theoretical physics research. We evaluate it on two benchmarks: the comprehensive TPBench and the frontier-level PRL-Bench. While TP-Agent demonstrates strong problem-solving capabilities on TPBench, it shows significant discrepancies with reference answers on PRL-Bench. To investigate this, we develop an LLM-based judge equipped with literature search and mathematical analysis. Our analysis reveals that many discrepancies actually stem from algebraic, conceptual, or logical inconsistencies in the benchmark's reference answers, as well as ill-posed problem statements. These findings underscore that evaluating AI on frontier physics requires rigorously validated benchmarks alongside capable agents. All resources are publicly available.