EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction

Authors/venue/date: Yuling Shi, Zhensu Sun, Junsen Dong, Chengcheng Wan, David Lo and Xiaodong Gu; arXiv preprint; 2 September 2026. Problem: costly full-length agent evaluations. Contribution: LightGBM success/failure classifiers stop runs once calibrated confidence is reached. Authors report 13–26% fewer steps and up to 44.1% fewer input tokens across three benchmarks at 89–97% prediction accuracy. Significance: potential evaluation cost reduction. Limitations: author-reported, preprint, and early stopping can distort resolve rates. Paper