Auditing MCQA Benchmarks through Probability Landscapes

Authors/institution/venue: Minsoo Song and Chanjun Park; affiliation not specified on the abstract page; arXiv preprint submitted 31 August. Problem: flawed multiple-choice benchmark questions can distort LLM evaluation. Contribution: probability landscapes combine top-answer probability and residual entropy, with noise injection to flag suspect items. Practical significance: cheaper benchmark triage before human review. Limitations: preprint, dataset/model scope, author-reported alignment with MMLU-Redux and no independent reproduction. Paper