The selection of the past 24 to 48 hours on arXiv is intentionally broad today: Instead of a single dominant theme, four papers stand side by side, each addressing a different aspect of AI operations – oversight of AI systems, training planning, agent endurance over long tasks, and evaluation by AI judge panels. The curation was based on substance and diversity: Each paper provides a comprehensible method with robust numbers instead of mere announcements, and no two papers address the same subfield.
Shared review tasks outperform individual reviewers
Victor Akinwande, J. Zico Kolter, and Aran Nayebi investigate why so-called LLM judges – language models that decide on the quality of texts or works as automated reviewers – become less reliable the more individual criteria they are asked to evaluate simultaneously in a single call (Sharding Prevents LLM Oversight Failures and Adversarial Exploitation). Across replications of research results, legal work, and the evaluation of clinical studies, the agreement with human experts decreases as the number of verdicts per call increases – even with identical token and tool budgets. The authors report that their remedy “Sharding” – dividing the review criteria into smaller groups with their own model calls and subsequently aggregating the verdicts – allows a weaker judge model to catch up to or even surpass a significantly more capable, holistically judging model, with the same overall budget. Additionally, sharding mitigates so-called best-of-N attacks, where attackers only vary the presentation of a work to induce an overloaded judge model to accept unmet criteria: Where sharding reduces the fundamental error, this attack advantage largely disappears as well. A previous digest contribution already showed how a single false hint drove the false alarm rate of AI safety committees to 100 percent – the new work provides a structural counterproposal.
A coupled scaling law saves computation time
Mathurin Videau, Badr Youbi-Idrissi, David Lopez-Paz, and Kartik Ahuja address a weakness of classical scaling laws – those formulas that predict the performance of large language models from small training runs (Skaling: Chinchilla’s Exponents Meet Kaplan’s Coupling). Standard formulas assume that model size and training data volume independently influence training error, which systematically distorts predictions at the margins – with very little data or severe overfitting. The proposed “Skaling” formula couples both quantities through a common interaction exponent and, according to the authors, reduces the average percentage prediction error by 1.5 to 3 times, both within and outside the originally measured range. In combination with a sampling strategy specifically limited to low-computation regimes, the method achieves a reliable extrapolation to large models with about ten times less computational effort than usual uniform parameter searches. For labs that need to plan training budgets in the millions in advance, this is a directly usable lever.
Why long agent tasks suffocate on success signals
Mingguang Chen, Licheng Wang, and Bo Qu evaluate 1,547 arXiv papers from 2024 to 2026 in a broad overview to systematically capture where AI agents fail in multi-hour, multi-step tasks (The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents). The authors distinguish three often-confused terms: long task horizons (many necessary steps), long context windows (more processable tokens), and long-term memory (information retained across individual steps). A central finding of the overview: Success signals that only evaluate the final outcome of a task become increasingly unusable as the number of steps grows, which is why the field is increasingly focusing on denser, step-by-step signals through process evaluation models and credit-assignment methods. Concrete approaches for such step memory were already shown in a previous digest contribution about a proactive memory agent that increased the success rate at Terminal-Bench and τ²-Bench by 6.8 to 8.3 percentage points. Those planning agents for multi-step business processes will find here a structured map of open problems instead of a single solution.
Judge panels: Verification helps almost only in close decisions
Yang Shu investigates when external verification signals – such as actually executing a test suite – really improve the accuracy of LLM judge panels (Blind to the Pivotal Vote: Aggregate Independence Metrics Miss Where Verification Actually Helps). The starting point is the finding that nine judge models in a panel often provide only the informational content of about two actually independent judges because their misjudgments are correlated. Aggregated metrics across all decisions therefore show little benefit from additional verification – but when broken down by voting ratio, the effect concentrates almost entirely on narrow one-vote majorities: There, verification improves accuracy across three code benchmarks and four panel sizes by 10.4 to 23.3 percentage points, while it hardly helps in clear majorities. A policy that selectively uses verification only in close votes can therefore manage with a deployment in about 16 percent of requests. For all those using LLM judges for automated quality assurance, this provides a directly implementable savings lever: targeted verification instead of blanket checks.
All four papers are non-peer-reviewed preprints – the usual restriction of arXiv publications applies here as well. Whether the reported effects carry beyond the respective test environments will only become apparent when other teams replicate the methods independently.
