From the arXiv new submissions of the past 24 to 48 hours in cs.AI, cs.CL, and cs.LG, this digest selects five preprints that demonstrate how fragile common measurement and training procedures of today’s AI systems are upon closer inspection: how reliable the consensus of multiple AI judges actually is, how uniform AI judgments in applications can exacerbate discrimination, whether computer-operating agents can jointly learn safety and task success, how safety-harming training data can be filtered across multiple languages, and whether shared experts can make transformers more efficient without losing performance. Curation was based on traceable methodology in the abstract, concrete core results, and thematic diversity instead of fivefold repetition of the same subfield.
Evaluation and Fairness
AI judges are more in agreement than their errors would be independent
Elias Hossain, Niloofar Yousefi, and Ser-Nam Lim investigate in their study how reliable it is to conclude the correctness of a judgment from the consensus of multiple AI judge models (LLM-as-a-Judge, where one language model evaluates the output of another model). According to the authors, this practice assumes that the judges make their errors independently – an assumption they test on a bank of ten open and frontier judge models. The average pairwise error correlation is 0.21, meaning that the ten judges together provide only about as much statistical information as 3.5 actually independent judges; among particularly high-performing frontier judges, the dependence is even more pronounced, even across different providers. If shared errors are ignored, one can conclude in up to 28 percent of model comparisons that a system is significantly better, while considering error dependence no longer supports this difference. This matters because an earlier digest finding on day-to-day instability of AI judges already showed how fragile individual judge decisions can be – this study provides the structural reason why even connecting multiple judges does not automatically solve this problem.
AI consensus in applications triples systemic exclusion
Matthew Bone, Fabian Stephany, and Maria del Rio-Chanona show in their investigation what happens when many employers use similarly trained language models for applicant pre-selection. They compared ten language models’ base versions with their fine-tuned (post-trained) counterparts in simulated hiring decisions. According to the authors, fine-tuned models invite older applicants to interviews 3.6 percent less often on average than their base versions – an effect that was observed in eight of the ten tested models and is primarily attributed to increased age discrimination in post-training. Because fine-tuned models make significantly more consistent decisions than base models, likely driven by features such as qualifications or field of study, the jointly excluded applicant group across all models rises from 5.6 to 17.3 percent; for applicants with multiple disadvantaged characteristics simultaneously, the exclusion rate even ranges between 12.2 and 21.7 percent. This matters because it reveals a discrimination risk that goes beyond individual faulty models: If many employers use similarly trained models, individual biases can coalesce into a uniform, market-wide exclusion pattern, according to the authors – a mechanism that points beyond the individual case of the AI-supported Meta layoffs against which 26 employees sued.
Agents, Safety, and Efficiency
Computer agents learn task success and risk assessment together
Zeyu Kang and a seven-member team present SCOPE (Safety and Capability Optimization for Policy Execution), a training method for computer-operating agents (CUAs, AI systems that perform tasks directly through graphical user interfaces) that trains task success and safety-conscious decision-making together rather than separately. For aligned training material, the authors additionally developed SCOPE-Gen, an automated pipeline that generates verifiable tasks and derives paired variants with environmental risk, as well as the resulting trajectory dataset SATraj-OS with capability demonstrations, safe continuations, and explicit refusals. Initially trained through supervised fine-tuning on all three trajectory types and subsequently improved through online reinforcement learning, the model Qwen3.5-9B achieved 54.17 percent task success on the OSWorld benchmark, 64.30 percent attack avoidance on OS-BLIND, and, with 58.80 percent, the highest measured combined capability-safety value among the compared agents. This matters because earlier digest findings on agent risks and guardrails showed that pure task training does not automatically deliver safety – SCOPE provides a concrete training approach that addresses both together instead of adding safety afterward.
Safety-harming training data hide in different language layers
Jiakun Li and five other authors show in their work that seemingly harmless fine-tuning data (post-training datasets meant only to convey new capabilities) can unnoticeably undermine the safety alignment of a language model – and that existing detection methods, which only consider a single safety-relevant model layer, fall short for multilingual models. Their cross-linguistic analysis shows that safety-relevant layers only partially align between languages and that safety signals often distribute across multiple layers instead of bundling in a single one. With MMSAFE, they present a multilayer framework that captures both shared and language-specific safety signals to identify safety-harming training data. In experiments across multiple models, languages, and safety benchmarks, MMSAFE reduced the average rate of harmful responses by 60 percent compared to random data filtering and, on average, also outperformed the strongest single-layer method. This matters because safety alignment has often been developed and tested primarily on predominantly English-language data – this study shows that protective mechanisms cannot be easily transferred from one language to another.
Shared experts make transformers more efficient without losing accuracy
Dian Jiao and five other authors address in their work a structural problem of today’s mixture-of-experts models (MoE, an architecture where only a part of specialized subnetworks is activated per input): because each transformer layer traditionally has its own, layer-isolated experts, neighboring layers often learn redundant functions. With CS-MoE, the authors instead propose a central, cross-layer shared expert pool that individual layers can access in addition to their own independent experts – a mechanism that makes the activation of parameters and computational effort per token flexibly controllable. In experiments, CS-MoE achieved a lower perplexity (a measure of a language model’s predictive quality) than equally sized dense transformers while activating only 55 percent of the parameters; performance also scales evenly with the number of activated experts and approaches classical MoE models with higher computational effort at a fixed computational budget. This matters because it creates a flexible trade-off between computational cost and model capacity that can be specifically useful for computation-constrained environments, similar to what an earlier digest finding on an MoE model running on a single edge graphics card showed.
Four of the five works are currently unreviewed preprints from the latest arXiv submission wave; the referenced figures are based on the abstracts and information from the respective author teams and have not been confirmed by independent review or replication. The study on multilingual safety signals is reportedly accepted for the main conference of EMNLP 2026, but at the time of this digest, it is still available as a preprint. How robust SCOPE’s safety-capability compromise and CS-MoE’s parameter savings hold outside their respective test environments remains to be shown by independent replications – the study on judge consensus in this digest itself provides a caution: even multiple consistent automated judgments are no substitute for independent verification.
