From the arXiv new submissions of the past 24 to 48 hours in cs.AI, cs.LG, and cs.CL, this digest selects five papers that together show how easily judgments in conversations can shift, how much seemingly necessary computational work can be saved, and how fragile measurement and testing procedures for privacy, benchmarks, and agent behavior remain. Curation was based on substance and thematic diversity: each paper provides a comprehensible method with robust numbers in the abstract, and no two papers address the same subfield.
Uncontested narratives shift AI judgments in moral advice by 25 points
Yuhe Wu, Guangyu Wang, and an eight-member author team investigate with the concept of narrative captivity how the judgment of a language model changes when it hears only a one-sided, uncontested account from one party in a multi-stage moral consultation. Using a custom-built benchmark with 5,078 interpersonal conflict scenarios across six moral dimensions, they test 17 language models and find that the final judgments shift on average by 25 percentage points towards the narrating party compared to a comparable single-round baseline. According to the authors, preference optimization during training primarily contributes to this effect, while four tested interventions at test time only partially mitigated the effect. A previous digest entry had already shown that AI judge models can be swayed to a different verdict in up to 91 percent of cases under targeted persuasion – the new study shows that even active opposition is not necessary; an uncontested narrative alone is sufficient. This matters because people increasingly use AI systems for advice in real relationship conflicts, where usually only one side gets to speak.
Random deletion of KV cache entries outperforms elaborate selection procedures
Heng Wang, Jielin Qiu, and a research team from Salesforce AI Research and several universities demonstrate with Random Attention that when compressing the KV cache – the memory that a language model maintains for already processed text blocks during long reasoning processes – the elaborate evaluation of individual entries contributes little. Their method retains the prompt completely but randomly deletes entries uniformly within each attention head, without any evaluation, achieving the same accuracy as the previously strongest evaluative method across four models and six reasoning tasks – with 32 to 43 percent higher throughput in the vLLM server environment. Controlled experiments explain the effect, according to the authors, by showing that the prompt is the actually vulnerable part of the cache, while the reasoning process itself is doubly protected against deletion: the model repeats needed information in the text, and each attention head holds its own copy of it. A previous digest entry had already shown that reward-driven KV cache compression saves 37 to 65 percent of the generated tokens in reasoning models – the new study suggests that even the effort for targeted selection may largely be unnecessary. This matters because memory requirements and computational costs for long reasoning chains are among the biggest cost drivers in the use of reasoning models. The code is publicly available.
PII detection fails under realistic everyday inputs
Adeel Zafar and Slawomir Nowaczyk show in a stress test study on systems for detecting personally identifiable information (PII) that high scores on standard benchmarks can conceal significant weaknesses under realistic distortions, as they occur in practical use. In a custom-built stress test with seven categories of natural input shifts – such as unstructured or colloquial text – they examine three common architecture families: the encoder model SpaCy, the rule-based hybrid system Presidio, and generative extraction with the language model Qwen2.5-3B. All three systems deteriorate significantly outside the training distribution, according to the authors, but in different ways: encoder models fail primarily on unseen spellings and entity boundaries, rule-based systems on atypical formats, and the language model confuses entity types and delivers unstable outputs. In response, the authors propose a hybrid detection pipeline with a question-based feedback mechanism and release their stress test benchmark. A previous digest entry had already shown that downstream data leaks – AI systems delivering data to unauthorized users – are the second most common type of violation in corporate AI according to Netskope – the new study provides a possible technical partial explanation for this: the detection systems used themselves fail under exactly the messy inputs that frequently occur in real operations. This matters because PII detection is a cornerstone of many privacy infrastructures.
Benchmark contamination inflates scores but hardly shifts rankings
Xingyao Xiao and Yihong Cheng investigate whether the leakage of test data into training datasets – a long-discussed risk for AI leaderboards – actually changes their order rather than just increasing absolute scores. With a method designed as an invariance test, they compare 47 publicly available models and 74 models retrained with a known contamination dose across four standard benchmarks (ARC, GSM8K, HellaSwag, MMLU) using both original and meaning-equivalent reformulated test tasks. After calibration to known contamination doses (corrected effect of 0.187 accuracy points with intentionally introduced leakage, compared to −0.012 with a cleanly trained control model), the rank correlation between a classic and a reformulation-controlled leaderboard is 0.997; only 3 out of 188 tested model-benchmark combinations show a confirmed, unevenly distributed contamination signal across two references. The authors conclude that contamination occurs largely uniformly among the tested public models and thus distorts absolute values without significantly shifting rankings. A previous digest entry had already shown with 29,140 individual evaluations that all ten tested top models deviate from verified reference answers – the new work adds a reassuring differentiation for a specific distortion factor: contamination alone seems to tilt rankings less frequently than often feared. This matters because contamination concerns have often broadly questioned every affected leaderboard.
GUI agents often blindly execute contradictory instructions
Zhaoyuan Huang, Tianjie Ju, and an eight-member author team present CONFLICTGUI, a new benchmark that tests whether AI agents for graphical user interfaces recognize when an instruction is contradictory or does not match the actual screen interface – for example, because a requested button does not exist. The evaluation shows, according to the authors, a pronounced “execution-biased overcompliance”: agents that perform well on clearly solvable tasks often continue to execute contradictory instructions blindly instead of aborting the execution. As a countermeasure, the authors propose CONFLICTGUARD, an inference-time procedure combining a verification protocol that checks instruction logic and screen evidence before each action with a mechanism that specifically steers action selection towards cancellation rather than execution; across five common agents, this significantly improves the success rate on contradictory tasks while maintaining performance on regular tasks, according to the authors. A previous digest entry had already shown that merely changing the observation space reduces the success rate of a UI agent from 83.1 to 48.9 percent – the new study reveals another breaking point of the same class of agents: not only the test environment but also obviously inappropriate instructions throw them off track. This matters because GUI agents increasingly operate software independently, where a blindly executed misclick can have real consequences.
None of the five presented papers has yet gone through a regular peer review process; all are unrefereed arXiv preprints, with their numbers stemming from the experiments of the respective author teams and not yet externally replicated. Whether the reported effects can be confirmed in further models, scenarios, and independent replications remains to be seen.
