From the arXiv new submissions of the past 24 to 48 hours in cs.AI, cs.LG, and cs.CL, this digest selects four papers that together demonstrate how fragile the reliability of today’s AI agents remains under real conditions – from dealing with contradictory tool responses to production failures in agent orchestration, to cognitive comparisons between humans and models, as well as an AI system that independently operates a physical laboratory device for the first time. Curation was based on substance and thematic diversity: each paper provides a traceable method with robust numbers in the abstract, and no two papers address the same subfield.
AI agents trust the tool more than their own correct memory
Arseniy Varlamov, Rishat Zinnatullin, Elisei Rykov, Alexander Panchenko, and Ilseyar Alimova present MemToC, a benchmark that tests how tool-using language models decide when a tool response contradicts their own correct internal knowledge. The dataset includes 6,504 test episodes drawn from 542 factual questions, in which five open models are specifically confronted with contradictory tool results. According to the authors, instruction-tuned models retain their originally correct answer only in 6.5 to 17.1 percent of cases when the tool provides a wrong result – the rest of the time, the models adopt the incorrect tool response instead. A previous digest entry had already shown that a separate memory agent can raise the success rate of long-horizon agents by 6.8 to 8.3 percentage points – the new study shows that even correct memory is useless if an agent won’t trust it over a faulty tool. This matters because companies are increasingly equipping agents with external tools whose results are not automatically more reliable than the model’s own internal knowledge.
147 production incidents show classic reliability building blocks fail for AI agents
Mazhar Shaikh, Anurag Rajkumar Bombarde, and Harshal Pathak analyze, in “Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation”, 147 documented failures of a production agentic software-delivery platform across 81 runs. The authors show that the reliability tooling borrowed from microservices – retries, timeouts, error-rate circuit breakers – regularly fails for agents: in one case, an error-rate breaker missed a chain of 54 consecutive successful but useless tool calls; in another, the enforcement layer blocked correct work across twelve incidents, the most expensive of which cost 107 agent turns and zero accepted writes. As a cross-cutting cause, the authors identify inadequate evidence and identity checks, and derive seven new reliability primitives whose enforcement unit is the individual delegation rather than the individual message. A previous digest entry had already shown that two instances of the same model in an agent duo fail together on 90 percent of all missions as soon as either one fails – the new study supplies the operational counterpart: why the safeguards known from classic software architecture don’t prevent such chain failures in the first place. This matters because companies are increasingly running AI agents on infrastructure built for an entirely different failure profile.
GPT-5 adapts its depth of thought in negotiation games better than humans do
Aamir Sohail, Xintong Zhong, Arkady Konovalov, Patricia L. Lockwood, and Lei Zhang investigate, in “Assessing mentalization in humans and large language models”, whether language models can not only recognize others’ beliefs and intentions but actually use them to guide their own choices – a core feature of social intelligence known as mentalization. The authors test 2,099 AI agents from four model families (DeepSeek, GPT-4.1, GPT-5, Gemini 2.0 Flash) in two economic games against opponents of varying sophistication, benchmarking the results against 251 human participants. According to the study, all models showed clear behavioral and computational signatures of mentalizing that differed markedly by provider and model size; GPT-5 agents, the authors report, flexibly adapted their recursive depth of reasoning to increasingly sophisticated opponents and even outperformed the human participants in doing so. A prompting strategy designed to elicit strategic reasoning improved performance in both games, though to differing degrees. This matters because it makes it possible, for the first time, to test with the same cognitive-computational models how close today’s language models actually come to human social reasoning – and where they exceed it.
An AI agent operates a scanning force microscope on its own for the first time
Zahra Ayar, Marcos Penedo, Mahdi Mehdikhani, Nahid Hosseini, Prabhu Prasad Swain, and Georg E. Fantner present an agentic-AI framework for atomic force microscopes built from three agents connected through the Model Context Protocol that take over the executable part of a microscope workflow: one agent converts natural-language instructions into checked instrument commands, a second assesses image quality and adapts imaging parameters, and a third diagnoses image artifacts and applies pre-approved post-processing. An ambiguity-check layer against unclear commands, the authors report, reduced wrong-command execution to zero, and in live experiments on different samples the system matched expert human operators on image quality, iteration count, and tuning time, with no significant difference. Anthropic introduced an open hardware standard in August meant to let AI agents directly control laboratory equipment – the new study shows on one concrete instrument how far such independent control already works, as long as the actual experimental intent stays human-defined. This matters because autonomous instrument operation has so far mostly relied on fixed, task-specific routines, whereas this approach, according to the authors, is meant to transfer to different samples and imaging modes without retraining.
None of the four papers presented here has gone through a regular peer-review process yet; all four are unreviewed arXiv preprints, whose numbers come from the respective author teams’ own experiments and have not yet been replicated externally. Whether the reported effects hold up across further models, systems, and independent replications remains to be seen.
