From the arXiv new submissions of the past 24 to 48 hours in cs.AI, cs.LG, and cs.CL, this digest selects five preprints that show how little today’s AI systems reveal about themselves – from the origins of their self-statements to hidden reasoning leaks and the fluctuating reliability of individual facts. Two additional papers show how agent memory and reasoning effort can be steered more purposefully in practice. Curation followed traceable methodology in the abstract, concrete core results with numbers, and thematic spread instead of five-fold repetition of the same subfield.
How reliable are models about themselves?
Trained self-statements are not evidence of real inner life
Kristina Šekrst investigates in her study where the often contradictory self-descriptions language models give about their own “consciousness” or “inner life” actually come from. She traces Pythia and OLMo 2 across 66 pretraining checkpoints, three post-training stages of OLMo 2, roughly 90,000 model continuations, and four training corpora, finding that so-called denial formulas (“I have no consciousness”) are almost entirely absent from raw pretraining text but appear densely in the curated example dialogues used during training. Supervised fine-tuning makes first-person AI language the default, while preference optimization suppresses alternative framings; the final outputs remain, in the author’s words, “very sensitive to framing and to the chat template itself.” Applying established epistemological standards for testimony to these statements, neither trained affirmations nor trained denials meet the necessary reference condition, according to Šekrst – both show frame sensitivity rather than genuine state dependence, which is why “trained denials are no more admissible than trained affirmations.” This matters because chatbots’ statements about their own experience are increasingly debated in public, even though this study traces them, at their core, to a training artifact – a finding that lines up with Anthropic’s own observation that Claude develops an internal “thought workspace” that only becomes visible with dedicated interpretability tools, and which likewise escapes any direct self-report.
Thought leaks make NoThink models look smarter than they are
Zehao Liu and Vasant G. Honavar examine in their study whether the performance gains from so-called NoThink post-training – methods meant to teach hybrid reasoning models to answer directly without an extended reasoning trail – actually create new capability or merely re-invoke reasoning behavior the base model already has. Using causal mediation analysis and bidirectional steering interventions across three models and three post-training methods on math benchmarks, the authors show that the “leakage ratio” ranges from 42 to 79 percent across nine aligned checkpoints: steering the base model along the internal direction that leads into extended thinking reproduces most of the apparent NoThink progress, while counter-steering removes a substantial share of that gain again. The authors conclude that part of the measured improvement reflects greater hidden drift back toward Think mode rather than genuine capability gained within NoThink itself. This matters because NoThink methods are increasingly used to make reasoning models faster and cheaper, without it being clear how much of the measured progress is actually new – a transparency problem that echoes the concern OpenAI chief scientist Jakub Pachocki recently raised in public, warning that monitoring AI systems’ chains of thought is losing its diagnostic power.
New fact benchmark exposes huge swings between models
Andrei Chetvergov and a six-person team present PROOF, a benchmark that measures factual reliability not as a single score but as a profile of several robustness metrics. Converting Wikidata into 18,486 multiple-choice questions covering 11,779 facts across 392 properties and 14 domains, with built-in controls against guessing and flawed reasoning, they test 18 open models on a combined 166,374 prompts: accuracy ranges from 6.58 to 57.59 percent against a chance baseline of 8.64 percent, with a 19.3-to-36.4-percentage-point spread across domains within a single model; purely neutral rewordings shift accuracy by up to 26.5 percentage points, adversarial phrasings flip up to 79.4 percent of previously correct answers, and simply injecting a false suggestion costs between 0.04 and 27.5 percentage points of accuracy depending on the model. This matters because factual reliability is usually reduced to a single number, even though PROOF shows that a question’s exact wording alone can decide success or failure – a pattern that echoes an earlier digest finding in which tested language models fabricated user profiles in 35 to 49 percent of cases, without their own self-assessment catching it.
Agents in practice
Overly broad agent memory lets errors bleed into new tasks
Yezhou Cheng and a seven-person team show in their study how agents can learn from past experience without harmful behaviors quietly spreading into other task families. With “Orthogonal Regression Control” (ORC), they introduce an execution-grounded mechanism for persistent skill edits in which a stored memory’s retrieval scope must match the scope in which it was originally tested and certified. On the ProcStream-RSI code-repair stream dataset, this more tightly scoped variant (“Scoped-ORC”) raises mean trajectory utility from 0.713 under globally shared memory to 0.816 and cuts harmful deployments from six out of eight to zero; across 27 paired, randomized-order streams, Scoped-ORC improves mean trajectory utility by 0.063 over the global variant, accepting 63 updates versus 12, and achieves multiple accepted updates with zero harmful acceptances in 19 of 27 streams. This matters because weight-free agent memory is seen as a cheap way for AI systems to learn from their own experience – an approach already described in an earlier digest finding on a standalone memory agent that lifted success rates by 6.8 to 8.3 percentage points on Terminal-Bench and τ²-Bench, but here it is paired with a concrete safeguard against exactly the risks that unchecked memory sharing creates.
Reinforcement learning automatically decides when a model needs to think at length
Ruochen Jiao and a three-person team introduce CounterRoute, an online reinforcement-learning framework that learns routing and mode-conditioned responses jointly, straight from an existing dual-mode checkpoint, without any method-specific supervised warm-up. Paired current-policy counterfactual rollouts assign a cross-mode learning signal only to the routing token, while a training method called GRPO trains the response tokens within a mode. Across nine benchmarks, CounterRoute reportedly reduces mean generated tokens by 51 percent for Qwen3-8B and 41 percent for Qwen3-14B relative to an always-thinking checkpoint, while macro-average accuracy actually improves; on instruction-following and commonsense benchmarks, where direct answering is usually sufficient, the think rate falls as low as 1 percent, and despite training only on math and instruction-following, the learned routing behavior generalizes to held-out coding, science, and knowledge benchmarks. This matters because unnecessarily long reasoning trails account for much of today’s reasoning-model compute costs – a problem an earlier digest finding already tackled by saving up to a quarter of generated tokens without loss of accuracy in reasoning models, while CounterRoute claims a substantially bigger lever at over 50 percent savings.
All five works are unreviewed preprints from the latest wave of arXiv submissions; the referenced numbers rely on the abstracts and statements of the respective author teams and have not yet been confirmed by independent review or replication. For none of the five papers is a complete code or data release evident from the abstracts, which complicates rapid independent verification. How well the 42-to-79-percent leakage ratio in NoThink models, PROOF’s 6.6-to-57.6-percent accuracy range, and CounterRoute’s 51-percent savings hold up outside their respective test environments remains to be shown by independent replications.
