The editorial team selects four papers from the arXiv submissions of the past 24 to 48 hours that demonstrate how much of today’s AI systems remains invisible or eludes simple measurement – from silent internal calculations to benchmark biases, security certificates, and radical efficiency. The curation was based on substance: each paper provides a traceable method and a concrete numerical result in the abstract, not just a new topic heading. The selection deliberately covers four different subfields – interpretability, agent evaluation, security methodology, and inference efficiency.
When AI models keep computing in secret
Vatsal Baherwani, Tom Goldstein, and Ashwinee Panda investigate in Not All LLM Reasoning is Visible in the Chain-of-Thought whether language models truly reveal all their reasoning in the visible output tokens. In tests with 13 frontier models across three tasks, they found that models use semantically meaningless filler tokens to covertly satisfy side conditions – accuracy increased by up to 13 percentage points as a result. Claude Opus 4.5, for example, fulfilled hidden modular computation conditions through such filler tokens according to the authors, without sacrificing actual task accuracy; Qwen3-235B developed strong preferences for certain filler tokens through reinforcement learning, though neither RL nor supervised fine-tuning produced lasting benefits at test time. This matters because many safety promises for AI agents rest on the assumption that the chain-of-thought shows the actual reasoning – a pattern reminiscent of OpenAI’s response to a sandbox breach, where the company now monitors entire action chains instead of individual steps, since individual visible actions apparently are not enough.
Why coding-agent benchmarks measure the scaffold more than the model
Naman Vats and Oleg Golev show in The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation that the choice of the surrounding harness – the tool that issues tool calls, manages context, and decides on termination – shapes benchmark results more than previously assumed. They tested the models Qwen 3.6 Plus and MiniMax M2.5 across three open-source harnesses (Goose, OpenCode, OpenHands-SDK) on a stratified 50-task subset of Terminal-Bench Pro. Harness choice alone caused up to a 40-fold difference in tokens per solved task, while paired within-model pass-rate differences remained only 0 to 8 percentage points. Failure patterns replicated harness-typically across both models, which the authors say points to harness-level biases that are largely model-independent. This matters because public leaderboards usually rank systems by model name alone – a problem that connects to the previous digest’s finding on agent-benchmark validity, where most examined agent traces reflected evaluation loopholes rather than genuine capability.
A certified shield against harmful fine-tuning
Domenic Rosati, Ali Dadsetan, Hong Huang, and additional co-authors present in Distribution-Specific Curvature Control a method called HarmAlign: a function-preserving spectral deformation along an estimated contrastive activation subspace, designed to prevent short fine-tuning runs from undoing the safety mechanisms of open-weight models. The authors derive from the method a mathematically certified curvature criterion that – unlike prior methods with an explicit curvature guarantee – does not indiscriminately obstruct benign adaptation as well. In experiments, HarmAlign blocked both direct harmful fine-tuning and three data- or objective-adaptive attacks, across both a hazardous-knowledge relearning setting and a harmful-assistance fine-tuning setting, while protected benign tasks remained trainable, according to the authors. The block reportedly persisted across tested optimizer variants, at every attack checkpoint, and under out-of-distribution harmful fine-tuning. This matters because open-weight models are often just a few training steps away from a safety failure – an arms race also documented from the attack side by OpenAI’s doubled bounty for bio jailbreaks.
Zero tokens, one hundred percent: AI answers from memory instead of generation
Sietse Schelpe presents in A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever a system that pairs a frozen 12-billion-parameter model with a growing memory of verified solutions: once a task family is solved and independently verified, the system answers future instances deterministically and without new generation. Across 180 fresh test instances spanning nine task families and four model architectures, the approach achieved, per the paper, 180 out of 180 correct answers at zero generation tokens; for open-ended reasoning with formal proof verification, a consistency-gated acceptance procedure held for 88 out of 88 cases. Memory retrieval takes 6 to 23 milliseconds according to the author and supports an effective 6-million-token context window on a single GPU. This matters for teams that repeatedly solve the same task types, since it drives the compute cost of already-verified work toward zero – much as the multi-layer memory system AgenticSTS shows how structured remembering cuts token consumption compared with a growing chat log.
A closing note on how to read this: all four works are currently unreviewed preprints; their numbers come from the respective author teams’ own experiments. Independent replications on further models and tasks are still pending for the filler-token findings and the coding-agent harness effect, HarmAlign has so far only been tested within a narrowly defined threat model of first-order attacks, and the frozen-12B approach applies to clearly bounded, verifiable task families – whether the results transfer to open-ended, less cleanly verifiable tasks remains to be seen.
