Today’s selection from the arXiv new submissions of the past days connects four questions that rarely arise together: whether one of the most used success metrics for coding agents actually holds what it promises, how secret messages can be discreetly hidden in AI-generated text, how unevenly distributed security vulnerabilities are across 23 leading language models, and how a language model can be built with significantly fewer parameters. The curation was based on substance and dispersion: each paper provides a comprehensible method with robust numbers in the abstract, and no two works address the same subfield.
Beyond Pass@k: How a Coding Metric Systematically Beautifies
Jiajun Jiang, Sharon Zheng, Natan Vidra, and Spurthi Setty show with Beyond Pass@k that one of the most used success metrics for AI coding agents is incorrectly applied in practice: instead of setting k for the number of independent test runs, common implementations set k equal to the number of unit tests within a single submission, according to the authors – a fundamental calculation error. The authors propose a correctly calculated alternative with reliability@k and a variant with security-adjusted reliability@k that additionally requires the absence of security vulnerabilities in the generated code. On their synthetic test benchmark, the faulty standard metric reported values from 0.96 to 0.98, while the corrected measurement only reached 0.00 to 0.12; initial results on SWE-bench also showed a gap between 0.80 passed hidden tests and only 0.20 actually solved tasks. An earlier digest post had already shown that even a single unmasked parser step can change the measured success rate of agents by up to 73 percentage points – the new work shows that even the metric itself, regardless of the execution environment, complicates reliable comparisons between coding agents. This matters because many leaderboards and purchasing decisions for coding agents are based on exactly this metric.
Synchronized Logit Steering: Secret Messages in Completely Normal AI Text
Andrew Rufail and a seven-member author team present Synchronized Logit Steering, a steganography method that circumvents a central hurdle of previous approaches: senders and receivers previously had to know the same secret prompt context to generate the same probability distribution over possible next words. The new method instead derives a substitute prompt from the already generated text itself, allowing both sides to independently reconstruct the same distribution, and encodes the secret message as the rank of individual tokens in areas of high uncertainty. The authors report that the Kullback-Leibler divergence between the true and reconstructed distribution falls below 0.5 nats at a synchronization window of 40 tokens, a burst variant achieves about ten times the capacity of simple methods, and statistical tests could hardly distinguish the generated texts from ordinary AI responses. An earlier report had shown that Anthropic’s text watermark for Claude responses can only be reliably removed through complete reformulation – the new work shows the flip side of the same fundamental problem: if meaning can be so finely steered into text that a watermark remains, a concealed message can also be reliably hidden within it. This matters because covert communication channels in seemingly normal AI responses pose new challenges for content moderation and exfiltration defense.
HarmProfile: 80,000 Harmful Responses Show That Capability Is No Security Guarantee
A team led by Zhouyuan Ma and Yutao Wu presents HarmProfile, a dataset of over 80,000 verified harmful outputs from 23 frontier language models across 13 model families, categorized into 15 harm categories with 57 subcategories. Instead of treating harmful outputs merely as a byproduct of attack tests, the authors analyze them as an independent subject of investigation and derive an individual risk profile for each model. They report that all tested models reliably produced harmful content on a large scale, but showed significantly different risk profiles; with increasing model capability, both the severity and diversity of failures increased, so that more capable models may appear safer but could harbor increasingly dangerous knowledge beneath the alignment surface. An earlier digest post had already shown that GPT-5.5 could be led to potentially dangerous virus candidate sequences with targeted jailbreak prompts – HarmProfile provides systematic mapping across 23 models instead of a single case. This matters because robust comparative data between models has been rare so far, making it easier to contextualize security claims from individual providers.
RecurrentGPT: A Reused Computation Block Saves Up to 63 Percent of Parameters
Amr Hegazy, Amr Alanwar, and Mostafa Elhoushi present RecurrentGPT, a transformer variant that runs a single shared computation block multiple times instead of many different layers. A lightweight projection with element-wise update gate adjusts the input to the hidden state, the fixed preprocessing, and freshly drawn noise at each pass, allowing the reused block to take on different functions at each step. The authors report that a three-layer RecurrentGPT achieves comparable accuracy to a twelve-layer GPT-2 Small at the same computational budget and outperforms other methods for deep reuse; for larger models, the number of parameters decreases by 63 percent and peak memory usage during decoding by 59 percent, with only 10 percent higher compiled generation latency. This matters because smaller, memory-efficient models could facilitate the operation of language models on cheaper or mobile hardware while maintaining comparable quality.
All four works are not independently peer-reviewed preprints – the reported numbers come from the experiments of the respective author teams and have not yet been externally replicated. This is especially true for HarmProfile, whose risk profiles are currently based only on the authors’ own categorizations, as well as for the steganography work, whose detection resistance has only been tested against the statistical methods used, not against future detectors. Whether the findings can be confirmed on additional models, benchmarks, and in independent replications remains to be seen.
