Today’s selection from the arXiv new submissions of the past 24 to 48 hours connects five questions that rarely appear together: how easily sensitive data can leak through harmless AI responses, how reliably agents detect silent tool errors, whether the training signals commonly used to shape agent decisions actually measure the right thing, how AI can be certified for safety-critical aviation systems for the first time, and whether creative AI responses are becoming more uniform over time. Curation followed substance and spread: each paper offers a traceable method with solid numbers in the abstract, and no two works cover the same subfield.
Silent context leakage: secrets seep through harmless answers
Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg, and Saeed Mahloujifar investigate with their study on inadvertent context leakage whether sensitive data in a language model’s context window – such as calendar entries, credentials, or health details – leaves hidden patterns in its otherwise harmless outputs, even when the model correctly refuses a direct request for that data. In controlled experiments across eight commercial models, the authors report that two-digit secrets could be reconstructed with near-perfect accuracy, and four-digit secrets still at an 82 percent exact-match rate – purely from responses to ordinary, non-adversarial requests. More capable models leaked more, they find, because stronger instruction-following heightens sensitivity to in-context secrets; building on this, the authors demonstrate both a classifier that infers health or financial details from routine outputs and a reinforcement-learning-trained attacker that extracts complete Social Security numbers from a production-style agent. An earlier piece had already shown that encrypted reasoning traces from major providers can be read in plaintext via weaker models – the new study shows that even ordinary, visible outputs can serve as a covert channel for sensitive context data. This matters because AI agents increasingly handle real credentials and personal information whose disclosure is hard to prevent with classic output filters.
Outcome Monitors: guarding agents against silent tool failures
Sugam Panthi and Rabab Abdelfattah present Outcome Monitors, a mechanism that protects AI agents from silent tool failures – such as a cached error page or a negative price that arrive in the expected format and get treated as fact instead of being flagged as errors. The monitor checks tool responses against “outcome contracts” mined from task-disjoint traces or derived from public schemas, and on a violation preserves the result while issuing a receipt naming the violated property and pointing to recovery tools. In frozen, pre-specified tests with injected failures, the mechanism raised completion on the ToolMaze benchmark, the authors report, from 10.9 to 28.1 percent across four models from two provider families, replicating with a third, and improved tau-bench retail tasks by 14.0 and 12.0 percentage points on two tiers; control experiments showed the recovery-tool suggestions actually drove the gain, while detection outside the mined error vocabulary fell to 46 percent coverage. An earlier digest piece had already shown that stale but plausible-looking conversation history flips otherwise-correct tool decisions in nearly a third of cases – Outcome Monitors target a related weak spot by checking the concrete tool result itself rather than the history. This matters because agents increasingly depend on external data sources whose failures can look almost identical to correct results.
Deceptive training signals: what agent training credits for success is often wrong
Haiyue Zhang examines with a study on credit assignment in LLM agents – the attribution of which individual steps in a multi-step agent task actually contributed to success or failure – whether common training signals such as LLM-judge scores, outcome-conditioned logprob ratios, or a policy’s own confidence actually identify the causally decisive steps. As ground truth, the paper uses “executed replay” in the ALFWorld tool environment: at each decision point, the policy’s own alternative actions are resampled and rolled forward to measure what actually changes about the outcome. None of the examined signals identifies these causally important steps better than chance, the study finds; causal contribution is itself sparse (only 30.5 percent of decision points with defined ground truth show a measurable effect) and how often it is even measurable varies by model. Instead, implicit credit mostly echoes how fluently the policy writes (rank correlation +0.75), while conditioning on the actual outcome adds essentially no causal information (partial correlation −0.004); in a seven-arm pre-registered training experiment, no arm reliably beat the untrained policy. An earlier digest piece had already shown that a simple “always safe” baseline beats several widely used agent-safety benchmarks – the new work shows that the training signals meant to teach agents in the first place often fail to measure the right thing either. This matters because many agent training procedures build directly on such credit signals without ever testing their actual causal validity.
Safety Nets: certifiable AI for airborne collision warning, for the first time
Johann Maximilian Christensen, Thomas Stefani, Elena Hoemann, Frank Köster, and Sven Hallerbach investigate with their study on Safety Nets how neural networks can be certified for safety-critical aviation systems the way the European Union Aviation Safety Agency (EASA) requires for AI-based systems: a compressed neural network is paired with a lookup table that corrects remaining errors, guaranteeing 100 percent correct outputs across the entire discretized operational domain. The authors say they provide the first systematic study of the trade-off between network size and table size, testing it on collision-avoidance systems for manned and unmanned aircraft (HCAS and VCAS). They identify architectures with three to five hidden layers of roughly 50 to 100 nodes each, combined with one-hot encoding, as the best balance, where the neural network alone correctly covers at least 97 percent of cases and a compact table handles the rest. The resulting system is nearly three orders of magnitude smaller than a pure lookup table, fits the memory budget of current avionics hardware, and still meets the full EASA certification requirement; the authors also release what they describe as the first open-source Safety Nets implementation for HCAS and VCAS. This matters because it shows a concrete technical path for certifying learning-based systems for manned aviation despite formal 100-percent-correctness requirements.
Creative AI outputs are converging over the years
Nirav Patel, Josiah Crossman, Eva Aggarwal, and Emily Wenger investigate with their analysis of LLM creative diversity how different AI models’ responses to open-ended, creative tasks have evolved over three years of model releases – tasks where originality and diversity matter as much as raw quality. They draw on real-world open-ended queries from the Infinity-Chat100 collection and the Alternate Uses Task, an established psychometric creativity assessment, measuring trends via sentence-embedding similarity across model responses. The authors report a statistically significant decrease in output diversity over time, suggesting that creative outputs across different models are increasingly converging in substance. If the trend persists, they write, this homogenization could progressively diminish human agency in human-AI co-creative work. The study explicitly frames itself as a preliminary analysis and does not address causes of the homogenization or possible countermeasures; it still matters because AI systems are already widely used for ideation and creative groundwork today.
All five works are non-peer-reviewed preprints – the reported numbers come from the respective author teams’ own experiments and have not yet been externally replicated. This applies especially to the context-leakage study, whose attacks have so far only been shown on eight commercial models under controlled lab conditions, and to the credit-assignment work, which relies on a single tool environment (ALFWorld) and predominantly two model families. The creativity study, too, is explicitly described by its authors as preliminary. Whether these findings hold up across further models, environments, and independent replications remains to be seen.
