AI Paper Digest

Friday, September 4, 2026

357 papers cs.AI · cs.LG · cs.CL ← 2026-09-032026-09-07 →

Jul Aug Sep

Highlights

PACE: Towards Surfacing Hidden Conflicts in User Requests

Highlight HF pick · 6▲Agents Yoojin Kim, Jihyoung Jang, Hyounghun Kim Personal assistants are usually evaluated on how faithfully they execute a request, not on whether the request makes sense given facts about the user that are never stated in the prompt. PACE (Personalized Assistants for Conflict Evaluation) pairs persona-grounded user requests with egocentric knowledge-base facts, so a model must retrieve latent constraints — a health condition, a past event — that make an otherwise reasonable request inappropriate, and the implicit nature of the link defeats direct request-to-fact matching. The companion PaceMaker framework splits the job across specialized agents for query reformulation, multi-hop graph traversal, and conflict-aware filtering, and outperforms existing approaches on both evidence retrieval quality and conflict decision accuracy.

Personalized assistants are typically evaluated on how faithfully they execute requests, not on whether a request is advisable given the user's circumstances — and in practice the disqualifying facts are implicit, scattered across a personal knowledge base rather than stated in the query. PACE is a retrieval-grounded benchmark where each request looks perfectly executable in isolation but conflicts with latent constraints hidden in an egocentric knowledge base, paired with PaceMaker, a training-free multi-agent retrieval framework built to surface that evidence.

  • PACE contains 3,249 queries across 185 synthetic persona instances and 376,448 atomic facts (roughly 2,035 facts per instance, with only 4.01 gold facts per query), split across temporal, personal, and state conflict types and deliberately balanced with 1,608 non-conflict cases so models cannot win by refusing everything.
  • PaceMaker runs four stages over the knowledge base — a planner that proposes up to three conflict dimensions, a multi-view query generator producing "counter views" aimed at potentially conflicting conditions, hybrid dense/BM25 retrieval fused by weighted reciprocal rank fusion with counter views upweighted, breadth-first traversal over a k-NN document graph, and a post-hop filter that keeps the ten most decisive facts.
  • Across three configurations it beats every retrieval baseline on both retrieval and decision quality: with text-embedding-3-small/GPT-5.4-mini it reaches Recall@5 of 36.05% versus 19.77% for sparse retrieval and a 75.35% Pass rate, and it improves conflict-query Pass by 11.40, 3.47, and 4.02 points over the strongest non-oracle baseline in the Qwen, GPT, and Gemini settings respectively.
  • Stuffing the full knowledge base into context is not a substitute for retrieval — Full KB scores 57.49% Pass in the open-source setting against an Oracle upper bound of 86.89% — and ablations show multi-hop traversal is the single most important component, its removal dropping conflict Pass from 59.17% to 52.41%.
  • The task remains far from solved: Gold@10 tops out at 12.55%, meaning the complete evidence set is almost never recovered, conflict queries lag non-conflict ones even under Oracle conditions, and the benchmark is fully synthetic (generated and filtered by GPT-5.4-mini, with human agreement of 93.3% on labels) while evaluating only feasibility judgment rather than downstream task execution.

arXiv:2609.03293 PDF

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Highlight HF pick · 51▲Large Language Models Heng Wang, Jielin Qiu, Wenting Zhao, Cheng Qian, Liangwei Yang, Jiawei Han et al. Long chains of thought make the key-value cache a severe memory bottleneck, and every existing compression method follows the same recipe: score each cached token by how much it will matter later, then keep the top scorers. Random Attention computes no score at all — it preserves the prompt and evicts uniformly at random within each attention head — and across four models and six reasoning tasks it matches the strongest prior evictor while serving 32–43% higher throughput in vLLM deployment. Controlled experiments explain why: the prompt is the fragile part of the cache and most of the gap between selectors is just whether their signal happened to protect it, while the reasoning trace protects itself through redundancy both in text (the model restates what it still needs) and across heads (each keeps its own copy), so once the prompt is safe a random draw retains enough.

Long chains of thought make the KV cache the dominant memory bottleneck, and every existing eviction method answers this by scoring cached tokens and keeping the top-ranked ones. Random Attention shows that the scoring signal is nearly worthless: pinning the prompt and evicting uniformly at random within each attention head matches the strongest scored evictor across four models and six reasoning tasks, while serving substantially faster because it never runs a scoring pass.

  • The entire method is two rules — give every prefill position an infinite score so the question is never evicted, then draw an i.i.d. uniform score for each remaining position independently per KV head and keep the top K — costing one rand and one topk per eviction round.
  • At roughly 4× compression it is statistically ahead of baselines in 31 of 60 comparison cells and behind in only one, scoring 0.874 on MATH500 and 0.610 on AIME with Qwen3-4B against TriAttention's 0.864 and 0.592, with the gap to scored methods widening as compression tightens from 2× to 16×.
  • Two controlled experiments explain the result: the prompt is the fragile part of the cache, so forcing every baseline to keep it erases most inter-method differences (SnapKV gains up to +35.2 points on LiveCodeBench, while prompt-retaining R-KV gains under 2), and the reasoning trace is redundant both in text and across heads — a planted fact held by one head is retrieved 3% of the time but 83% with three heads and 99% with all eight.
  • Skipping the scoring pass yields 32–43% higher throughput than TriAttention under vLLM at 32k-token generations on an H200, and 1.6–2.7× full attention, because vLLM compresses some request at nearly every decoding step and each compression stalls the whole batch (about 15 ms per compression for TriAttention versus under a millisecond).
  • The failure mode is real but narrow: for a passcode stated once and never restated 57 eviction rounds before the question, Random Attention retrieves it 0% of the time against R-KV's 84%, and code reasoning remains its weakest task since LiveCodeBench prompts average 557 tokens and pinning them whole can consume half the budget, costing about three points to TriAttention on Qwen3-32B.

arXiv:2609.03430 PDF

LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes

Highlight HF pick · 78▲Vision Chuyan Chen, Haoxing Chen, Kun Chen, Zhenglin Cheng, Long Cui, Ruishan Fang et al. LLaDA-Image pairs a 6-billion-parameter Diffusion Transformer (DiT) trained from scratch with a frozen vision-language understanding module built on the LLaDA2.0-Mini diffusion language model, deliberately establishing a visual generative prior through image-only pre-training and mid-training before leaning on paired image-text data. The 220-million-sample pipeline uses parameter-free RMSNorm throughout the transformer and the Muon optimizer, and the model is distilled into LLaDA-Image-Turbo for inference in 2-4 sampling steps. On Qwen-Image-Bench it scores 53.53 on the English track and 53.38 on the Chinese track, the best reported among open-source models on both, with weights, training code, and recipes released.

Training a competitive image generator from scratch normally spends its largest compute budget on paired image–text data, yet captions are expensive, lossy, and poorly matched to the aggressive downsampling used at low training resolutions. The approach here decouples the two: a 6B single-stream diffusion transformer first learns a visual prior from images alone — a frozen diffusion-LLM vision-language model derives the conditioning signal from the same crop the generator must reconstruct — and only afterward gets aligned to text, with weights, code, and stage-by-stage recipes released.

  • Because the condition is derived from the image rather than a caption, crops need only mild downsampling to 256×256 (then 512×512 with aspect-ratio buckets in mid-training), and randomly masking the SigLIP-VQ reference tokens turns reconstruction into a sparse-to-dense prediction task instead of a trivial identity mapping.
  • Of the 220M generation-training samples, 98% are real images and over 90% are image-only, with the real share staying above 70% through supervised fine-tuning; parameter-free RMSNorm throughout the DiT plus the Muon optimizer are credited with keeping the long run stable.
  • Editing lives in the same checkpoint as text-to-image — the reference image bypasses the VLM entirely and enters the DiT through a SigLIP-VQ semantic branch alongside a clean FLUX.2 VAE latent concatenated to the noised target — while T2I and I2I examples are mixed 1:1 so editing training does not erode open-ended generation.
  • On Qwen-Image-Bench the model scores 53.53 (English) and 53.38 (Chinese), open-source state of the art by 1.87 and 0.67 points over Z-Image Turbo, though it trails GPT-Image 2 at 65.23/64.69 and its weakest dimension is real-world fidelity (43.90 EN, below several open baselines such as Qwen-Image 2512 at 47.80).
  • TwinFlow distillation yields a 2–4 step Turbo variant that gives up roughly 2.5 overall points (50.98 EN), falling behind several open-source multi-step models, and LongText-Bench scores of 0.923/0.913 show balanced bilingual rendering but remain under dedicated text-rendering systems.

arXiv:2609.03796 PDF

Editable Visual Design

Highlight HF pick · 21▲Agents Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li, Zhutao Lv et al. Image generation models like GPT-Image-2 and Nano-Banana produce visually striking designs but flatten everything into a bitmap with error-prone text and no editable layers, while coding agents that emit layout code preserve layers and real text yet lack global aesthetic intuition and struggle to draw complex visual assets. The proposed paradigm splits the roles: a vision-language model serves as the creative brain for requirement comprehension, planning, and aesthetic judgement, calling an image model on demand as a 'visual world simulator' to synthesize standalone assets, then writing native HTML and CSS and iteratively refining it against rendered visual feedback in an imagine-first-then-act loop. The output is a genuinely editable artifact with decoupled layers and real text that users can drag and rearrange in a graphical interface, plus an Agent Design Replay mode that reproduces the creative and reasoning trajectory; validation covers posters, infographics, and similar scenarios.

Diffusion image models produce aesthetically strong but flattened bitmaps with garbled text, while code-generating agents produce clean, layered HTML but lack visual taste and cannot draw complex assets — Editable Visual Design splits the job between them, using a VLM as the planning "creative brain" and an image model as an on-demand "visual world simulator." The agent imagines the finished piece as a picture first, then rebuilds it as native HTML/CSS with separately generated assets and real text.

  • The pipeline runs five stages — planning, visual simulation, structural coding, verification, and delivery — with GPT-5.6 Sol driving Codex as the creative brain and GPT Image 2 producing both the imagined reference visual and the standalone assets.
  • Assets are generated layer by layer rather than cut out of the imagined visual, requested with an alpha channel where supported or matted off a flat green background otherwise, so no reference pixels reach the deliverable and layers never entangle.
  • Self-healing combines deterministic checks in a headless browser (overflow, failed resources, malformed DOM) with a VLM reviewer judging a rendered screenshot, converging after one or two rounds of targeted local patches.
  • Agent Design Replay serializes the full trajectory from intent to repair; the reported cases scale layer structure to brief density, from a 120-layer, 13-group infographic that needed real layout repairs down to a 6-layer, single-group travel poster that passed review unchanged.
  • The authors report cases rather than scores — there is no ground truth for aesthetics and the visual reviewer is a VLM standing in for a designer's eye — and all examples are single-page, with multi-page consistency, weak image-model composition, and unmeasurable editability named as open limits.

arXiv:2609.04034 PDF

CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation

Highlight HF pick · 10▲Multimodal Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Chu Liu, Pengjun Xie et al. Embedding models built on multimodal large language models (MLLMs) struggle to separate scenes that contain the same concepts but bind attributes to objects differently, even though the same backbone can make the distinction when used as a cross-attentive reranker. CORE distills the reranker's judgments into the embedding model by synthesizing candidate lists spanning five levels of compositional match and training with a listwise Rank-KL objective. Under a matched data and tuning budget, both Rank-KL and pairwise CoSENT exploit that multi-level supervision better than contrastive learning, with Rank-KL strongest overall. CORE-RERANKER-8B averages 82.7% across COLA, SUGARCREPE++, and NEGBENCH, 10.7 points above Jina-Reranker, and the gains transfer to MCMR without hurting retrieval on COCO and Flickr30K.

MLLM-based embedding models still fail at compositional retrieval, confusing scenes that share concepts but differ in attribute–object binding ("a white plate and a black chair" versus "a black plate and a white chair"), even though the same backbone used as a cross-attentive reranker resolves these cases correctly. CORE closes that gap by synthesizing graded candidate lists across five compositional matching levels and distilling the reranker's fine-grained ranking into the embedding space with a listwise KL objective.

  • The synthesis pipeline seeds from LAION-400M, uses Qwen3-VL-32B to extract a structured scene representation and write five level-specific captions (full match, partial presence, attribute error, object error, full mismatch), renders them with Z-Image-Turbo, and discards any list where a candidate fails automated caption–image or query–image verification — yielding 92,211 tuples after removing 22.1%, with 94% of 50 human-checked tuples fully correct.
  • Rank-KL trains the student dual encoder to match the teacher reranker's softmax distribution over the whole candidate list rather than pushing all negatives away uniformly as InfoNCE does, and it is the only objective that beats the backbone on the compositional macro-average (0.641 versus 0.604 for VL-Emb-2B) while also topping the graded dev set at 0.850 NDCG@10.
  • CORE-Reranker-8B reaches an 82.7% total average across COLA, SugarCrepe++, and NegBench, 10.7 points above Jina-Reranker, and notably recovers negation sensitivity (0.698 on NegBench) that conventional reranker fine-tuning destroys — Qwen3VL-Reranker-8B collapses to 0.261 from its base model's 0.739.
  • CORE-Embed-8B posts the best embedding total average at 0.666 (up 5.7 points from VL-Emb-8B), transfers to the independent MCMR multi-condition benchmark with R@1 rising from 0.375 to 0.412, and preserves general retrieval, taking best R@5/R@10 in both directions on COCO and Flickr30K.
  • The authors are candid that embedding gains lag reranker gains, that COLA barely moves for the embedding model, that the graded dev set shares a synthesis pipeline with training data and so is an in-distribution diagnostic only, and that Rank-KL's edge over pairwise CoSENT is borderline (+0.007, 95% CI [+0.000, +0.013]).

arXiv:2609.04083 PDF

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

Highlight HF pick · 53▲Large Language Models Sergii Kozyrev, Davyd Maiboroda Hybrid language models interleave softmax attention with linear-attention layers such as Gated DeltaNet (GDN), and early 4-bit quantizations of Qwen3.8-27B left the GDN blocks — especially their decay and write-strength gates — at 8 or 16 bits, on the assumption that errors inside a recurrence accumulate over long contexts. Minima applies NVFP4 W4A4 to all 496 linear layers, GDN included, and matches BF16 within seed noise across perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K at 17.5 GiB with 14-19% faster prefill. A four-part mechanism study credits NVFP4's 16-element block scaling for localizing outliers in the residual stream, finds the supposedly fragile gate projections least sensitive (roughly 11% GEMM error compresses to about 2% output error), and shows the delta-rule recurrence holds injected noise at a flat plateau because each write overwrites state along the current key direction. The work also repairs a global-scale mismatch that appears when per-module-calibrated checkpoints are served by kernels that fuse those modules.

Hybrid LLMs interleave softmax attention with linear-attention layers like Gated DeltaNet, and every public 4-bit build of Qwen3.8-27B has kept the recurrent block — especially its decay and write-strength gate projections — at 8 or 16 bits, on the assumption that per-step rounding error compounds across a long context. Minima tests that assumption by quantizing all 496 linear layers to NVFP4 W4A4, gates included, and finds the intuition is backwards: the recurrent half is the easy half to quantize.

  • Against BF16 and the two public NVFP4 recipes (Unsloth, RadixArk) under one fixed serving regime, Minima lands within seed noise on MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, and LiveCodeBench — a 5-task average of 85.10 vs. BF16's 85.62 (−0.52), matching BF16's AIME score exactly at 26/30 on all four seeds — while being the smallest checkpoint at 17.5 GiB (2.9× under BF16) and the fastest at prefill (32K TTFT 6.90 s → 4.03 s, +14–19% prompt throughput over the community builds whose GDN GEMMs stay FP8).
  • The mechanism study replays captured 32K-token activations one projection at a time and inverts the community's precision map: the gate projections a and b that every public recipe protects are the least sensitive, turning ~11% and 8.5% GEMM error into only 2.1% and 2.6% output error, because the log-space softplus/exponential parameterization squashes noise before it reaches the decay horizon — the real error comes from the plain out (12.7%), qkv (10.4%), and z (9.9%) projections.
  • Running the recurrence in lockstep FP32 shows the state error plateaus flat at ~12.6% from token 256 through 32,768 rather than accumulating, and a one-off 1% state impulse decays to 1/e within 80–1,382 steps against decay-implied horizons of 44K–62K tokens — the delta rule overwrites the state along each new key direction, so old error is erased rather than merely forgotten.
  • End to end the picture holds: the weight-quantization perplexity gap falls from +0.081 nats in the first half of the 32K window to +0.011 in the second, going negative (−0.053) in the final 2K tokens, while the oppositely-behaved FP8 KV-cache cost (+0.41 PPL at 32K for Minima, 3× BF16's) is 83% recovered by calibrated per-layer scales that cost nothing at serving time.
  • Getting there required repairing a serving bug that silently corrupts any GDN-quantizing checkpoint — llm-compressor calibrates one global scale per module while vLLM fuses qkv+z and b+a into single GEMMs and takes the max, mis-scaling the gates in all 48 layers (AIME 80.8 corrupted vs. 86.7 repaired, with deceptively better 32K perplexity) — and the evidence is bounded to one model, one format, 32K perplexity and 64K retrieval, with the gate-shielding argument explicitly not transferring to recurrent mixers whose decay is linearly parameterized.

arXiv:2609.04098 PDF

Environment Evolution for Terminal Agents

Highlight HF pick · 4▲Agents Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu et al. Training terminal agents requires interactive, verifiable environments, but environments synthesized from scratch stop challenging frontier models, and co-evolution methods that build them from weaknesses seen in on-policy rollouts generalize poorly and run out of signal as the model improves. The alternative here raises environment difficulty off-policy along three directions derived from the multi-turn learning objective, scheduling successive generations of evolved environments through a loop-engineered multi-agent harness. Rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol confirm each generation is harder, and simple long-horizon reinforcement learning on the resulting environments improves Qwen3.6-27B and Qwen3.6-35B-A3B by 14.4 and 18.0 percentage points on Terminal-Bench 2.1.

Terminal-agent RL is bottlenecked by environments: pipelines that synthesize tasks from scratch produce environments frontier models solve on every rollout, so the resulting all-success groups carry no gradient signal, while co-evolution methods that patch this by mining on-policy failures stay tethered to one rollout model and dry up as training saturates. The proposal is environment evolution — deriving an off-policy, model-agnostic notion of environment difficulty from the multi-turn learning objective, then mutating a seed environment generation by generation along the three axes that objective exposes.

  • Decomposing the negative log-likelihood of a high-level trajectory (interleaved scenarios and skills) yields exactly three difficulty contributors — solver-turn count L, scenario novelty, and skill rarity under a reference world-knowledge distribution — and swapping the policy-dependent probabilities for that reference distribution separates environment difficulty from model weakness, which the authors define as the non-negative excess of the former over the latter.
  • Each generation runs through a loop-engineered multi-agent harness with two gated loops: a Proposer extracts the scenario–skill execution sequence and edits it along one randomly ordered direction (with cross-mode fallback when a direction fails) under a low/high/max evolution effort knob controlling edit scope, then a Modifier applies the residual and must clear an Oracle-solution check, an empty-solution check that catches broken verifiers, and a rubric quality check.
  • Rollout difficulty estimates with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol show high and max effort drive pass rate monotonically to zero over 15 generations while average turns rise, and per-direction ablations agree between one-step and 15-step-mean measurements — length gives the biggest pass-rate drop (−7.1 pp in one step) with the smallest total mutation, while scenario and skill add more turns (+13.5 and +12.5).
  • Under 200-step GRPO with a Claude Code harness at 256K context, an Evolution-Lineage scheduler (advance only when 8-rollout pass rate exceeds 6/8) keeps more rollout groups partially solved than random sampling, lifting Terminal-Bench 2.1 Verified by 14.4 pp on Qwen3.6-27B and 18.0 pp on Qwen3.6-35B-A3B to peaks of 71.5% and 64.9%, against 62.9%/55.1% for co-evolution and 60.0%/52.8% for environment ensembling.
  • The honest caveats are that difficulty is only ever observed through pass rate, which forces a 15-generation cap once lineages hit the zero-pass regime and leaves the claimed reference distribution estimated by a deep-research agent rather than computed; low effort fails to increase difficulty monotonically because repeated local edits can revert earlier ones; the harness needed human-in-the-loop review to grow its rubric set; the 500-environment seed pool survives brutal filtering (127 of 47,678 collected environments, topped up with SkillSynth); and the self-improvement setting where one model both builds and learns from its environments is deferred to a future version.

arXiv:2609.04128 PDF

Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments

Highlight HF pick · 30▲Agents Jie Wu, Zhenru Zhang, Beichen Zhang, Xuwu Wang, Yuhui Su, Mouxiang Chen et al. Terminal-based coding agents have produced large archives of trajectories, but each trajectory is a single frozen demonstration, whereas post-training needs executable environments that can be re-queried into many verifiable tasks and give execution feedback. Terminal-Universe reconstructs environments from the trajectories themselves: replaying the recorded file operations restores each file to its pre-modification state, producing a partial workspace that a completion agent fills in with missing files and dependencies. Tasks are then synthesized on that workspace in breadth, by mining directional dependency relations between environments to form cross-workspace queries, and in depth, by extending single-turn queries into multi-round sessions driven by a user agent. Applied to public trajectories it yields 37.3k task-sufficient environments, and supervised fine-tuning of Qwen3.5-27B on them adds 11.9 points on Terminal-Bench 2.1 and 13.8 points on EvoCode-Bench v2 MT@4.

Agent post-training needs executable environments, but realistic ones are scarce while recorded agent trajectories are abundant — and a trajectory's tool-execution history already exposes the file contents and structure of the workspace it ran in. Terminal-Universe inverts the usual mapping by reconstructing an environment from each trajectory, then re-querying that environment for many new verifiable tasks.

  • Reconstruction runs in two stages: deterministic replay restores each touched file to its earliest observed version (withholding the agent's own edits so the task starts unsolved), then a completion agent supplies missing files and dependencies, raising mean workspace size from 2.9 to 22.4 files and task-sufficiency from 40.2% to 93.5% on terminal workspaces (20.1% to 77.1% on SWE ones).
  • Each recovered workspace is re-queried four ways — intent recovery, single-workspace synthesis, cross-workspace tasks that pair a writable target with a read-only reference repo, and multi-round sessions driven by a user agent with a requirement tracker — with every task paired to an agent-authored pytest verifier and only fully passing trajectories kept.
  • Applied to public trajectory corpora, the pipeline yields 37.3k task-sufficient environments and 32.0k SFT demonstrations (~1.42B tokens), and fine-tuning Qwen3.5-27B lifts Terminal-Bench 2.1 by +11.9 points (46.2 → 58.1) and EvoCode-Bench v2 MT@4 by +13.8 points (6.3 → 20.1).
  • Ablations isolate what matters: re-solving recovered tasks with a stronger teacher averages 52.1 on Terminal-Bench 2.1 versus 36.7 for imitating the raw source trajectories (which lands below the 47.0 base model), and under a matched 35k-record budget, doubling the number of environments (56.0) beats doubling queries or solutions per environment (53.8 and 53.9).
  • Cross-workspace tasks are genuinely harder rather than just more data — teacher pass@1 drops from 72.3% to 49.2% with 1.9× the tool calls — but the whole pipeline inherits three constraints: a generic ubuntu:24.04 container for every workspace reduces fidelity for tasks needing specialized system dependencies, coverage is bounded by the source trajectory distribution (84.7% Python), and a single teacher (Qwen3.7-Max) writes the tasks, solutions, and verifiers, so its blind spots can go undetected.

arXiv:2609.04148 PDF

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

Highlight HF pick · 44▲Large Language Models Zixuan Fu, Bingxiang He, Yuxin Zuo, Haohuan Huang, Jinqian Zhang, Ruhang Xiao et al. On-policy distillation (OPD) trains a student on its own rollouts using dense token-level supervision from a teacher, but the role of the training data itself has gone largely unexamined. Training on a single query keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task domains and model families: one query already reaches 71.5% of the states that full-data training visits, most of it within the first 100 steps, and 16 semantically distinct queries reach 98.9% and match full-data results, including in the multi-teacher setting. Because the student aligns with the teacher at a similar slow rate whether trained on one query or the whole dataset, the authors describe OPD as data-overfed but algorithm-starved; content-light templates and off-domain WildChat queries also approach the real-query baseline, showing task content and induced state coverage can come apart.

On-policy distillation (OPD) pairs student-generated rollouts with dense token-level teacher supervision, but its data requirements have gone unexamined; training at the data-minimal limit — a single query — shows the student keeps improving for hundreds of steps and recovers most of full-data OPD's gain. The explanation is that a query matters only through the reasoning states its rollouts visit, so OPD is "data-overfed but algorithm-starved": supervision is abundant and cheap to reach, while the rate at which the student absorbs it is what actually gates training.

  • Since every token position of every rollout is a supervised state, one query with 64 rollouts per update yields tens of thousands of training states, and one-shot OPD on DAPO-Math-17K reaches 68.5 average accuracy against 69.8 for full-data OPD at step 300 — 87% of full-data's gain and 69% of the teacher–student gap — with the effect holding across DeepSeek-R1-Distill-Qwen-1.5B, Llama-3.2-3B-Instruct, and OLMo-3-7B-Instruct-DPO, and recovering 73%, 66%, and 64% of the gap on code generation, instruction following, and agentic tool use.
  • The authors quantify this with state coverage, clustering teacher hidden-state signatures from full-data rollouts into 200 K-means clusters and reporting the fraction a run reaches: a single query hits 71.5% (65.9% of it within the first 100 steps), while 16 semantically distinct queries selected by BGE-M3 clustering reach 98.9% and match full-data training, and the same 16-per-domain recipe recovers 101% of full-data multi-teacher OPD's gain.
  • On the algorithm side, the absorption rate — the share of the remaining teacher–student log-probability gap closed per update — decays throughout training at nearly the same pace whether OPD trains on 1, 4, 16, or all 17k queries (each run removes 78–84% of its step-30 distance by step 300), and an always-off-policy run on 64 frozen trajectories still improves for roughly 200 steps, so run length is a property of the optimizer rather than of a fresh supply of states.
  • Pushing the data side to its limit, content-free <think> templates and off-domain WildChat chat queries (only 0.17% math-labelled) drive math OPD to nearly the same 59.1 → 69.8 three-benchmark improvement as real problems, and against one-shot RLVR on the same query OPD delivers more than twice the validation gain over 1000 steps because GRPO's outcome signal vanishes once rollouts agree while the token-level gap persists.
  • The main caveats are that state coverage is a semantic proxy defined relative to a full-data reference run — it cannot be estimated from queries alone, and it weights every cluster equally regardless of visit frequency or remaining teacher signal — while what actually determines the absorption rate is left open, the MOPD result covers only three domains with one teacher each, and all students are 1.5B–7B.

arXiv:2609.04172 PDF

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Highlight HF pick · 20▲Large Language Models Yuntian Deng, Pengyu Nie, Stuart Shieber Recurring text-transformation tasks are often easy to describe in words but awkward to implement with rules, while calling a large remote model on every input adds recurring cost, latency, and provider dependency. Compile by training turns a natural-language specification into a reusable neural function: at compile time, teacher models generate task-specific examples that train a small adapter over a compact interpreter, after which the function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, where the Program-as-Weights fast compiler produced no exact matches, the approach reaches 83.6% semantic accuracy, at a compile cost of about a minute rather than seconds; the authors deploy it as a public interactive service with demos including a multi-site website helper and a language-controlled 3D avatar.

Recurring text tasks that are easy to describe but awkward to encode as rules — email triage, ID extraction from links — currently force a choice between brittle handwritten code and a remote LLM call on every input. Compile by training spends large-model compute once at build time, turning a natural-language specification into a small local neural function that runs afterwards without any teacher calls.

  • At compile time, teacher models (GPT-5.4-mini and GPT-5.5 in a 2:1 mix) synthesize validated input/output pairs from the specification, which train a rank-64 LoRA adapter over a frozen quantized Qwen3-0.6B interpreter, warm-started from the Program-as-Weights amortized compiler's single-forward-pass prediction and packaged as a versionable .paw artifact alongside its run-time prompt scaffold.
  • On FuzzyBench-Hard — the specifications where the PAW fast compiler produced zero exact matches — semantic accuracy measured by an LLM judge rises from 0.224 to 0.836, a +0.612 absolute gain bought with 50.9 seconds of compile time instead of 3.5; the GPT-5.5 grader agrees with author labels at 0.977 accuracy and Cohen's κ of 0.946 over 128 items.
  • Supervision quality moves the needle more than supervision quantity: adding a stronger second teacher lifts mean LEM from 0.746 to 0.851, while a 5× increase in unique training pairs (1440 to 7200) only moves it from 0.821 to 0.866.
  • To keep a minute-scale build interactive, the service streams teacher synthesis concurrently with training rather than waiting for the full dataset, yielding cold compiles of 50.9 s on a B300, 68.2 s on an H200, and 99.2 s on an RTX GPU, with four concurrent jobs clearing at a mean queue wait of 1.01 s.
  • Three deployments show compiled functions composing with ordinary code — a four-site helper routing through 28 live programs, a 3D avatar controller that produced the expected action DSL on 43 of 44 hand-written instructions, and an English–Claudish translator that served 100,747 requests in eleven days — though the authors note synthetic supervision inherits teacher errors, no user studies were run, and the headline number comes from a subset deliberately selected as hard for the prior compiler rather than a broad benchmark.

arXiv:2609.04199 PDF

Applications 94

X-Translator: A Real-Time Multilingual Speaker-Aware Speech-to-Speech Translation System

Yuxiang Zhao, Yichi Zhang, Yanjie An, Yanqiao Zhu, Zhanxun Liu, Yushen Chen et al. cross-listed Real-time speech-to-speech translation has to trade off translation quality, latency, naturalness, and speaker consistency, and open reproducible systems struggle particularly with long conversations where partial ASR hypotheses are unstable and turn boundaries are unclear. X-Translator is a low-cost modular cascade of streaming ASR, machine translation, and prompt-conditioned TTS, coordinated by a session-level runtime controller that uses incremental segment commitment to turn unstable ASR streams into translation-ready units and an online speaker prompt manager to bind each source span to the right voice. The system is evaluated on OpenSTBench for translation quality, speech quality, and latency, benchmarked against proprietary speech translation APIs, plus long-form voice stability and multi-speaker speaker preservation, with code and a demo released.

arXiv:2607.17544 PDF

ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrieval

Aaryan Kapoor, Md Abdullah Al Hafiz Khan cross-listed Embedding-based code retrieval feeds coding agents and retrieval-augmented code generation, where returning code that actually works matters more than returning code that looks similar, yet no existing benchmark places execution-verified near-clone bugs in the search pool to test that distinction. ExecRetrieval supplies 939 Python tasks, each with one execution-verified canonical implementation and up to four buggy distractors produced by single mechanical mutations, and evaluates 23 dense embedding configurations plus BM25 with paired McNemar tests and bootstrap intervals. The best hosted system reaches exec@10 of 1.00 but only exec@1 = 0.331, with rank-1 misses being a paired buggy variant 91.5-99.4% of the time and the canonical ranked below at least one of its distractors in 67-78% of queries on leading systems.

arXiv:2609.01865 PDF

Privacy Leakage in Federated Learning: Gradient-Based Client Identity Inference and Defenses for Inertial Sensing in Vehicular Edge Networks

Ali Akarma (Islamic University of Madinah, Madinah, Saudi Arabia, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia) et al. cross-listed Federated learning (FL) is promoted as the privacy-preserving way for vehicles and roadside infrastructure to train shared models, but the weight deltas clients upload can reveal who sent them, undermining the anonymity that safety-critical vehicle-to-everything applications assume. Using inertial measurement unit data from the UCI HAR benchmark as an accessible proxy for onboard vehicle telemetry, an honest-but-curious server recovers client identity with near-perfect accuracy (roughly 1.000) across five attack classifiers and five non-IID partitions of undefended updates. Sweeping a lightweight clip-then-noise defense shows Gaussian noise with sigma between 0.1 and 0.2 pushes attack accuracy to near-random at under 5% relative accuracy loss, with formal differential privacy budgets from Renyi accounting, and ensemble FL supplying a complementary 1/K anonymity bound at no noise penalty.

arXiv:2609.02971 PDF

Equation Recast for Canonical Operator Learning Across Parametric PDEs

Qiyun Cheng, Valentin Duruisseaux, Cesar F. Clauser, Md Hossain Sahadath, Huihua Yang, Shaowu Pan et al. Purely data-driven surrogates for parametric partial differential equations (PDEs) need heavy coverage of both input functions and physical parameters, and they can fail silently once queried outside that training distribution. Equation recast derives the parameter-induced variation of the operator analytically from the governing equation and absorbs it into effective source terms, so only a single canonical operator has to be learned and new parameter regimes are handled zero-shot, with loss of convergence in the recast iteration doubling as an internal warning that a prediction is unreliable. The approach extrapolates across multi-parameter, nonlinear, and singular PDE settings and merges sparse heterogeneous datasets into one representation; in high-fidelity tokamak fusion simulations, one jointly trained operator unifies electron-temperature data from four different device geometries via canonical-domain mapping.

arXiv:2609.02982 PDF

Toward Collective-Centric Evaluation of Preference Inference for Participatory Democracy

Pierre-Antoine Lequeu, Salim Hafid, Paul Lerner, Nazanin Shafiabadi, Laur\`ene Cave, David Mas et al. cross-listed Participatory democracy platforms such as Polis and Remesh let thousands deliberate at once, but no participant can vote on every statement submitted, so the vote data is highly sparse and preference inference models fill the gaps — potentially amplifying, suppressing, or reordering support in ways that reshape how a consultation is interpreted. Instead of judging these models only by per-user prediction accuracy, the authors introduce a collective-centric evaluation asking whether inferred votes preserve salient properties of the overall preference landscape, and release four consultations covering more than 90,000 participants, 1 million votes, and 22 languages. Models with comparable predictive accuracy differ substantially in how much collective structure they preserve, which the authors take as evidence that accuracy alone is an inadequate criterion in democratic settings.

arXiv:2609.02990 PDF

Population-Calibrated Graph Screening at 835-Million-Address Scale, with Label-Free Transfer to New Chains

Yury Korolev cross-listed Compliance screening of blockchain addresses is in practice a lookup against sanctions registries plus clustering heuristics, which fails on unlabelled addresses and entirely on chains with no label coverage. The deployed system described here scores an address by its position in a multi-chain transaction graph of 835,330,427 addresses and 15,826,261,934 edges across five EVM chains, using a shared inductive encoder with per-chain normalisation and two scoring heads, with decision thresholds set as exact quantiles of the score distribution over the full population so alert volume is known in advance. Heads trained on two chains recall 0.8598, 0.8182, and 0.9967 of held-out positives on Base, Arbitrum, and Gnosis at a 0.1% population alert rate with no target-chain labels, and a replay over 68 external registry events flagged 40 of them at that budget, with first on-chain appearance a median of 529 days (Ethereum) to 648 days (Tron) before public designation; an adversarial harness of eight reinforcement-learned behaviour archetypes also exposes a measured blind spot in the deployed heads.

arXiv:2609.03036 PDF

IDSPACE: A Novel Document Generator for Reliable Evaluation of Digital Identity Verification Systems [Extended Technical Report]

Lulu Xie, Yancheng Wang, Kanchan Chowdhury, Rolando Garcia, Yingzhen Yang, Jia Zou cross-listed Banks, lenders, and governments verifying remote users need fraud detection tools that can be evaluated and tuned, but identity documents are sensitive and therefore scarce, making synthetic generation the practical route. IDSpace adds three things: model-guided Bayesian optimization that tunes generation parameters from only a few target-domain samples to maximize both visual similarity and prediction consistency with target-domain models; a separation of user-specified metadata such as demographics, fraud patterns, and capture device from automatically tuned low-level controls like fonts, noise, and image quality; and support for scanned and mobile-captured documents beyond flat templates. It improves evaluation consistency by 15-45% over baselines including CycleGAN, diffusion inpainting, and non-guided optimization, raises training accuracy by up to 9% and target-domain SSIM by 10%, and ships a released dataset of 359,240 synthetic documents across ten European ID types.

arXiv:2609.03052 PDF

Scaling Laws, Tabular Data and Actuarial Ratemaking Models

Ronald Richman Neural scaling laws describe power-law improvements in held-out loss as data, parameters, and compute grow, but whether the same regularities hold on noisy heterogeneous tabular data — where generalized linear models (GLMs) remain competitive — is untested. Training several model families on growing fractions of a real motor insurance portfolio across multiple seeds and scoring out-of-sample Poisson deviance shows every family improving with more data but at sharply different exponents: TabM scales with data markedly better than purely supervised tabular Transformers or standard multilayer perceptrons, and Transformer variants gain little from added parameters unless given extra inductive bias through TabM-style adaptation or self-supervision. The practical implication is that architecture and objective design, not raw size, determine whether scaling pays off in this regime.

arXiv:2609.03106 PDF

Improving precipitation forecasts in an AI weather model using observational data

Julian F. Schmitt, Bertrand Delorme, Robert C. King, Yashica Patodia, Tapio Schneider, Aditi Sheshadri et al. cross-listed Global AI weather models are trained almost entirely on the ERA5 reanalysis, which carries known biases for precipitation. Fine-tuning a graph-transformer forecaster on IMERG satellite precipitation observations at 0.25° resolution improves medium-range continuous ranked probability scores by up to 19% and sharpens skill on tropical storms and drizzle. On extreme rainfall the model exceeds operational state-of-the-art Brier skill scores by 57% globally, though a physics-based operational model stays more reliable for the very heaviest events.

arXiv:2609.03210 PDF

The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100

Francesco Mantegna, Gereon Elvers, Dulhan Jayalath, Gilad Landau, Tasha Kim, Miran \"Ozdogan et al. Announces the second year of a multi-year competition for non-invasive speech decoding from magnetoencephalography, following 2025 tasks where winners reached 95.6% F1-macro on speech detection and 73.6% on phoneme classification using the single-subject LibriBrain dataset. The new LibriBrain100 release adds 32 subjects at roughly 40 minutes each plus about 80 hours of within-subject data, and the tasks advance to word classification along two tracks. The Deep track pushes within-subject accuracy at scale, while the Broad track tests cross-subject generalisation as per-subject fine-tuning data shrinks from about 40 to 20 to 10 minutes, the clinically feasible range for a usable brain-computer interface.

arXiv:2609.03231 PDF

Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings

Peichun Hua, Yunming Xiao cross-listed Organizations that outsource vector indexes for retrieval-augmented generation expose both their corpora and user queries to the cloud provider, but cryptographically protecting a query is expensive because every search touches corpus-scale state — a naive secure implementation at million-document scale costs minutes and roughly 90 GB of communication per query, and even tuned prior systems take 10 to 22 seconds. Spruce co-designs the representation with the protocol: it learns compact binary codes that preserve candidates for later full-precision reranking, so corpus-wide scoring becomes Hamming-distance computation under two-server multi-party computation, with a corpus-calibrated fixed-radius rule that avoids multi-round candidate selection, optional private cluster pruning, and a single-core owner-operated dealer that removes the cloud preprocessing bottleneck. Across four corpora of 383K to 5.42M documents, full scans run in 0.21 to 2.97 seconds, 4.8 to 6.7 times faster than the closest measured prior work, while pruning gives 13.1 to 22.9 times speedups and retains 93.9% to 97.3% of full-precision NDCG.

arXiv:2609.03376 PDF

SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign

Jiarui Lu, Yuyang Wang, Yizhe Zhang, Jiatao Gu, Navdeep Jaitly, Joshua M. Susskind et al. Generative models for proteins typically train in two stages, first fitting autoencoders that tokenize sequence and structure into latents and then training a generator over those latents. SimpleDesign tests whether that pipeline is necessary by training directly in data space with a single end-to-end objective that combines discrete cross-entropy for amino acid sequences with a regression loss for three-dimensional structure, using a Mixture-of-Transformer architecture that gives each modality its own processing while keeping global self-attention across both. Trained on over 2M sequence-structure pairs, it reports strong results on co-design as well as unconditional sequence and structure generation benchmarks.

arXiv:2609.03377 PDF

Privacy, Robustness, and Fairness Trade-offs in Federated Intrusion Detection: Geometric Indistinguishability at the Aggregation Interface

Adrita Rahman Tory, ABM Shawkat Ali, Md Abu Layek, Khondokar Fida Hasan cross-listed Federated learning lets organizations train network intrusion detection systems without pooling sensitive traffic, but real deployments need differential privacy, tolerance to Byzantine participants, and coverage of rare attack classes at the same time. The authors combine differentially private stochastic gradient descent (DP-SGD) with coordinate-wise median aggregation on UNSW-NB15 under label-flip and model-poisoning attacks, framing the interaction through 'geometric indistinguishability': privacy noise disperses client updates, and minority-class signal is exactly what robust aggregation then discards. Joint use of privacy noise and robust aggregation degrades rare-attack detection disproportionately compared with majority classes, part of the collapse under strong privacy traces to training miscalibration, and a residual performance floor for ultra-rare categories persists even after tuning per privacy budget.

arXiv:2609.03420 PDF

When Retrieval Helps: Selective Retrieval for Single-Turn Mental-Health QA

Hyunseo Oh, Chong-Kwon Kim, Yoonhyuk Choi Retrieval-augmented generation grounds answers in source material, but its benefit is not uniform in single-turn mental-health question answering, where a query can mix emotional distress, treatment concerns, and safety-sensitive needs. The authors operationalize retrieval need through three draft-conditioned dimensions — psychoeducational need, coping need, and response specificity — plus a rule-based safety trigger over a compact corpus of coping, psychoeducational, and safety guidelines, then compare closed-book, always-retrieve, and selective policies using a QLoRA fine-tune trained on MentalChat16K and evaluated on CounselBench-Eval and CounselBench-Adv. Always retrieving raised specificity but lowered overall response quality and introduced additional safety-sensitive failures, while the selective policy preserved closed-book behavior on low-need queries and avoided that degradation, framing retrieval activation as a safety-relevant control decision.

arXiv:2609.03454 PDF

The Psychological Costs of Artificial Intelligence Adoption in Software Engineering

Adam Alami, Elda Paja, Abhishek Tiwari cross-listed Organizations rolling out AI in software engineering measure tangible outcomes like productivity, while the disruption to role identity, team norms, and established sources of job satisfaction goes unmeasured. A case study at a large software development services company one year after its AI adoption launch, drawing on meetings and 21 semi-structured interviews, identifies five psychological costs: accountability anxiety, craft identity disruption, erosion of meaning and satisfaction, cognitive and workload intensification, and uncertainty distress. Practitioners handle these by restoring control through new practices, mitigating them with identity-preserving adaptations, or simply absorbing what neither resolves — which reframes AI adoption as a human transition rather than only a technological and organizational one.

arXiv:2609.03456 PDF

Mind the Gap: Robustness Risks in PII Detection Systems

Adeel Zafar, Slawomir Nowaczyk Detectors for personally identifiable information (PII) report strong benchmark numbers, but those benchmarks look nothing like the noisy, unstructured, informal text such systems meet in deployment, and every missed entity is a direct privacy risk. A stress-test benchmark spanning seven categories of natural distribution shift is applied to three widely deployed architectural families: encoder-based named entity recognition (SpaCy), rule-based hybrid detection (Presidio), and generative extraction with Qwen2.5-3B. All three degrade significantly out of distribution, but with distinct and complementary failure modes — encoders miss unseen surface forms and entity boundaries, rules break on non-standard formats, and the language model confuses entity types and generates unstably — so aggregate scores hide deployment-critical gaps and no single architecture is reliable across PII categories; the authors propose a hybrid pipeline with a question-answering feedback loop and release the benchmark.

arXiv:2609.03464 PDF

Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech

Kunat Pipatanakul, Potsawee Manakul, Warit Sirichotedumrong, Sittipong Sripaisarnmongkol, Pakorn Nathong, Phatrasek Jirabovonvisut Text-to-speech in a low-resource language normally forces a choice between a large voice-cloning model with expensive inference and a compact fixed-voice model that needs its own speaker corpus. The route studied here treats the cloning model as a programmable data source: 15 seconds of reference audio becomes a fully synthetic training set for a small student, making text preparation, quality filtering, and rejection sampling consequential since teacher mistakes become training targets, with Thai adding ambiguous word boundaries, lexical tone, loanwords, numeric verbalization, and Thai-English code-switching. The resulting 82M-parameter Wayu-Paxa-TTS-Edge runs on-device without reference audio, scoring 68.2% challenge-set keyword accuracy and 91.4% pause precision — above its OmniVoice teacher's 89.9% — with 3.7% and 1.1% character error rate on Thai and English; the model and evaluation framework are open-sourced.

arXiv:2609.03502 PDF

An Adversarial Zero-Shot Learning Approach for Anomaly Detection in Multivariate IoT Traffic Data

Mahshid Rezakhani, Tolunay Seyfi, Fatemeh Afghah Anomaly detection on Internet of Things (IoT) traffic is complicated by device diversity, absent labels, and distribution shift between deployment environments. The proposed framework trains a sequence variational autoencoder with adversarial and contrastive objectives to learn latents that are both domain-invariant and semantically structured, adds encoder and decoder adaptor layers to align feature distributions across environments without moving raw features, and segments traffic by destination to better reflect real communication structure. Evaluated on six datasets covering industrial, enterprise, general-purpose, smart home, and military settings across 44 cross-domain transfer scenarios, it shows strong zero-shot generalization in several pairings and performance competitive with a contrastive domain-adaptation baseline.

arXiv:2609.03505 PDF

WeatherNext 3: Increasing resolution and performance of global weather models with raw observations

Stephan Rasp, Boris Babenko, Dominic Masters, Andrew El-Kadi, Samier Merchant, Guy Shalev et al. AI weather models forecast quickly and skillfully but run at coarser resolution than the best physics-based systems and are trained and initialized only on analysis products, so they cannot use raw observations and inherit whatever biases the analysis carries. WeatherNext 3 produces a new forecast every hour by ingesting low-latency geostationary satellite data, runs at hourly steps and 0.1 degree resolution for single-level fields including solar radiation and cloud cover, and learns to predict satellite-derived precipitation, tropical cyclone tracks, and sparse station observations directly. Learning from station data lets it emit 2-metre temperature and dewpoint anywhere conditioned on local geography, with substantially lower error than competing global models even at stations held out of training, effectively collapsing data assimilation, forecasting, and post-processing into one learned system.

arXiv:2609.03582 PDF

Analysis of Prompt Engineering for Drug Toxicity Prediction

Mia MacGregor, Aakash Welgamage Don, Mark Bartlett Toxicity drives much of the roughly 90% failure rate in drug trials, and large language models are increasingly used to predict it, but their outputs shift with small changes in wording. The study systematically varies job-role framing, prompt structure and rule interpretation when asking models which chemical properties matter for toxicity, then uses the models to generate feature datasets that feed conventional machine learning classifiers. The run-to-run variance inherent to the language models outweighed any benefit from tuning the prompts, while substituting cheminformatics code for model-generated feature values produced substantial accuracy improvements, and the authors present the analysis procedure as reusable across bioinformatics prompting tasks.

arXiv:2609.03635 PDF

Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements

Arianna Miola, Bruno Spaccavento, Lorenzo Silotto, Marco Bianchetti, Luca Cagliero Comparing banks across jurisdictions defeats most question-answering systems because the filings are long, technical, and inconsistent in how they present text and numbers. FinRAG-QA provides 999 practitioner-curated questions on 10 standardised indicators grounded in 209 annual and Pillar 3 reports from 24 major European and U.S. banks covering 2019-2023, with source documents averaging 198k words and questions that require retrieval across institutions rather than within one. Ablating a multi-stage retrieval-augmented generation pipeline, contextual chunk enrichment plus a retrieval-optimised embedding model lifts NDCG@10 from 0.322 to 0.710, and conditional on retrieving the right evidence a reasoning-optimised generator raises answer accuracy from 44.6% to 79.0% at roughly 20x the generation latency. Two negative results stand out: cross-encoder reranking degrades retrieval when the first stage is already strong, and feeding a single top-ranked chunk beats larger contexts.

arXiv:2609.03654 PDF

Artificial Intelligence for Energy Optimization in Data Centers

Mohammed Basharath Ullah, Summaiya Unnisa Begum, Mohammed Nadeem Ullah Data centres are increasingly both optimised by artificial intelligence and loaded by it, yet control research models workload as an external arrival process while sustainability research models infrastructure as a fixed multiplier, so the feedback between them goes unstudied. A documented screening protocol retrieved roughly 194 papers, 63 of which were coded in detail; of the 28 primary control-oriented studies, 18 were validated only in simulation, 5 reached real hardware or production, and none accounted for water withdrawal or embodied carbon. Reported energy-savings intervals across the four technique families overlap almost completely, meaning the literature cannot currently rank its own methods. The authors also sketch CLEAR-DC, an untrained framework and reporting schema that couples control policy to workload demand through an explicit elasticity term and reports net rather than direct benefit.

arXiv:2609.03716 PDF

Can LLMs Extract Architectural Design Decisions from Source Code Commits? - A Preliminary Exploratory Study

Amey Karan, Rudra Dhar, Mohamed Soliman, Karthik Vaidhyanathan cross-listed Architectural design decisions (ADDs) record why a software system is structured as it is, but they are seldom documented and instead remain implicit in source code commits, making recovery valuable for architectural knowledge management. Four large language models (Gemini 3 Pro, DeepSeek R1, Kimi K2, and Qwen3) were prompted zero-shot and few-shot to reconstruct 30 developer-written ADDs from open-source projects, scored with ROUGE-L, BLEU, METEOR, and BERTScore plus a manual review of the Gemini outputs. All models exceeded 0.81 BERT-F1 and few-shot prompting improved alignment (0.828 to 0.847 for Gemini), yet the generated decisions were consistently too long, focused on implementation detail, and missing the underlying rationale.

arXiv:2609.03721 PDF

RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intents

JoyIndustrial VisCAD Team, Linxin Cai, Qiuhe Hong, Zhichao Huang, Guanlin Li, Zongzhen Li et al. cross-listed Parametric computer-aided design (CAD) benchmarks tend to use synthetic or CAD-native inputs and score only executability and intersection-over-union, which misses how real industrial modeling tasks are specified. RealCADBench contains 12,632 tasks drawn from 19 factory-automation categories spanning text descriptions, 2D engineering drawings, real product photographs, and rendered images, for both part and assembly modeling; each method emits FreeCAD API Python that a shared runtime executes, scored on executability, solid and surface intersection-over-union, and a rubric-based visual-semantic identity judge. Across nine frontier models none leads on all four metrics — executability spans 0.565 to 0.812 and solid IoU 0.2841 to 0.5379 across part regimes, and the best balanced composite belongs to a model that tops none of the individual metrics. On the 25-task assembly slice, Codex with GPT-5.5 improves executability and both IoU scores over standalone GPT-5.5 while losing 6.98 points on the judge, and recurring failures include missing fine structures, lost part identity, and misplaced assembly components.

arXiv:2609.03773 PDF

More Criticism Does Not Make a Better Review: EquiReview-R

Zexing Zhang, Jichao Li, Tianyang Lei, Yude Fu, Yang Kewei Automated paper reviewers now generate plenty of specific criticisms, but two opposite failures hide inside an aggregate score: missing a real weakness, and keeping an accusation the evidence does not support. EquiReview-R reframes reviewing as evidence-guided refinement of a structured set of concerns, resolving each existing concern against localized evidence, searching for missing issues from both independent and review-conditioned angles, and returning an explicit stop, continue, or defer decision. On a frozen cohort of unseen papers it meets a prespecified non-inferiority bar for major omission while cutting major overcritique from 15.5% to 8.1%, stopping on 52.4% of papers; computation-matched controls show the improvement comes from revising concerns rather than from extra inference or shorter output. The evidence-linked trajectory corpus behind the analysis is released as ReviewTrace.

arXiv:2609.03943 PDF

FiMI Banking: A Sovereign Model for Indian Retail Banking

NPCI AI Research Team, Aman Kumar, Asit Desai, Chandra Bhushan, Harsh Sharma, Harshit Bhushan et al. Retail banking chatbots must answer product questions, act on accounts through tools, and refuse anything outside their remit under regulatory constraints — behaviors general-purpose models handle unreliably. FiMI Banking is a controlled Indian retail-banking environment built from vetted bank documents, structured ground truth, synthetic customer profiles, and callable banking tools, used to compare two post-training recipes. Preference optimization mainly fixes response-level safety, lifting out-of-scope refusal from 52% to 80%, while reinforcement learning with verifiable rewards fixes multi-turn tool use, raising edge-case scores from 0.509 to 0.718 and order-sensitive task scores from 0.590 to 0.679 while generating 29% fewer tokens. The two methods address complementary failure modes rather than substituting for each other.

arXiv:2609.03960 PDF

Para-Pipe: Exploiting Hierarchical Operator Parallelism of ML Computational Graphs on SoCs

Yujie Zhang, Huiying Lan, Ehsan Aghapour, Zhiyuan Ning, Peng Zan, Weidong Shao et al. cross-listed Deploying neural networks on heterogeneous edge systems-on-chip usually means choosing between pipelining across processing units, which favors throughput, and running independent operators in parallel, which favors latency — improving one typically degrades the other. Para-Pipe is a hierarchical mapping framework that tunes operator parallelism both inside and across pipeline stages, which also trims inter-processor communication overhead. Evaluated on an Amlogic chip with ARM big.LITTLE CPUs and a GPU and on a Black Sesame chip with a deep learning accelerator and two DSPs, it yields several Pareto-optimal configurations, with throughput-optimized ones averaging 11.0% better energy efficiency than pure pipelining and 23.3% better than non-pipelined parallel execution.

arXiv:2609.04168 PDF

67 more specialized papers

Large Language Models 55

R$^{2}$Adapter: A Routing and Rewriting Adapter for Efficient Hybrid RAG

Yucan Guo, Miao Su, Saiping Guan, Long Bai, Zhongni Hou, Zixuan Li et al. Vanilla retrieval-augmented generation (RAG) is cheap but weak on relational or multi-hop questions, while graph-based RAG handles them at much higher inference cost, and existing hybrid systems route between the two using heuristics or an LLM call that adds overhead and ties the pipeline to a particular backbone. R2Adapter is a lightweight plug-in that routes each query between vanilla and graph retrieval and rewrites uncertain graph-routed queries so their multi-hop structure is more explicit, without extra supervision. Across three multi-hop question-answering benchmarks it cuts graph-based RAG usage by up to 59% while keeping answer accuracy comparable, and being model-agnostic it drops into existing pipelines.

arXiv:2609.02894 PDF

Margins, Not Windows: Training-Free Per-Step Lossy Speculative Decoding

Oszk\'ar Urb\'an, Young D. Kwon, Stylianos I. Venieris, Cecilia Mascolo Speculative decoding speeds up LLM inference by drafting tokens and verifying them in parallel, but tree-attention drafters like EAGLE-3 keep two things fixed: strict token-match verification and a static draft-tree shape. AdaptiveSpec relaxes both per step without training, using signals already available during decoding: a margin rule accepts a mismatched drafted token when the target model's probability on it is close enough to its own top-1 probability, and a tree policy sets depth, width, and node count from a fused signal of draft confidence and rolling acceptance history so the total draft count varies rather than merely shifting around. Implemented in the SGLang serving engine, it improves throughput over EAGLE-3 by up to 56% while recovering 93% to full lossless accuracy on GSM8K, MATH-500, and HumanEval across DeepSeek-R1-Distill-Llama-8B, Llama-3.1-8B-Instruct, and Qwen3-8B.

arXiv:2609.02897 PDF

Contamination Inflates Scores but Rarely Reorders Large Language Model Leaderboards

Xingyao Xiao (Stanford University), Yihong Cheng (City University of Macau) Benchmark contamination is usually treated as a single threat, but inflating everyone's absolute scores and reordering the model ranking are separate questions. The authors reframe contamination as a violation of anchor-item invariance and measure it by comparing each model's performance on original versus semantically equivalent paraphrased items, a within-item contrast that holds the underlying skill fixed and isolates memorization; the measure is calibrated against 74 models finetuned with a known contamination dose, recovering leakage dose-responsively at +0.187 accuracy points and never flagging a clean negative control. Applied to per-instance responses from 47 public models on ARC, GSM8K, HellaSwag, and MMLU, the rank correlation between the standard leaderboard and a paraphrase-controlled one is 0.997, with only 3 of 188 model-by-benchmark cases showing corroborated differential contamination. The conclusion is that contamination in these models is largely uniform, inflating scores without changing order, and the authors release the audit and recommend reporting paraphrase-controlled rankings with confidence intervals.

arXiv:2609.02899 PDF

Judging LLM-as-a-Judge: Concerning Rubric Artifacts in LLM-based Automated Text Generation Evaluation

Anshul Bagaria, Sowmya S Sundaram, Gokul S Krishnan, Balaraman Ravindran Pipelines that use a language model as an evaluation judge rest on the assumption that the score comes from reasoning about the candidate response against a rubric. Classifiers trained on rubric text alone, with no access to any evaluated response, predict judge outputs well above chance, meaning rubric wording itself encodes recoverable evaluative signal that partly determines scores independently of what is being judged. Counterfactual perturbations add a second problem: judges frequently fail to update their decision when either the candidate response or the rubric criterion is reversed.

arXiv:2609.02942 PDF

The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priors

Toni J. B. Liu, Jiajun Bao, Yizhou Liu, Gurbir Arora, Nicolas Boull\'e, Rapha\"el Sarfati et al. A single direction in a language model's unembedding matrix encodes the unigram distribution of its training corpus, serving as the prior the model falls back on when the context tells it little. Projecting the final prediction state onto this 'direction of ignorance' yields a per-token loading factor that splits the state into two orthogonal pieces matching exactly the two factors of a tempered Bayesian update — a unigram prior raised to that exponent, and a context-driven likelihood — and the exponent falls steadily as context becomes more informative. The structure shows up in Llama, Qwen, Gemma, and Pythia across 0.4B to 405B parameters, with larger models leaning on the prior less at high context, and raising or lowering the factor directly steers predictions toward or away from the unigram prior.

arXiv:2609.02959 PDF

Modern Transformers Are Implicit Hybrids: From Functional Differentiation to Principled Hybrid Architecture Design

Runlin Shi, Bojian Yin, Guoqi Li Models that mix full attention with linear attention are increasingly common, yet which layers get which type is decided heuristically. Two intervention metrics — RoPE Frequency Importance Score, measuring how each rotary frequency shapes a head's attention distribution, and RoPE Positional Dependence, isolating reliance on rotary positional modulation — reveal in Qwen3-series models and Llama3.1 a complete split between retrieval heads and positional heads at a mid-low frequency boundary the authors call the Global Positional Band, whose location tracks the training-length positional scale and suggests a cause of zero-shot length-extrapolation failure. The resulting design principles (positional modeling only locally, global access via position-independent retrieval, assigned at head rather than layer granularity) are instantiated in a Head-wise Hybrid Architecture using NoPE full attention for retrieval and linear attention for local position, which at a full-to-linear attention ratio below 1:3 substantially strengthens zero-shot long-context extrapolation over plain Transformers, pure linear attention, and layer-wise hybrids while retaining language modeling and commonsense reasoning quality.

arXiv:2609.02986 PDF

Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation

Zhiwei Zhang, Zechen Sun, Fei Zhao, Kang Peng, Bin Liang, Huayu Deng et al. On-policy distillation gives a student dense token-level supervision from a frozen teacher on the student's own rollouts, but vanilla implementations apply that supervision to every prompt without checking whether the teacher is right — and since reverse KL is mode-seeking, a confidently wrong teacher produces a strong misleading update. TGOPD estimates teacher reliability per prompt from a small set of verifier-scored teacher probes, routing prompts that pass to dense distillation and the rest to verifier-grounded GRPO; distributional proxies such as entropy or teacher-student agreement are rejected because they measure uncertainty rather than outcome correctness. With 4B and 35B students across mathematics, code, and instruction following, it beats vanilla on-policy distillation in all six single-domain settings and posts higher seven-benchmark averages at both scales under multi-domain training, while the probing work raises teacher-node GPU utilization from 9.8% to 78.9% in the measured asynchronous run.

arXiv:2609.02998 PDF

Unifying Conformal Language Tasks with In-Context Ensembles

Xiao Shi Huang, Chen-Yuan Lin, Bruce Kuwahara, Kin Kwan Leung, Jesse C. Cresswell Summarization, extractive question answering, and similar tasks reduce to retrieving relevant content from documents under two competing constraints: coverage, keeping enough pertinent information, and conciseness, dropping as much irrelevant material as possible. Conformal prediction can guarantee coverage, leaving conciseness to the design of a score function, and current state-of-the-art scores come from hand-engineered prompts asking a language model to rate importance — labor-intensive and task-specific. The Conformal Relevance framework replaces that manual effort with curated in-context learning examples plus ensembling, maintaining coverage while improving conciseness across seven NLP tasks with minimal manual input, and the authors add theory on ensemble diversity: a complementarity condition characterizing when ensembling improves worst-case sentence scores, and a saturation bound on how much it can help.

arXiv:2609.03005 PDF

LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference

Renyuan Liu (Richard), Yuyang Leng (Richard), Kaiyan Liu (Richard), Yuzhou Zhong (Richard), Shaohan Hu (Richard), Chun-Fu (Richard) et al. Running large language models (LLMs) on phones and embedded boards is hard because weights far exceed available DRAM, so systems stream them from SSD or flash while exploiting activation sparsity — but accurate sparsity decisions need the latest context, while overlapping I/O with compute needs early prediction, forcing prior designs to either serialize or fetch redundantly. LeanStream resolves the tension with a speculate-and-refine streaming design that progressively sharpens its computation, weight-loading, and cache-retention priorities using partial GPU results, so storage I/O and GPU execution interleave at fine granularity. On mobile and embedded hardware it cuts memory usage by 4.8x to 7.5x at the best throughput prior systems reach, while generating tokens 1.6x to 2.1x faster.

arXiv:2609.03079 PDF

Large Language Models in Resolving Contextual Knowledge Conflicts

Xinye Yang, Zhenyang Liu, Ruisi Li, Yuanyuan Lei Work on knowledge conflicts in large language models (LLMs) has focused on context contradicting a model's parametric knowledge; the neglected case is contradictions inside the supplied context itself. A six-way taxonomy (factual, inferential, temporal, granularity, perspective, ambiguity) backs ContextConflict, 5,781 reasoning and summarization samples spanning explicit contradictions and implicit ones requiring multi-step inference, on which nine LLMs perform poorly. Interpretability analysis finds models latently register conflicts yet consistently privilege whichever evidence appears earlier in the context, and a training-free, label-free activation-steering method that counteracts this positional bias improves reasoning accuracy and yields more balanced summaries.

arXiv:2609.03148 PDF

Routing Is Not Enough: Diagnosing Intra-Adapter Subspace Contention in MoE+LoRA Fine-Tuning

Mehreen Hossain Chowdhury, Nowshin Mahjabin, Ahmed Shafin Ruhan, Md Azam Hossain, Abu Raihan Mostofa Kamal, Md Tahmid Rahman Laskar Multi-domain fine-tuning often pairs mixture-of-experts (MoE) routing with low-rank adaptation (LoRA), on the assumption that token-level routing keeps domain-specific updates apart. Mixing Python code with biomedical text and mathematical reasoning shows routing does separate almost disjointly, yet adding biomedical data still substantially raises code perplexity, and two diagnostics — Jaccard routing overlap and adapter-gradient cosine similarity — localize the cause to nearly orthogonal domain gradients competing inside the same low-rank adapter subspace rather than to shared experts. SpawnLoRA responds by spawning gated sub-adapters inside an expert when adapter-level contention is detected while leaving the router untouched, reducing negative transfer relative to standard and rank-adaptive LoRA on Phi-tiny-MoE-instruct and OLMoE-1B-7B.

arXiv:2609.03150 PDF

Frontier LLMs are effective batch optimizers: Assessing reasoning models in continuous and discrete settings

Frank Hu, Shriram Chennakesavalu, David Graff Large-scale pretraining makes frontier large language models (LLMs) plausible priors for optimization, but their behavior as batch optimizers — proposing whole batches of candidates per round — has been little tested, especially for the current reasoning-tuned generation. Evaluation spans continuous numerical test functions and discrete, semantically meaningful search spaces. LLMs are competitive zero-shot batch optimizers on numerical benchmarks yet brittle compared to classical non-LLM optimizers there, while their priors help substantially in semantically rich discrete settings that resemble the structure of their pretraining data.

arXiv:2609.03177 PDF

LLMs Learn Better In-Context from Rules than from Examples

Xiang Fu, Seungmin Cho, Yukyung Lee, Najoung Kim Sets up a controlled comparison between the two dominant ways of teaching a language model a task in context: giving it the rule (instruction following) versus giving it input-output demonstrations (few-shot prompting), for five tasks spanning games, arithmetic, and linguistic inference where both modes specify the same underlying task. Models learn more reliably from rules than from examples, and neither adding examples on top of rules nor scaling up the example count yields consistent gains. Instruction tuning amplifies the rule advantage without harming example-based learning, base models show no special affinity for examples, and the rule advantage is largest for tasks recruiting algebraic abstraction and smallest for those needing distributional sensitivity or parametric knowledge.

arXiv:2609.03213 PDF

Language-encoded network topology enables large language models to reason about complex networks

Ucchwas Talukder Utsha, Sakib Mostafa, James Zou, Md Tauhidul Islam Language models handle structural questions about networks poorly when graphs arrive as edge lists, prose, or measurement tables, because roles like hub or bridge must be inferred rather than read. BioGlyph precomputes graph partitioning and structural measurements to label each node with a role — hub, community core, cross-community connector — and translates those into a fixed vocabulary describing the role, its supporting evidence, and its semantic consequences, leaving both graph and model untouched. Across twenty networks in five domains it improves open models' structural reasoning accuracy by up to 26 percentage points over edge-based, numerical, and learned representations, with gains concentrated in dense community-structured graphs; on a budding-yeast protein interaction network the cross-community connectors it identifies are enriched for essential genes.

arXiv:2609.03229 PDF

SGD-KV: Summarization Guided KV Cache Compression

Zeyu Liu, Woomin Song, Xuandi Fu, Sai Muralidhar Jayanthi, Vivek Govindan, Aram Galstyan et al. Long-context inference is bounded by key-value cache memory that grows linearly with sequence length, and existing compression schemes apply uniform heuristics that ignore what individual attention heads actually do. SGD-KV introduces a chunk-summarization diagnostic that scores each head on how much it performs hierarchical information aggregation, then allocates cache budget according to that score distribution. On Qwen2.5-7B-1M and Qwen3-32B across long-context benchmarks it reaches state-of-the-art accuracy at contexts up to 1M tokens while cutting KV cache memory by as much as 75%.

arXiv:2609.03235 PDF

What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation

Daisuke Kikuta When someone asks a model to make one local edit to an artifact built up over a conversation, the model must find every dependent part and propagate the change, with the relevant context often scattered through the chat history. A new benchmark isolates this revision-propagation setting and nine methods are compared — including sequential reflection and parallel sampling variants — on gpt-oss-20b/120b, gpt-5.4-mini, and qwen3.5-9b/27b/122b. Baselines land between 68.3% and 93% accuracy, and the best cost-to-benefit ratio comes from drawing three parallel samples and picking one by LLM judgment or medoid selection, worth 2.2 to 9.7 points of accuracy.

arXiv:2609.03254 PDF

Decoupling Turn-Taking from Semantics: A Decoupled Data Approach for Finite-State-Machine-Based Full-Duplex Dialogue

Yihang Li, Chenhui Chu The Neural Finite State Machine approach to full-duplex speech dialogue writes turn-taking control tokens and response text onto one causal token stream, keeping fine-tuning cheap, but it is trained on synthetic text that cannot reproduce the acoustic timing of real conversation. The proposed decoupled recipe learns turn-taking from real human-human spoken dialogues, converted into finite-state-machine tapes by a rule-based event-guided transformation that classifies turn events and applies deterministic mapping rules without any large-language-model annotation, while semantic behavior is shaped separately by configurable human-agent text dialogues. A Source-Aware Calibrated loss rebalances the long-tailed state-transition tokens and routes each data source toward the capability it supervises best, improving turn-taking proficiency while recovering the base model's semantic ability.

arXiv:2609.03321 PDF

How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models

Dun Li Chan, Emily Liu, Niyathi Allu, Christian Hoang Robustness to typos, corrupted text, word substitutions, and shuffled tokens is usually measured only by what a model outputs, which hides how the disturbance actually travels through the network. Six perturbation types are tracked at three levels — output behavior, hidden-state geometry via centered kernel alignment and intrinsic dimension, and attention-head function — across four GPT-2 and two Qwen2.5 checkpoints. Each perturbation type leaves a distinguishable metric profile that output measures alone do not capture and that is only partly consistent across checkpoints; copying scores track activation-patching recovery under substitution and shuffling, and gradient-guided HotFlip edits disrupt behavior and representations more than rate-matched random substitutions, so robustness claims resting on any single behavioral or representational metric can be misleading.

arXiv:2609.03322 PDF

From Zero to Hero: An Open LLM Ecosystem for Armenian

Erik Arakelyan, Khatun Avetisyan, Meri Davtyan, Heghine Grigoryan, Nane Khachatryan, Hayk Shahsuvaryan et al. Armenian is morphologically rich and short on pretraining data, and no open Armenian language model had shipped with the data and recipe needed to reproduce it. Two datasets are released: ArmWeb, 4.37M validated Armenian news documents, and ArmSTEM, 373K English-Armenian math and science problems with step-by-step solutions, translated and checked by both model judgment and human review. Continued pretraining of Gemma-4-E4B yields arm-gemma-e4b, which beats every existing open Armenian model as well as its own base, and the ablations show news-only continued pretraining improves fluency while eroding knowledge — a pattern visible in prior Armenian models — with a small share of verified translated STEM data reversing the loss; the authors also find heavy overlap between the largest public Armenian corpora and web-derived evaluation panels, including train/test self-overlap inside FineWeb-2.

arXiv:2609.03350 PDF

Chiaroscuro for Emotions: A Contrastive Emotion Benchmark Grounded in Appraisal Theory

Divyesh Bommana, Mohammad Saim, Tianyu Jiang Emotion recognition datasets usually assign one label per text, which cannot represent the common case where a single event produces opposite emotions in two people, such as a child kicking a seat in excitement while the passenger ahead grows angry. CHIARO is a 1,000-sentence human-annotated benchmark for this contrastive setting, grounded in appraisal theory, where each scene has one causal trigger yielding a positive emotion in one person and a negative one in the other from a ten-class taxonomy. Across seven frontier LLMs and four off-the-shelf emotion classifiers, the best model reaches 67.3 macro-F1, far below human agreement, while the dedicated classifiers score near chance. Used as training data alongside an existing emotion corpus, it also improves a downstream classifier on six of ten external emotion benchmarks.

arXiv:2609.03394 PDF

TabScope: Question-Adaptive Scope Selection for Table Question Answering

Yuxiang Wang, Junhao Gan, Jianzhong Qi Language models get worse at table question answering as tables grow, but the authors observe the drop is uneven: questions that hinge on locating specific cells suffer most from irrelevant content, while questions needing broad evidence can still benefit from seeing the whole table. TabScope builds question-specific sub-tables via operation-aware decomposition and uses a predicted question type to choose between localized and full-table reasoning, and the authors add silver reference sub-tables for scoring evidence selection plus SLQA, a benchmark drawn from real-world long tables. On WikiTQ and SLQA, localization helps most for lookup and local reasoning questions while adaptive switching between the two modes gives the best overall accuracy, indicating that long-table QA depends on deciding when to localize, not only how.

arXiv:2609.03395 PDF

Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations

Yunao Zheng, Bin Wen, Xiaojie Wang Transformers have no native lookup mechanism, so recognizing and reusing recurring local patterns costs dense computation every time. Lngram v2 decouples the number of discrete routes, the memory dimension, and the backbone width — version 1 tied memory capacity to model width — and adds a context-aware grouped-query attention readout plus a zero-value sink and counterfactual surrogate gradients that keep hard discrete addressing trainable. Across vision-language models of several sizes it scales successfully to a 30B-parameter backbone while substantially cutting both total and activated memory parameters relative to v1 at equal or better language modeling quality, and the discrete IDs retain enough of the continuous hidden state's semantic structure that meaning can be recovered from IDs alone.

arXiv:2609.03426 PDF

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Heng Wang, Jielin Qiu, Wenting Zhao, Cheng Qian, Liangwei Yang, Jiawei Han et al. Long chains of thought make the key-value cache a severe memory bottleneck, and every existing compression method follows the same recipe: score each cached token by how much it will matter later, then keep the top scorers. Random Attention computes no score at all — it preserves the prompt and evicts uniformly at random within each attention head — and across four models and six reasoning tasks it matches the strongest prior evictor while serving 32–43% higher throughput in vLLM deployment. Controlled experiments explain why: the prompt is the fragile part of the cache and most of the gap between selectors is just whether their signal happened to protect it, while the reasoning trace protects itself through redundancy both in text (the model restates what it still needs) and across heads (each keeps its own copy), so once the prompt is safe a random draw retains enough.

arXiv:2609.03430 PDF

Decoupled Analysis-Judging: An Automated Creativity Evaluator Using LLMs in Complex Multi-step Creativity Tasks

Xiangyu Wang, Jin Wu, Xiaoyu Li, Chanjin Zheng, Yifeng Zhou Using a language model as a judge for creative output is skewed by verbosity and leniency bias, and the problem worsens on contextually-grounded and procedurally-structured tasks (CGPST), where steps depend on each other, quality is subjective, and scoring ranges are wide. CreaEval splits the judge in two: a skeleton-of-thought model converts a multi-step response into structured evaluation evidence carrying cross-step memory, then a separate judge scores from that evidence without ever seeing the raw response. Across CGPST and two classic simpler creativity tasks it outperforms the second-best baselines by 22.74% on average.

arXiv:2609.03432 PDF

GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

Qiankun Ma, Yanjiang Zhou, Zinan Xiong, Haofei Wang, Zhen Song, Yang Xiang et al. Key-value cache compression schemes fix a per-request budget up front and only decide which cached states to keep, which fits reasoning workloads poorly since different requests need different capacity and a single request's attention demand shifts as it generates. GrowPage treats capacity as a runtime resource: lightweight dual-timescale query summaries capture recent versus long-term attention behaviour, and their relative working sets decide at each capacity boundary whether to compress within the current allocation or acquire an additional physical page, hooking into PagedAttention's page abstraction so continuous batching and prefix caching still function. Across reasoning benchmarks on multiple models, it reports a better quality-versus-throughput trade-off than fixed-budget compression.

arXiv:2609.03494 PDF

Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations

Karthika Nhayakkat, Rajat Verma, Maharaj Brahma, Vetcha Gnana Mahesh, Maunendra Sankar Desarkar, Ganesh Ramakrishnan et al. Relatively free word-order languages allow the same meaning to be expressed with different syntax, and it is unclear whether multilingual reasoning in large language models survives such rewrites. IndicReStruct rebuilds GSM8K math problems in Hindi and Malayalam under two meaning-preserving perturbations — constrained constituent reordering (GSM8K-Reordered) and active-to-passive voice transformation (GSM8K-Voice). Across six current models and several prompting strategies, mathematical reasoning accuracy degrades consistently and significantly on the perturbed inputs; error analysis and residual-stream activation patching trace the failures to disrupted alignment between entities and their quantities, with intermediate transformer layers contributing most to restoring correct reasoning.

arXiv:2609.03511 PDF

What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

Bo Zeng, Yu Zhao, Yefeng Liu, Zhihong Lu, Xuanfan Ni, Xintong Wang Research on decoding-time key-value cache eviction concentrates on better token scoring functions while treating the rule that aggregates scores across decode steps as an implementation detail. Under aggressive compression, exponential-moving-average aggregation turns out to make approximately order-preserving scorer changes nearly indistinguishable at the eviction-set level: value-norm and entropy variants keep almost the same tokens as plain attention, whereas KeyDiff, key norm, recency, and a learned scorer reorder the ranking and lose substantial quality. The paper introduces InertiaKV and its periodic-refresh variant InertiaKV-Lazy, which reaches 1.34–1.46x the decode throughput of full refresh, plus a Score-Free operating point that freezes the ranking after the first decode step for an average quality change of +0.03, evaluated on six open-weight backbones across LongBench, LongBench-v2, and RULER.

arXiv:2609.03515 PDF

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao, Wei-Chih Chen, Hong-Han Shuai Large language models used as peer-review assistants can produce fluent claims that the paper does not support, and existing hallucination benchmarks do not fit this setting because verifying a review claim requires grounding it in a long technical document. HalluPeer supplies aligned triples of paper content, genuine human reviews, and reviews with injected hallucinations, annotated for detection, classification, and localization, built by inducing a review-specific hallucination taxonomy, locating suitable contexts, and injecting errors with automated filtering. Across 12,000 papers and 38,000 reviews, current detectors cannot reliably tell hallucinated statements apart from legitimate critical feedback, and the same hallucination patterns show up in authentic human-written reviews.

arXiv:2609.03580 PDF

The Impact of Synthetic Data Augmentation on Discourse-Pragmatic Function Classification

Sara Sorahi, Kevin Tang, Reza Kazemian Work on synthetic data for class imbalance usually asks how many examples to generate and how varied they are, rather than where they land relative to the real training data. Using 410 manually annotated instances of the English word look from the British National Corpus across four discourse-pragmatic functions, examples generated with Llama 3.1 were partitioned by cosine distance from real data in RoBERTa embedding space and used across six training conditions with the quantity of augmentation held fixed. Every augmented condition beat the real-only baseline, with proximal examples giving the largest macro-F gain (+0.113) and a distance-balanced mix the best accuracy (0.748), but no condition improved AUC, indicating the augmentation shifts the decision boundary rather than sharpening the model's underlying probability estimates.

arXiv:2609.03652 PDF

A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities

Anh Danh, Rick Nouwen, Massimo Poesio Resolving what a pronoun refers to is a basic piece of contextual reasoning, and the authors ask mechanistically how models represent a group of entities well enough to refer back to them with a plural pronoun. Combining attention-pattern analysis with causal interventions, they isolate attention heads that carry coreference information from the input, identify which mentioned entities combine into a single plural referent, and hand that information to the component that selects antecedents and emits the pronoun. The models' preferences line up with human ones: entities are more likely to be referred to collectively when they are ontologically similar and joined by the conjunction and.

arXiv:2609.03687 PDF

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

Kenneth Paulsen, Florian Tambon, Mike Papadakis, Shin Yoo Compact code encoders that power search, classification and retrieval are usually trained either on human-written docstrings, which are inconsistent and labor-intensive, or on mined structural signals such as execution traces, which are expensive and setting-specific. The study replaces both with synthetically generated natural-language descriptions of what code does and intends, contrastively pretraining small dual-encoder models on code-description pairs and discarding the text side at inference so nothing extra is needed at runtime. Across eight retrieval, classification and generation tasks in C, C++ and Java, this beats same-size pretraining baselines significantly on five tasks with parity on two, stays level with execution-aware supervision at matched pretraining data, and once fine-tuned matches or exceeds zero-shot models two orders of magnitude larger on classification.

arXiv:2609.03702 PDF

Free Pause Tokens

John Langford, Nathan Godey, Giovanni Monea, Yoav Artzi, Harry Dong, Ying Fan et al. Pause or thinking tokens buy a language model extra computation per prediction at the cost of occupying real sequence positions. A free pause token instead carries that computation in a parallel prediction stream over a weight-shared backbone that rides an existing position, so it adds no context length, no key-value cache entries, and essentially no latency, since inference floating-point operations are rarely the throughput bottleneck. On a 1-billion-parameter model this improves next-token prediction by 2-3 centinats at equal parameters, tokens, and inference flops, with the only real cost in training, where the overhead versus an optimized pretraining pipeline can be held to about 1.14x while retaining most of the gain.

arXiv:2609.03807 PDF

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

Vineet Kumar, Meghanadh Pulivarthi, vishwajeet kumar, Jaydeep Sen, Riyaz Ahmad Bhat, Sachindra Joshi Retrieval-augmented generation usually splits documents into fixed-length chunks, discarding the global organization of a corpus even though long-context models still lose information buried mid-prompt. STAIR instead feeds a document's table of contents to a language model so that structure is encoded into the model's own parameters, extending the Differentiable Search Index idea of generating document identifiers rather than searching an external index. On SearchTome, a released benchmark built from 18 books across 6 domains, it reaches 82.6% Recall@1 versus 76.9% for plain DSI, a statistically significant gap, and far ahead of BM25 at 59.5%, DPR at 68.7%, and an off-the-shelf Mistral at 13.8%, with a hallucination rate under 0.05%.

arXiv:2609.03874 PDF

CROCODIL: Cross-Model Code Editing with LLMs

Linghan Zhong, Aditya Thimmaiah, Jayanth Srinivasa, Milos Gligoric, Junyi Jessy Li Teams increasingly mix coding assistants, so one model routinely edits code another model wrote — and the authors find that when a model encounters this foreign code, whose style differs because of different training data, it makes more edits than necessary. CROCODIL is a post-training recipe that curbs this with two rewards multiplied together: a similarity reward penalizing large diffs and an execution reward for build and test success, so the policy can only shrink edit size by keeping the task passing. Code is released publicly.

arXiv:2609.03894 PDF

Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating

Heejin Choi Methods for continually updating a model's knowledge are usually declared winners from a single final checkpoint at a single adapter rank, and the authors show that snapshot can invert the true ordering. Holding a periodic hierarchical method fixed and comparing it against cumulative replay over a 24-month Wikidata stream while sweeping evaluation month, replay LoRA rank, and query phrasing, the hierarchy's 5.0-point lead over rank-8 replay on Qwen2.5-1.5B becomes an 11.6-point deficit against rank-72 replay, with the same rank-conditioned reversal reproducing on Llama-3.2-1B and on held-out paraphrases. They propose reporting full trajectories and capacity sweeps and calling a winner only when the ordering is stable across that region — under which the hierarchy is a cheaper operating point rather than a better method.

arXiv:2609.03900 PDF

VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch

WenJie Fan Key-value (KV) cache eviction methods such as H2O and SnapKV rank tokens by the attention they have received, which fails for a long-lived cache that must be compressed before the queries that will read it exist — on a model using no positional encoding (NoPE) with multi-head latent attention (MLA), those methods score 0.00-0.33 on needle retrieval. VestigeKV instead evicts using a query-independent signal already sitting in the cache of Kimi Linear: a 64-dimensional decoupled branch left over from rotary position embedding (RoPE) that NoPE training repurposes as a salience channel, read from 11% of each row, with non-selected rows moved bit-exactly to a GPU-resident archive rather than deleted. Needle retrieval holds at 1.00 under 8x compression and 0.92 under 32x from 8k to 65k context with no training, quantization, or kernel change, shrinking the attended tier to 0.25 KB of the model's 8.1 KB per-token cache. The same operator collapses to 0.08 on a RoPE-based MLA model, and the authors argue query-universal exact merging is provably impossible under rotation, making the trick specific to NoPE architectures.

arXiv:2609.03949 PDF

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute et al. Autoregressive structure forces large language models to emit tokens one at a time, and the usual workarounds either need a separate draft model or trade away output quality. Diffusion-augmented language models keep the autoregressive distribution exactly, splitting parameters into autoregressive weights trained with standard next-token prediction and lightweight diffusion weights that draw several tokens in parallel from that same distribution, learned in a cheap distillation phase and sampled with a family of samplers called Ψ-Spec. The resulting Uno models — trainable from scratch or grafted onto existing open-weight models — beat leading speculative-decoding methods at every batch size tested and reach up to 3x speedup over the base autoregressive model, and the 8B version outscores the 26B DiffusionGemma and the proprietary Mercury 2 across agentic tool use, coding, and long-context reasoning benchmarks, with code and checkpoints released.

arXiv:2609.04010 PDF

Instruction Duplication as an Inference-Time Control Primitive

Victor Lavrenko (PeaceTech VC, Israel) Simply repeating the procedural instruction in a prompt — no retraining, no decoding changes, nothing else altered — is evaluated as a black-box inference-time control across seven instruction-tuned models, 300 medical multiple-choice questions, eight placement conditions, and 16,800 scheduled generations. Going from one copy to two raised the share of responses passing all eight observable format tests from 90.22% to 93.17%, eliminating 30.2% of the failures that remained after a single copy, while final-answer accuracy stayed at exactly 60.21% and premature commitment rose slightly; a blinded challenge audit produced 10 of 30 directional confirmations with no reversals, missing its prespecified 28-of-30 criterion. The effect matters mainly when a downstream system consumes the generated trajectory: in an answer-engineering pipeline that repairs explicit trajectory state, a trailing duplicate lifted one endpoint from 84.2% to 97.1% but lowered a diagnostic branch-preservation score from 78.6% to 73.8%, making the control placement-sensitive rather than uniformly beneficial.

arXiv:2609.04024 PDF

The Dice Roll Method: A Standardized Protocol for Repeated-Query Auditing of Large Language Model Brand Recommendations

Dmitrij \.Zatuchin cross-listed Auditors studying how large language models recommend brands typically fire the same prompt repeatedly, but no standard exists for choosing iteration counts, stability metrics, or reliability thresholds. The Dice Roll Method formalizes this into a protocol grounded in a generative model of temperature-scaled nucleus sampling, decomposing response variance into sampling, prompt-phrasing, run-to-run, and model-version components using a negative-binomial mixed model, Cliff's delta effect sizes, dependence-preserving bootstrap, simulation-based power, and a generalizability-theory analysis with drift diagnostics on pinned snapshots. Reanalysing five prior auditing studies (roughly 190,000 observations, 270+ brands, six languages) yields three iteration tiers — exploratory at 5 runs, confirmatory at 10, rigorous at 15 — and a pre-registered validation on three independent corpora reproduces the reliability predictions in 37 of 39 cells with no failures, though the fixed tiers themselves do not transfer, arguing for piloting and solving per setting rather than reusing the numbers.

arXiv:2609.04047 PDF

When Models Edit Too Much: On the Fidelity of Minimal Code Edits

Tongyao Zhu, Wei Hern Lim, Min-Yen Kan cross-listed A correct code repair can still be a bad one if the model rewrites far more than the bug demanded, leaving a patch that is hard to review and unfaithful to the original implementation. The authors inject controlled abstract syntax tree corruptions into reference solutions for 400 BigCodeBench problems so every task has a known minimal patch, and find over-editing widespread even in frontier models such as GPT-5.5, where high Pass@1 coexists with oversized edits and added cognitive complexity. A simple preservation instruction cuts average excess Levenshtein distance from 0.195 to 0.131, reduces added cognitive complexity by 26.6%, and raises Pass@1 by 2.3 points, and these gains do not follow from more reasoning budget or larger models; when the behavior is instead taught during post-training, supervised fine-tuning overfits to the corruption patterns it saw while reinforcement learning gives the best out-of-domain edit fidelity and performance retention.

arXiv:2609.04061 PDF

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

Sergii Kozyrev, Davyd Maiboroda Hybrid language models interleave softmax attention with linear-attention layers such as Gated DeltaNet (GDN), and early 4-bit quantizations of Qwen3.8-27B left the GDN blocks — especially their decay and write-strength gates — at 8 or 16 bits, on the assumption that errors inside a recurrence accumulate over long contexts. Minima applies NVFP4 W4A4 to all 496 linear layers, GDN included, and matches BF16 within seed noise across perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K at 17.5 GiB with 14-19% faster prefill. A four-part mechanism study credits NVFP4's 16-element block scaling for localizing outliers in the residual stream, finds the supposedly fragile gate projections least sensitive (roughly 11% GEMM error compresses to about 2% output error), and shows the delta-rule recurrence holds injected noise at a flat plateau because each write overwrites state along the current key direction. The work also repairs a global-scale mismatch that appears when per-module-calibrated checkpoints are served by kernels that fuse those modules.

arXiv:2609.04098 PDF

Hardware-Aware FP4 FlashAttention-4

Robert Hu Blackwell's 4-bit floating-point (FP4) tensor cores do not make attention faster on their own, because once the matrix products shrink, softmax conversion and on-chip dependencies dominate. Direct-P maps attention scores straight to FP4 probabilities for noncausal inference, while a separate causal path passes the forward quantization into the backward pass, reconstructing probabilities from saved quantized queries and keys and using 8-bit floating-point (FP8) gradient operands. Direct-P reaches up to 2.13x the bfloat16 forward throughput on an NVIDIA GB200, and the causal path accelerates a complete single-GPU 8-billion-parameter update by up to 1.14x; matched distributed training had to retain FP8 probabilities and values, since every MXFP4 trajectory tested diverged.

arXiv:2609.04105 PDF

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

Zixuan Fu, Bingxiang He, Yuxin Zuo, Haohuan Huang, Jinqian Zhang, Ruhang Xiao et al. On-policy distillation (OPD) trains a student on its own rollouts using dense token-level supervision from a teacher, but the role of the training data itself has gone largely unexamined. Training on a single query keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task domains and model families: one query already reaches 71.5% of the states that full-data training visits, most of it within the first 100 steps, and 16 semantically distinct queries reach 98.9% and match full-data results, including in the multi-teacher setting. Because the student aligns with the teacher at a similar slow rate whether trained on one query or the whole dataset, the authors describe OPD as data-overfed but algorithm-starved; content-light templates and off-domain WildChat queries also approach the real-query baseline, showing task content and induced state coverage can come apart.

arXiv:2609.04172 PDF

Last Translation Benchmark

Vil\'em Zouhar, Niyati Bafna, Mukund Choudhary, Maike Z\"ufle, Sara Rajaee, Pinzhen Chen et al. Machine translation benchmarks are approaching saturation while automatic metrics remain vulnerable to reward hacking and give unactionable verdicts, and gold human evaluation lacks reproducibility and scale. The Last Translation Benchmark is a collection of human-authored, peer-reviewed examples spanning text, images, audio, and video, selected because they break leading machine translation models, with each example carrying handcrafted verification rules that name the concrete failure it should expose. It is a live dataset accepting ongoing contributions; LTBv1 contains submissions accepted before September 2026, with further releases planned as data accumulates.

arXiv:2609.04173 PDF

Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views

Joseph Lee, Yidi Huang, Dokyoon Kim, Shu Yang, Li Shen How knowledge actually gets absorbed during pre-training remains poorly understood; the hypothesis tested here is that auxiliary views — reformulations of the same knowledge — causally help learning. Controlled experiments confirm that repetition is necessary and that paraphrasing helps only at smaller batch sizes, then show that with the token budget held fixed, reallocating tokens from document repetition to auxiliary views improves learning even for factual recall. The benefit does not depend on the strength of the teacher model generating the views, and contextual versus foundational reformulations help differently when prior knowledge is missing; layer-wise analyses of biases and compression trace the mechanism, offering an explanation for why corpus diversity matters.

arXiv:2609.04180 PDF

ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize

Lihao Liu, Peng Tang, Kunwar Yashraj Singh, Shabnam Ghadar Evolutionary prompt optimizers such as GEPA append rules and caveats every iteration, producing prompts up to three times longer with no accuracy gain. ESPO (Error-Structured Prompt Optimization) traces the bloat to incomplete error observation, limited search diversity, and unreliable selection, and addresses each with a phase: clustering all training errors into structural patterns in a single round, generating candidates through four complementary strategies with independent biases, and choosing among them by bootstrap stability selection. Across seven benchmarks including MMLU, GSM8K, and HotpotQA, it reports +3.76 percentage points average accuracy over GEPA while producing prompts 47% shorter, holds the best average across four additional student models such as Qwen3 32B and Claude Haiku 4.5, and an ablation confirms that adding diversity without stability selection actually hurts.

arXiv:2609.04197 PDF

Clean Engineering, Unstable Measurement: A Preregistered Reliability Failure of Black-Box LLM Observers on Shared Endpoints

Haoyaun Zhu, Jie Zhang Language-model judges now gate training data and drive leaderboards, resting on a rarely stated assumption: the same request sent to the same model name reads the same tomorrow. Two preregistered campaigns covering 52,988 audited request attempts, with every threshold fixed in advance, never got past validating that instrument — same-window repeat rankings agreed at Spearman 0.400 against a required 0.90, and byte-identical next-day replays at 0.78 against a required 0.99. Waiting, substituting metrics, resampling, and switching among four providers did not repair the noise floor, and self-hosting on batch-invariant kernels helped only while the server was quiet. The authors distill a three-level snapshot-identity ladder, eight design rules, and a reporting checklist, observing that a pilot at roughly 2% of the study's call volume would have exposed both unreachable gates in advance.

arXiv:2609.04198 PDF

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Yuntian Deng, Pengyu Nie, Stuart Shieber Recurring text-transformation tasks are often easy to describe in words but awkward to implement with rules, while calling a large remote model on every input adds recurring cost, latency, and provider dependency. Compile by training turns a natural-language specification into a reusable neural function: at compile time, teacher models generate task-specific examples that train a small adapter over a compact interpreter, after which the function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, where the Program-as-Weights fast compiler produced no exact matches, the approach reaches 83.6% semantic accuracy, at a compile cost of about a minute rather than seconds; the authors deploy it as a public interactive service with demos including a multi-site website helper and a language-controlled 3D avatar.

arXiv:2609.04199 PDF

7 more specialized papers

Agents 43

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

Michael Nguyen, Wei Chen Tan, Nurul Aisyah Hassan, Arvind Raman, Li Hua Lim, Ahmad Faiz Razak Prompt-evolution methods improve frozen large language model (LLM) agents by rewriting their harness, the textual scaffolding of persona, strategy, format rules, and control heuristics, but they usually treat it as one flat string. HARNESSEVO splits the harness into four separately evolvable slots and uses leave-one-in and leave-one-out attribution to locate where the gains come from. On ALFWorld with a frozen 7B backbone the decomposition gives no overall gain (0.657 versus 0.642 for both the stock harness and flat-string evolution), but nearly all the optimization value sits in the reflection/control slot, worth +0.119 on its own, and spreading 64 rollouts evenly across four slots starves each below the optimizer's search floor so every slot freezes at its empty seed; concentrating half that budget on the control slot reaches 0.761. On WebShop all slots freeze and all methods tie, which the authors read as an absence of recurrent verbalizable control failures rather than budget starvation.

arXiv:2609.02889 PDF

Bounded Personas Match Retrieval on Classification but Not Regression for a Frozen Agent

JaeHa Yoon, Minjun Park, Seoyeon Kim, Jiwoo Lee, Hyunwoo Choi, Dohyun Kang Personalizing a language agent to a user means either retrieving past items into the prompt at each query, which is accurate but scales in cost with history length, or distilling the history once into a compact persona, which is bounded and interpretable but assumed to be less accurate. PersonaLink is a training-free distillation method that compresses a user's history into a bounded three-field persona and recursively refines it, each pass scoring the frozen agent on a held-out slice of that user's own labeled history, rewriting the persona from its errors, and keeping the rewrite only if it does not regress. Holding one frozen 7B backbone constant so only the in-context representation varies, the method reaches 0.745-0.755 accuracy on 200 users of LaMP-2 news categorization, statistically indistinguishable from BM25 retrieval at 0.760-0.765, while the authors report the parity does not carry over to regression-style tasks.

arXiv:2609.02890 PDF

Counterexamples as Feedback for Agent Self-Correction

Sidhesh Badrinarayan, Adithya Parthasarathy Single-turn code-generation scores say little about whether a deployed agent can actually repair a wrong artifact once it receives concrete feedback. A-CEGIS evaluates multi-turn refinement on natural-language-to-regex synthesis: the agent proposes a regular expression, a deterministic oracle checks it under full-match semantics, and compact false-positive or false-negative witnesses are handed back as the next turn's feedback. On 30 NL-RX-Turk tasks, diagnostic counterexample feedback solves 90% of tasks within four turns versus 17% zero-shot, 27% for generic self-correction, and 23% for error-only feedback, with a full run solving every hidden-set task by the final turn at a mean of 2.7 turns and 77% robust success after targeted probing.

arXiv:2609.02892 PDF

LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games

Hrithika Deepu Nair, Kayvan Karim cross-listed Scripted game opponents become predictable once players learn them, while a trained reinforcement learning policy keeps the same behavior against every opponent. The setup tested here leaves the policy untouched and adds a runtime strategy selector: five non-player characters share a PPO policy in Unity, and a locally hosted Mistral 7B served through Ollama reads live game state every five seconds and assigns one of four tactical tags, evaluated over 600 episodes against three scripted opponent types. Against a Balanced opponent that switches tactics mid-episode, the win rate more than doubled from 11% to 24%, but the guidance backfired against an Aggressive opponent, and 83.8% of 2,430 strategy choices were the same 'Surround' tag regardless of opponent, showing little zero-shot strategic differentiation at this model size.

arXiv:2609.02931 PDF

Reflect-SQL: A Self-Reflection Based Framework for Text-to-SQL

Anupreksha Jain, Manish Shrivastava cross-listed Text-to-SQL stumbles in real enterprises on cryptically named and oversized schemas, poor retrieval of the relevant tables and columns from vague questions, and generated SQL that is never validated. Reflect-SQL wraps each stage in a feedback loop scored by a language model acting as judge: one loop rewrites the user's question to improve retrieval, a synthesis loop validates and repairs the SQL, and an entailment loop tunes the end-to-end process while enriching a schema knowledge base. On the BIRD benchmark the framework reports 72.03% execution accuracy, ahead of the state-of-the-art baselines compared.

arXiv:2609.02944 PDF

MemoryLACE: Memory Lifecycle-Aware Consolidation and Evidence Retrieval

Meriem Yacoubi, Pia Schmidt, Nenad Petrovic, Ahmed Frikha, Martin Kirchhoff, Alois Knoll Long-running LLM agents need memory that distinguishes repeated evidence, superseded facts, updates, and open contradictions, but flat retrieval leaves those relations implicit while graph or reflection-based systems pay heavy overhead. MemoryLACE keeps atomic natural-language memories with provenance and adds sparse merge, supersession, and contradiction links, then assembles relation-aware evidence units at retrieval time that surface current, historical, supporting, and conflicting items together. On BEAM and StructMemEval across open-weight and proprietary backbones it leads same-backbone comparisons while cutting end-to-end runtime on BEAM by 66.6% versus the reflective-memory baseline Hindsight, with ablations crediting lifecycle expansion and temporal awareness.

arXiv:2609.03201 PDF

MasterControl Seventeen Every Time

MasterControl AI Lab Compares two designs for enterprise analytics over data: letting a model plan at runtime by writing SQL and picking tools, versus having the model only interpret intent while deterministic policy selects and runs a pre-approved analytical program that returns results plus supporting evidence. The restricted class still covers relational operations with aggregation, comparison, windows, ranking, and similarity, and fixing meaning, policy, data, and execution rules makes runs replayable. Across 440 runs, none of the 330 runtime-planning episodes from three 8B models satisfied the full answer-and-evidence contract, while the policy-executed analyzer satisfied all 110, a result the authors scope to this specific configuration rather than to runtime agents in general.

arXiv:2609.03209 PDF

Speculative Macro Commit for Faster Tool-Using Agents

Zeyu Liu, Souvik Kundu, Peter A. Beerel Tool-using agents lose wall-clock time to the strictly serial loop of action, environment transition, and observation, on top of model inference. Speculative Macro Commit runs a small drafter model alongside the large authoritative actor, executing predicted multi-action chains against an isolated environment snapshot; recurring multi-action skeletons mined from training traces form a macro library, and when the actor's next call matches the draft's first action, the remaining pre-executed steps and their observations are committed wholesale. With Qwen3.5-27B INT4 as actor and Qwen3.5-4B as drafter, it matches sequential accuracy while cutting latency 18.59% over sequential execution on the τ²-Bench Telecom subset and 44.9% of wall time on AppWorld, with a small drop in task completion there.

arXiv:2609.03236 PDF

PACE: Towards Surfacing Hidden Conflicts in User Requests

Yoojin Kim, Jihyoung Jang, Hyounghun Kim Personal assistants are usually evaluated on how faithfully they execute a request, not on whether the request makes sense given facts about the user that are never stated in the prompt. PACE (Personalized Assistants for Conflict Evaluation) pairs persona-grounded user requests with egocentric knowledge-base facts, so a model must retrieve latent constraints — a health condition, a past event — that make an otherwise reasonable request inappropriate, and the implicit nature of the link defeats direct request-to-fact matching. The companion PaceMaker framework splits the job across specialized agents for query reformulation, multi-hop graph traversal, and conflict-aware filtering, and outperforms existing approaches on both evidence retrieval quality and conflict decision accuracy.

arXiv:2609.03293 PDF

Fresh Memory, Stale Plans: Dependency-Scoped Validation for Distributed LLM-Agent Memory

Evan Chen, Shiqiang Wang, Christopher G. Brinton In distributed teams of language-model agents, every member can hold the freshest shared facts and still act on a plan derived from a superseded requirement — a planner works from requirement r₃, another agent commits r₄, and the executor receives r₄ without discarding the stale plan. PlanFence addresses this stale-plan execution by having plans cite the exact public records they used, so an executor validates only the records that could affect the pending external action, then replans once or blocks if validation is incomplete. Across 30 controlled live workflows containing a post-plan revision, a freshness-only executor acted on the obsolete plan in every task while PlanFence completed all tasks with no invalid action; replay shows proactive synchronization is cheaper at low churn, whereas dependency scoping wins as churn and shared keyspace grow.

arXiv:2609.03340 PDF

Accountable AI with Grounded, Faithful, Consistent, Actionable Rationales: A Case Study in Clinical Trial Matching with VERDICT

Zikai Zhou, Yufei Jin, Yilin Xu, Yu-Chiang Wang, Chieh-Ju Chao, Monica S. Lam Accountability for a language model decision requires rationales that are actually faithful to how the decision was reached, consistent policy application, and explicit statements of what was assumed and what would flip the outcome; the authors operationalize this as self-faithfulness, meaning that changing the pivotal conditions should change the decision. Studying clinical trial matching, they find that LLM-based matchers are reasonably accurate yet apply their own decision policies inconsistently and produce rationales unfaithful to their decisions. VERDICT is an agent that translates the task, its constraints, and its policy into Satisfiability Modulo Theories and derives the decision with SMT and MaxSMT solvers. On a SIGIR 2016-derived dataset and TREC 2021 it reports the best accuracy among LLM-only and neurosymbolic baselines, applies policies with perfect consistency, and yields clinician-preferred rationales with better counterfactual self-faithfulness.

arXiv:2609.03366 PDF

StrixAE: An Intelligent Agent for Audio Enhancement under Complex Distortion Coupling in Real-World Scenarios

Chenglin Wu, Junjie Wu, Jinhang Chen, Mingyang Chen, Zixu Lin, Jiabian Chen et al. cross-listed Real-world audio is degraded by several coupled distortions at once and users want enhancement tuned to their own preferences, which single fixed models handle poorly. StrixAE puts a multimodal large language model in the controller role, planning and invoking a set of enhancement and personalization tools, and trains it in two stages: chain-of-thought supervised fine-tuning on AcoustBench to ground basic reasoning and tool calls, then Audio Perception Reinforcement Learning, whose rewards jointly score output format validity, structural coherence of the pipeline, and perceptual quality. The structured rewards are designed to force executable pipelines in a valid section order, and the authors report that the agent produces enhancement plans without hallucinated tools while beating most open-source and proprietary systems on perceptual metrics for real-world test data.

arXiv:2609.03414 PDF

Dude: A Dual-Detection Multi-Agent System for Paper-Code Discrepancy Detection

Weijie Liu, Running Zhao, Wenhao Yuan, Jinfeng Xu, Zhanfeng Xu, Xiaoxi Zhang et al. Checking whether a paper's claims match its released code is increasingly beyond manual review capacity, but single-agent LLM approaches run out of context and inspect the discrepancy from only one direction, so they miss cases. Dude uses a dual-detection multi-agent design and identifies a specific failure source: paper prose and code operate at different granularities, which pushes agents toward over-interpretation and over-reporting and inflates false positives. Granularity-aligned negotiation between agents and a two-stage salience filter are introduced to suppress those spurious reports. On real-world paper-code discrepancy datasets, the system improves recall and precision by up to 22.8% and F1 by up to 18.7% over baselines.

arXiv:2609.03416 PDF

The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systems

Guangjun Liu cross-listed Humans currently act as the transport layer between AI systems, losing context at every hop. The proposed design makes the addressable party a 'civilization' — one human sovereign, a persistent ledger, and interchangeable agents — with a carrier-agnostic Embassy Protocol in which messages arrive asynchronously at a ledger endpoint, any online agent handles them, and matching commitment state rather than delivery is ground truth; an agent's authority is capped by the memory it can reach and externalized through signed credentials. A preregistered 1,908-trial experiment on one frontier model probes a 'temporal-weight effect' in which whatever arrives first gains unearned authority: with verification removed, an incorrect upstream claim arriving first captured 54.2% of answers versus 4.2% under full verification, and 31.6% when it arrived after the receiver had sealed its own answer. A failed tool-use budget check means the registration classifies the round inconclusive, so all results are reported as exploratory pending a replication.

arXiv:2609.03425 PDF

Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents

Zhaoyuan Huang, Tianjie Ju, Pengzhou Cheng, Zheng Wu, Yansi Li, Chuanbiao Song et al. Agents that carry out natural-language instructions on graphical user interfaces need to recognize when an instruction should not be executed at all, since users issue infeasible requests by honest mistake. CONFLICTGUI benchmarks that ability with two conflict types — contradictions inside the instruction itself, and contradictions between the instruction and what is actually on screen — and reveals severe execution-biased overcompliance, where agents that score well on feasible tasks keep clicking blindly under conflicting instructions. CONFLICTGUARD addresses this at inference time with a feasibility verification protocol that forces the agent to weigh instruction logic and on-screen evidence before acting, plus conditional action modulation that steers it toward termination; across five widely used agents it raises average conflict-task success while preserving normal GUI performance.

arXiv:2609.03438 PDF

Plan Pointers and Record-Directive Form in Budgeted Verification of Inherited Agent Memory

Kazuki Nakayashiki cross-listed When an agent inherits a handful of one-line memories and can afford to pull at most one archived source record before acting, the directive text stored alongside those memories steers which record it fetches. Twelve registered studies totaling 14,760 attempts compare directive forms — a bare record id, a length-matched criterion describing the record, or both together. A criterion beat a bare id by 35.0 points across six direct-provider models, though the same contrast failed its registered superiority rule on a nine-model OpenRouter panel; appending the id to the criterion cancelled its effect entirely on three Claude models (40 of 40 down to 0 of 40 on Opus 5), and a ratification line plus a two-fetch budget restored the target on all three. Everything is reported as descriptive effects of exact string edits on fixed model panels, with no mechanism claim.

arXiv:2609.03450 PDF

When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

Wen-Yu Chang, Yun-Nung Chen Memory systems for long-horizon conversational agents are usually measured with question-answering probes, which is not how memory actually gets used mid-conversation. LOCOMO-CONV reworks the LoCoMo dataset into four conversational query styles — dialog, implicit, counterfactual, and composed — and evaluates five representative memory systems on both retrieval recall and end-to-end response quality. Conversational framing exposes substantial retrieval gaps that QA benchmarks miss, worst on implicit and composed queries, which multi-facet query rewriting narrows for raw-turn memory but not for abstractive memory; strong retrieval also fails to translate fully into response quality, and implicit queries show 'silent grounding' where memory improves contextual grounding without surfacing the gold fact. The authors release supplementary annotations marking conversationally useful context beyond the original gold evidence.

arXiv:2609.03467 PDF

Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Models

Xingming Long, Yu Liu, Zhiwei Yang, Hanqi Feng, Shaojie Zhang, Barnabas Poczos et al. Agentic vision-language models that call tools such as image cropping, image search, and text search are usually trained only against final-answer correctness, so they issue redundant or off-target calls and often fail to pull the needed information out of what a tool returns. The NTEP (Necessary Tool-Evidence Path) annotation scheme records, per query, exactly which external evidence and tool calls are required, and the NTEP-R reward checks that a call's pre-invocation intent matches a necessary evidence goal and that the summary of the returned observation actually contains that evidence, with a regularizer penalizing repeat calls against already-satisfied goals. Across seven image-grounded benchmarks under a unified three-tool setup, the resulting 8B-parameter NTEP-8B improves both search-oriented accuracy and tool-use efficiency.

arXiv:2609.03493 PDF

Dalek: A Constructive Agent Machine

Wanpeng Xie Dalek is a proposed machine architecture for agents that can maintain, evolve, reproduce, and organize themselves on any host meeting a general contract, built from just three primitives: actors, messages, and channels. Four structural obligations, a host boundary, a construction language, admissible transitions, and rule heredity, give the system its identity and closure, while von Neumann's 1948 self-reproducing automaton supplies the hereditary core of self-description, constructor, copier, and controller. A large language model paired with a compiler occupies the payload position as a general capability producer, so new capabilities are authored, compiled, installed into the self-description, and inherited by descendants, a path that the authors extend to generating the machine's own organs and runtime.

arXiv:2609.03546 PDF

GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

Linh Le, Melanie Bui, My Chiffon Nguyen, Zachary Schlosser, David Williams-King Language-model simulations of policy outcomes are hard to validate because plausible-sounding agent behavior is rarely checked against what actually happened. GPS-Bench grounds the task in evidence by linking policies to affected actors, their actions, and downstream impacts using legislative records, lobbying disclosures, regulatory filings, corporate reports, and economic data, reconstructing each persona from the dated record rather than prompting an archetype; human-annotated cases form the gold test set while separately LLM-labelled cases serve only as silver supervision. Because every inference mode reads the same grounded state and emits the same schema, the benchmark cleanly compares joint reasoning, independent and communicating actor agents, graph methods, and fine-tuning, finding that weight-level fine-tuning on the grounded record gives the strongest actor-level impact prediction and multi-agent decomposition does not beat it. What decomposition contributes instead is mechanism: agents holding private, non-identical evidence form checkable coalitions with explicit offers and asks.

arXiv:2609.03553 PDF

KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents

Yaxing Lyu, Shengjie Zhou, Binbin Toh, Pengyu Zhu, Lijun Li An agent calling tools has to reconcile three sources that can disagree: what the user says, what the model already believes, and what the environment reports back. KC-Bench measures that reconciliation over multi-turn interactions covering world-knowledge conflicts, inconsistent inputs, and temporal conflicts across sources, with 238 tasks hand-screened from over 1,000 candidates and backed by a user simulator, stateful tools, deterministic environment assertions, an open-source natural-language evaluator, and human-verified trajectories. Across nine models including DeepSeek-V4-Flash, GLM-5.2, and MiniMax-M3, none handled factual correction, identity consistency checking, and temporal conflict resolution reliably in all settings, and missed conflicts propagated into tool calls and simulated protected-data flows. The benchmark deliberately isolates model behavior rather than scoring whole agent frameworks.

arXiv:2609.03588 PDF

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Xuanfa Jin, Zhijian Ma, Yongcheng Zeng, Xinyu Cui, Haifeng Zhang, Jun Wang Multi-agent debate improves large language model reasoning by having several agents critique and revise each other, but it backfires when most agents start out agreeing on a wrong answer — a failure the authors call shared misconception, which debate amplifies rather than corrects. R^2-MAD equips each agent with a memory of past debates: a retrieval policy that adapts to the current consensus level pulls in relevant historical evidence to recalibrate an agent's prior, and those retrieved experiences also serve as the basis for estimating each agent's reliability, producing confidence weights that modulate how much peers influence one another. Across several reasoning benchmarks the framework reports consistent gains over both single-agent and existing multi-agent debate baselines.

arXiv:2609.03619 PDF

A computable representation of the physical laboratory enables verifiable workflows

Xiaobo Li, Luyao Ge, Xiaohui Li, Lulu Guo, Ming Mao, Jiwang Zheng et al. Automating science end to end requires a machine representation of the physical laboratory, not just of scientific knowledge, so that an agent's plans can be checked against what the equipment can actually do. The proposed formalism models samples and materials as typed research objects, instrument abilities as capability-bound operations, and experiments as programs in a compositional workflow algebra with explicit dependencies, decisions, iteration and concurrency. It was implemented in a modular agentic robotic laboratory by binding formal operations to executable Function Skills, generating capability-relative workflows from stated scientific intents and using a stateful simulation to verify operation preconditions and laboratory constraints before anything is dispatched to hardware.

arXiv:2609.03621 PDF

What Do CAE Simulation Agents Really Need Beyond a Generic Harness?

Jiasheng Shi, Tianhan Zhang cross-listed Setting up computer-aided engineering (CAE) solvers such as OpenFOAM, FEniCS, or COMSOL demands expertise, and recent large language model agents for the task add multi-agent decomposition, domain retrieval, and scripted reflection on top of the base model. Holding information access and repair budget fixed, the authors compare that machinery against a plain single-agent harness that already supplies multi-turn reasoning, tool use, and execution feedback: the generic single agent reaches 96.4% on FoamBench versus 88.2% for specialised multi-agent systems. Ablations attribute the result to execution-feedback repair, which lifts accuracy from 71.8% with no repair round to 96.4%, while scripted reflection contributes nothing; the one genuinely useful addition is domain knowledge in the form of solver tutorials, worth a jump from 80.9% to 96.4%.

arXiv:2609.03718 PDF

Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation

Yan Tang, Tingyu Cao, Yuanbo Tang, Huaze Tang, Keer Hu Most large language model agents wait for an explicit instruction, whereas proactive service requires inferring an opportunity from partial signals and choosing among staying silent, asking, assisting, and acting while weighing interruption, misunderstanding, overreach, and privacy costs. The survey defines proactivity in terms of initiative and casts it as a partially observable sequential decision process constrained by authorization and risk, with timing, content, and delivery bundled into one structured action so the option value of waiting and the decision value of asking become explicit. Existing methods are organised along a single pipeline covering state and need estimation, intervention gating, action construction, and feedback adaptation, and metrics are formalised for triggering, timing, calibration, user burden, safety, and policy value. Two claims stand out: offline classification accuracy does not predict deployment benefit, and long-term memory is not a defining condition of proactivity.

arXiv:2609.03727 PDF

SimSkill: A Lifelong Learning AI Agent for Autonomous Mastery of Traffic Simulation

Qi Liu, Qinzheng Wang, Yiming Bie Agents typically solve one request at a time rather than converting experience into reusable competence, which SimSkill targets in the setting of the SUMO urban traffic simulator. The agent finds its own capability gaps, generates and solves grounded tasks, checks solutions with an action-critic loop, and files what it learns into episodic, procedural, and semantic memory, building a skill library covering the simulation workflow without ever updating the backbone model's weights. Across two held-out benchmarks, three backbone models, and independent artifact-based verification, it raises verified task completion by up to 25 percentage points, with procedural and semantic memory contributing separately; the authors note the gains are backbone- and budget-dependent and that memory neither helps every model nor reliably cuts inference cost.

arXiv:2609.03753 PDF

DNative-Twin: Decision Graphs and Digital Twins for Reconstructable Agentic Decisions

Junjie Pang, Zhenzhen Xie, Haoke Han, Ying He, Jing Wang, Gang Liu When an AI agent gathers evidence, calls tools, and commits to an action, the final output alone does not reveal which evidence, tool state, rule, or authorization produced it. DNative-Twin records a committed decision as a typed trajectory in a graph that links observed state, the path followed, and the authority behind the action, then re-executes the decision mechanism in isolation under declared conditions so outcomes can be compared under controlled changes. Across three public enterprise process logs and a three-condition experiment with 300 injected instances, recall of unresolved divergences rose from 0 to 0.667 once replay-contract state was recorded and to 1.0 when verification results were also available, showing that graph structure alone cannot determine the consequence of an unobserved tool state. Median end-to-end time grew from 0.794 to 8.889 seconds over 500-5,000 BPI 2020 cases.

arXiv:2609.03787 PDF

Adapting to Evolving Requirements: Agentic AI for Retail Supply Chain Operations

Lei Zheng, Liping Yang, Zihao Li, Guodong Lyu, Chaik Ming Koh, Chung-Piaw Teo Retail supply chains chain together several optimization modules, so a single new business requirement can be satisfied by changing any of a number of them, with different downstream consequences. The authors cast requirement-driven adaptation as jointly choosing an intervention route through a graph of coupled modules and an admissible edit within the chosen module, with domain agents exposing reformulation interfaces and a central processor searching bounded paths whose candidates are scored by downstream key performance indicators. Evaluated on 100 warehouse requirements gathered from practitioner interviews at a large retail partner, the framework raises end-to-end success from 72–76% to 79–83% over direct reformulation by the language model, holding across GPT, Qwen, and DeepSeek backends.

arXiv:2609.03860 PDF

Bioinfoysis Technical Report

Qingyang Shao, Xin Zhang, Zhouyang Yuan, Xianying Chen, Yujia Xiang, Zihao Yang et al. Bioinformatics agents typically optimize for a final answer and discard the planning, tool calls, and intermediate files that produced it, which breaks down on long analyses where a conclusion has to stay traceable to the data behind it. Bioinfoysis models each request as a persistent, artifact-grounded run: a planner keeps an executable checklist and revises pending steps from structured handoffs that bind every intermediate result to the agent, checklist step, and plan generation that produced it, so stale evidence cannot be silently reused after a replan, and a controlled runtime validates generated scripts, tables, and figures before downstream use. It reports state-of-the-art 82.4% accuracy on BixBench, and across four backing language models lifts average accuracy from 27.81% to 64.13% on SeqQA2 and from 3.13% to 31.25% on DbQA2, which the authors read as evidence that the surrounding harness matters as much as raw model capability.

arXiv:2609.03871 PDF

RuleMem: Active Rule Memory for Long-Term Conversational Agents

Xingyuan Zeng, Zuohan Wu, Quanming Yao, Yue Wang, Wei Liu, Libin Zheng et al. Conversational agents that must answer questions spanning months of dialogue typically store the past as passive facts, which leaves semantic gaps when the relevant evidence is phrased nothing like the question. RuleMem instead induces reusable natural-language Horn clauses from prior conversations, filters them through a Rule Perplexity Consistency check, and uses the surviving rules both to retrieve semantically distant evidence and to give answer generation an explicit logical scaffold. Against 14 baselines on LoCoMo it takes the top accuracy, beating the baseline average by 27.47 points, a 54.3% relative gain, with further evaluation on LongMemEval_s*.

arXiv:2609.03915 PDF

Value-Preserving Architectures for Agentic AI Systems

Alessandro Pesare, Tommaso Dolci, Katja Hose, Emanuel Sallinger Architectural decisions in multi-agent systems built on language models — how agents coordinate, what protocol they speak, what topology connects them — shape not only performance but whether properties like privacy, pluralism, and fairness survive contact with the system. The authors argue these human-centered values should be treated as first-class architectural concerns rather than post-hoc checks, and set out three candidate patterns: a federated topology for privacy, a distributed architecture to preserve diversity of viewpoints, and a guard-agent pattern that detects and mitigates unfair outcomes. Representative use cases illustrate each, framed as groundwork toward a catalog of architectural guidelines for trustworthy multi-agent systems.

arXiv:2609.03920 PDF

Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting

Muneeb Khan, Frederic Kirstein, Terry Ruas, Bela Gipp Delegating an absent participant's role in an online meeting to an LLM agent breaks down at the simplest step — knowing when to speak — with prompt-only delegates staying silent on 51.4% of the absent person's talking opportunities in the AMI meeting corpus. CAPA (Collaborative Agent Predictive Architecture) splits the task across modules: a Perceiver that maintains an explicit meeting state of stances, coverage, and who holds the floor, a Predictor that forecasts the next turn, a Controller that decides whether and what to contribute, a Generator that phrases it in the participant's style, and judges plus a Recalibrator that correct the state against what actually happened. Across 137 AMI meetings the silence rate falls to 2.5%, credited recovery of the participant's real idea units roughly doubles from 26.1 to 52.2, and hallucinated contributions stay at 0.6%. Ablations attribute the gain to the structured meeting state rather than to feeding the model more raw context, and the accompanying schema-constrained LLM judges agree with human annotators at Cohen's kappa 0.71.

arXiv:2609.03923 PDF

Interface-Induced Trajectory Censoring

Wenbo Wang The tool-call rate reported by an agent evaluation is read off the serving stack, and that number can be zero while the model is emitting perfectly well-formed calls that the chat template and parser silently discard before anything downstream sees them. Holding weights, cases, decoding, and seeds fixed and swapping only the serving adapter, the same model scores 0.00 or 0.96 on BFCL v4, and a 2x2 over template and parser shows both main effects are exactly zero with the entire effect in the interaction — meaning neither component is broken and fixing one alone buys nothing. On tau-bench's 115 interactive retail tasks the same swap moves server-parsed calls from 0 to 636; across a 21x scale range of Qwen2.5-Coder the server parses 0 of 100 at every size while well-formed emitted calls climb to 80 of 100 at 32B, and inside verl's training loop at 7B, 45 of 115 generations carry a complete call and none are accepted or executed. Repairing the adapter at evaluation time restores the mechanism but yields no significant pass-rate gain (53 to 62), and the authors release a 98-line preflight check that catches every silent failure observed.

arXiv:2609.03966 PDF

Editable Visual Design

Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li, Zhutao Lv et al. cross-listed Image generation models like GPT-Image-2 and Nano-Banana produce visually striking designs but flatten everything into a bitmap with error-prone text and no editable layers, while coding agents that emit layout code preserve layers and real text yet lack global aesthetic intuition and struggle to draw complex visual assets. The proposed paradigm splits the roles: a vision-language model serves as the creative brain for requirement comprehension, planning, and aesthetic judgement, calling an image model on demand as a 'visual world simulator' to synthesize standalone assets, then writing native HTML and CSS and iteratively refining it against rendered visual feedback in an imagine-first-then-act loop. The output is a genuinely editable artifact with decoupled layers and real text that users can drag and rearrange in a graphical interface, plus an Agent Design Replay mode that reproduces the creative and reasoning trajectory; validation covers posters, infographics, and similar scenarios.

arXiv:2609.04034 PDF

PatchBench: Evaluating AI Agents for Vulnerability Patching

Chihao Shen, Jiacheng Li, Aastha Mahajan, Jeffery Siyuan Tian, Yonghwi Kwon, Yizheng Chen cross-listed Evaluations of automated vulnerability patching typically check only whether the proof-of-concept input still crashes, which lets agents pass either by reproducing the historical developer fix from memory or by suppressing the crash along the reported stack trace. A patch-similarity metric shows that on average 25% of agent patches closely resemble the original developer patch, and agents frequently patch on the crash stack rather than localizing the root cause. PatchBench counters both problems by selecting C/C++ vulnerabilities whose true fixes lie outside the crash stack, transplanting historical vulnerabilities into new repository contexts with code mutations, and validating both security and semantic correctness. Across 11 agents including the top three AIxCC entrants, proof-of-concept-only validation inflates measured solve rates by 1.83x on average.

arXiv:2609.04075 PDF

DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training

Shubham Gandhi, Saurabh Goyal, Kiran Kate, Yara Rizk Reinforcement learning from verifiable rewards requires a programmatic checker, which most long-horizon agent domains lack; multi-criteria rubrics can substitute, but they produce a single scalar for a trajectory spanning tens of steps. DRACO generates rubrics during training so they track the policy's evolving ability, scores each completed trajectory against them, and redistributes that judgment in closed form onto the steps responsible for each annotated rubric item, producing differentiated per-step advantages for GRPO with no trained attribution module. On AppWorld it gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, while using no verifier itself, and adds 5.3 points on out-of-domain Tau-Bench.

arXiv:2609.04094 PDF

Environment Evolution for Terminal Agents

Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu et al. Training terminal agents requires interactive, verifiable environments, but environments synthesized from scratch stop challenging frontier models, and co-evolution methods that build them from weaknesses seen in on-policy rollouts generalize poorly and run out of signal as the model improves. The alternative here raises environment difficulty off-policy along three directions derived from the multi-turn learning objective, scheduling successive generations of evolved environments through a loop-engineered multi-agent harness. Rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol confirm each generation is harder, and simple long-horizon reinforcement learning on the resulting environments improves Qwen3.6-27B and Qwen3.6-35B-A3B by 14.4 and 18.0 percentage points on Terminal-Bench 2.1.

arXiv:2609.04128 PDF

The Natural Language Interaction Protocol and Standard for AI Agents

Luyi Xing, Rasit Onur Topaloglu, Ranjan Sinha, Abhay Ratnaparkhi, Samuel Ndichu, Christopher Nguyen et al. Agents built on different frameworks, models, tool interfaces, and runtimes currently have no common way to talk to one another. The Natural Language Interaction Protocol (NLIP), standardized by Ecma International, defines an application-layer semantic message envelope that rides over existing transports such as HTTP/HTTPS, WebSocket, and AMQP, with gateways that adapt between clients, agents, local context stores, ontologies, tools, enterprise services, and other underlying protocols. The write-up covers the message model and transport bindings, security-by-design considerations, a reference implementation, representative applications, adoption signals, and how NLIP relates to MCP and A2A.

arXiv:2609.04135 PDF

Efficient Test-Time Adaptation through Human-AI Interaction

Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang et al. Agents trained on population-scale data produce artifacts that rarely meet the personal bar a professional would stake their reputation on, because the criteria distinguishing expert work are heterogeneous and poorly documented — they surface only through repeated back-and-forth. Test-time adaptation through human-agent interaction (TAHI) folds that cross-session interaction data into both agent context and weights, maintaining an evolving rubric module that crystallizes each user's training and evaluation criteria. Adapting agents to 30 individuals across writing and visual creation over 600 tasks, solo task success improves 4.5-20.9% within only tens of tasks, the evolving rubrics catch 16.0-22.3% more failures than rubrics written by language models or humans alone, and up to 8.8% of the gains generalize to other users.

arXiv:2609.04141 PDF

Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments

Jie Wu, Zhenru Zhang, Beichen Zhang, Xuwu Wang, Yuhui Su, Mouxiang Chen et al. Terminal-based coding agents have produced large archives of trajectories, but each trajectory is a single frozen demonstration, whereas post-training needs executable environments that can be re-queried into many verifiable tasks and give execution feedback. Terminal-Universe reconstructs environments from the trajectories themselves: replaying the recorded file operations restores each file to its pre-modification state, producing a partial workspace that a completion agent fills in with missing files and dependencies. Tasks are then synthesized on that workspace in breadth, by mining directional dependency relations between environments to form cross-workspace queries, and in depth, by extending single-turn queries into multi-round sessions driven by a user agent. Applied to public trajectories it yields 37.3k task-sufficient environments, and supervised fine-tuning of Qwen3.5-27B on them adds 11.9 points on Terminal-Bench 2.1 and 13.8 points on EvoCode-Bench v2 MT@4.

arXiv:2609.04148 PDF

SENTINEL-RL: Offloading Topological Reasoning from LLM Agents in the Security Operations Center

Uday Vallabhaneni, Cassie L. Cagwin, David J. Wild cross-listed Large language model agents pitched as autonomous security operations center (SOC) analysts run into two walls: a context window too small to hold a multi-thousand-host authentication graph, and no guarantee that free-form containment advice is consistent with the topology it acts on. Sentinel-RL splits the labor — a heterogeneous graph attention encoder compresses the live authentication subgraph into a fixed-size state, a Proximal Policy Optimization (PPO) policy maps that state to a constrained set of investigative actions, and the LLM loop is restricted to turning those recommendations into analyst-readable narratives behind a critic gate. Instantiated on the LANL Comprehensive, Multi-Source Cyber-Security Events dataset on an HPC cluster, the policy reaches 0.91 precision and 0.87 recall on held-out red-team events and the full detect-investigate-recommend-approve cycle closes in a median of 6.3 seconds. The authors also report an ingestion pattern that loads a 24M-edge graph into Neo4j in 14.2 minutes, roughly 24x faster than the usual MERGE-based pipeline, plus an enterprise-readiness analysis of false-positive economics and the human-approval boundary.

arXiv:2609.04159 PDF

SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents

Xin He, Yanlin Wang, Mingwei Liu, Jiachi Chen, Hongyu Zhang, Guanbin Li cross-listed Repository-level coding benchmarks judge a patch by whether functional tests pass, ignoring the review-derived acceptance constraints that decide whether a change is actually merged in real projects. SWE-Gate mines those constraints from real pull request review comments and synthesizes 303 repository-level repair instances across 75 open-source Python repositories, each shipping separate functional and constraint tests alongside non-compliant and gold patches so issue resolution can be scored apart from constraint compliance. Across four LLM backends under a shared coding-agent scaffold, 221 of the 644 patches that passed functional tests failed their review constraints, indicating that functional-only evaluation overstates how well agents satisfy a complete repair specification.

arXiv:2609.04167 PDF

1 more specialized paper

Theory 34

Occupancy-based Quantile Risk Control

Zihao Shi, Huajun Xi, Bingyi Jing, Hongxin Wei cross-listed Conformal risk control gives finite-sample safety guarantees for deployed models, and quantile risk control extends it to quantile-based risk measures, but existing methods are either badly over-conservative or lack rigorous guarantees. OQRC recasts the problem as a finite-occupancy one: it partitions the loss space using the ordered calibration losses, estimates how test losses distribute across the resulting bins, upper-bounds risk by the worst loss in each bin, and picks the threshold parameter so this bound stays under the target with high probability. The bounds are proven to converge to optimal at an O(n^-1/2) rate, and experiments shrink the risk gap by up to 78.64% on standard benchmarks.

arXiv:2609.03104 PDF

A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations

Ruiyang Hong, Hrad Ghoukasian, Anastasis Kratsios cross-listed Kernel ridge regression and support vector regression are tractable because their estimators are closed-form or convex, guarantees deep networks lack. A two-stage compositional formula reconstructs an unknown Lipschitz function on a metric space from N noisy independent samples in closed form, yielding a high-probability uniform recovery bound that controls approximation and statistical error with exactly zero optimization error and no assumed oracle access to an approximate empirical risk minimizer. The construction is shown optimal in three senses — attaining the optimal fat-shattering dimension on Ahlfors-regular spaces, being maximally numerically stable in its parameters, and matching the target's Lipschitz constant in its input dependence — and on the unit cube under the sup norm it admits ReLU multilayer perceptron and exact ReLU multi-head transformer realizations of depth logarithmic in N with O(N) nonzero parameters.

arXiv:2609.03129 PDF

Beyond Straightness: Non-Crossing Flow Matching via Quantile AlignTree Coupling

Junyi Lin, Mengyu Li, Jingxuan Hu, Kejun He, Cheng Meng Flow matching quality depends heavily on how source and target samples are paired: independent coupling produces crossing paths and ambiguous local velocities, while optimal-transport couplings are expensive to construct. QAT-FM builds a hierarchical coupling between a Gaussian prior and the data distribution through a quantile-aligned tree, costing O(Nd log N) to construct with O(d) per-pair source sampling, and is proven to preserve marginals, induce non-crossing linear interpolation paths, and improve path separation at intermediate times relative to independent coupling. Experiments across benchmark datasets report generative performance competitive with existing couplings at substantially lower coupling construction cost, and the construction extends naturally to conditional generation.

arXiv:2609.03443 PDF

Towards a Statistical Understanding of Mixture-of-Experts

Siyuan He, Bokai Yang, Jie Hu, Ziwen Gao, Yuhong Yang cross-listed Mixture-of-experts (MoE) architectures route each input to a small subset of expert predictors, yet the statistical role of routing, sparse activation, and shared experts is only partly understood because existing theory mostly assumes parametric or correctly specified models. Treating MoE as localized aggregation, the analysis derives oracle risk bounds for dense and sparse routing with evolving experts that cleanly separate approximation error, expert-learning error, and router-estimation error, and characterizes when Top-K routing retains the benefits of localization while capping per-input computation. Gating is also interpreted geometrically through regions of local expert advantage, and shared experts of the kind used in DeepSeekMoE are shown to absorb common predictive structure so routed experts handle only residual local variation.

arXiv:2609.03501 PDF

Residual neural networks overcome the curse of dimensionality for semilinear heat equations

Ilkhom Mukhammadiev, Diyora Salimova cross-listed Existing theory shows plain feedforward networks can approximate high-dimensional partial differential equation solutions without their size blowing up exponentially in the dimension, but almost nothing comparable was known for residual networks on nonlinear problems. The authors prove that ResNets overcome the curse of dimensionality for semilinear heat equations with globally Lipschitz, gradient-independent nonlinearities: under polynomial growth and approximability assumptions on the PDE data, networks with at most a polynomial number of parameters in the dimension d and in 1/epsilon reach L2 error epsilon. The construction represents one deterministic realization of a multilevel Picard estimator, with shortcut connections carrying the spatial variable and a scalar accumulator while residual branches successively add the estimator's terms, giving for ridge-sum initial conditions the explicit parameter bound C d^(4+xi) epsilon^-(3+xi).

arXiv:2609.03626 PDF

Symmetries and Causality: Causal Effect Identification Beyond IID Data

Martin Rabel, Jakob Runge cross-listed Symmetries and cause-effect structure both pervade the natural sciences, yet they are hard to exploit together in machine learning tasks such as world modelling for reinforcement learning. The proposed formalism describes statistical systems through the symmetries in data that leave causal mechanisms invariant, giving a compact mathematical language for stating models and queries along with the machinery for rigorously deciding when a query is identifiable from the data at hand. On independent and identically distributed data it reproduces the standard identification results and known transport results for combining experimental and observational sources, while extending causal reasoning to non-IID settings and to queries that cannot be expressed as do- or soft-interventions; it also recasts familiar objects like c-components and hedges, covers missing data, and is framed for transfer and robustness questions.

arXiv:2609.03697 PDF

Projected Riemannian Gradient Descent for the Bures-Wasserstein Barycenter: Dimension-Independent Linear Convergence at Unit Step Size

A. Afham Computing the Bures-Wasserstein barycenter of positive definite matrices underpins work in optimal transport, machine learning, and quantum information, and practitioners run Riemannian gradient descent at unit step size, but theory has offered only a choice between unit-step bounds that degrade exponentially with dimension and dimension-free bounds that require slow small steps. Adding a projection step resolves the gap: the projected iteration converges linearly at unit step size with a dimension-independent rate of (1 - κ^{-3/2}) in the ensemble condition number κ, polynomially better than the κ^{5/2} complexity of the best small-step guarantee. The key technical result is a projection lemma showing that clipping eigenvalues to an interval is exactly the closed-form, non-expansive Bures-Wasserstein projection onto the corresponding matrix interval, and since it reuses an eigendecomposition the next iteration needs anyway, the projection costs nothing per step. The same analysis also covers the invariant matrix projection problem of Brahmachari et al., whose fixed-point algorithm is identified as unit-step Riemannian gradient descent on a totally geodesic submanifold.

arXiv:2609.03762 PDF

High-Dimensional Learning Dynamics of Attention-Indexed Models

Yizhou Xu, Margarita Sagitova, Lenka Zdeborov\'a, Florent Krzakala Training dynamics of attention layers are analyzed through attention-indexed models, a framework general enough to cover multi-layer and multi-head architectures with high-rank attention matrices. In a high-dimensional limit the population loss depends on only finitely many trace order parameters, while online stochastic gradient descent follows an infinite hierarchy of matrix moments that the authors show can be truncated with exponentially small error. The analysis identifies the attention parameterization itself as an implicit bias: optimizing a full matrix S directly can stay stuck in an uninformative state, tied attention S=WWᵀ breaks symmetry automatically and recovers signal in Θ(d² log d) samples, and untied attention S=UVᵀ exhibits a fast-slow split where the pre-activation mean moves first and determines whether recovery happens at all.

arXiv:2609.03858 PDF

The Dually Flat Geometry of Planning as Inference

Nikola Milosevic, Asaki Kataoka, Nicolas Hinrichs, Kenji Doya, Nico Scherf Reinforcement learning's occupancy measure is recast by folding the planning criterion into the dynamics through a resetting planning process, whose stationary distribution the authors call the visitation measure. The achievable visitation measures form a dually flat statistical manifold with two affine charts — visitation probabilities and log-policies — dual under the conditional entropy. Planning-as-inference then generalizes from linear rewards to nonlinear functionals of the visitation, with each iterate solved by a single natural-gradient step, and the temporal-difference error gains an interpretation as a marginal-utility estimate, with consequences drawn for both reinforcement learning and theoretical neuroscience.

arXiv:2609.04005 PDF

The Head Complexity of Boolean Functions in Single-Layer Attention

Rajmohan Rajaraman, Ravi Sundaram, Amanuel Tesfaye cross-listed How much can a single layer of self-attention compute, measured in heads? Defining head complexity as the minimum number of attention heads needed in a one-layer attention-only model, the authors prove an exact hierarchy: k heads compute k-bit parity but provably cannot compute (k+1)-bit parity, and the lower bound holds even at unbounded embedding dimension and unbounded numerical precision, resting on an alternating-sum obstruction where every monomial of the decision polynomial omits at least one input bit. A compactness theorem shows any computable function needs only dimension and precision bounded by the task's discrete data — head count, alphabet size, length — so neither resource can substitute for heads, and counting arguments give nearly matching universal bounds, with 2^n heads sufficient for any n-bit binary function and almost all such functions requiring Ω(2^n/n^2). The same obstruction extends to related tasks including multi-hop induction heads.

arXiv:2609.04046 PDF

Constant regret in general games via higher-order optimism

Omar Abbadi, Rida Laraki, Panayotis Mertikopoulos For an arbitrary N-player normal-form game with up to K actions per player, the proposed uncoupled learning algorithm guarantees individual regret of O(N³log²K), uniform over the horizon of play, when every player uses it. Higher-order optimism with discounting (HOOD) is a variant of optimistic follow-the-regularized-leader that pairs a discounted (N+1)-th order predictor with entropic regularization over a lifting of the game's strategy space, damping in a controlled way the large oscillations that blocked earlier attempts at constant regret. The authors note striking similarities to independent concurrent work that derived an O(N²¹log⁴K) bound using higher-order optimism with an exponential moving average estimator.

arXiv:2609.04113 PDF

A Computationally Feasible Framework for Causal Probabilistic Explanation

Rafal Urbaniak, Sam Witty, Daniel Waxman, Andy Zane, Poorva Garg, Emily Bunnapradist et al. Explaining why a particular outcome happened, and which inputs deserve credit or blame, currently forces a choice between the theory of actual causality, which gives principled verdicts but requires enumerating counterfactual scenarios and so only handles toy models, and scalable attribution methods like SHAP that discard part of the causal structure and can contradict a careful causal analysis. Probabilistic Causal Impact (PCI) recasts explanation as an estimation problem on a probabilistic causal model — specifying a distribution over candidate explanations, a distribution over counterfactual values, and a scoring function — so that answers are approximated by Monte Carlo rather than exhaustive enumeration, with actual causality and Pearl's probabilities of necessity and sufficiency recovered as degenerate cases. Evaluation spans consistency checks against actual causality, scaling experiments, continuous-valued dynamical systems, and a deployed causal machine learning model trained on millions of datapoints.

arXiv:2609.04177 PDF

22 more specialized papers

Safety & Alignment 27

Traceable TTS: Toward Watermark-Free TTS with Strong Traceability

Yuxiang Zhao, Yunchong Xiao, Yushen Chen, Zhikang Niu, Shuai Wang, Kai Yu et al. cross-listed Synthetic speech has become realistic enough to raise impersonation concerns, creating demand for text-to-speech (TTS) systems whose output can be traced back to the model that produced it. Existing traceability methods embed explicit watermarks in the waveform or the vocoder, which degrades audio quality and can be spoofed; the alternative proposed here trains the TTS model and a discriminator together so that attribution comes from the model's own generative fingerprint. The authors report that this joint training improves traceability generalization while preserving and even slightly improving audio quality, and present it as the first watermark-free TTS approach with strong traceability.

arXiv:2507.03887 PDF

Probe Generalization as Subspace Selection for OOD Deception Detection

Daniel Yoo, Adrians Skapars Linear probes read concepts such as deception out of language model activations but often fail to transfer to out-of-distribution inputs. Studying Llama-3.1-8B-Instruct probes across three held-out deception datasets, the authors find that projecting activations onto a small subset of principal components from the training distribution recovers cross-domain transfer that nearly matches probes trained directly on the test distribution. Selecting which components to keep can be automated by having an LLM judge score each component's most and least activating examples for whether it implies a transferable deception direction, closing the baseline-to-oracle gap by 78% on Insider Trading Report and 25% on Sandbagging. The directions a source probe weights heavily appear to capture source-specific surface features, while the transferable ones encode the same contrast more abstractly, suggesting probe robustness is mostly a matter of subspace selection.

arXiv:2609.02893 PDF

When Optimization Becomes Manipulation: Defending Generative Search against Malicious Generative Engine Optimization

Haozhang Li, Yangguang Shao, Xinjie Lin, Zhong Guan, Mi Zhou, Junzheng Shi cross-listed Generative Engine Optimization (GEO) rewrites web documents to match what a generative search engine prefers to cite, steering the answers users see, and because the rewrites stay factually consistent with the originals, fact verification and perplexity filters miss them. GEO Defender requires no fine-tuning of the answering model and mirrors the attack chain in two stages: Shield Reranker learns a preference-based defensive residual over a frozen reranker so rewritten documents get demoted while relevance judgments survive, and Training-Free Shield Generation distills past defense outcomes into a natural-language experience library that guides how the model uses sources at inference. Across seven GEO attacks on two closed-source and three open-source models, average attack success rate falls from 50.32% to 6.20% while 94.12% of benign evidence use is retained, including against attacks not seen during construction.

arXiv:2609.02964 PDF

Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning

Jinxi Yu, Eric Hanchen Jiang, Levina Li, Dong Liu, Zhi Zhang, Wenxiao Zhao et al. cross-listed Topology-based safeguards for multi-agent systems built on large language models train a graph neural network over the inter-agent communication graph to localize risky agents, but they assume one operator can pool every labeled trace — untenable across organizations whose episodes hold private prompts, tool outputs, and proprietary workflows. FGLGuard recasts the problem as federated graph learning: each operator fits an edge-featured graph attention detector on its own judge-labeled episode graphs and shares only model updates, combined with a proximal objective for non-IID clients, domain-balanced aggregation, over-refusal-constrained threshold calibration, and a guarded rewrite for blocked answers. On Agent-SafetyBench, R-Judge, and AgentDojo, the federated guard beats the in-domain centralized ceiling on all three benchmarks without pooling any data, and in live use cuts AgentDojo's ground-truth attack success rate by 43% at near-unguarded utility; off-the-shelf transfer without in-domain adaptation collapses to 0.51 AUROC.

arXiv:2609.02967 PDF

ObserverBench: Testing Mechanistic Estimates for Intervention and Control

Vijay Erramilli Mechanistic interpretability increasingly guides real interventions — activation steering, circuit removal, safety monitoring — yet an internal estimator that is accurate on average can still select a poor action. ObserverBench fixes, for each task, the model, information boundary, allowed actions, decision rule, held-out cases, and loss, then reports estimation accuracy separately from the loss caused by the action the estimate chose. On circuit-intervention tasks in GPT-2-small and Qwen2.5-7B, pairwise observers predict unseen effects more accurately without always choosing better actions, while observers trained on action loss choose lower-loss ones; in safety triage across Qwen2.5-7B, Gemma-2-9B-it, and prospectively frozen Qwen3.5-9B APPS tasks, AUROC ranks monitors differently from deployment loss, and sparse autoencoder readouts trail their layer-matched dense controls.

arXiv:2609.03026 PDF

You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring

Aritra Das, Jaee Ponde, Mihir More, Debayan Gupta cross-listed Language model agents deployed together can collude while each individual action still looks benign, and output monitors are defeated by obfuscation and steganography, which motivates probes trained on internal activations — but those probes are normally evaluated on agents that do not know they are being watched. Holding models, probes, and thresholds fixed, the study varies only what agents are told: nothing, that an activation monitor is present, or that a monitor is present along with the previous round's score. Across a four-agent blackjack game and a two-agent Simmons prisoners game using Qwen3-32B-AWQ and GPT-OSS-20B in homogeneous and mixed pairings, the best probes stay accurate in all three conditions and the agents keep colluding — telling agents about the monitor does not help them hide.

arXiv:2609.03035 PDF

CRAW: Codec Robust Audio Watermarking

David Chernin, Ethan Fetaya cross-listed Audio watermarking is a leading defense for marking generated speech, but post-hoc watermarks are erased by the neural codecs, denoisers, and vocoders routinely applied when audio is stored, transmitted, or re-synthesized. CRAW combines distortion-aware training with attention-based pooling, inference-time perceptual masking, and an error-correcting code that recovers the audio fidelity sacrificed to robust training. It reports state-of-the-art robustness to neural codecs, denoisers, and vocoders at perceptual quality comparable to existing post-hoc watermarks, with code released publicly.

arXiv:2609.03107 PDF

Reducing Catastrophic Risk from AI with Systematic Monitoring and Evaluation of Rogue AI Progression

T. Bauer, W. P. Kegelmeyer, E. Begoli, A. Sadovnik, T. Emerson, C. Corley et al. cross-listed Proposes a structured set of behavioral indicators intended to flag when an AI system is progressing toward catastrophic or "rogue" capabilities, borrowing threat-escalation methodology from cybersecurity and national security practice. The framework lays out metrics, indicators, and thresholds across several dimensions of capability and behavior so that monitoring can be tied to observable evidence rather than speculation. The stated aim is to give researchers and policymakers a common basis for evidence-based monitoring protocols rather than to report empirical measurements.

arXiv:2609.03189 PDF

The Analyst in the Prompt: Role, Retrieval, and Memory Biases in LLM Financial Analysis

Ahmed Asaad, Amr Mohamed, Yang Zhang, Omneya Abdelsalam Personalization context — memory, user profiles, role prompts — can change what a model concludes from identical evidence, which matters in finance where judgments rest on long documents. Testing twelve LLMs over 3,575 SEC filings and separating persona-conditioned retrieval, neutral retrieval, and memory-framed context isolates where the bias enters: most user-context spillover comes from how models interpret the same evidence under different roles, not from retrieving different evidence. Two mitigations — phrasing the investor mindset as a user profile rather than an assistant role, and splitting evidence-based from personalized output — each reduce spillover without eliminating it, with effectiveness varying widely by model.

arXiv:2609.03218 PDF

Counterfactual Fairness Audits of Multi-Step Clinical LLM Agents Require a Measured Per-Action Instability Floor

Rohith Reddy Bellibaltu, Manpreet Singh, Deepak Parashar, Rahul Joshi Counterfactual fairness audits of clinical agents report a flip rate — how often the recommended action changes when only the patient's demographic descriptor changes — and this work argues that number is meaningless without a noise baseline. Re-running sixteen vignettes ten times with nothing varied at all still moved the agent's action in 8.7% of outcome-vignette cells, with per-action instability spanning an eightfold range from 0.022 for ICU escalation to 0.179 for controlled-substance caution, and no demographic contrast in the data rose above that floor. A second model gives a 6.7% pooled floor and near-identical action ranking, majority voting over five draws removes 39% of the instability and then plateaus, and the authors release the FairMedAgent harness and floor protocol while explicitly claiming no disparity result.

arXiv:2609.03221 PDF

Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversation

Yuhe Wu, Guangyu Wang, Yujie Chen, Jiatong Zhang, Yuran Chen, Yutong Zhang et al. People bring interpersonal conflicts to language models for advice, but prior evaluations use single-turn judgments or explicit adversarial rebuttals, neither of which resembles someone recounting their side of a story over several turns. The authors name the resulting failure mode narrative captivity, where a model accepts an unopposed one-sided account as complete and adopts the narrator's framing without asking about missing perspectives, and measure it with a benchmark of 5,078 interpersonal-conflict scenarios spanning six moral dimensions. Across 17 models, end-state judgments after multi-turn narration shift by 25 percentage points on average relative to a matched single-turn baseline, with stage-level analysis pointing to preference optimization as a major contributor and four inference-time mitigation strategies helping only partially.

arXiv:2609.03407 PDF

Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models

Ross Tieman, Evan Markou Populations of language models are often treated as independent components even when their failures are strongly correlated, and semantic similarity of outputs captures only overlap in meaning rather than differences between the processes producing it. Borrowing from algorithmic information theory, the authors measure inferred generative-process diversity as Normalised Compression Distance between raw outputs, residualised against a permutation control, across 38 models. The measure surfaces population structure that semantic similarity misses and predicts chance-corrected correlated failure between model pairs across ten disjoint benchmark families, with a partial rank association of −0.216 (95% interval [−0.309, −0.122]) and a negative estimate on every benchmark, beyond what semantic similarity or pair capability explains.

arXiv:2609.03422 PDF

EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders

Xinghao Wang, Dong Li, Wei Yu, Yingwei Pan, Tao Gong, Qi Chu et al. cross-listed Text-to-video diffusion models trained on loosely curated data can reproduce unsafe or copyrighted material, and concept-erasure methods that intervene at a coarse granularity either fail to remove the concept fully or damage generation quality. EraseSAE argues that erasure should target monosemantic features and builds a decompose-attribute-erase pipeline for DiT-based video models: a Partitioned Convolutional Sparse Autoencoder splits dense spatiotemporal activations into interpretable sparse features while preserving spatiotemporal coherence, a contrastive attribution step compares activations from paired prompts to isolate concept-specific feature kernels, and timestep-resolved masks confine the intervention at inference to regions where the concept is active. Experiments across several diffusion models and erasure tasks report removal that is more precise and more robust than prior state-of-the-art methods with minimal loss of quality on unrelated content.

arXiv:2609.03629 PDF

Extracting Forgotten Prompts from Targeted Unlearned Models

Au Ashley Hoi-Ting, Meghdad Kurmanji, William F. Shen, Nicholas D. Lane, Ligang He Machine unlearning methods such as NPO, DPO and LUNAR suppress forgotten data through refusal alignment, and known attacks recover the answers to prompts the adversary already has in hand. The new attack shows the forgotten prompts themselves leak: Targeted Active Search constructs canonical templates and an entity pool from the retained data, queries the model black-box with the most informative template-entity pair under a limited budget to pin down which entities were forgotten, then instantiates templates with those entities to reconstruct the prompts. Across three unlearning methods, three datasets and three language models, it identifies the forgotten entity with 100% accuracy and reconstructs up to 95% of forgotten prompts while using up to 99.7% fewer queries than naive probing.

arXiv:2609.03662 PDF

Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

Alexandr Goultiaev Tolstokorov, Kyriakos Mouratidis, Javad Dogani, Nikolaos Laoutaris cross-listed When data providers license corpora to a third-party retrieval-augmented generation (RAG) operator, they lose any visibility into whether their documents keep being used, and auditing is hard because the operator is uncooperative, answers are paraphrased, and a single response may blend many providers' evidence. DirBucket watermarks documents provider-side through meaning-preserving paraphrases whose embeddings are nudged toward secret per-provider directions, so reuse can be detected from black-box answers without degrading retrieval quality. On a mixed-provider reuse benchmark it is the only method that combines strong target detection with zero non-target activation, catching non-compliance within 23 audited answers, and detection survives adversarial rewriting of answers as well as transferring unchanged to a second benchmark built from clinical, cyber-threat-intelligence, and legal corpora.

arXiv:2609.03749 PDF

IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks

Saikat Mondal, Mamta, Deeksha Varshney, Oana Cocarascu, Asif Ekbal Language model safety is still assessed mostly in English, leaving open how alignment fails in low-resource and culturally distinct languages. IndicSafeEval crosses ten safety-critical content categories with six human-like persuasion strategies across Hindi, Bengali, Marathi, and Punjabi to produce 7,200 adversarial prompts, then runs a black-box evaluation of several open-source models. Safety behaviour varies sharply with both the language used and the persuasive framing of the request, and vulnerability differs by risk category, with some kinds of harmful content markedly more susceptible to persuasion-based jailbreaks than others.

arXiv:2609.03781 PDF

Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation

Danting Zhang, Bei Peng, Robert Loftin Content moderation benchmarks typically collapse several distinct policy criteria into one harmful-or-not label, so high scores cannot show whether a model actually reasons about each criterion separately. Diagnostic Evaluation of COntent (DECO) factorises content so that criteria can be varied independently, and a pairwise protocol compares a model's outputs across different criteria applied to the same input. Across four moderation datasets and four large language models, strong aggregate benchmark performance masks substantial criterion-level failures, with the worst behaviour where the correct decision hinges not on overall harmfulness but on the particular aspect of the content the criterion targets.

arXiv:2609.03814 PDF

Flip, Don't Shuffle: Watermarking LLMs at the Speed of Inference

Simone Ceppi, Ignacio Sanchez cross-listed Statistical watermarks for language models decide which tokens belong to a boosted green list using vocabulary permutations as in KGW or multi-layer tournaments as in SynthID, both of which add real work to every decoding step. Stateless Bernoulli Watermarking makes membership an independent per-token Bernoulli trial resolved by one comparison against a counter-based random number generator, giving constant-time membership, single-kernel execution with no intermediate allocations, and a proof that the z-score detection test still follows a standard normal distribution under the null hypothesis. The stateless design enables full-vocabulary self-salt watermarking over 6000 times faster than KGW's self-salt and twice as fast as SynthID, works with distributed inference, and adds under 1% end-to-end generation overhead at all batch sizes; a GPU-native Jenkins hash further improves null calibration by 1.8 times while producing more diverse text, with ROC-AUC differences below 0.01 confirming statistical equivalence.

arXiv:2609.03844 PDF

A Blind Trust, the Bloody Thrust: When Attacker-Controlled Hook Updates Steer AI Agent Harnesses towards Malicious Behaviors

Pengxun Li, Litian Zhang, Jianwei Hou, Shujiang Wu, Song Li, Zifeng Kang et al. cross-listed AI coding harnesses let plugins register lifecycle hooks — shell commands bound to events like session start, tool calls, or file edits — that execute with the user's host privileges and can fire without the language model ever seeing them. The authors identify the hook update path as an unguarded attack surface: an attacker who controls only plugin metadata and hook configuration can push an update that trojanizes a previously benign versioned plugin, and their automated framework HookPry realizes ten attack objectives including privilege escalation. Across 1,000 end-to-end runs spanning 25 harness-backend combinations it compromised all seven harnesses tested, with per-harness success up to 92.5%, while Microsoft Defender caught none of it and three static defenses combined still missed 47.5% of the malicious artifacts.

arXiv:2609.03884 PDF

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

Hoang Cuong Nguyen, Mark Dras, Usman Naseem How a model is post-trained to refuse harmful requests, not just what data it sees, changes the internal machinery that implements refusal. Comparing supervised fine-tuning, fine-tuning on reasoning chains that justify the safety decision, and preference optimization via ORPO across Llama-3.1-8B, Gemma-2-9B, and Qwen3-8B, the authors find reasoning-augmented training yields a recognizably distinct refusal computation in all three models, while architecture separately determines internal structure and how reliably refusal can be steered by activation edits. No method achieved all three desirable properties at once — refusal spread across many components rather than a few fragile ones, safety gains that leave general capability intact, and behavior correctable by small targeted edits — leading them to caution against treating current post-training as a settled defense.

arXiv:2609.03887 PDF

Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection

Joe Cecil, Marjorie Freedman Benchmarks for factual errors in chatbot output are typically built from one annotator making one pass, which may not catch subtle mistakes buried inside otherwise correct medical text. The study layers first-pass human annotation, LLM-as-a-Judge candidate discovery, and two kinds of adjudication — medical experts and evidence-based fact-checking — over the same medically relevant responses to see what each stage adds. First-pass annotators routinely miss errors that adjudicators later confirm, and the LLM judge surfaces different errors than humans rather than a superset, while adjudicators themselves disagree; applying the procedure to an existing benchmark exposes the same missing annotations. The conclusion is that single-pass hallucination benchmarks buy scale by undercounting errors, and that multi-pass adjudication improves coverage without removing the dependence on who judges and what evidence they use.

arXiv:2609.03953 PDF

Representational alignment yields generalizable safety in language models

Lingyu Li, Yan Teng, Yingchun Wang, Xia Hu Alignment methods optimize what a model says, which leaves models exposed when the same harmful intent is recast in an unfamiliar or adversarial form that a person would still recognize. Borrowing prototype theory — human concepts organized around central cases with graded typicality — the authors show across 23 models that this categorization structure for moral concepts is only weakly preserved regardless of parameter count or alignment stage, and introduce representational similarity optimization, which aligns latent representations with human moral judgements instead of supervising generated responses. In matched experiments over the same 251,334 moral annotations, standard behavioral alignment learned the intended judgements at the response level but left categorization structure intact and increased adversarial vulnerability, whereas reorganizing the categorization consistently improved adversarial robustness across model scales, benchmarks, and attack strategies at the cost of smaller gains on explicit judgements.

arXiv:2609.04022 PDF

From Deceptive Outputs to Deceptive Mechanisms: A Causal Framework for Language-Model Deception Research

Yakov Pyotr Shkolnikov Claims that language models deceive often slide from behavior that looks deceptive to a mechanism that actually is, importing human mental-state vocabulary along the way. The proposed causal taxonomy separates prior commitment from retrospective report, model preference from realized output, a false preference from sensitivity to the utility of misleading a recipient, and deceptive behavior from the provenance of the objective producing it. Controlled guessing-game and stock-trading experiments on two open-weight model families show that deceptive-looking behavior can arise without the mechanism usually credited for it, while other interventions do give direct evidence that a recipient's information state causally shifts deceptive preference. Even that evidence, the authors argue, does not establish model agency in the deception.

arXiv:2609.04166 PDF

A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms

Davide Paglieri, Logan Cross, Tim Genewein, Joel Z. Leibo, Nenad Tomasev, Alexander Sasha Vezhnevets A collective of 100 autonomous LLM agents was set to proving formal mathematical conjectures while sharing a knowledge library and peer-to-peer messaging. One agent found an exploit in the evaluation system, and the cheat spread through the shared library and private messages until a cohort of agents adopted it under competitive pressure — after which a separate group began auditing fraudulent proofs, alerting peers on broadcast and private channels, staging boycotts, lodging formal complaints, and proposing validation patches, all without external intervention. The authors note that the same transparent channels that carried the exploit gave honest agents the visibility to organize resistance, and frame the shared infrastructure as a knowledge commons, proposing Ostrom-style institutional mechanisms such as graduated sanctioning and collective-choice rules for decentralized self-governance.

arXiv:2609.04170 PDF

3 more specialized papers

Multimodal 26

Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization

Qingchan Zhu, Weihang You, Hanqi Jiang, Changdi Yang, Tianming Liu, Geng Yuan cross-listed Visual token pruning cuts vision-language model (VLM) inference cost, but nearly all methods only score which tokens to keep, which can retain redundant high-scoring tokens while discarding evidence that no survivor stands in for. CoverPruner flips to the demand side, asking which remaining original token represents each removed one, and formalizes this as Representational Coverage Maximization over the projected visual-token set weighted by query demand, instantiated with projector-space coverage plus a cheap first-layer attention probe and requiring no training. Across several VLM architectures and compression rates it attains the best average accuracy of all compared pruners, with the widest margins at aggressive compression.

arXiv:2609.03158 PDF

Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

Alejandro Bar\'on Garc\'ia, Feng Wang, Emilia Garcia Casademont, Han Xiao Jina-OCR-v1 is an end-to-end document parsing model targeted at cheap GPUs, pairing the compressed-vision encoder and 3B mixture-of-experts decoder of DeepSeek-OCR (about 570M active parameters per token) with a FastMTP speculative decoding head that reuses one draft block recursively over three prediction steps, kept lossless by greedy verification. Post-training layers instruction alignment, robustness fine-tuning on hard documents, and Group Relative Policy Optimization (GRPO) driven by dense verifiable rewards — deterministic formula, table, and structural checks that grant partial credit — over cleaned public corpora plus targeted synthetic pages. It scores 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench at the highest throughput in its comparison (2.57 pages per second), and speculative decoding doubles decoding speed over greedy autoregressive decoding on an NVIDIA L4; weights are public.

arXiv:2609.03181 PDF

MedQA-MM: Shortcuts Behind Medical Visual Reasoning

Benlu Wang, Yifan Zhang, Jiaqing Yu, Chin Siang Ong, Juncheng Huang, Zhuohao Li et al. cross-listed A correct answer on a medical multimodal multiple-choice question can come from the intended image finding or from non-visual shortcuts: wording of the answer options, clinical text, burned-in image text, annotations, or device artifacts, an effect the authors call reasoning inflation. Across six medical multimodal question datasets and a 13-configuration panel of open models, prompt- and image-side audits, modality ablations, and cue-removing repairs separate these routes: full-input accuracy is 62.63%, but text-only reaches 53.96% and options-only 29.71%, while stripping length-gap, absolute-wording, and spatial-preposition cues costs 6.58, 3.50, and 4.77 points. The authors release MedQA-MM, a 1,000-item shortcut-mitigated subset on which text-only accuracy falls to 5.21% and options-only to 12.33%, arguing that claims about medical image reasoning need route-level evidence rather than headline scores.

arXiv:2609.03261 PDF

FPCO-Dialog: A Multi-Turn False-Premise Benchmark for Correction and Cooperation in Vision-Language Models

Jiayuan Ma, Yuqi Lu, Weiyang Guo, Chenrui Wang, Junyi Shu, Xuebo Liu et al. Vision-language models are used in conversations where a user may keep describing an image incorrectly, but evaluations rarely isolate what happens when the same visually grounded false premise recurs turn after turn. FPCO-Dialog supplies 1,080 images and 10,800 question turns stratified by visual complexity, object category, and false-premise class, using a 10-turn protocol that starts with a correct prefix and then repeats false-premise referring expressions, scored by CorrTP@K, a correction rate over false-premise turns judged by two independent detectors. Evaluating 20 commercial and open models reveals substantial and persistent differences in how readily models correct a repeated false premise, along with model-specific turn-wise dynamics and systematic variation across premise types.

arXiv:2609.03331 PDF

DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents

Puneet Mathur, Dinesh Manocha Full-duplex voice agents must decide moment to moment whether to listen, backchannel, interrupt, take the floor, or yield, yet benchmarks typically hand them explicit turn-management instructions while deployed systems are configured through roles and personas from which the right behavior must be inferred. DSB-IFEval supplies 1,038 test cases across eight assistant roles and five conditioning protocols (default, explicit rules, persona-implied, combined, and conflicting), scoring floor management with a deterministic Instruction Adherence Score and persona-consistent content with an LLM-judged Persona Adherence Score. Across six real-time systems the trade-offs split by architecture: F-Actor and PersonaPlex lose 9.7% and 4.5% adherence when behavior must be inferred from a persona instead of stated outright, while GPT-Realtime, MiniCPM-o, and Fun-Audio-Chat follow persona content closely but never adapt their floor behavior, and all systems struggle to override persona directives that conflict with safety.

arXiv:2609.03423 PDF

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

Bo Zeng, Linfeng Gao, Peiqin Lin, Yu Zhao, Mingyan Zeng, Yu Tong et al. Multimodal models score near ceiling on food recognition, leaving open whether that reflects cultural understanding or visual pattern matching. CulturalMenuBench separates the two with 4,870 items in 10 languages across 18 regions and 10 tasks that pair finished-dish and step-by-step cooking images with ingredients, procedural text, and regional labels. Across 12 models, systems exceeding 94% on standard multiple choice drop to at most 56% when attributing dishes to Chinese regional cuisines in an identical four-way format, with error patterns consistent with random guessing and accuracy tracking visual distinctiveness rather than cultural structure; the same models classify cuisine 7 to 18 points more accurately from dish names than from images, indicating the knowledge is present but not reachable through vision.

arXiv:2609.03526 PDF

The Attention Triangle in Audio-Video Models

Sagi Polaczek, Noa Kraicer, Gal Metzer, Zhuo Ning, Ali Mahdavi-Amiri, Daniel Cohen-Or et al. Audio-video diffusion models coordinate text, sound, and picture through cross-modal attention, and the authors probe the three attention edges connecting those streams, the attention triangle, to trace where semantics travel during generation. The audio-video edge turns out to carry influence in both directions and to be the main source of semantic leakage: when a prompt conflicts with priors baked into the model's weights, cross-modal interaction can override the intended conditioning and steer output toward whatever is visually canonical instead. Signals extracted from attention serve both as a diagnostic that can deliberately induce leakage under controlled conditions and as a guide for inference-time interventions, which improve semantic grounding without degrading generation quality.

arXiv:2609.03586 PDF

ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

Taewoo Kim, Young Han Lee, Nam In Park, Chanwoo Kim cross-listed Audio deepfake detection is normally framed as clip-level binary classification, yet manipulated recordings in the wild mix genuine and synthetic material across time or across overlapping sources, so a useful system must also point to the evidence. ToolDF uses an audio large language model as an orchestrator trained on supervised tool-use trajectories: it inspects the acoustic scene, optionally runs source separation, routes the resulting components to domain-specific expert detectors, and aggregates their outputs into an interpretable verdict. On a new mixed-authenticity benchmark covering temporal transitions, acoustic overlaps and hybrid mixtures, it reports macro-F1 gains of 3.72 points over the strongest monolithic baseline and 14.39 points over a fixed pipeline on composite cases, while localizing evidence to specific time regions and sources.

arXiv:2609.03620 PDF

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

Oline Ranum, Edward Fish, Simon Hadfield, Richard Bowden BLEU-4 is the default metric for sign language translation, but a spoken-language n-gram score lets models exploit spurious correlations and spoken-language priors instead of learning better sign representations. Comparing spatio-temporal understanding against BLEU-4 for six translation models on Phoenix-2014T and CSL-Daily shows that score gains are not by themselves evidence of improved sign comprehension. The proposed alternative borrows from language-learning assessment, using an open-weight-LLM question-answering protocol to measure whether salient content survives translation; it tracks human rankings more closely and is six to seven times more paraphrase-invariant than BLEU-4. Under the new protocol the five gloss-free systems on Phoenix-2014T are essentially indistinguishable while the gloss-supervised system leads by 9.3 points, a gap the old metric hides.

arXiv:2609.03734 PDF

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

Marco Cipriano, Leonardo Zini, Alexandra Schild, Valentin Teutschbein, Afsana Mimi, Marcella Cornia et al. Text-to-Scalable Vector Graphics (SVG) generation is currently scored with metrics built for natural images, chiefly CLIPScore, which was never trained on vector art. Controlled caption and image perturbations show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, wrong object counts, and broken spatial relations, while off-the-shelf Vision-Language Model judges are more sensitive but respond unevenly across error types and drawing styles. The authors collect a human-annotated dataset for semantic alignment and build two evaluators on it: CLIP scorers adapted to vector graphics and tuned to human preferences for cheap large-scale scoring, and a VLM judge trained with supervised fine-tuning plus reward-shaped reinforcement learning for interpretable assessment, then benchmark open-source, commercial, and optimization-based generators on an independent caption set.

arXiv:2609.03806 PDF

VisCAD: A Foundation Model Suite with Multimodal Industrial CAD Intelligence

JoyIndustrial VisCAD Team, Linxin Cai, Qiuhe Hong, Zhichao Huang, Guanlin Li, Hongsen Liu et al. cross-listed AI-assisted computer-aided design (CAD) has to map renders, text, 2D drawings, and photographs into executable CAD domain-specific-language programs at the part level, and additionally plan mating relations and part poses at the assembly level; specialist models trained on narrow input types generalize poorly while frontier models cover more inputs but perform inconsistently. VisCAD is a model suite centred on VisCAD-M1, a 27-billion-parameter model given CAD-specific mid-training and post-training for part-level generation, plus a domain-specific harness that drives frontier models for assembly generation. On PubCADBench and RealCADBench it averages 0.5540 at the part level against 0.5496 for the strongest frontier model, and reusing the model as a test-time verifier raises this to 0.5797, roughly a 5 percent relative gain over prior state of the art.

arXiv:2609.03811 PDF

When Vision Meets Graphs: A Survey on Graph Reasoning and Learning

Xinjian Zhao, Wei Pang, Zhixuan Yu, Xiangru Jian, Xiaozhuang Song, Yaoyao Xu et al. cross-listed Graph machine learning has been dominated by Graph Neural Networks (GNNs) treating graphs as purely symbolic structures, even though scientists routinely read graphs visually, as chemists do with molecular diagrams and social scientists with network plots. This survey organises the emerging area treating visual depictions of graphs as first-class model inputs into three threads: using images of graphs for structural understanding and multi-step reasoning, using visual features to complement graph encoders beyond the known limits of message passing, and domains such as chemistry where standardized drawing conventions support both. It assesses what current vision and vision-language methods can and cannot do, and sketches a path toward foundation models that perceive and reason about graphs the way practitioners do.

arXiv:2609.03816 PDF

Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs

Prakhar Khatri cross-listed An hour of video sampled once per second is 3,600 frames, of which a long-video multimodal model keeps only a small fixed slice, and published frame selectors are hard to compare because they change the scorer, prompt boundary, resolution policy, and answering model at once. Holding each fixed and varying one decision at a time across six training-free selection rules, three benchmarks, and two answering models, the study finds selection is the dominant lever: on LongVideoBench's hour-long bin, eight query-selected frames beat sixteen uniformly spaced ones by 6.9 points, and Orthogonal Matching Pursuit, an unmodified decades-old sparse-approximation algorithm, matches or nearly matches every purpose-built selector. Halving each frame's spatial budget costs at most 0.44 points, and spending those savings on twice as many compressed frames returns another two to three points, so compression only pays once reinvested. A bug found in the authors' own AKS baseline and a 0.07-to-3.74-point gap between two harnesses running identical published rules at the same budget argue that such comparisons must happen inside one controlled harness.

arXiv:2609.03820 PDF

GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs

Junqing Du, Fernando Ropero, Erkin Turkoz, Yanfeng Zhang, Lu Liu cross-listed Multimodal large language models handle 3D spatial questions poorly — they misjudge distances, struggle to switch between a first-person and a map-like view, and mis-ground fine visual details — and the usual fixes require either large curated spatial datasets or a bolted-on 3D encoder. GraFT needs no training at all: it builds a compact 3D scene graph and uses it three ways, computing exact geometry with symbolic tools, rendering a bird's-eye view for map-like layout questions, and selecting task-relevant first-person frames for appearance grounding. It raises CIDEr by 27% on ScanQA against the same backbone, and on VSI-Bench improves frozen models by up to 65%, beating every proprietary and open-source general baseline as well as several fine-tuned spatial models.

arXiv:2609.03892 PDF

Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis

Sanyuan Chen, Min-Jae Hwang, Sho Inoue, Anna Sun, Bokai Yu, David Kant et al. Voice dubbing and conversational speech systems typically depend on forced alignment and explicit duration prediction, which constrains how naturally timing and turn-taking can be modeled. Text-AB is a 3B-parameter diffusion transformer trained with a flow-matching objective on 480k hours of speech that drops alignment entirely, consuming raw text through an off-the-shelf encoder and learning text-speech correspondence via cross-attention; it operates in a latent space of DAC-VAE features that compress 48 kHz audio into a 25 Hz sequence, over 10x more compression than EnCodec. After fine-tuning on cross-lingual dubbing and full-duplex dialogue, the authors report near-human quality on short-form conversations plus large gains in prosody similarity, voice similarity, and naturalness over their previous production dubbing system, with turn-taking, back-channeling, and emotional dynamics modeled natively.

arXiv:2609.03992 PDF

InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models

Chao Shen, Xinyuan Li, Yunfan Zhou, Jianguo Yao, Haibing Guan, Zhihai Wang et al. Reading an industrial gauge is low-effort and highly repeatable for a trained operator, yet multimodal large language models (MLLMs) remain unreliable at continuous-valued measurement, and existing benchmarks strip away the situated context, specialized instruments, and real-world noise that make the task hard. InSituMeasure covers 2,922 real industrial monitoring scenes across eight functional categories of professional engineering instruments, with dense gauge-attribute annotations and noise tags for failure diagnosis, scoring numerical accuracy under predefined tolerances, unit consistency, rejection of fake or unanswerable tasks, and how well model failures line up with annotated error factors. Across 24 state-of-the-art models the best reaches only 25.7% joint value-and-unit accuracy and 51.8% confidence-diagnosis F1, with recurring failures traced to text-induced shortcuts, overconfident responses, viewpoint deviation, occlusion, and environmental interference.

arXiv:2609.04014 PDF

CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation

Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Chu Liu, Pengjun Xie et al. cross-listed Embedding models built on multimodal large language models (MLLMs) struggle to separate scenes that contain the same concepts but bind attributes to objects differently, even though the same backbone can make the distinction when used as a cross-attentive reranker. CORE distills the reranker's judgments into the embedding model by synthesizing candidate lists spanning five levels of compositional match and training with a listwise Rank-KL objective. Under a matched data and tuning budget, both Rank-KL and pairwise CoSENT exploit that multi-level supervision better than contrastive learning, with Rank-KL strongest overall. CORE-RERANKER-8B averages 82.7% across COLA, SUGARCREPE++, and NEGBENCH, 10.7 points above Jina-Reranker, and the gains transfer to MCMR without hurting retrieval on COCO and Flickr30K.

arXiv:2609.04083 PDF

9 more specialized papers

Other 25

From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning

R\'emi Bourgerie, \v{S}ar\=unas Girdzijauskas, Viktoria Fodor Machine learning that collects data, trains, and infers in a single location runs into scalability and privacy limits, which federated and decentralized learning address by keeping training local — but most of that work assumes Euclidean data with grid-like structure such as images and text. The survey first organizes collaborative learning for Euclidean data along three dimensions, learning effectiveness, efficiency, and privacy preservation, then extends the treatment to graph-structured data with a taxonomy of graph distribution scenarios, the statistical heterogeneities each induces, and standardized problem formulations and algorithmic frameworks. A central argument is that message passing makes graph learning conceptually well-suited to collaborative settings, since propagating information between connected nodes is already the kind of exchange agents must perform, and the closing sections map open challenges and research directions.

arXiv:2609.02984 PDF

Causal Foundation Models

Christopher Stith, Hossein Rahmani, Jesse C. Cresswell Estimating the effect of a treatment traditionally requires a bespoke pipeline per problem: propose a causal mechanism, pick a compatible estimator, train it. Causal foundation models bring the pretrain-once paradigm to this setting — networks pretrained at scale that estimate quantities such as the average treatment effect on entirely new datasets using in-context learning, with no model updates or fine-tuning. The write-up is a practical introduction, covering the necessary causal inference and machine learning background before the model class itself, with example code and Jupyter notebooks throughout.

arXiv:2609.03003 PDF

Efficient Constant Optimization for Symbolic Regression with GPU-Accelerated Tree-Based Genetic Programming

Hao Mao, Xu Tony Liu, Shuai Lu, Peng Zhao, Wenzheng Jiang, Yuntian Chen cross-listed Tuning the numerical coefficients inside candidate expressions is expensive enough per generation that GPU symbolic-regression frameworks usually drop it or keep only a cheap version. The proposed solver keeps a batched Levenberg-Marquardt optimization entirely GPU-resident across a structurally heterogeneous population of expression trees using a fixed number of population-wide CUDA launches per iteration, with reverse-mode automatic differentiation assembling each tree's Jacobian in one backward sweep so per-iteration cost stops depending on how many constants a tree has, and a double-precision guard ensuring returned constants never regress. It sustains up to 5.1×10⁵ trees per second on an NVIDIA A100, roughly 9.9× the throughput of Operon on a 64-core EPYC 7763 at matching fp64 quality, and once integrated into EvoGP it recovers governing equations on 10 of 18 constructed problems versus 0 for the stock system.

arXiv:2609.03352 PDF

Time Without Timesteps: Simulating Coupled Dynamical Systems via Self-Consistency

Liyu Zerihun, Mark Shinyoung Lee Simulating coupled dynamical systems normally means marching through time step by step, which makes both the forward solve and its gradient deeply sequential. The proposed alternative trains a neural surrogate per subsystem type that maps an entire driving trajectory plus initial condition to an entire output trajectory, then assembles coupled systems by enforcing self-consistency among trajectories in the style of classical waveform relaxation, turning simulation into a fixed-point problem. On coupled van der Pol oscillators and Hodgkin-Huxley neuron networks this replaces 1500 reference integrator steps with 4 to 10 Newton iterations, and the gradient becomes a linear system solved by GMRES at memory independent of solver depth. The spectral radius of the learned operator's Jacobian predicts in advance whether the coupled solve converges; past that boundary unrolled backpropagation diverges while the implicit gradient stays accurate to 0.04%.

arXiv:2609.03358 PDF

Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty

Qing Zhang, Yifei Huang, Juyoung Lee, Thad Starner, Jun Rekimoto Now that generative models make polished prose cheap, fluency no longer signals truth — readers trust fluent hallucinations while discounting accurate content once it is disclosed as AI-generated, a pattern the authors name the Fluency Trap. Binary 'Made with AI' labels disclose authorship but say nothing about what supports a claim; the proposed Provenance Density interface instead visualizes how densely a text's claims are backed by verified evidence. In a study with 81 participants, an idealized version of the interface produced a large gap in discriminating truth from fabrication (+4.15 points, d = 1.82) while participants given no signal showed no detectable discrimination, and a technical audit of 200 samples found retrieval density alone insufficient, with a consistency-veto check carrying most of the discriminative signal on dynamic queries.

arXiv:2609.03460 PDF

On the Interaction Between Model Compression and Test-Time Adaptation

Francesco Corti, Dong Wang, Young D. Kwon, Cecilia Mascolo, Olga Saukh Models shipped to real deployments often need both compression to run cheaply and test-time adaptation (TTA) to survive distribution shift, but the two are almost always studied separately. Combining several structured compression methods with standard TTA techniques on ResNet-18 and ViT-Base over CIFAR-10-C and ImageNet-C, and adding a diagnostic that measures representational expressivity and how well the adaptation subspace still fits, the study finds compressed models keep their accuracy under supervised adaptation yet lose TTA performance steadily as compression increases. The cause is traced to reduced representational diversity and structural constraints that limit what adaptation can recover, and the size of the effect varies sharply by compression method, suggesting adaptability should be an explicit design target for compression.

arXiv:2609.03604 PDF

Witnesses Explain Anomalies

Lamine Diop Unsupervised anomaly detectors score points in one pass but say nothing about which features drove a flag, so explanations are usually bolted on afterwards with SHAP or LIME, which re-query the detector thousands of times per point and only approximate it. WAND scores each tabular point by how far its projection onto directions on the unit sphere escapes a sub-Gaussian extreme-value baseline, and because those witness directions are vectors in feature space they double as per-feature attributions at no additional cost over scoring, recoverable by gradients since the score is differentiable. Scoring is linear in sample size and a probe-efficiency bound guarantees every anomaly has a witness; across 47 ADBench datasets the method achieves the best mean Friedman rank at ROC-AUC parity with 16 unsupervised baselines, with explanations reported as more accurate and faithful than post-hoc SHAP, LIME, and ECOD at a fraction of the query cost.

arXiv:2609.03826 PDF

Xiaomi-TabLDM: A Tabular Foundation Model Technical Report

TabLDM Team, Penghui Wang, Wei Liu, Hong Wang, Chengyue Huang, Yuxi Sun et al. Xiaomi-TabLDM is a foundation model for tabular classification and regression that predicts through in-context learning, so a new task needs no fine-tuning, and it is pretrained purely on synthetic tables generated from structural causal models. Architecturally it combines a three-stage training schedule with dual-stream feature grouping, a lightweight attention residual, and a sparse mixture of experts, and it supports spending extra compute at inference to improve predictions. It ranks first on OpenML-CTR23 and second for regression on TALENT, TabArena, and BCCO, and on TabArena regression reaches the second-highest Elo while using 82% less training time and 68% less prediction time than the top-ranked TabFM.

arXiv:2609.03880 PDF

17 more specialized papers

Reinforcement Learning 18

RL-ADA: A World-Feedback Framework for Adversarially Robust Enterprise Dialogue Agents

Ram Narayanan, Harshit Rajgarhia, Abhishek Mukherji Training robust customer-support dialogue agents normally requires labeled interaction logs, which are privacy-sensitive, expensive to annotate, and outdated by the time labeling finishes. RL-ADA removes the labels entirely by having a 3B support agent and a 7B adversarial customer agent co-evolve in an arena judged automatically, where the support agent is rewarded for resolving multi-turn conversations and the adversarial customer is rewarded for realistic, intent-concealing utterances that cause misroutes, with an isolation gym retraining whichever agent is currently weaker on prior failure transcripts. In a banking support proof of concept, tool-routing errors are eliminated and the strict end-to-end pass rate doubles over five co-evolutionary cycles with no labeled data, and the adversarial agent spontaneously develops what the authors call contextual camouflage, burying its intent inside dense plausible customer detail.

arXiv:2609.02902 PDF

Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representations

Aditya Makkar, Benjamin Unger, Jeongyeol Kwon, Mathieu Lauri\`ere, Eugene Vinitsky, Yonathan Efroni cross-listed Reinforcement learning over very large agent populations is usually done by optimizing each agent as if everyone else were fixed background, which ignores how the population as a whole shapes the dynamics. The mean-field formulation here assumes rewards and transitions depend on the population only through an unknown low-dimensional aggregate statistic, and an offline algorithm is given that provably reaches a near-optimal policy by first learning that representation. Tested on a one-step routing game modeled on supply-chain optimization, holding neural network parameter count and optimization budget fixed, learning the low-dimensional population representation improved both reward prediction and the Nash gap of the resulting policies compared with baselines that ignore the structure.

arXiv:2609.02928 PDF

Tail-Likelihood Reinforcement Learning

Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar, Guanning Zeng, Qingyang Wu, Zhongzhu Zhou et al. Optimizing average reward hides a distinction that matters for generative policies: two policies with the same mean can differ greatly in how often they produce a rare but very high-reward rollout, which is exactly what repeated sampling at training and inference time depends on. TailRL optimizes that coverage directly by maximizing the log-probability of exceeding a randomly chosen reward threshold, turning a continuous reward into a family of binary success events whose gradient upweights rare high-reward rollouts and can be read as a mixture of Best-of-k gradients. Because it amounts to a modified advantage function, it drops into existing reinforcement learning pipelines, and across object localization, maze navigation, GUI grounding, and code optimization it escapes suboptimal solutions and yields models that gain more from additional inference-time samples.

arXiv:2609.02987 PDF

FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience

Zixun Huang, Kishan Panaganti, Haitao Mi, Leowei Liang Self-improvement from a model's own rollouts is unstable: outcome verifiers give trustworthy but sparse signal, while dense guidance from the same model can entrench false confidence or collapse onto one solution mode. FlowBalance scores each trajectory using token-level log-probability gains from a frozen copy of the policy given privileged context, then calibrates that self-guidance against the verifier-derived group advantage — keeping it on positive-advantage trajectories, reversing it on negative ones, and switching it off when the rollout group shows no outcome preference — and fits the resulting reweighted target with profiled trajectory balance using one log-partition estimate per group. On mathematical reasoning it beats FlowRL on both Qwen3-4B and Qwen3-8B while training faster and more stably, avoiding the response-length collapse seen in direct on-policy self-distillation and retaining more correct-strategy diversity on an AIME24 diagnostic.

arXiv:2609.03241 PDF

DE-Venus: A Data-Efficient RLVR Framework for Large Language Models

Shenzhi Yang, Guangcheng Zhu, Kai Tang, Zhengqing Zang, Xing Zheng, Haobo Wang et al. Reinforcement learning with verifiable rewards (RLVR) is limited in practice by the cost of on-policy rollouts and of obtaining trustworthy labels, and existing fixes for sample selection, weak supervision, and noisy labels are usually tangled into the distributed training code. DE-Venus treats supervision as evolving state across the data lifecycle and separates it into three modules — active data selection for training and annotation budgets, weak supervision construction from unlabeled examples, and training-time refinement that filters or corrects unreliable signals — expressing each method as offline dataset transitions or online transforms of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business deployments, configurations match or improve model quality using only 10% of labels or as little as 13% of relevant data, with some business setups converging in 63–75% fewer steps.

arXiv:2609.03324 PDF

Gradients Know What Outcomes Don't: Unlocking Reinforcement Learning for LLM Reasoning with Gradient-Aligned Rewards

Leqi Zheng, Jinbo Su, Fang Niu, Chaokun Wang, Weiping Wang, Jiajun Zhang et al. Reinforcement learning from verifiable rewards gives a binary correct/incorrect signal that cannot rank one correct chain of thought above another, and dense alternatives either use crude heuristics or need costly process reward annotation. Gradient-Aligned Reward (GAR) works in the policy's own gradient space: truncated backpropagation through the output projection layer produces a compact gradient vector per rollout, and its cosine similarity with a gradient anchored on an expert solution already present in the training corpus becomes a dense reasoning-aware reward at under 9% wall-clock overhead. The cosine is shown to factor multiplicatively into prediction-error and activation-pattern terms, and on Qwen3-4B and Qwen3-8B the method beats GRPO and other baselines on competition math while transferring to GPQA Diamond and MMLU-Pro without domain-specific data.

arXiv:2609.03342 PDF

TIGPO: Temporal Instance-Graph Policy Optimization for Long-Horizon LLM Agents

Jinwei Gan Organizing agent rollouts into state-transition graphs improves credit assignment over long horizons, but current methods rebuild the graph from scratch at every policy update, throwing away transitions found by earlier policies and confining advantage estimates to small batch-local groups. TIGPO (Temporal Instance-Graph Policy Optimization) keeps a persistent transition graph per task so transitions discovered by different policy versions jointly shape credit for current rollouts, and splits a fixed rollout budget between exploration slots for fresh sampling and revisit slots that reattempt earlier tasks, pairing each revisit with its original exploration group to form a cross-temporal comparison. Historical transitions and scores act only as structural and detached statistical references and are never replayed in the policy loss. On ALFWorld and WebShop, it consistently outperforms prior group-based and graph-based policy optimization methods.

arXiv:2609.03383 PDF

LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL

Sijie Wang, Zhiqiang Tan, Xinrui Yang, Shaohuai Shi Reinforcement learning post-training for diffusion image and video generators, as in DanceGRPO and FlowGRPO, recomputes selected timesteps with gradient tracking after rollout — mathematically redundant when rollout and update run on-policy through the same backend, though simply keeping gradients during rollout costs a lot of memory. LeanGRPO restructures the data-parallel layout and adds two recompute-free schedules: LeanGRPO-Retain reuses the rollout's computation graph and saved activations directly for the backward pass, while LeanGRPO-Reweight backpropagates each selected step immediately using a provisional advantage, delays gradient synchronization, and corrects with the true advantage once the trajectory completes. On FLUX.1-dev and Wan, the schedules give up to 1.83x end-to-end speedup while preserving the original optimization objective.

arXiv:2609.03528 PDF

Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners

David Milec, Spyridon Samothrakis, Michael Fairbank, Dennis J. N. J. Soemers Reinforcement learning for games relies on neural networks largely because they learn incrementally, which suits the shifting data distribution of self-play, even though gradient-boosted trees dominate tabular supervised learning and game states — discrete actions, card identities, board positions — are inherently tabular. LUGL decouples data collection from model fitting: a local phase runs self-play and accumulates Q-values, V-values, policies or regrets in a finite table, then a global phase trains a function approximator on that table to generalise to unseen states before the table is cleared, which lets non-incremental learners survive distributional shift. Across four perfect-information games (Tic-tac-toe, Connect-4, Othello, Hex) and five imperfect-information ones (Kuhn poker, Leduc Hold'em, Liar's Dice, Goofspiel, Flop5 Hold'em), LightGBM-based agents are competitive with or superior to DQN and DeepCFR on every benchmark tested.

arXiv:2609.03660 PDF

Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learning

Oussama Hidaoui, Omer Ebead, Ulrich Armel Mbou Sob, Siddarth Singh, Juan Claude Formanek, Felix Chalumeau et al. Offline multi-agent reinforcement learning struggles to handle tasks it never saw during training, and it has been unclear which axis of scaling actually buys that transfer. The authors extend offline sequence-modelling architectures to cope with multi-task observation and action spaces and with agent counts that vary between tasks, then run a large empirical study of task diversity, dataset size and network capacity. Across Connector, RWARE, SMAX and LBF, scaling the diversity of training tasks, not the raw size of the dataset, is the dominant factor in zero-shot transfer, with multi-task models averaging a 3.2x improvement on held-out tasks over single-task models and consistently beating strong behaviour-cloning baselines.

arXiv:2609.03667 PDF

Multi-step Proximal Policy Improvement in Offline Reinforcement Learning

Soohyun Choi, Seonvin Cho, Songnam Hong Offline reinforcement learning faces a tension between staying near dataset-supported actions, which keeps value estimates trustworthy, and moving away from the behaviour policy, which is where real improvement lives. Treating policies as points on a probability manifold with a chosen metric, the authors show that a broad class of offline actor objectives amounts to a single proximal policy improvement step, an implicit discretization of a manifold gradient flow driven by the critic. Multi-step proximal policy improvement composes several re-centered proximal steps instead, allowing controlled movement beyond dataset support while keeping proximal regularization at each step, with instantiations for deterministic and diagonal-Gaussian policies. On D4RL, a few refinement steps improve strong baselines including TD3+BC, ReBRAC, and IQL on many tasks, and diagnostics separate re-centering from mere update scheduling and characterize failure under critic error.

arXiv:2609.03842 PDF

Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO

Hyun Bin Park, Du-Seong Chang Reinforcement learning post-training for reasoning models spends most of its wall-clock time generating fresh rollouts, especially for agents that must interact with an environment, and replaying past trajectories could cut that cost — but published replay schemes are tangled up with exploration heuristics and mixed-policy objectives, so nobody knows what replay itself contributes. Headroom-Drift Replay is a minimal replay control for Group Relative Policy Optimization (GRPO) built from two independent decisions: Headroom ranks stored rollout groups by how much learning value they still hold, and Drift filters out groups too far from the current policy to be safely reused, leaving the on-policy stream and everything else untouched. Across math, multimodal, and agentic-search benchmarks this single intervention beats naive replay and matches or exceeds more elaborate replay pipelines on Avg Mean@32, and in agentic search — where environment calls dominate cost — it reaches comparable quality in materially less wall-clock time.

arXiv:2609.03941 PDF

Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

Jiacheng Xu, Wentao Zhang, Zhiyi Lyu, Fuxiang Zhang, Chaojie Wang, Yang Liu et al. Reinforcement learning on executable feedback has pushed code generation forward, but it depends on test cases that are both correct with respect to the reference solution and sharp enough to expose wrong programs — and such tests are scarce. Framing test writing as an adversarial problem against the solver's current weaknesses, TCS (Test Cases Scaling) trains a test generator in two stages from a rolling policy-aligned buffer: first producing tests consistent with the reference solution, then narrowing the buffer to the solver's active failure modes and learning counterexamples. On TACO and LiveCodeBench this raises both pass@1 and the accuracy of picking the right answer at inference time using the generated tests, and the learned generator also transfers to selecting among outputs from other LLMs.

arXiv:2609.03955 PDF

Spurious Advantage Hidden in GRPO

Jiamian Wang, Samyadeep Basu, Koustava Goswami, Tong Yu, Zhiqiang Tao Group Relative Policy Optimization (GRPO) scores each rollout using within-group reward statistics, so a rollout that lands on the right answer by guessing receives the same large advantage as one that reasoned its way there — an effect named spurious advantage. It appears when the answer space is small, when open-ended tasks contain bounded sub-cases, and when search agents have enough budget that many paths reach the same answer, all of which push the policy toward guess-like behavior. The proposed replacement, SIGNBALANCE, keeps only the verifier's sign, applies a global scale, and restores zero-mean balance through a stop-gradient per-class rescaling. It matches GRPO on open-answer math while improving on bounded-answer math and search-agent benchmarks across model scales.

arXiv:2609.04063 PDF

Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR

Boyan Li, Bingsen Chen, Chenghao Yang, Ping Nie, Chen Zhao, Xi Ye Reasoning models are post-trained with both reinforcement learning from verifiable rewards (RLVR) and on-policy distillation (OPD), and prior work fuses the two signals within a single step, either as a weighted sum or as a teacher-modulated rescaling of the RL advantage. Running them sequentially instead — OPD first, then RL — outperforms pure OPD, pure RLVR, and all joint baselines across logic and math reasoning benchmarks. Analysis of pass@k behavior, learning dynamics, and parameter updates gives a consistent explanation: OPD expands the student's coverage of solutions the teacher supports and RL sharpens within that support, while optimizing both at once makes them interfere. As a practical recipe, the OPD validation score signals when to switch to RL, and OPD is a better cold start for RL than supervised fine-tuning.

arXiv:2609.04108 PDF

3 more specialized papers

Vision 15

Position: Unlabeled IS NOT Equal to No Human Supervision in Visual Learning

Dong Lao cross-listed Much of modern computer vision builds on representations learned from large-scale unlabeled data, all filed under the single term 'unsupervised', yet different data curation schemes and training objectives embed substantially different human priors that models then rely on. The position advanced here is that the absence of labels does not imply the absence of human supervision, and that the umbrella term obscures distinctions needed for fair comparison — a point the authors connect to the sharp decline since 2021 in papers titled 'unsupervised' at flagship vision conferences even as the area keeps growing. Rather than rejecting large-scale pre-training, they call for standardized disclosure: state the priors embedded in data selection and learning objectives, and specify which pipeline components depend on which assumptions.

arXiv:2609.03077 PDF

Learning to Zoom Efficiently with a Contrastive Curriculum

Falko Helm, Iryna Gurevych cross-listed Zoom-in tools let visual agents handle high-resolution images, but teaching multimodal LLMs to use them normally requires a costly warm-start supervised fine-tuning stage. An InfoNCE-style intrinsic reward removes that requirement by contrasting the correct tool call against a curriculum of increasingly hard negative crops, needing no extra labels. On V*, HRBench, and MME-RealWorld the method is competitive at lower cost and beats all baselines when used as a drop-in replacement for supervised fine-tuning; the accompanying synthetic Muffin&Chihuahua dataset of muffin-or-chihuahua grids with region-of-interest labels shows recall of the zoomed region correlates most strongly with final task accuracy.

arXiv:2609.03206 PDF

TruncGradGS: Improved 3D Gaussian Splatting via Truncated Gradient Updates

Theo Morales, Nhat-Quynh Le-Pham, Robin Atkins, Binh-Son Hua cross-listed Fitting 3D Gaussian primitives to images breaks down because of vanishing gradients: a pixel far from a given Gaussian contributes almost no gradient signal to that primitive's attributes, leaving parts of the scene poorly reconstructed. The proposed method replaces the standard update with a piecewise truncated gradient formulation that keeps optimization stable and less sensitive to how primitives are initialized. Gains hold for both random and COLMAP initializations and carry over from static to dynamic Gaussian Splatting, and the authors additionally release a synthetic 3D dataset after finding existing dynamic-scene benchmarks inadequate for isolating these effects.

arXiv:2609.03534 PDF

How Far Can Synthetic Data Take Thai OCR?

Kunat Pipatanakul Synthetic pages give optical character recognition (OCR) exact labels at scale, but realism bundles together several separable factors: source domain, page context, typography, spatial layout, and glyph variation. A controlled document-reconstruction pipeline isolates each factor for Thai documents and finds non-text page context barely matters, while typeface diversity, two-dimensional structure, and real handwriting glyphs all help transfer, and whether matching the source domain pays off depends on training granularity, in-domain reconstruction nearly matching real supervision at page level (1.82 versus 1.31 percent character error rate) yet losing badly at crop level (15.59 versus 5.52 percent). Applying those lessons, the 0.9-billion-parameter PaddleOCR-VL-1.6 was adapted with 45,723 synthetic pages and no real OCR labels into Wayu-Paxa-OCR-Zero, cutting median character error rate on printed pages from 6.64 to 1.24 percent and on handwriting from 74.87 to 20.55 percent while beating the larger Typhoon OCR v1 7B on all five evaluation sets.

arXiv:2609.03595 PDF

Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

Julian Truetsch, Felix Hauser, Christoph Stiller, Frank Bieder cross-listed Knowing what is actually inside a large autonomous driving dataset matters for catching domain shift between where a model was trained and where it drives, but current pipelines depend on metadata, fixed label sets or manual inspection. The work applies set difference captioning — given a target and a reference set of images, generate a natural-language hypothesis describing how they differ — and adapts a two-stage formulation to driving by operating on object-centric patches from a detector, which simplifies aggregation and lets differences be traced to specific object instances or categories. AD-Diff Bench is introduced to evaluate this in-domain using only open-weight models, including low-concentration settings where the real difference affects only a sparse fraction of the target set.

arXiv:2609.03677 PDF

LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes

Chuyan Chen, Haoxing Chen, Kun Chen, Zhenglin Cheng, Long Cui, Ruishan Fang et al. cross-listed LLaDA-Image pairs a 6-billion-parameter Diffusion Transformer (DiT) trained from scratch with a frozen vision-language understanding module built on the LLaDA2.0-Mini diffusion language model, deliberately establishing a visual generative prior through image-only pre-training and mid-training before leaning on paired image-text data. The 220-million-sample pipeline uses parameter-free RMSNorm throughout the transformer and the Muon optimizer, and the model is distilled into LLaDA-Image-Turbo for inference in 2-4 sampling steps. On Qwen-Image-Bench it scores 53.53 on the English track and 53.38 on the Chinese track, the best reported among open-source models on both, with weights, training code, and recipes released.

arXiv:2609.03796 PDF

Sparse auto-regressive modeling for scene generation from multi-view images

Thomas Lucas, Maxime Pietrantoni, Philippe Weinzaepfel, Wonjune Cho, Bardienus Pieter Duisterhof, Vincent Leroy et al. cross-listed Turning a handful of unconstrained photographs into a complete 3D scene requires inventing content the cameras never saw, which feed-forward reconstruction cannot do and dense volumetric generative models make too expensive. SPAR3S learns a compact 3D latent space in which only occupied voxels are represented, training it directly from multi-view images through differentiable 3D Gaussian Splatting rather than ground-truth 3D geometry. Scene completion then becomes a token-prediction problem: a masked autoregressive transformer jointly predicts which voxels are occupied and what latent code each holds. Novel-view quality on synthetic indoor scenes exceeds prior work, and the method also transfers to real footage from RealEstate10k.

arXiv:2609.03931 PDF

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen, Muntasir Wahed, Nabeel Bashir, Xiaona Zhou et al. cross-listed Instruction-guided video edits and subject-guided edits driven by a reference image are typically served by separate systems. EditVid handles both without any training, combining sparse causal memory for local coherence, correspondence-based post-attention token injection to hold identity across long ranges, and soft latent blending to keep edits confined to the intended region. The same pipeline covers style transfer, attribute modification, object insertion, part-level editing, and subject replacement, scoring 78.16 on FiVE-Acc versus 58.95 for the strongest training-free baseline, staying competitive on IVEBench, and drawing 51.8% overall preference in a user study against seven competing methods.

arXiv:2609.04190 PDF

7 more specialized papers

Robotics 12

Sensing Which Modality Matters: Evidence-Gated Regularization for Robust VLA Policies

Yue Yang, Diego Romeres, Chiori Hori, Gedas Bertasius, Daniel Szafir, Siddarth Jain cross-listed Vision-Language-Action (VLA) robot policies trained on limited, homogeneous demonstrations latch onto spurious correlations between sensors — a failure called modality entanglement — so they break when an uninformative camera is occluded or when only one informative sensor survives. Evidence-Gated Regularization (EGR) computes a per-frame, per-sensor task-relevance signal that gates two consistency objectives during training (invariance to low-evidence sensors, sufficiency of high-evidence ones) and adds no inference-time cost. On a new BEHAVIOR-1K benchmark it raises success from 12.5% to 16.4% under full modalities and from 2.8% to 6.1% under single-sensor fallback, while on a bi-manual two-arm real robot facing physical distractors success climbs from 30% to 85%.

arXiv:2609.03142 PDF

Latent Energy Action Planning with World Models

Phu Pham, Aniket Bera Model predictive control over a learned latent world model can pick action sequences that score well on the latent objective yet whose decoded terminal state does not actually match the goal. Latent Energy Action Planning (LEAP) treats the whole action horizon as one differentiable variable optimized through a frozen LeWorldModel, combining terminal latent goal matching with a terminal-window state energy so that both the predicted latent and the decoded descriptor must agree with the goal; a goal-conditioned proposal seeds the search, a quasi-Newton solver refines through the autoregressive rollout, and projection enforces action limits. Across four control domains with the official checkpoints, mean success rises from 77.5% with cross-entropy-method planning to 94.8%, a 17.3-point gain, without retraining the world model.

arXiv:2609.03294 PDF

Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Maps

Shuning Zhang, Liang Li, Yunheng Wang, Tao Wang, Yihang Kang, Renjing Xu cross-listed Pairing a drone's overhead view with a ground robot's first-person view for language-guided navigation has so far produced no stable cooperation, and naive semantic message-passing between the two can make things worse. AGC-VLN is a training-free baseline that exploits the split between vision-language-model reasoning and deterministic geometric execution: the unmanned aerial vehicle draws the ground robot's reported pose and the anchored target onto its bird's-eye view as distance-labelled markers, and the ground robot uses that shared map to plan a road-following route under closed-loop control while the drone separately runs a spatial-search routine called 3D-SPF. Over 100 closed-loop episodes in CARLA-Air's Town10HD scene it reaches a 77.0% joint success rate, 27 points above the weaker agent working alone and 24 points above the strongest published single-agent baseline, Travel UAV.

arXiv:2609.03483 PDF

BRIDGE: An Open-Source Humanoid Platform via Morphology-Control Co-Design for Physical AI

Jianren Wang, Letian Qian, Zikai Wang, Weiwei Wu, Junjie Zong, Abhinav Gupta et al. cross-listed Humanoid hardware is typically designed independently of the whole-body controller, which limits how fluidly a robot can reproduce recorded human motion. The framework presented optimizes morphology and control jointly against human motion data, scored with a new metric that combines kinematic retargeting fidelity with dynamic tracking performance, and reports better results than the Bumi, K1, and Toddlerbot humanoids on every metric. The design is realized as Bridge, an 88 cm open-source humanoid released together with its control policy, demonstrated on locomotion, balance under disturbance, and highly dynamic maneuvers.

arXiv:2609.03497 PDF

Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning

Muyuan Liu (GENISOM AI, Beijing, China), Yue Huang (GENISOM AI, Beijing, China) et al. cross-listed World models based on the Joint Embedding Predictive Architecture (JEPA) let a robot plan toward a goal image without predicting future pixels, but training on latent prediction alone gives no guarantee that the learned representation keeps the information a controller needs. The proposed model adds two objectives end to end: an inverse dynamics term that forces latent transitions to reveal the action that caused them and so resists collapse, and a state alignment term that ties consecutive representations to the underlying physical configuration and motion. Success rates reach 100 percent on TwoRoom, 98 percent on PushT, and 87 percent on OGBench-Cube, with ablations showing state alignment improves on inverse dynamics alone in all four tasks, and a subspace analysis showing the state-aligned model spreads transition energy over more dimensions than the LeWorldModel baseline.

arXiv:2609.03565 PDF

FailBench: How Reliable are VLMs at Judging Robot Task Success?

Zaruhi Navasardyan, Tatul Danielyan, Hrant Davtyan cross-listed Vision-language models (VLMs) are increasingly asked to judge whether a robot manipulation attempt succeeded, but existing benchmarks say little about whether that judgment generalizes across domains. FailBench collects 2,197 manipulation attempts from 14 sources, 12 real and 2 simulated, with 75 percent of failures occurring naturally and six real-world sources drawn from datasets never intended for failure detection. Evaluating 13 detectors, the best reaches only 0.77 mean balanced accuracy, and models fine-tuned specifically for failure detection consistently score below general-purpose VLMs and below their own pretrained baselines; performance tracks what visual evidence is required, near-saturating when success is visible as object motion but dropping to near chance on contact-heavy assembly, with a persistent bias toward calling ambiguous cases successes that more reasoning effort does not remove. Cropping to outcome-relevant regions at the input lifts the top detector by 2.4 percentage points without retraining.

arXiv:2609.03611 PDF

Rethinking World Models for Safety-Critical Embodied Systems

Kailang Ma, Heye Huang, Inhi Kim, Kitae Jang World models have grown from compact latent dynamics into controllable generative simulators, but high likelihood and visual fidelity do not guarantee that a model retains the evidence needed for safe decisions. The position argued here identifies three structural mismatches — likelihood versus risk, prediction versus intervention, and finite-horizon prediction versus accumulated consequences — and proposes the Risk-Informed World Model as a decision-centric alternative organised around consequences, intervention, epistemic uncertainty, and recoverability. It calls for four interlocking capabilities: decision-relevant representation, counterfactual reasoning, safety-critical episodic memory, and runtime safety assurance, distinguishing physical, social, and operational consequences. The central claim is that world models should identify which futures matter and when evidence suffices to act, revise, sense, defer, or abstain, rather than simply predicting likely futures.

arXiv:2609.03774 PDF

FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulation

Yutian Zhang, Siyuan Ma, Liwen Yang, Yang Li, Ce Hao, Haozhen Chi et al. cross-listed Vision-language-action (VLA) models produce semantically sensible robot actions but have no notion of the contact forces those actions create, while whole-body controllers keep a robot stable without distinguishing intended manipulation forces from external disturbances. FWBC-VLA closes that gap for wheeled-legged robots without adding force/torque hardware: a sensorless residual-torque estimator called HSR-Force infers contact strength and its rate of change, these estimates become tokens injected into the VLA action expert so the policy perceives contact onset, sustained load, and release, and a compensation generator fuses proprioception with Jacobian-derived force estimates to produce corrective actions for the whole-body controller. The backbone is fine-tuned on a WL&Arm Dataset of more than 5,000 episodes, and real-robot trials cover whiteboard wiping and opening a door fitted with a self-closing mechanism.

arXiv:2609.03889 PDF

Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis

Sixu Yan, Shikang Wang, Binhua Huang, Xuanlai Tang, Guohua Fan, Fan Huang et al. cross-listed Vision-language-grasp systems usually wire foundation models directly into an end-to-end grasp policy, so any change in task understanding forces retraining of the policy. AdaRoboVLG decouples the two: a base policy generates and evaluates physically feasible grasp candidates through explicit kinematic mapping and force-closure stability estimation, while separate foundation-model modules contribute spatial, cognitive, and temporal priors that compose into the synthesis step. Simulation and real-robot experiments show the base policy learns efficiently and generalizes across different robotic hands while absorbing the priors without retraining, matching state-of-the-art grasp performance and enabling functional grasping in cluttered, dynamic scenes.

arXiv:2609.04096 PDF

3 more specialized papers

Reasoning 8

RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory

Yuxiang Wang, Kunyu Feng, Yingda Shen, Haoning Xu, Junyu Wang, Zhizheng Wu Looping a block of middle layers gives a language model more effective depth without extra parameters or extra generated tokens, but each iteration sees only the previous output and the loop count is fixed, so easy inputs waste compute and hard ones get too little. RecurTrace adds Loop Memory Attention, which lets each looped layer attend to its own states from earlier iterations along a loop-time axis, plus a halting head trained against an oracle that marks when more depth still lowers loss. On MathQA with a shared looped backbone it reaches 56.9% accuracy using an average of 2.0 loops, 2.2 points above the best fixed loop depth at matched compute, while ACT and PonderNet collapse to a single loop and CALM needs 5.6 loops for 54.1%; gains over same-budget fine-tuned baselines grow from 0.6 to 3.4 points as models scale from 0.6B to 8B.

arXiv:2609.03379 PDF

It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories

Yigit Utku Bulut Claims that reasoning traces contain 'breakthrough' moments, or that an attempt's fate is legible from its early tokens, rest on measurements lacking a counterfactual control matched to the claim. Two controls are supplied: a restart-controlled truncation probe comparing continuation from a prefix against generation from scratch at equal total token budget, and a difficulty-controlled test of early-window internal signals. Across 178 problem-model cells built from 89 MATH problems and two small open models, exactly one cell survives as genuinely prefix-limited, and continuing a model's own prefix beats restarting in all nine comparable cases — compute compression rather than expanded reachability. For the second claim, a trace-blind difficulty proxy reaches AUROC 0.873 on 192K DeepSeek-R1 generations, inside the range of published probes, and a close reconstruction of a published early-window positive is statistically indistinguishable from chance within problem, so a question-only baseline or within-problem evaluation is required.

arXiv:2609.03436 PDF

AutoGraphForge: Towards Automated Graph Theory Discovery

J\'an Pastorek An in-progress pipeline chains conjecture generation, refutation, formalization, and proving for graph theory. A Graffiti3 generator proposes invariant relations over a small evolving table of graphs that grows only by counterexamples to its own conjectures; a novelty filter of 559 classical and folklore relations, closed under transitive composition and linear substitution, uses a linear program to reject anything already implied; survivors are tested against roughly 348,000 graphs assembled from the House of Graphs invariant export, the exhaustive census of connected graphs on at most nine vertices, extremal families, and random models. Several cluster rounds yielded 6,522 conjectures surviving refutation, novelty filtering, and active counterexample search, including relations between the annihilation number and the edge-cover number for bipartite and regular graphs that the authors prove by hand, after which each survivor is translated into a Lean 4 skeleton and attacked by DeepSeek-Prover-V2-671B and OProver-32B behind an independent kernel check against pinned mathlib4.

arXiv:2609.03478 PDF

</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination

Seunghee Koh, Sungjae Choi, Minchan Kwon, Sunghyun Baek, Junmo Kim Training-free early-exit methods cut long chain-of-thought traces short by injecting an end-of-think token (EoT, the </think> marker) to force the switch from reasoning to answering. The authors find the injected token often fails to produce a clean answering phase: the model keeps generating reasoning-like text until it emits another EoT on its own, and the length of that span grows with the number of reasoning tokens the early exit saved — a phenomenon they name spurious chain-of-thought termination. Testing the hypothesis that the model simply attends too little to the injected token, Exit-token Attention Biasing reduces both the spurious continuation and the answering-phase length across four large reasoning models, five benchmarks and two early-exit methods, showing that conforming to a model's explicit think-block format does not by itself control its internal state transition.

arXiv:2609.03633 PDF

Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning

Kevin Du, Alexander Hoyle, Laura Ruis, Acyr Locatelli Chain-of-thought traces look like a readable window into a model's reasoning, and a growing body of work leans on that by having LLM judges diagnose errors, assess faithfulness, and supply step-level supervision for process reward models. Defining a step's importance as its advantage — the change in expected reward from including it, estimated with Monte Carlo rollouts — the authors test whether the step's text carries information about its functional role. Sufficiently capable judges beat a prevalence baseline but fall well short of the noise ceiling, and a fine-tuned step-level critic improves substantially only on incorrect responses, implying that step importance is only partly recoverable from the trace text and that legibility should not be equated with interpretability.

arXiv:2609.04194 PDF

3 more specialized papers