Research Reality Cards

AVA improves performance on vision-language benchmarks by an average of +1.7 points, with gains of up to +2.4 points on VizWiz, while maintaining robustness against noisy demonstrations.

+1.7 points average improvement on five vision-language benchmarks with interleaved 4-shot prompts.

No specific authors or detailed methodology provided, which may hinder reproducibility.

EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline while producing higher-quality synthetic experiences.

Improved average task success by 13.3 points.

The paper does not specify potential limitations or reproducibility concerns.

MIRAGE achieves accurate metric trajectory estimation, outperforming COLMAP in computational efficiency for vision-only benchmarks.

MIRAGE establishes accurate 6-DoF reference trajectories for a new vision-only benchmark.

Scarcity of long-horizon monocular benchmarks for large-scale indoor reconstruction may limit reproducibility.

The proposed method allows for training compact student encoders that are up to 10x more efficient than larger teacher foundation models while matching or outperforming their performance.

Achieved up to 10x efficiency in training compact student encoders compared to larger teacher models.

The paper does not specify potential limitations or reproducibility concerns.

SWE-Duel successfully demonstrates a self-scaling benchmark where coding agents evaluate each other, leading to a significant performance difference based on task authoring capability.

GPT-5.5 with Codex CLI repaired 98.3% of the defects it reviewed.

The study's results may be limited by the specific repositories and languages used, which may not generalize to all coding environments.

The proposed CircuitTree method enables the composition of multiple circuits into a single model that can classify syllables with 64.8% accuracy.

Achieved 64.8% accuracy in transcribing open syllables, within 1.2 points of a 66.0% ceiling.

The method's effectiveness may depend on the specific circuits used and their pruning from HuBERT.

Algebraic integer quantization (AIQ) achieves competitive accuracy compared to the MX format for large language model inference while enabling error-free integer arithmetic.

AIQ demonstrates competitive accuracy across diverse models and benchmarks at equivalent bit-per-parameter budgets.

Generalizations to higher-order algebraic integers are still a promising direction and may not be fully explored in the current work.

The input-dependent phase improves performance on longer sequences, reducing perplexity from 12.81 to 3.76 when tested beyond training length.

Achieved a perplexity reduction from 12.81 to 3.76 on 1024 tokens after training on 256 tokens.

The learned phase does not improve retrieval and is limited by the model's exposure to training history.

MCTS is 2.88% worse than randomized-restart list scheduling under equal budget conditions, indicating that simpler methods can outperform MCTS in certain scenarios.

At a budget of 2048 evaluated schedules, MCTS is 2.88% worse than the randomized-restart method.

Results are confined to a controlled synthetic domain with limited task and processor configurations, raising concerns about generalizability.

The paper identifies five major shortcomings of existing faithfulness metrics in XAI and suggests four research directions to enhance their reliability.

The paper outlines five major shortcomings of current faithfulness metrics.

Existing metrics for quantifying faithfulness are either infeasible or computationally expensive.

The deployment regime significantly influences error variation in PV disaggregation models, more so than model choice itself.

The best model achieved a daylight mean absolute error (MAE) of 0.104 kW on seen households, increasing to 0.185 kW on unseen households.

The study's findings may vary based on the specific deployment regime and the adaptation to new regions, which could limit generalizability.

AVA improves average accuracy by up to 2.40 points over the strongest same-setting baselines while significantly reducing the prior-only answer rate and hallucination rate.

Increases visual-dependency score from 0.28 to 0.49.

The framework's performance is evaluated without fine-tuning, which may limit its adaptability to specific use cases.

DCOCO achieves up to 6.6x speedup over vanilla decoding while maintaining high throughput across various configurations.

DCOCO outperforms competing methods by 6-23% across 4-32 concurrent requests.

The paper does not specify the authors or provide detailed reproducibility guidelines.

EliteTable demonstrates significant improvement in candidate scores across six Gymnasium control tasks, with some tasks achieving score ceilings in the first generation.

Candidate-score medians rose in all six tasks over ten generations.

Comparisons and ablations are needed to establish EliteTable's relative performance and the contributions of its components.

AutoIndexer enables post-generation context-editing capabilities that surpass those of most autoregressive and diffusion models without sacrificing performance on standard benchmarks.

Exhibits post-generation context-editing capabilities that most autoregressive and diffusion models lack.

The robustness to erroneous context may vary based on the quality of perturbation training data.

The CLUE framework effectively automates event type induction and annotation, reducing reliance on manual processes.

The framework utilizes contrastive loss guided clustering for event type induction.

The effectiveness of the method may vary based on the dataset's domain and scope, which could affect reproducibility.

SpireBench demonstrates that while frontier models can match average human players in Slay the Spire, they significantly lag behind expert players, particularly in long-term strategic decision-making.

20 large language models were tested, with frontier models matching average human performance but falling short against experts as run length increased.

Models do not improve from the memories they write across runs, often leading to incorrect and overly cautious decisions.

The implementation of Kairos optimizes convergence for a byte-level foundation model across multiple modalities, achieving a minimum loss near $10^{-3}$ with specific attention mechanisms.

The two attention variants tested reach a minimum loss near $10^{-3}$ after 3,000 steps.

The study does not address generalization, multimodal training, or comparison with existing models, which raises concerns about the reproducibility of results.

Coral-KV improves PSNR by up to 4.95 dB while reducing cache memory usage by up to 48% compared to existing methods.

Improved PSNR by 4.95 dB while using 37% less cache memory on MAGI-1.

The paper does not provide specific details on the implementation or datasets used for reproducibility.

ELR enables looped language models to achieve non-recurrent model accuracy at one loop without additional cost, while allowing for improved accuracy with more loops.

At one loop, ELR models are 0.52 points below and 1.04 points above their matched baseline on a ten-task average.

The paper does not specify author names or provide detailed reproducibility metrics.

The dataset includes 1,211 samples of stereo video clips and stills, released in camera-native formats along with lens calibration and conversion tools.

Native-fisheye coding required 38% more bitrate than half-equirectangular coding at equal viewport quality across all tested clips.

The dataset is limited to recordings from specific cameras, which may affect generalizability to other systems.

SLVR significantly improves multimodal reasoning benchmarks with structured latent supervision, achieving absolute gains of +9.4 on MMVP and +14.2 on BLINK Relation.

Achieved absolute gains of +9.4 on MMVP and +14.2 on BLINK Relation.

The method's reliance on specific training signals may limit generalizability to other tasks or datasets.

The method achieved a 50% reduction in cumulative input tokens and significantly lowered API costs while maintaining functionality.

Used 50% fewer cumulative input tokens, resulting in an estimated API cost of USD 1.28-1.44 versus approximately USD 4.38.

The method did not fully satisfy follow-up evaluations and showed variability in performance across different tasks.

The framework significantly improves the efficiency of TMP preparation by fine-tuning multiple open-source LLMs.

Fine-tuning improves performance across standard text generation metrics.

LLMs tend to over-generate strategies and struggle with project-specific justifications and accurate cost estimates.

MESSENGER achieves state-of-the-art performance in long-horizon future extrapolation, reducing EPE3D by 71.6% on nuScenes and 67.7% on Argoverse 2.

Reduces EPE3D by 71.6% on nuScenes.

The computational overhead may increase with more input frames, potentially affecting reproducibility in resource-constrained environments.

The proposed autoencoder achieves compressed representations comparable to lossless text compression algorithms at 2.24 bits per byte while maintaining good reconstruction and performance on downstream tasks.

Achieves 2.24 bits per byte compression on web text data.

The paper does not specify the authors, which may affect reproducibility and validation of results.

The proposed architecture effectively discovers disentangled object and attribute representations, demonstrating empirical evidence for the Linear Representation Hypothesis in slot space.

Improvements in DCI scores over state-of-the-art methods.

The paper does not specify the authors, which may limit reproducibility and verification of results.

The study proves that independent majority amplification preserves well-defined languages in the context of agentic AI self-improvement.

Exact verification is established as the zero-randomness case, with placement and completeness results.

The framework's reliance on explicit complexity assumptions may limit reproducibility and general applicability.

The Synthesis Through Simulation (STS) framework guarantees structural validity of generated data while achieving 0.88 average marginal fidelity and 100% constraint satisfaction across ten environments without access to database schemas.

Achieved 0.88 average marginal fidelity and 100% constraint satisfaction across all ten environments.

The framework's effectiveness may be limited by the specific environments and the lack of access to database schemas.

Diffu-LoRA achieves improved subject fidelity and prompt alignment in personalized diffusion models compared to traditional fine-tuning methods.

Experiments show improved overall subject fidelity and prompt alignment relative to evaluated fine-tuning baselines.

The paper does not specify the authors or provide detailed reproducibility guidelines.

Machine learning models can serve as 'cognitive thermometers' that bridge the gap between symbolic logic and connectionist AI, offering a unified approach to measuring semantic complexity.

Emerging evidence shows that logic and machine learning often yield converging results on relative complexity.

The reliance on specific logical languages may limit the generalizability of the findings.

Router telemetry can significantly enhance membership inference attacks on fine-tuned MoE models, increasing true positive rates by 2.7–9.4 percentage points.

Router telemetry improves membership inference over a strong output-signal ensemble by 2.7–9.4 percentage points at 1% false positive rate.

The study's findings may be limited by the specific architectures and data domains tested, which could affect generalizability.

The number of synthetic samples needed for activation probes is determined by coverage rather than the difficulty of the data, with a median half-gain size of 7-11 samples across different concepts.

Probes for high-stakes and harmful situations plateau with as few as 80 samples, while instruction probes require several times more.

The variance in results is significantly influenced by the concept and distribution, which may affect reproducibility across different contexts.

SPERA achieves the highest average balanced accuracy across nine downstream tasks in clinical, cognitive, and BCI applications, demonstrating its effectiveness as a general-purpose EEG model.

Pretrained on approximately 80,000 hours of EEG data from 29,048 subjects across 106 datasets.

The model's performance may vary across different recording conditions, which could affect reproducibility.

OcuFlow achieves superior reflection removal quality and ocular fidelity, outperforming existing methods significantly in user studies.

OcuFlow received 67.32% of user selections in a blind study, 6.2 times the next-best share.

Existing datasets cover limited reflection conditions, which may affect generalization to complex real-world scenarios.

The framework effectively maps headers to 39 interpretable types and assesses data quality issues, providing a reusable workflow for semantic annotation and quality monitoring.

Evaluated across 120,000 header columns with broad practical coverage across noisy real-world metadata.

The official evaluation results were modest, indicating potential issues with benchmark granularity and ontology-selection effects.

Self-evolution of LLM agents can be effectively managed by confining changes to a runtime harness, allowing for reviewable adaptations without altering model weights.

The gated loop admitted 144 of 7,449 candidate changes, none of which worsened error on held-out history.

The gate's fixed tolerance blocked correct replacements in half the seeds at the highest structural severity.

The proposed framework improves anomaly detection in in-vehicle networks by effectively capturing subtle temporal and contextual anomalies through a lightweight Transformer encoder and federated learning.

Achieved robust anomaly detection performance while maintaining efficiency and privacy.

The reliance on open-source datasets may limit generalizability to real-world scenarios.

Freezing the decoder and healing only the encoder achieves parity with other methods while using 3x fewer trainable parameters and 3x less optimizer-state memory.

Achieved a real measured saving of 3x fewer trainable parameters and 3x less optimizer-state memory.

The results are based on a specific compression ratio and may not generalize across different configurations or models.

The authors created a 6.75 billion token corpus from 294,369 documents, which is over three orders of magnitude larger than existing annotated datasets for pre-1918 Polish.

The character error rate on a hand-corrected sample is 0.68%, with 45% of sampled passages being uncorrectable.

The models trained on this corpus reproduce period-specific prejudices, including antisemitic statements.

The MambaXray-PRB framework significantly enhances report generation performance and model interpretability through a multi-stage pre-training and reasoning approach.

Extensive experiments validate the effectiveness of MambaXray-PRB across multiple datasets.

The CheXpert Plus dataset lacks baseline implementations and evaluation results, hindering standardized training and fair comparisons.

The current reliance on a few benchmark datasets limits the potential of autonomous driving research, necessitating a shift to a community-driven data paradigm.

Over 600 autonomous driving datasets exist, yet most remain underused due to fragmentation and limited visibility.

The fragmentation of datasets and incompatible protocols hinder reproducibility and broad research impact.

VCR-Bench integrates 30 video classification models, 14 adversarial attacks, and 10 defense wrappers under a common evaluation protocol, facilitating reproducibility in video classification robustness research.

Evaluated video classifiers on Kinetics-400, reporting metrics such as clean accuracy and attack success rate.

The benchmark's effectiveness may be limited by the diversity of video classification models and the specific configurations used.

The introduction of the Sextant localization head significantly improves pedestrian tracking performance, raising the HOTA score from 47.30 to 49.49 in the PanoPed-S test.

Sextant improves the MOTIP baseline by 2.19 HOTA points in the PanoPed-S test.

The method relies on synthetic data for training and may not generalize well without fine-tuning on real data.

CoDR improves average accuracy across various model-sampler configurations while using far fewer forward passes than prior remasking methods.

CoDR improves accuracy across all evaluated model-sampler configurations with modest overhead.

The method's effectiveness may vary based on the specific model and sampler configurations used.

Tokka-Bench provides a standardized framework for evaluating the efficiency of various tokenizers across 100 natural and 20 programming languages.

The study compares seven BPE tokenizers and finds that vocabulary allocation strategy is more critical than raw vocabulary size.

The paper does not specify the authors, which may limit reproducibility and attribution.

Child ASR adaptation is essential for improving recognition quality in non-native Arabic and English child speech, but direct adaptation can negatively impact adult ASR performance.

Weight-space merging improves the adaptation-retention trade-off, particularly for encoder-CTC and AudioLLM-based ASR systems.

The study may face challenges in generalizing results across different languages and speech contexts.

The BRaVeS framework successfully enforces bounded governance behavior, achieving finite-step convergence without safety-guard violations in simulations.

The LBCF process achieved finite-step convergence across tested parameter-grouping strategies and thresholds.

The safety of the full continuous neural activation space is not proven, which raises concerns about the generalizability of the results.

The study identifies baseline severity as the strongest predictor of depression outcomes and highlights the differing influences of clinical context and demographic factors on short-term and long-term outcomes.

Ridge Regression achieved the best 12-week performance with an RMSE of 5.186 and R^2 of 0.474.

The study's modest sample size and the use of model-based stochastic imputation for missing follow-up outcomes may limit reproducibility.

The Prithvi crop classification model's accuracy significantly declines when observation windows do not align with local crop phenology, but adjustments can recover accuracy without retraining.

Consolidating 13 classes into 10 raised the mean overall accuracy by 8.4 percentage points.

The model's performance is highly sensitive to the alignment of observation windows with local growing seasons, which may limit its reproducibility across different regions.

Adaptation improves in-domain diarization performance but introduces inconsistencies in speaker identity over time.

Adaptation evaluated on 7.5 hours of two-party conversation shows substantial improvement in in-domain performance.

The inconsistency in performance across different evaluation scenarios raises concerns about the generalizability of the adaptation method.

KVFetch recovers verbatim copying performance from 0.8 to 78.4 on RULER-16K, demonstrating substantial improvements in tasks requiring sequential access.

Increases the 13-task average performance by +8.4.

The paper does not provide specific details on the reproducibility of the results or the dataset used.

The BMA provides a foundational framework for emergent consciousness by defining a closed update mechanism that binds self-representations to abstract world models.

The paper defines a Causal Learning Regime (CLR) that diagnoses learnable causal structures in the environment.

The framework may exhibit multiple coherent basins, complicating reproducibility and generalization of results.

Emo-Jev outperforms standard Jev classification with a 67.28% average macro-F1 score while maintaining lower latency and cost.

Standard Jev achieved 62.93% average macro-F1 versus 67.28% for the strongest LLM baseline.

The framework's effectiveness against leading LLMs remains an open question.

MoR-MLLM significantly reduces training memory and computation complexity compared to existing tiny MLLMs while achieving high performance on vision-language tasks.

Achieved a reduction in training memory and computation complexity while retaining high performance on various vision-language tasks.

The paper does not specify the authors, which may hinder reproducibility and validation of results.

The system successfully generated dashboards from natural language requests with a strict success rate of 6/8 on metadata and SQL tasks.

Passed 4/4 single-panel dashboard tasks and met specified outcomes in all ten fault scenarios.

The results do not establish production readiness or general text-to-SQL accuracy.

FluidPD improves overall SLO attainment by up to 94.6 percentage points compared to static configurations.

FluidPD's mechanisms improve SLO attainment by up to 94.6 percentage points.

The paper does not specify the authors or provide detailed experimental setups, which may limit reproducibility.

UniPro achieves strong performance across diverse segmentation settings by enabling annotation-efficient 3D segmentation from sparse 2D initialization.

Extensive experiments show that UniPro reduces slice-by-slice correction effort significantly.

The model's reliance on specific reference-conditioned prediction mechanisms may limit its generalizability across all clinical scenarios.

GameGoCoder, trained on GameGoData, outperforms matched baselines and is comparable to frontier models across game development benchmarks.

GameGoData consists of 55,060 development trajectories across various game types.

The reliance on synthetic trajectories may limit the generalizability of the results to real-world game development scenarios.

The introduction of 'Anchor Divergences' allows for the specification of context-specific semantic geometries in contrastive learning, enhancing the modeling of semantic similarity.

Anchor divergences provide an effective and efficient way to specify context-specific semantic similarity, as demonstrated in retrieval experiments.

The paper does not specify potential limitations or reproducibility concerns.