A simple change in loss weighting can make multimodal MT models pay attention to the image when the source text is ambiguous, rather than quietly ignoring the visual input. In parallel, G²PTQ tackles post-training quantization by combining global supervision with gradient compensation, and the paper argues that stale Hessian-only guidance is part of why GPTQ-style methods plateau as quantization progresses.

Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting

For image-guided MT, weight loss toward tokens whose probabilities change with the image, because PCXMI-based weighting improves CoMMuTE accuracy by up to 7 points without hurting general translation.

Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. While models, through multimodal fusion, are able to accept images related to the source text, they can ignore this information. Therefore, increasing their visual sensitivity remains an active research area. In this work, we introduce a training method, Metric-based Loss Weighting, that improves visual grounding of translations by increasing the loss function for tokens that benefit from the accompanying image. We identify these tokens using the Point-wise Cross-mutual Information (PCXMI) metric, which compares the model's output probabilities with and without visual context. We introduce a Congruency-based PCXMI metric and experimentally show that both metrics working in combination yield the best results. We evaluate our method by fine-tuning three pretrained Multimodal Large Language Models on the task of Image-guided Machine Translation for three language directions. Metric-based Loss Weighting outperforms other tested methods on the CoMMuTE contrastive dataset, improving accuracy by up to more than 7 percentage points compared to standard fine-tuning, while maintaining strong general translation performance.

G$^2$PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

For LLM compression, refresh gradient and Hessian estimates block by block during PTQ, because G$^2$PTQ’s globally supervised compensation aligns quantized models better than GPTQ-style baselines.

Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations. Methods with local, layer-wise objectives lack global supervision; while methods with global objectives fix their Hessian estimates at the start and ignore first-order gradients, so their guidance grows stale as quantization proceeds. This paper presents G^2PTQ, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective. By refreshing gradient and Hessian estimates before quantizing each Transformer block, G^2PTQ avoids the staleness of prior global methods. Furthermore, to stabilize the exact first-order compensation, we introduce a trust-region scaling mechanism that dynamically bounds the gradient step to prevent exploding weight updates. Finally, we derive efficient implementations for block-wise Hessian approximation and exact gradient compensation. Experimental results on various model families and bit-widths demonstrate that G^2PTQ enables better alignment with the full-precision model, outperforming state-of-the-art baselines. Code is available at: https://github.com/G2PTQ/G2PTQ.

Papers announced by arXiv on Sunday, 27 September 2026, covering submissions from 24 September to 25 September 2026.