A Jetson Nano–based speech-to-speech pipeline cut weak English→Spanish outputs back into line without retraining, using a Whisper-tiny ASR front end, Opus-MT, and a multilingual-BERT QE gate to trigger correction; on 1,012 FLORES-200 sentences, the minimum Bayes-risk pass at τ=0.90 gave a statistically significant improvement. Separately, work on Indic MT evaluation finds that COMET is not script-invariant: re-encoding the same content into Latin script explains 22.9% of native-script COMET variance across five languages and reduces agreement with human ratings, pointing to tokeniser-induced bias rather than a harmless formatting effect.

Quality-Aware Self-Correcting Speech Translation on an Edge Device

On-device speech translation can use a QE gate to trigger correction without retraining, and M2 at τ=0.90 is the only variant here that beats greedy decoding while freeing 680 MB by skipping QE ranking.

We present a fully offline speech-to-speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper-tiny ASR feeds an Opus-MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary-pass correction when confidence falls below a pre-defined threshold τ. We compare three correction methods: QE reranking (M1), Minimum Bayes-Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES-200 sentences (English-Spanish), M2 at τ=0.90 produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1->M2) does not hurt quality and frees 680 MB from the critical path. Using a gain-to-edit ratio adapted from the post-editing-effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six language pairs.

Making COMET Comparable Across Scripts: Diagnosis and Correction of Tokeniser-Induced Script Bias in Indic MT Evaluation

COMET scores are script-sensitive in Indic MT, so compare scripts only after COMET-QN normalises each language-script pair; it lifts annotator agreement from 0.300 to 0.399 while preserving within-language orderings.

COMET reports translation quality as a single number, and that number is routinely compared across target languages written in different scripts. Such a comparison assumes Script Invariance: the score should not depend on the writing system that carries the target. We test it on IndicMT Eval by re-encoding the target into Latin script, which changes orthographic form while holding content and human ratings fixed. Script identity then accounts for 22.9% of native-script COMET variance, and agreement with annotators falls in all five languages studied. We trace the effect to the tokeniser and measure it with three label-free diagnostics. The bias is two faults, not one. Scores from different scripts occupy incompatible ranges, and within a single script the metric orders translations less accurately. No order-preserving transform of the score can repair the second fault. The first is removed exactly by COMET-QN, which maps the score distribution of each (language, script) pair onto a shared reference. Pooled agreement with annotators rises from 0.300 to 0.399, which is what makes scores from different scripts safe to place on one axis, and every within-language ordering is provably preserved. A regressor over parity features recovers a further 17.1% of the lost sensitivity. The remainder belongs to the encoder, and no post-processing can reach it. We therefore recommend publishing the normalised score, the three diagnostics, and the identity of the tokeniser they were computed against, so that a reader can tell how much of a score reflects translation quality and how much reflects the writing system.

Papers announced by arXiv on Tuesday, 6 October 2026, covering submissions from 5 October to 6 October 2026.