Reliable uncertainty estimates can sit beside a quality score in a single forward pass, and BEiTScore++ shows that with conformalised intervals for reference-free image captioning evaluation on human-rated benchmarks. On the medical side, the CliTe and ParaCliTe corpora provide synthetic multilingual cardiology records in Czech, Dutch, English, Italian, Romanian, Spanish and Swedish, built with hospital clinicians to support AI work under privacy constraints.

BEiTScore++: Reference-free Image Captioning Evaluation with BEiT-3 and Conformalized Quantile Regression

For caption evaluation, BEiTScore++ adds conformal confidence intervals to a single forward pass, and its uncertainty tracks error and domain shift better than Monte Carlo dropout on most benchmarks.

Reliable uncertainty estimation is important for deploying evaluation metrics in real-world settings, where both input ambiguity and domain shifts are common. In this work, we propose an uncertainty-aware extension of the BEiTScore image captioning metric, producing conformally calibrated confidence intervals alongside alignment scores. Our approach models predictive distributions over human judgments, enabling both accurate scoring and principled uncertainty quantification in a single forward pass. We evaluate our BEiTScore++ metric across multiple benchmarks with human-annotated ratings, showing that it preserves the ranking correlation of BEiTScore, while producing intervals whose width tracks the prediction error more closely than Monte Carlo dropout on most benchmarks. The estimated uncertainty also grows under image degradation and on out-of-domain artwork images, making BEiTScore++ a practical tool for uncertainty-aware evaluation of image-caption alignment.

Multilingual Artificial Datasets for AI-Driven Cardiology: the CliTe and ParaCliTe corpora

For clinical MT or terminology work, CliTe and ParaCliTe provide 699 artificial cardiology reports plus a seven-language parallel subset with sentence-level alignment, including manually translated and auto-translated material.

Background and Aims. Comparable, multilingual, validated cardiology corpora are critical to advance trustworthy natural language processing (NLP) models for cardiology but such resources are scarce due to privacy concerns. This study aimed to cover this gap by presenting suitable artificial corpus creation guidelines and by releasing 3 multilanguage corpora. Methods. Following purpose-specific publicly-available corpus construction guidelines, cardiologists from 7 European hospitals generated artificial clinical records in Czech, Dutch, English, Italian, Romanian, Spanish and Swedish, ensuring clinical diversity and sex balance, resulting in the CliTe corpus. A CliTe subset underwent a multi-stage manual translation into the other 6 languages, resulting in the parallel ParaCliTe corpus, while the remaining reports were automatically translated, constituting ParaCliTe Silver. Results. CliTe is a collection of 699 artificial cardiology reports, mostly outpatient notes (340 documents), discharge reports (263), and inpatient notes (89), describing fictional patients with ischaemic heart disease (199 patients), heart failure (188), and valvular disease (79), among others. ParaCliTe contains reports from 94 fictitious patients, each available in seven languages with sentence-level alignment, resulting in 658 documents. The corpus mainly comprises discharge reports (322 documents), and outpatient (196) and inpatient (140) notes, describing patients with heart failure (399 documents), ischaemic heart disease (287), and valvular disease (238). Empirical results illustrate the utility of ParaCliTe and highlight the limitations of state-of-the-art NLP models adapted to the cardiology domain. Conclusions. The creation of multilingual, equitable and realistic corpora of artificial patients capable of strengthening NLP performance for cardiology is feasible when supported by detailed guidelines.

Papers announced by arXiv on Sunday, 8 November 2026, covering submissions from 5 November to 6 November 2026.