Pith. sign in

REVIEW 4 cited by

Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2205.12522 v2 pith:YNHGXOP6 submitted 2022-05-25 cs.CV cs.CL

Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset

Ashish V. Thapliyal , Jordi Pont-Tuset , Xi Chen , Radu Soricut This is my paper
classification cs.CV cs.CL
keywords languagesmassivelymultilingualacrossannotatedcaptioningcaptionscrossmodal-3600
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Research in massively multilingual image captioning has been severely hampered by a lack of high-quality evaluation datasets. In this paper we present the Crossmodal-3600 dataset (XM3600 in short), a geographically diverse set of 3600 images annotated with human-generated reference captions in 36 languages. The images were selected from across the world, covering regions where the 36 languages are spoken, and annotated with captions that achieve consistency in terms of style across all languages, while avoiding annotation artifacts due to direct translation. We apply this benchmark to model selection for massively multilingual image captioning models, and show superior correlation results with human evaluations when using XM3600 as golden references for automatic metrics.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting

    cs.CV 2025-08 unverdicted novelty 7.0

    The paper offers a comprehensive survey and proposes a new taxonomy for continual learning strategies in VLMs and MLLMs to combat catastrophic forgetting beyond traditional methods.

  2. PaLI: A Jointly-Scaled Multilingual Language-Image Model

    cs.CV 2022-09 conditional novelty 7.0

    PaLI jointly scales a 4B-parameter vision transformer with language models on a new 10B multilingual image-text dataset to reach state-of-the-art results on vision-language tasks while keeping a simple modular design.

  3. GRAPE: Let GRPO Supervise Query Rewriting by Ranking for Retrieval

    cs.CV 2025-09 conditional novelty 6.0

    GRAPE applies GRPO to an LLM query rewriter with a corpus-relative ranking reward to improve frozen CLIP retrieval by an average 4.9% Recall@10 on shifted benchmarks without retraining or re-embedding.

  4. BamiBERT: A New BERT-based Language Model for Vietnamese

    cs.CL 2026-07 unverdicted novelty 5.0

    BamiBERT is a new base-sized Vietnamese BERT model trained on raw text that outperforms PhoBERT on 11 of 15 metrics across 8 benchmarks.