pith. sign in

arxiv: 1505.04870 · v4 · pith:SBOSNPG6new · submitted 2015-05-19 · 💻 cs.CV · cs.CL

Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models

Bryan A. Plummer , Liwei Wang , Chris M. Cervantes , Juan C. Caicedo , Julia Hockenmaier , Svetlana Lazebnik This is my paper
classification 💻 cs.CV cs.CL
keywords flickr30kimageentitiesbaselinebenchmarkcaptionsdescriptionmentions
0
0 comments X
read the original abstract

The Flickr30k dataset has become a standard benchmark for sentence-based image description. This paper presents Flickr30k Entities, which augments the 158k captions from Flickr30k with 244k coreference chains, linking mentions of the same entities across different captions for the same image, and associating them with 276k manually annotated bounding boxes. Such annotations are essential for continued progress in automatic image description and grounded language understanding. They enable us to define a new benchmark for localization of textual entity mentions in an image. We present a strong baseline for this task that combines an image-text embedding, detectors for common objects, a color classifier, and a bias towards selecting larger objects. While our baseline rivals in accuracy more complex state-of-the-art models, we show that its gains cannot be easily parlayed into improvements on such tasks as image-sentence retrieval, thus underlining the limitations of current methods and the need for further research.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

    cs.CV 2026-05 unverdicted novelty 6.0

    LocateAnything proposes Parallel Box Decoding for unified generative visual grounding and detection, paired with a 138M-sample dataset, to raise both speed and high-IoU accuracy.

  2. Florence: A New Foundation Model for Computer Vision

    cs.CV 2021-11 unverdicted novelty 6.0

    Florence is a new vision foundation model that learns universal visual-language representations from web-scale data and reports state-of-the-art results on 44 benchmarks including 83.74% zero-shot ImageNet top-1 accuracy.

  3. TuringViT: Making SOTA Vision Transformers Accessible to All

    cs.CV 2026-06 unverdicted novelty 5.0

    TuringViT uses Turing Linear Attention, VISTA-Curation, and dynamic-resolution pretraining to outperform open ViT baselines with 10% data while improving VLM performance and high-resolution latency.

  4. TuringViT: Making SOTA Vision Transformers Accessible to All

    cs.CV 2026-06 unverdicted novelty 5.0

    TuringViT claims a new ViT design with linear attention and curated data that matches SOTA performance using 10% of typical pretraining data while supporting dynamic resolutions and improving VLM integration.

  5. Grounded Scaling: Why Agentic AI Needs Deterministic Environments

    cs.AI 2026-06 unverdicted novelty 5.0

    Agentic AI scaling requires deterministic environments because per-step success probability below 1 causes exponential degradation in k-step chains, addressed via new metrics SCI and DMM plus formal bounds.

  6. Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

    cs.CV 2026-05 unverdicted novelty 4.0

    A native multimodal embedding model from Gemini achieves reported state-of-the-art results on retrieval benchmarks across modalities via large-scale contrastive learning.