Silberer, C., Zarrieß, S., & Boleda, G. (2020). Object naming in language and vision: A survey and a new dataset. In Calzolari N, Béchet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Moreno A, Odijk J, Piperidis S, editors. Proceedings of the 12th Language Resources and Evaluation Conference; 2020 May 13-15; Marseilles, France. Stroudsburg (PA): ACL; 2020. p. 5792-801. ACL (Association for Computational Linguistics).

University of Michigan - Ann Arbor

A central issue in Language & Vision (L&V) is how speakers refer to objects. This is most prominent for referring expression generation and interpretation, but it also pervades virtually any other L&V task, such as caption generation or visual dialogue. One of the central components of referring expressions are object names.

Object Naming in Language and Vision

Object Naming in Language and Vision: A Survey and a New Dataset

The act of naming an object amounts to that of picking out a nominal to be employed to refer to it. Since an object is simultaneously a member of multiple categories, all the various names that lexicalize these constitute a valid alternative, meaning that the same object can be called by different names.

Object Naming as a Linguistic Phenomenon

Previous work in L&V has collected and used datasets where annotators produced free and natural utterances for a given image. These datasets typically record utterances that are more complex than a single word, such as image captions, referring expressions, visual dialogues or image paragraphs. 

While object names occur in all of these datasets, they are not necessarily marked up and linked to the corresponding image regions.

Object Naming in L&V

There are three previously existing resources that can be of use for analysis and modeling of object naming: RefCOCO (and a variant, RefCOCO+), Flickr30k Entities, and Visual Genome.

Visual Genome: It is one of the most densely and richly annotated resources currently available in L&V. VG aims at providing a full set of descriptions of the scenes which images depict in order to spur complete scene understanding. The data collection followed a complex procedure, involving many different rounds of annotation.

Learn Before

Related