The images are selected from seven domains: ANIMALS PLANTS, BUILDINGS, CLOTHING, FOOD, HOME, PEOPLE, VEHICLES.

Within each domain, the dataset aimed at collecting instances at different taxonomic levels to cover a wide range of phenomena. It defined a set of 52 synsets that are used to collect object instances from VG.

ManyNames: Sampling of Instances

ManyNames gathers many more names per object than previous datasets: 35.3 on average. It also contains the most variability, since objects have on average 5.7 names. ManyNames shows high potential use for studying the degree of inter-subject naming agreement, and what factors influence variation. 

Despite the care in filtering out objects that are occluded or have unclear bounding boxes, there are still many examples where annotators identified different objects for the same box. Typically, workers named an adjacent object or one supported by the target object, or a part of the target object. While some of these cases are arguably annotation errors, in many cases it is not possible to distinguish which object is being indicated by the box. Referential uncertainty of this kind is a roadblock for the use of L&V resources to study naming variation.

Pros and Cons of ManyNames

The data is taken from VisualGenome (VG) because its dense annotations of images can be beneficial to analyze the factors affecting naming, and how these impact the modeling of natural language in L&V. VG suits the purpose of collecting names for naturalistic instances of common objects, as it has images of varying complexity, with close-ups as well as images with many objects.

University of Michigan - Ann Arbor

There are three previously existing resources that can be of use for analysis and modeling of object naming: RefCOCO (and a variant, RefCOCO+), Flickr30k Entities, and Visual Genome.

Visual Genome: It is one of the most densely and richly annotated resources currently available in L&V. VG aims at providing a full set of descriptions of the scenes which images depict in order to spur complete scene understanding. The data collection followed a complex procedure, involving many different rounds of annotation.

Object Names in Existing L&V Resources

Silberer, C., Zarrieß, S., & Boleda, G. (2020). Object naming in language and vision: A survey and a new dataset. In Calzolari N, Béchet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Moreno A, Odijk J, Piperidis S, editors. Proceedings of the 12th Language Resources and Evaluation Conference; 2020 May 13-15; Marseilles, France. Stroudsburg (PA): ACL; 2020. p. 5792-801. ACL (Association for Computational Linguistics).

Learn Before

Related

Learn After