The analysis of naming variation shows that, on the one hand, there is a fair bit of consistency in the names chosen for objects, and, on the other, also consistent variation. 

University of Michigan - Ann Arbor

ManyNames gathers many more names per object than previous datasets: 35.3 on average. It also contains the most variability, since objects have on average 5.7 names. ManyNames shows high potential use for studying the degree of inter-subject naming agreement, and what factors influence variation. 

Despite the care in filtering out objects that are occluded or have unclear bounding boxes, there are still many examples where annotators identified different objects for the same box. Typically, workers named an adjacent object or one supported by the target object, or a part of the target object. While some of these cases are arguably annotation errors, in many cases it is not possible to distinguish which object is being indicated by the box. Referential uncertainty of this kind is a roadblock for the use of L&V resources to study naming variation.

Pros and Cons of ManyNames

Silberer, C., Zarrieß, S., & Boleda, G. (2020). Object naming in language and vision: A survey and a new dataset. In Calzolari N, Béchet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Moreno A, Odijk J, Piperidis S, editors. Proceedings of the 12th Language Resources and Evaluation Conference; 2020 May 13-15; Marseilles, France. Stroudsburg (PA): ACL; 2020. p. 5792-801. ACL (Association for Computational Linguistics).

Object Naming in Language and Vision: A Survey and a New Dataset

Naming Variation and Agreement analysis

Previous work on object naming has assumed that variation is mostly along a taxonomic axis, and in particular hierarchical. This parameter does not seem to explain the variation in ManyNames.

As shown in the table, in the vast majority of cases, no hierarchical relation between the name and the synset can be retrieved from WordNet. Even factoring in the noise introduced by referential uncertainty, it is clear that a good portion of our data cannot be explained by variation in the level of abstraction of the chosen name.

Learn Before

Related