RQ1: To what extent can text-based conversational interfaces support the execution of different types of crowdsourced microtasks?

Researchers addressed RQ1 by comparing execution time, quality of results, and satisfaction of workers who used the standard Web interface and with those who used the conversational interface.

Chatterbox: Research Question 1

RQ2: How do different types of UI input elements in conversational interfaces affect quality-related outcomes in microtasks?

Researchers addressed RQ2 by comparing different implementations of conversational UI elements for single and multiple input selections in microtasks.

Chatterbox: Research Question 2

- The goal of the researchers is to further the understanding of how text-based conversational interfaces could serve as an alternative to the standard Web interfaces typically used for microtask crowdsourcing.
- Researchers carried out experiments to gauge the interest and acceptance of automated, text-based conversational work interfaces by crowd workers, while assessing their performance within different task types.
- Participants were recruited  from the FigureEight (F8) microwork platform.
- Researchers addressed five typical microtask types: information finding, human OCR (captcha), speech transcription, sentiment analysis, and image annotation.

Boston College

- The availability and usage of messaging applications (such as Telegram, Facebook Messenger, etc.), as well as their popularity and recent advances in machine learning capabilities, has sparked new interest in conversational interfaces and chatbots.
- The crowing popularity of conversational interfaces has coincided with flourishing crowdsourcing marketplaces.
- Microtask crowdsourcing allows the interaction with a large crowd of diverse people for data processing or analysis purposes.
- Microtask crowdsourcing is commonly executed by means of dedicated Web platforms, where all the published microtasks are publicly presented to workers, and upon selection of the task, users are directed to a different webpage that the task requires.
- Recent work has shed light on the importance of task design choices made with respect to user interface elements; and on how such choices can influence the quality of work produced and satisfaction among workers.
- Although conversational interfaces have been effectively used in numerous applications, the impact of conversational interfaces in microtask crowdsourcing marketplaces has remained unexplored.
- Lowering the entry barrier for workers to participate effectively in crowdsourcing tasks is an important step towards securing the future of crowd work.
- The availability of effective automated text-based conversational interfaces as an alternative to the traditional Web UI, could broaden the pool of available crowd workers by easing their unfamiliarity with the interface elements.

Chatterbox: Introduction

Mavridis, P., Huang, O., Qiu, S.. Gadiraju, U., & Bozzon, A. (2019) Chatterbox: conversational interfaces for microtask crowdsourcing. Association for Computing Machinery, 243–251. DOI:https://doi-org.ezproxy.lib.uconn.edu/10.1145/3320435.3320439

Learn Before

Related

Learn After