Phase 1 Task Design (Crowdsourcing Complex Engineering Tasks)
In this phase of the study, several tasks were assigned to the participants. The first task was only assigned to the crowd workers (and not the graduate students) in order to acquaint them with the simulation. This task consisted of a tutorial, a survey, and practice questions (shown in the image below). The remaining 13 tasks were assigned to both groups, and each of these tasks contained a set of data (3 items) from a single simulation. Each task consisted of one question with the choices “yes,” “no,” or “unable to determine” regarding whether the simulation results were valid or not. The tasks were published on Amazon Mechanical Turk for the crowd workers and graduate students to complete, and they received monetary compensation for every task they completed.

0
1
Tags
Psychology
Social Science
Empirical Science
Science