Concept
Crowdsourcing Workflows: Challenges: Task Derailment, Emergent Complexity, and Cycling
- In the majority of cases, the unsupervised crowd produced unsuitable workflows, or unsuccessful results.
- The most frequent type of failure was derailment, a phenomenon that occurred when the PDS algorithm failed to terminate and continued to produce steps indefinitely.
Derailment can happen for multiple reasons:
- First, workers were confused about appropriate task granularity.
- Second, workers authored subdivisions that could be executed by a single worker, but not split across separate workers.
- Finally, workers who had lost the context of the overall workflow generated decompositions that restated previous tasks as subtasks, leading to cyclic behavior.
0
1
Updated 2021-07-05
Tags
Psychology
Social Science
Empirical Science
Science
Related
Crowdsourcing Workflows: Table 1: Experimental Results
Crowdsourcing Workflows: Snap Judgements
Crowdsourcing Workflows: Successful Planning
Crowdsourcing Workflows: Challenges: Task Derailment, Emergent Complexity, and Cycling
Crowdsourcing Workflows: Task Starvation
Crowdsourcing Workflows: Standard Quality Assurance Techniques are Insufficient