The researchers had a few design recommendations for future crowdsourcing platforms, including:
- Letting crowdworkers design their own workflows specific to their situation 
- Supporting ongoing reconfiguration of the workflow by crowdworkers (i.e. by treating the workflow as an editable scaffold rather than a strictly pre-specified work plan)
- Making it easier for crowdworkers to communicate and adapt collectively (instead of including features solely to minimize the need for coordination)


New York University

- Workflows are limited tools in unpredictable situations with evolving goals that require open-ended adaptation, making them problematic for complex work
- However, as the results show, the structure that workflows enforce are still important. Future designs must find a balance between structure and flexibility.
- The researchers suggest a few possible ways to improve future designs for crowdsourcing platforms that would incorporate their findings, as well as ways to improve existing platforms.


Implications for Future Design (How Crowdsourcing Workflows Constrain Complex Work)

Retelny, D., Bernstein, M. S., & Valentine, M. A. (2017). No workflow can ever be enough: How crowdsourcing workflows constrain complex work. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1-23.

No Workflow Can Ever Be Enough: How Crowdsourcing Workflows Constrain Complex Work

Supporting Open-Ended Adaptation in Future Crowdsourcing Platforms

The researchers included several concrete design changes that could be implemented in existing platforms, such as:
- A panic button for workers to indicate when they believe the current task design needs to be adjusted
- Allowing workers downstream to inspect and return a task for an earlier stage in the workflow
- Hiring project managers to monitor the workflow and with any uses of the panic button or revision requests
- Informing workers of the structure and default assumptions of the workflow so they can understand any consequences their decisions might have for those downstream


Learn Before

Related