The researchers had a few design recommendations for future crowdsourcing platforms, including:
- Letting crowdworkers design their own workflows specific to their situation 
- Supporting ongoing reconfiguration of the workflow by crowdworkers (i.e. by treating the workflow as an editable scaffold rather than a strictly pre-specified work plan)
- Making it easier for crowdworkers to communicate and adapt collectively (instead of including features solely to minimize the need for coordination)


Supporting Open-Ended Adaptation in Future Crowdsourcing Platforms

The researchers included several concrete design changes that could be implemented in existing platforms, such as:
- A panic button for workers to indicate when they believe the current task design needs to be adjusted
- Allowing workers downstream to inspect and return a task for an earlier stage in the workflow
- Hiring project managers to monitor the workflow and with any uses of the panic button or revision requests
- Informing workers of the structure and default assumptions of the workflow so they can understand any consequences their decisions might have for those downstream


Supporting Adaptation in Existing Crowdsourcing Platforms

- Workflows are limited tools in unpredictable situations with evolving goals that require open-ended adaptation, making them problematic for complex work
- However, as the results show, the structure that workflows enforce are still important. Future designs must find a balance between structure and flexibility.
- The researchers suggest a few possible ways to improve future designs for crowdsourcing platforms that would incorporate their findings, as well as ways to improve existing platforms.


New York University

- Complex tasks require situated action: constant reflection, revision, and adaptation toward one’s goals. The more complex the task, the more necessary adaptation becomes.
- Workflows streamline the work process, reducing the margin of error. 
- However, workflows are static, limiting situated action. As such, complex tasks have traditionally been difficult to achieve using workflows.  

Tradeoffs of Using Workflows to Complete Complex Tasks

Retelny, D., Bernstein, M. S., & Valentine, M. A. (2017). No workflow can ever be enough: How crowdsourcing workflows constrain complex work. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1-23.

No Workflow Can Ever Be Enough: How Crowdsourcing Workflows Constrain Complex Work

Implications for Future Design (How Crowdsourcing Workflows Constrain Complex Work)

The strict pre-arranged structure of workflows inhibit the crowd’s ability to:
- Respond to contingencies
- Consider and implement alternative solutions

Workflows as an Obstacle to Adaptation

Workflows support coordination in three main ways:
- Encoding the best practices
- Influencing interdependence styles
- Minimizing coordination challenges


Learn Before

Related

Learn After