- To enable requesters to gain insight into partially completed work, Turkomatic provides a workflow visualization that shows the current state of an ongoing job as either an indented list or a decomposition graph.
- This list view allocates the most screen space to task content; the graph also shows hierarchy, as well as parallel and serial decompositions.
- Both visualizations inform requesters how much of the work has been accomplished, what strategies have been taken, and whether subtask solutions or decompositions are of sufficient quality.

Boston College

This section reviews interaction with Turkomatic from the requester’s perspective.

Additional Sections:
- Requesting Work
- Visualizing Ongoing Work
- Editing Workflows

Crowdsourcing Workflows: Requester Interfaces

Kulkarni, A., Can, M., & Harmann, B. (2012). Collaboratively crowdsourcing workflows with Turkomatic. Association for Computing Machinery, 1003-1012. DOI: https://doi.org/10.1145/2145204.2145354 

Collaboratively crowdsourcing workflows with Turkomatic

- Requesters post new Turkomatic jobs through a natural-language web interface; it offers a single text box where requesters specify what they want to accomplish.
- Expert requesters can also author decompositions that serve as a starting point for the PDS algorithm, which may be preferred if the requester believes that crowds will not be able to properly decompose a given task.
- Once a new task prompt or task tree is loaded, the PDS solve the leaf tasks, and eventually merge solutions to accomplish the root tasks.

Crowdsourcing Workflows: Requesting Work

Crowdsourcing Workflows: Visualizing Ongoing Work

- Ongoing work can be unsatisfactory for multiple reasons: a crowd-authored decomposition may be flawed, or a solution to a subtask may be of low quality.
- Requesters can edit existing workflows in real-time to address such challenges.
- Requesters edit the task description, decomposition, or solution for any one node; once an edit is completed, Turkomatic computes which tasks will have to be performed again by additional workers.
- When a task description is changed, any subtasks created for this task by the crowd may also no longer be valid, so Turkomatic then invalidates the entire subtree below the edited task and reissues the tasks.
- When a requester edits a task solution or decomposition directly, the entire subtree of that task is discarded.

Learn Before

Related