What mechanisms from structured communication in the writing domain effectively transfer context within a crowd work setting ?

Macalester College

This article explores the notion of structured communication: using structured microtasks to support communication in the domain of complex writing. Different mechanisms are presented and compared in order to find the most effective and less costly ways to communicate with the crowd in the domain of writing. 
Proposal: limiting the requester’s interactions can result in less costly interactions that are just as beneficial


Communicating Context to the Crowd for Complex Writing Tasks 

Salehi, N., Teevan, J., Iqbal, S., Kamar, E., (2017) Communicating Context to the Crowd for Complex Writing Tasks. Stanford University, Microsoft Research. 
http://dx.doi.org/10.1145/2998181.2998332

1. General Questions 
2. Specific Questions 
3. Free form Comment and edit 
4. Highlight strength and weaknesses 
5. Main Problem identification

Examined Methods (Communicating Context to the Crowd for Complex Writing Tasks)

Two studies designed to quantify the value and cost of each mechanism. Study 1 measures the added value to crowd workers in completing the tasks, and Study 2 measures the cost of each mechanism for requesters.

Methodology (Communicating Context to the Crowd for Complex Writing Tasks)

Design based on three key findings: 1. Structured communication yields information that can improve the paragraph. 2. The effect size of improvement is dependent on the current state of the paragraph and the communication mechanism. 3. Communication mechanisms have different costs associated with them.

Workflow design (Communicating Context to the Crowd for Complex Writing Tasks)

The cost of communication with the system is relatively low, with an average of 10.6 minute to complete the task, 3.2/5 for mental demand of the task and 2.2/5 for hard work.
All researchers reported that they would be willing to use the system again in the future for writing tasks such as a blog post, the abstract, or introduction of a paper, but not for academic purposes or professional writing. 
They found that the paragraph matched what they would have written themselves in terms of content, but not writing style.

Result (Communicating Context to the Crowd for Complex Writing Tasks)

Neglected the communication cost on the worker’s side. 
A requester’s situations, for example, limitations of the mobile device, can affect communication cost.
Future study can be done to examine how to adapt design implication to other domains and other communication mechanisms.

Learn Before

Related