Because the data for both Growth Rules fit a power-law distribution, cost forecasting is a bursty process. This means that algorithm alternates between requesting new tasks in short bursts and receiving responses to existing tasks for quite some time.

Cost Forecasting Can Be Modeled as a Bursty Process

1. The maximum number of responses allowed
2. Larger seed sizes
3. Smaller seed sizes
4. Brief inaccuracy

Constraints on Cost Forecasting

Cost forecasting allows the crowdsourcer apply efficient algorithms to crowd-generated microtasks, which lets the crowdsourcer get high quality results even when the set of tasks is open-ended or if the budget is cut short unexpectedly.

Advantages of Cost Forecasting: Efficient Algorithms Applied to Crowdsourcing

Cost forecasting can result in higher accuracy for smaller budgets (≤ ≈5%) because it dynamically determines growth rate with past and current state of crowdsourcing.

Advantages of Cost Forecasting: Higher Accuracy for Smaller Budgets

- GRI: Request a new task when the expected cost of a new task is less than the minimum cost of any existing task.
- GRII: Request a new task when the expected cost of a new task is less than the median cost of the set of already-existing tasks.
- GRII is included in case of outliers that would make the overall minimum an inappropriate choice.

University of Rochester

Cost forecasting is a technique that allows the crowdsourcer to decide between having the crowd generate more tasks or letting the crowd complete some of the already-generated tasks to more efficiently balance resources in a situation where crowdworkers are both proposing new tasks and completing them.

(Proposed in the paper Efficient crowdsouring of crowd-generated microtasks.)

Cost Forecasting

Hotaling, A., &amp; Bagrow, J. P. (2020). Efficient crowdsourcing of crowd-generated microtasks. PLOS ONE, 15(12). https://doi.org/10.1371/journal.pone.0244245 

Efficient crowdsourcing of crowd-generated microtasks

Growth Rules to Test Cost Forecasting (Efficient crowdsourcing of crowd-generated microtasks)

1. Efficient algorithms applied to crowdsourcing
2. Higher accuracy for smaller budgets
3. Useful for dynamic costs

Learn Before

Related

Learn After