This study was conducted to explore the limits of crowdsourcing using complex technical engineering tasks. Two populations were analyzed in this experiment: anonymous crowd workers recruited from Amazon Mechanical Turk (a marketplace for crowdsourcing microtasks) and civil engineering graduate students. The completion of the tasks was compared between the two groups to assess the effectiveness of using untrained crowd workers for complex tasks that required expertise.

Lessons Learned from Crowdsourcing Complex Engineering Tasks

Bonney, R., Shirk, J. L., Phillips, T. B., Wiggins, A., Ballard, H. L., Miller-Rushing, A. J., & Parrish, J. K. (2014). Next steps for citizen science. Science, 343(6178), 1436–1437. https://doi.org/10.1126/science.1251554

Next Steps for Citizen Science

A new dawn for citizen science

Terminology usage and understanding impacts how knowledge is developed. Defining terms for understanding and applying citizen science can help with broad application and general understanding of the science.

The Importance of Terminology in Citizen Science

Direct communication with researchers and participants is important, as well as review one's understanding of the terminology being used in Citizen Science studies. There needs to be a recognition that one term is usually not applicable to all contexts. Further studies on the use of terminology or the review of it in citizen Science is recommended.

Lessons Learned from the Importance of Terminology in Citizen Science

Mueller, M., Tippins, D., Bryan, L., The Future of Citizen Science. Democracy & Education, vol 20, No 1. https://democracyeducationjournal.org/cgi/viewcontent.cgi?article=1026&context=home

The Future of Citizen Science

- community science
- crowd science
- crowd-sourced science
- civic science
- volunteer monitoring
Public Participation in Scientific Research (PPSR)

Alternative names for Citizen Science

This article focuses on the potential contributions of volunteers to scientific research on wildlife in their local communities, such as research on the mammals living along the Appalachian Trail. It also notes the increasing use of citizen scientists across studies and their critical impact on the advancement of ecological research and findings.

Citizen Science: Can Volunteers Do Real Research?

There are research gaps on citizen science in education. In addition, there is a lack of citizen social science projects. Citizen science is also mainly done by adults. Lastly, there is a lack of records of youth participation in citizen science in a school context. 


Gaps in Research on Youth Participation in Citizen Science 

Studies have shown that there have been positive educational outcomes because of youth participation in citizen science. As a result, educators want to promote citizen science projects in the educational system. This study took place in an Austrian school where students participated in projects that dissected what kinds of political topics young people encounter in their everyday lives. 

Citizen Science in Schools

Sexual and reproductive health research currently lacks:
- Collaboration that targets the gaps in evidence
- High quality tools for patients and the public to participate
- Patients and the public as stakeholders, not just as research subject, to create incentive and recognize their contributions

The Need for Citizen Science in Sexual and Reproductive Health Research 

The article examines how would citizen science develop when coupled with emerging technologies, in terms of its research processes, program and participant cultures, and scientific communities. 

The future of citizen science: emerging technologies and shifting paradigms

This Dutch study explores the question of whether citizen science can be effectively used in decision making regarding public health policies. To this end, challenges that come with incorporating citizen science into public health are discussed, as well as ways these issues can be remedied. Finally, a framework for using citizen science in public health is suggested.

Citizen Science for public health

Citizen science projects that utilize technology and are predominantly worked online are considered to be citizen cyberscience. Some citizen science projects strictly collect data in a physical manner, restricting them from falling into the subset of cyberscience. 

Citizen Cyberscience: a subset of citizen science 

Citizen science is research contributed by nonprofessional participants via crowdsourcing. In citizen science projects, participants either collect data with mobile technologies or analyze existing data through internet-based Virtual Citizen Science (VCS) platforms, while in the accompaniment of a professional scientist or researcher.
This research improves the scientific community's capacity and increases the public's understanding of science.

Carleton College

Ohio State University, Columbus

University of Michigan - Ann Arbor

A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information. It can be depicted as a sequence of operations, the work of a person or group,the work of an organization of staff, or one or more simple or complex mechanisms. From a more abstract or higher-level perspective, workflow may be considered a view or representation of real work. The flow being described may refer to a document, service, or product that is being transferred from one step to another. Workflows may be viewed as one fundamental building block to be combined with other parts of an organization's structure such as information technology, teams, projects and hierarchies.

Workflows

Obtaining information or input into a task or project by enlisting the services of a large number of people.

Crowdsourcing

Sprinks, J., Wardlaw, J., Houghton, R., Bamford, S., & Morley, J. (2017). Task workflow design and its impact on performance and volunteers’ subjective preference in virtual citizen science. International Journal of Human-Computer Studies, 104, 50-63. https://doi.org/10.1016/j.ijhcs.2017.03.003

Task Workflow Design and its impact on performance and volunteers' subjective preference in Virtual Citizen Science

Wikipedia

Jennett, Charlene; Kloetzer, Laure; Schneider, Daniel; Iacovides, Ioanna; Cox, Anna; Gold, Margaret; Fuchs, Brian; Eveleigh, Alexandra; Methieu, Kathleen; Ajani, Zoya and Talsi, Yasmin (2016). Motivations, learning and creativity in online citizen science. Journal of Science Communication, 15(3), article no. A05. 

Motivations, learning and creativity in online citizen
science

The process of preparing jobs for  the crowdsourcing marketplace requires planning, as well as the development of an efficient workflow design. Researchers of this study introduce the program Turkomatic, which aims to recruit crowdworkers to develop workflows for larger tasks, where their goal is to break a job down into microtasks which can then be completed by other crowdsource workers. Results suggest that allowing independent crowdworkers to develop the workflow design is partially successful without requester supervision, but requester supervision during workflow planning and execution improves results dramatically. Researchers argue that Turkomatic’s design will be more successful than the conventional workflow design process.

Collaboratively crowdsourcing workflows with Turkomatic

Citizen science

The practice of completing microtasks by using a group of remote workers.

Microtask Crowdsourcing

Crowdsourcing Applications

A distributed innovation process based on managed knowledge flows across organizational boundaries.

Learn Before

Related

Learn After