MTurk populations tend to be more demographically diverse than traditional sampling populations, as MTurk relies on a much larger population, located in countries around the world.

- A majority of workers are American and Indian
- 65% of workers are female
- Average Age ~ 36 years
- 14% of workers rely on MTurk as a primary source of income
- 61% consider money an important factor in completing a task (lower payment on MTurk has been associated with it being harder to find workers to complete a task)


MTurk Demographics

1. Internet participants would exhibit more signs of depression than traditional samples (Kraut et. al, 1998). This would be disproven by Gosling et. al, in 2004.
 
2. Online participants are less motivated than traditional samples, partially due to their anonymity.
This was prevented by instructional manipulation checks. i.e, “Select Option C for this question”. 
Anonymity allowed participants to be more comfortable discussing difficult, personal topics, like mental health for example.

3. Since MTurk is powered by Amazon, a multinational, financially motivated business, is the data from MTurk reliable?
Studies show that there is little to no difference in reliability between studies done through MTurk and through labs using college students


Concerns against Online Sampling on Amazon MTurk


MTurk is a platform powered by Amazon that allows ‘employers’ (researchers) to post HITs, or ‘Human Intelligence Tasks’ to complete for payment. MTurk was designed to be a marketplace for people to post simple jobs that cannot be automated, and require human input. These HITs can be surveys, transcriptions, and other functions.  

University of California, Davis

The most common participants in behavioral/psychological studies tend to be American college students, but with the rise of the internet, other forms of sampling have become more common, like using Amazon MTurk.

Separate but equal? A comparison of participants and data gathered viaAmazon’s MTurk, social media, and face-to-face behavioral testing

 https://www.sciencedirect.com/science/article/pii/S074756321300160X?casa_token=jI2BChgu7ggAAAAA:-4S7-fZdImGk74J_dcMoh4Ligrpyzq1UT32vNGALA-loTV-uUURnV9CGRCefvhLEm8ikqqmljA

Separate but equal? A comparison of participants and data gathered viaAmazon’s MTurk, social media, and face-to-face behavioral testing (reference)

What is Amazon MTurk?

This study compared three different groups completing the same object selection and categorization task:
1. college undergrads completing the task in a University lab room
- 24 students, 58% female, majority White, received $5 
2. adults recruited through social media completing the task at their own leisure
- 30 adults, 83% female, majority White
3. adults recruited through MTurk completing the task at their own leisure
- 32 adults, 35% female, majority Asian (with an overall more diverse breakup), received $0.50 on Amazon

to compare the difference between tasks completed in-person with the researcher vs. independently online


Separate but Equal Study Methods

When comparing results based on behavioral expectations (that for teaching trials, participants would choose randomly and for non-teaching trials, participants would choose the novel tool), online and in-person outcomes were overall very similar. The main difference between online and in-person participants came in demographic data, where MTurk was more racially diverse and presented diverse socioeconomic groups.

Learn Before

Related

Learn After