Survey:
Invited ~11,000 new editors to use the tutorial by using the Wiki talks page. Afterward, they were sent a survey that asked about their confidence and engagement in editing Wikipedia.


Field Experiment:
Selected 1,967 accounts for this experiment, 1,751 accounts were randomly selected for the treatment group and were invited to use the tutorial. The remaining 216 accounts were assigned to a control group and were not invited to the tutorial.

Each group was being observed for their editing behavior 180 days since they were invited/not invited


Methods

- 91% of respondents said that TWA is an useful introduction to edit Wikipedia

- 90% of respondents agreed (strongly agreed) that TWA made them feel more confident as an editor and helped them develop a better understanding of Wikipedia

- 85% said that TWA is very useful on reviewing Wiki's policies

- 80% responded that TWA encourages them to make more edits. They also reported that they felt welcome and supported

Survey Results

- 22% of those invited complete some part of the tutorial
- 46% editors played till the seventh and final mission
- The first mission lost the most editors at 24% after completion. 
   The dropout rates were decreasing after the first mission 

Field Experiment Results

After using TWA, new editors will:
- Increase the number of overall contributions
- Increase the number of contributions on talk pages
- Create contributions of greater quality

Hypotheses

Purpose:
- How new users perceive the system design and tone (via survey~11,000 editors)
- If using TWA led to making more contributions prior to being invited to participate in the tutorial (via field experiment~1,967 editors)

Results:
- From the survey, users found TWA engaging and well-designed for new users.
- Even though new users say that TWA is very useful and helps new editors, the results from the field experiment report that TWA did not effect editing contributions)


University of Michigan - Ann Arbor

Wikipedia launched an interactive tutorial that has both a structure and video game elements that helps new editors get accustomed to Wikipedia's technical requirements and rules/policies

The Wikipedia Adventure (TWA): Field Evaluation of an Interactive Tutorial for New Users

University, Sneha Narayan Northwestern, et al. “The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for New Users.” The Wikipedia Adventure | Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 1 Feb. 2017, dl.acm.org/doi/10.1145/2998181.2998307.

The Wikipedia Adventure: Field Evaluation. of an Interactive Tutorial for New Users 

Purpose and Summary of Results

- Points
- Leaderboards
- Achievements/ Badges
- Levels
- Stories/Themes
- Clear goals
- Feedback
- Rewards
- Progress
- Challenges


10 Common Motivational Affordances of Gamified Systems (Hamari, et al.)

Having a dedicated space for new users to make contributions without being penalized for mistakes

Creating Spaces for New Users to Practice Making Contributions

When making contributions, most newcomers do not follow any external structure and they adopt their own techniques that stick overtime 

Newcomers Follow their Own Rules when Making Contributions

Wikipedia relies on individualized socialization where it is up to the user to learn the norms and policies. New members learn by asking for help/feedback and referring to Wikipedia's help and policy page

Also learn what not to do by receiving warning and advisory messages they receive from senior members. 

Learn Before

Related

Learn After