- some papers had a small sample size
- the psychometric measurements used when conducting surveys were not proper or validated
- some experiments did not have a control group and only used user evaluation for their analyses 
- in terms of implemented motivational affordances: there were sometimes no controls between them; they were sometimes analyzed as a group, not at the individual level
- several papers gave descriptive stats and could have presented details about the links between constructs but did not
- short time frames when conducting the experiments (novelty could be a confounding factor) 
- reports of results were unclear 
- none of the experiments used all levels of measurements (motivational affordances, psychological outcomes, and behavioral outcomes)
- it's important to be aware of these limits so future studies on gamification can be more clear and refined 

Discussion - Limits of Reviewed Studies (Hamari et al. 2014)

- broaden our understanding of the role context plays in gamification, and how context affects the gamification outcomes
- to do that, future gamification research should consider the social environment (specifically the theory of planned behavior), the nature of the system, and the user's involvement 
- additionally, individual differences between each participant of the gamification experiment need to be carefully observed and analyzed. Different player types, approaches, attitudes, motivations, and interactions may lead to different gamification outcomes or lead users to experience the same motivational affordances differently. 

Discussion - Future Research (Hamari et al. 2014)

- this paper only reviewed studies done explicitly on gamification, not related topics such as intrinsic motivation
- similarly, there might have been papers that discussed gamification but with different terms and were thus not included in this literature review
- a formal meta-analysis could not be conducted due to a large variety in the methods of the reviewed studies

Discussion - Limits of this paper (Hamari et al. 2014)

- the reviewed articles suggest that yes, gamification does work, but there are some caveats 
- there are many layers to the effects of gamification that need to be studied 
- the reviewed papers contained two key aspects: 1) the role of the context, 2) user qualities
- for future research on gamification, studies are strongly recommended to include more complex and far-reaching methodologies 

Chapman University

This is a paper that peer-reviewed research articles on gamification.

Does Gamification Work? — A Literature Review of Empirical Studies on
Gamification

Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work?--a literature review of empirical studies on gamification. In 2014 47th Hawaii international conference on system sciences (pp. 3025-3034). Ieee.

Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification

- Gamification has become much more popular in recent years, not only in business models but also in academic research articles. 
- Empirical support on the effectiveness of gamification is much needed in order to dispel the confusion on how positive or negative gamification really is.
- This paper aims to review pre-existing empirical studies on gamification in order to contribute to our understanding of the topic.
- Due to the lack of clarity around which studies truly focus on gamification, the authors of this paper also aim to explore what papers who seemingly focus on gamification are actually studying as motivational affordances and psychological and behavioral outcomes.

Introduction (Hamari et al. 2014)

- search terms: "gamification," "gamif*," "gameful," “motivational affordance” 
- search engines: Scopus, ScienceDirect, EBSCOHost, Web of Science, ACM Digital library, AISel, Google Scholar, Proquest
- table 1 shows all search results, not filtering for academic content
- after the initial search, the researchers narrowed down the results to: 
     - peer-reviewed, 
     - empirical study, 
     - research methods were explained, 
     - motivational affordances were clearly identifiable, 
     - studied gamification, not games
- the scientists then checked through the references of the papers to catch any key papers they may have missed. They found 1 additional relevant paper.
- ended with 24 peer-reviewed academic study articles

Literature review methods (Hamari et al. 2014)

- 1 the motivational affordances (object traits that provide users with the experience of fulfilling their psychological needs, often used to facilitate extrinsic and intrinsic motivation) established as dependent variables by each study
- 2 any psychological outcomes that have been recorded (as independent or dependent variables) in the studies
- 3 the behavioral outcomes that were recorded as dependent variables in each study
- 4 discover which services gamification has been used with
- 5 which methods have been implemented to analyze the effects of the gamifications

Aims of the peer-review (Hamari et al. 2014)

- all articles addressed the question of "does gamification work?" but the 24 papers still had varying implementations of motivational affordances, measurements of effectiveness (measuring different types of outcomes), results (negative or positive), and contexts

Learn Before

Related

Learn After