- Tested how much Sans Forgetica feels disfluent.
- Participants: 151 undergraduate psychology students of a university.
- Design: manipulation of Sans Forgetica and Arial between-subjects.
- Procedure: two phases conducted online. Participants would read and answer questions on passages.
   - Phase One: participants were randomly assigned a 298 words long passage in Sans Forgetica or Arial (Topics: Creation of Baseball, and Georgia O’Keefe). Four pages, each with one paragraph.
   - Phase Two: researchers measure the difficulty of the passages from the participants on a scale from 1 to 7 (1 = very easy, 7 = very hard).
- Results: participants rated Sans Forgetica more difficult than Arial to read, therefore more disfluent.

Experiment 1 on the Effect of Sans Forgetica on Memory

- Replication of Sans Forgetica team's original experiment, but with the researchers own a set of delays (10s, 20s, and 30s).
- Participants: 156 participants from the TurkPrime platform.
- Design: mixed design - used Sans Forgetica and Arial as a within-subject factor, and 
used delay as a between-subject factor.
- Procedure: two phases conducted online.
   - Phase One: participants were presented 20-word pairs and each would be on the screen for 100ms. The pairs were highly associated (like accident - crash). Half of the pairs were in Sans Forgetica and the other half were in Arial.
   - Phase Two: the researchers used three delays (10s, 20s, and 30s) and each one was assigned to a participant. During the delay, multiplication problems would appear and the participants would be asked to solve them. After the delay, participants would be asked to recall the second pair word when the first word appeared on the screen.
- Results: participants recalled fewer word pairs in Sans Forgetica than Arial.
   - The results were the same for all three delays.
   - Suggest that Sans Forgetica impairs the ability to recall.
   - But, these results may be due to the fact that the pairs were presented for 100 ms.
Sans Forgetica word pairs could not be deeply processed in time.

Experiment 2 on the Effect of Sans Forgetica on Memory

- Replication of Sans Forgetica team's original experiment, but removed the delay, and had participants read at their own pace.
- Participants: 300 participants that are MTurk workers or psychology students from a university.
- Design: manipulation of Sans Forgetica and Arial within-subjects.
- Procedure: two phases conducted online.
   - Phase One: participants read five passages at their own pace. In each passage, there are four paragraphs. One is in Sans Forgetica and the other three are in Arial.
   - Phase Two: participants played a matching game for five minutes, and then their memory was tested for the passages. They were given 20 multiple-choice questions in Times New Roman. One question is related to one paragraph.
- Results: there was no significant difference in the number of correct answers read in Sans Forgetica and Arial.
   - This suggests that Sans Forgetica does not improve recall of information.

Experiment 3 on the Effect of Sans Forgetica on Memory

- Tested the effect of Sans Forgetica with higher-level conceptual questions.
- Participants: 319 MTurk workers (but after exclusions, 275 participants’ results were recorded).
- Design: manipulation of Sans Forgetica and Arial within-subjects.
- Procedure: the same as Experiment 3, except:
   - Phase One: participants read six passages and each is longer than experiment 3. In each passage, there are four paragraphs. Two are in Sans Forgetica, and the other two are in Arial. Two important paragraphs of the four will be used for questioning later which one is Sans Forgetica, and the other is Arial. 
   - Phase Two: participants were asked to answer 12 cued-recall questions that are meant to test the participants' understanding of the passage. For each of the six passages, the test will ask a question that relates to the Sans Forgetica paragraph and another question that relates to the Arial paragraph.
- Results: a coding scheme was used to grade the participants' answers. Answers were scored out from 0 to 2 points (0 = incorrect, 1 = partially correct, 2 = correct). 
   - Comparing the percentage of points, the researchers found no evidence supporting that Sans Forgetica boosts performance on conceptual questions.
   - The number of points between Sans Forgetica and Arial were almost the same.

Experiment 4 on the Effect of Sans Forgetica on Memory

- Sans Forgetica creates a feeling of disfluency, but it is not a desirable difficulty.

Implications 
- Disfluent fonts do not encourage deep and slow processing that is beneficial towards memory.
- Sans Forgetica is not the best level of disfluency.
- It is possible that Sans Forgetica is beneficial for specific individuals that enjoy challenging fonts and can understand and encode those fonts.
- It is also possible that the effects of Sans Forgetica occur after a longer period of time between the study and testing.
   - But this was tested by Xie et al., 2018: for disfluent fonts, it does not matter if the interval is more or less than 10 minutes for the effects of the font to occur.  
   - But another study by Weissgerber & Reinhard, 2017: for disfluent fonts, the effects on memory occur after two weeks.

- From the researchers’ findings, disfluent fonts are not effective in improving memory. There could be problems if students start using Sans Forgetica in hopes that it improves their studying after listening to the media covering Sans Forgetica.

General Discussion of the Effects of Sans Forgetica on Memory

- 2018 - an academic team created a new font that supposedly is the best disfluent font.
- The team conducted an experiment with four groups and tested how much the subjects remembered pairs of words:
   - Moderately disfluent font remembered 69% of the pairs.
   - Slightly disfluent font remembered 61% of the pairs.
   - Extremely disfluent font remembered 61% of the pairs.
   - Fluent font remembered 68% of the pairs.
- The team concluded that the moderately disfluent font (picture below) is the best and is a desirable difficulty.
- Sans Forgetica received a lot of media attention and awards, yet their claims have not been published in a peer-reviewed journal.

Background on Sans Forgetica (Disfluent Font)

- The paper used Sans Forgetica to see how much the font can improve memory performance.
- Subjects read information either in Sans Forgetica or Arial.
   - Experiment 1: subjects rated the difficulty of reading the information presented.
   - Experiment 2 - 4: subjects attempted to recall information from readings.


Rutgers University

Studying by practice tests improves encoding into long term memory more than rereading passages (while there is no difference in short term understanding).
- using tests as learning events 
- question asking
- flashcards

Retrieval practice ( testing effect )

Andrea Taylor, Mevagh Sanson, Ryan Burnell, Kimberley A. Wade & Maryanne Garry (2020) Disfluent difficulties are not desirable difficulties: the (lack of) effect of Sans Forgetica on memory, Memory, 28:7, 850-857

Disfluent difficulties are not desirable difficulties: the (lack of) effect of Sans Forgetica on memory

Roediger, H. L., & Karpicke, J. D. (2006). Test-Enhanced Learning. ​Psychological Science, 17​(3), 249-255. (Page 1)

Test-Enhanced Learning (Roediger & Karpicke): Taking Memory Tests Improves Long-Term Retention

Previous studies analyzing the testing effect fail to do so in an educational context.  Roediger and Karpicke set out to discover this gap in the literature and prove whether or not testing promotes the learning process beyond restudying.  To analyze their hypothesis, the researchers conducted two separate experiments.  Their findings proved that, within educational contexts, testing is a crucial element on both the learning and long-term retention of educational materials than simply the re-studying of these materials.  The researchers suggest that the outcome of massed study leading to short-term benefit, while spaced studying or testing having a greater effect on long-term retention, may be a reflection of desirable difficulties.  Learning in spaced presentations facilitates deeper learning than massed presentations, over a given period of time. Within this study, the results suggest that the testing and the spacing effect introduced a desirable difficulty during learning for the subjects (Roediger & Karpicke, 2006, page 249).  

Test-Enhanced Learning

- Sixth-grade social studies class
- International political economics course at Washington University in St. Louis

Studying the Testing Effect in the Classroom

The practice of retrieving information from memory to fortify and maintain one's knowledge. 
- using tests as learning events
- question asking 
- flashcards

Retrieval Practice (Testing)

Results of previous studies done in differing classroom environments and different quiz formats showed that there should be benefits of testing if questions on the final exam were related in content to questions on quizzes throughout the semester. The two main goals of the study were to determine the effects of quizzing content that would be on a final exam compared to no previous quizzing, and to determine the effects of quizzes on the retention of information that are conceptually related to quizzed content but had not been on quizzes. In short, to see if testing is a desirable difficulty for learning specifically tested material and/or material related to quizzed material.

Multiple-choice testing as a desirable difficulty in the classroom

Students take a mock exam with similar material to test their knowledge.

Rated as high utility. Practice testing is broadly applicable to types of material, and ages and abilities of learners. Moreover, it is one of the least time-consuming habits and evidence shows that practice testing is very effective in boosting academic performance.


Practice Testing

Gyllen, J. G., Stahovich, T. F., Mayer, R. E., Entezari, N., & Darvishzadeh, A. (2020). Priming productive study strategies with preparatory quizzes in an engineering course. Applied Cognitive Psychology.

Goal of the study is to focus on preparatory quizzes where they have to answer questions before they are exposed to the material 

Using preparatory quizzes to aid study strategies study

Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning?. Journal of Experimental Psychology: Applied, 15(3), 243.

This study looks at the pretesting effect in different contexts (compared to extra study time and memorizing questions)

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study

- The article discusses desirable difficulties in vocabulary learning as:
   - Learning that creates difficulty at first but later retention and transfer of what is learned are benefited.
   - Difficulties that are related to intrinsic motivation, engagement, and understanding. 
- The article also touches upon evidence of benefits from having difficulties in learning and the consequences for second-language learning. 

Desirable Difficulties in Vocabulary Learning

The act of taking a test generally improves future performance on the subject compared to simply rereading it or not being exposed to it again.

Learn Before

Related

Learn After