the idea that students do better when they complete practice questions before actually learning the material 

Pretesting Principle

- One positive to standardized testing is that testing helps improve memory for what is being tested 
- Testing in general has been shown to be better than re-testing the material 
- Testing should be better used as a tool for acquiring knowledge as opposed to measuring it 
- Testing is most beneficial when the items are answered correctly
- Testing potentially has the power to distort knowledge 
- The goal of the study was to encourage more errors and failed tests by testing before learning the material 
- Experiments-asked to read a text about vision

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study introduction

tested sentences were italicized

- Predicted that pretesting would be better than more time studying, “even when differences in attention direction were minimized by italicizing key sentences in the text in both conditions”
- 61 undergraduates who were given extra credit-sampled from a upper level psych course 
- Same study materials except that the 10 testable items were italicized-done so that the participants are equally alerted to what is important like the way textbooks do
- Same procedure as study 1
- Participants answered 22% of the pre-test questions correctly
- Recall for the pre-test condition with italicizing was better than the extra study time with italicizing-M=71% & SE=5.6 vs. M=54% & SE=3.7
- In the pre-test condition, pre-tested items were recalled more than the untested italicized items-M=71% & SE=5.6 vs. M=53% & SE=4.3
- This shows that the testing effect does still apply to failed tests-it has a benefit besides just directing attention to the things that matter the most in the text 

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study- experiment 2

They were either tested before they read the text or given additional time to study

- Predicted that testing beforehand would enhance later memory recall 
- 63 undergraduates who were given extra credit 
- Questions were fill in the blank or free response questions 
- Final test of all 10 testable items-participants in the test condition had 5 questions that were already pre-tested-all questions in the final test were new for the control group who had extended study time 
- Learning phase-participants in the treatment group were given one of two pretests and given 2 minutes to answer them 
- Everyone had to study the passage for 8 minutes 
- Participants in the control group with extra study time were given 10 minutes 
- All participants answered the Time 2 test which had 10 questions (untimed)
- Participants answered about 5% of the questions correctly on the pretest 
- The pretest ended up resulting in a better posttest score- M=75% & SE=3.2 vs. M=56% & SE=2.7
- Pre-tested items were better recalled than items that were not tested-<=75% & SE=3.2 vs. M=50% & SE=3.4
- This reveals that answering incorrectly (failed tests) can improve learning 
- This effect might be due to the fact that the test pointed attention to testable items 

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study- experiment 1

keyword from each sentence was bolded (might be more attention drawing than italics)

- Predicted that pretesting would be better than more time study even with bolded words 
- 64 undergraduates who were given extra credit 
- The test and study materials were the same as experiments 1 and 2 except that in the testable sentences, there were bolded words instead of italicized words 
- The bolded words served as the answers to fill in the blank and multiple choice questions 
- Same procedure as experiments 1 and 2-no specific instructions were given about the bolded text
- In the pre-test, participants answered 21% of the questions correctly-they removed two questions from further analysis so after the removal, the average performance level was about 11%
- In the pre-test condition, the bolded items were answered correctly on the final test significantly more than in the extended time condition with bolded items-M=82% & SE-3.8 vs. M=64% & SE=4.0
- On the final test, there was not a significant difference between the bolded items that were previously tested and the bolded items that were not previously tested


The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study- experiment 3

- Previous research has shown that the testing effect is especially powerful because people remember tested things better over time than other things that haven’t been tested 
- Made the final test one week after the pre-test-had to come in around the same time of day
- Manipulated bolding within subjects and treatment type (extended study versus pre-test) between subjects 
- Predicted that the final items tested at a delay would be better for pre-tested items than the bolded items in the extended study condition 
- Predicted that bolding would be better than non bolding in the extended study condition but pre-testing overall would be better than bolding 
- 158 undergraduates who were given extra credit for participating 
- Only 5 items in the study material were bolded-on the post test out of the 10 items that were tested, 5 had been bolded (and in the pre-test group, was also tested)
- Participants answered 10% of the questions correctly on the pre-test
- Testing effects analyzed by bolded and tested vs. just bolded-significant pre-testing effect-M=55% & SE=2.0 vs. M=445% & SE=3.0
- Effects of bolding analyzed by bolded vs. unbolded in the extended time condition-small difference showing that bolding is effective in drawing attention to a specific subject 
- Tested and bolded questions were recalled better than untested and unbolded questions
- This experiment shows that the testing effect works 1 week later-more specifically, bolding with testing significantly helped to improve performance 


The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study- experiment 4

used bold and analyzed testing versus more study time one week later

- Previous research has shown that the testing effect is especially powerful because people remember tested things better over time than other things that haven’t been tested 
- Made the final test one week after the pre-test-had to come in around the same time of day
- Manipulated bolding within subjects and treatment type (extended study versus pre-test) between subjects 
- Predicted that the final items tested at a delay would be better for pre-tested items than the bolded items in the extended study condition 
- Predicted that bolding would be better than non bolding in the extended study condition but pre-testing overall would be better than bolding 
- 158 undergraduates who were given extra credit for participating 
- Only 5 items in the study material were bolded-on the post test out of the 10 items that were tested, 5 had been bolded (and in the pre-test group, was also tested)
- Participants answered 10% of the questions correctly on the pre-test
- Testing effects analyzed by bolded and tested vs. just bolded-significant pre-testing effect-M=55% & SE=2.0 vs. M=445% & SE=3.0
- Effects of bolding analyzed by bolded vs. unbolded in the extended time condition-small difference showing that bolding is effective in drawing attention to a specific subject 
- Tested and bolded questions were recalled better than untested and unbolded questions
- This experiment shows that the testing effect works 1 week later-more specifically, bolding with testing significantly helped to improve performance 


studied the difference between reading test questions beforehand and actually answering them

- Just looking at test questions might be helpful because it can serve as a framework to interact with the text or guide looking behaviors 
- Had 3 conditions-pre-test questions, extra study time and a third condition where participants are asked to study questions
- 2 opposing predictions based on contradictory previous research-testing and question learning might have similar effects because they both involve deep processing & pretesting is better than trying to reproduce questions 
- 76 undergraduate participants who were given extra credits 
- For each participant, half of the key concepts are italicized (instead of the whole sentence so that it’s more precise)
- All questions were rewritten from previous experiments to be in fill-in-the-blank format
- In the memorizing questions condition, they were told to memorize because they would be testing someone else later on-they then were asked to write down the question before reading the passage 
- In the pre-test condition, participants answered 6% of the questions correctly
- The pre-test condition did better than the other two conditions and the question memorizing condition did better than the extra study condition 
- The largest difference in performance was between the pre-test condition and the extra study condition 
- This study shows that attempting to answer a question is better than simply reading it 


The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study- experiment 5

- Pretesting is effective and persists after 1 week
- “Pretesting did not seriously impair retention of untested items, as it has been posited to do previously”
- Unsuccessful tests might be beneficial because they strengthen the retrieval between the question itself and the actual answer 
- Unsuccessful tests might also be beneficial because they facilitate deep processing that doesn’t occur when you only read the question 
- Tests in schooling should be looked at as a learning opportunity instead of just a measurement opportunity 
- Standardized tests can still be helpful if students are taught about the content in general shortly after taking the test 


The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study discussion 

This study looks at the pretesting effect in different contexts (compared to extra study time and memorizing questions)

University of Michigan - Ann Arbor

Studying by practice tests improves encoding into long term memory more than rereading passages (while there is no difference in short term understanding).
- using tests as learning events 
- question asking
- flashcards

Retrieval practice ( testing effect )

Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning?. Journal of Experimental Psychology: Applied, 15(3), 243.

Roediger, H. L., & Karpicke, J. D. (2006). Test-Enhanced Learning. ​Psychological Science, 17​(3), 249-255. (Page 1)

Test-Enhanced Learning (Roediger & Karpicke): Taking Memory Tests Improves Long-Term Retention

Previous studies analyzing the testing effect fail to do so in an educational context.  Roediger and Karpicke set out to discover this gap in the literature and prove whether or not testing promotes the learning process beyond restudying.  To analyze their hypothesis, the researchers conducted two separate experiments.  Their findings proved that, within educational contexts, testing is a crucial element on both the learning and long-term retention of educational materials than simply the re-studying of these materials.  The researchers suggest that the outcome of massed study leading to short-term benefit, while spaced studying or testing having a greater effect on long-term retention, may be a reflection of desirable difficulties.  Learning in spaced presentations facilitates deeper learning than massed presentations, over a given period of time. Within this study, the results suggest that the testing and the spacing effect introduced a desirable difficulty during learning for the subjects (Roediger & Karpicke, 2006, page 249).  

Test-Enhanced Learning

- Sixth-grade social studies class
- International political economics course at Washington University in St. Louis

Studying the Testing Effect in the Classroom

The practice of retrieving information from memory to fortify and maintain one's knowledge. 
- using tests as learning events
- question asking 
- flashcards

Retrieval Practice (Testing)

Results of previous studies done in differing classroom environments and different quiz formats showed that there should be benefits of testing if questions on the final exam were related in content to questions on quizzes throughout the semester. The two main goals of the study were to determine the effects of quizzing content that would be on a final exam compared to no previous quizzing, and to determine the effects of quizzes on the retention of information that are conceptually related to quizzed content but had not been on quizzes. In short, to see if testing is a desirable difficulty for learning specifically tested material and/or material related to quizzed material.

Multiple-choice testing as a desirable difficulty in the classroom

Students take a mock exam with similar material to test their knowledge.

Rated as high utility. Practice testing is broadly applicable to types of material, and ages and abilities of learners. Moreover, it is one of the least time-consuming habits and evidence shows that practice testing is very effective in boosting academic performance.


Practice Testing

Gyllen, J. G., Stahovich, T. F., Mayer, R. E., Entezari, N., & Darvishzadeh, A. (2020). Priming productive study strategies with preparatory quizzes in an engineering course. Applied Cognitive Psychology.

Goal of the study is to focus on preparatory quizzes where they have to answer questions before they are exposed to the material 

Using preparatory quizzes to aid study strategies study

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study

- The paper used Sans Forgetica to see how much the font can improve memory performance.
- Subjects read information either in Sans Forgetica or Arial.
   - Experiment 1: subjects rated the difficulty of reading the information presented.
   - Experiment 2 - 4: subjects attempted to recall information from readings.


Disfluent difficulties are not desirable difficulties: the (lack of) effect of Sans Forgetica on memory

- The article discusses desirable difficulties in vocabulary learning as:
   - Learning that creates difficulty at first but later retention and transfer of what is learned are benefited.
   - Difficulties that are related to intrinsic motivation, engagement, and understanding. 
- The article also touches upon evidence of benefits from having difficulties in learning and the consequences for second-language learning. 

Desirable Difficulties in Vocabulary Learning

The act of taking a test generally improves future performance on the subject compared to simply rereading it or not being exposed to it again.

Learn Before

Related

Learn After