“A) do students who are assigned to take pre-quizzes in a college class exhibit more productive study strategies (e.g., visiting the explanatory text in the textbook more often and earlier?
B) do students who are assigned to take pre-quizzes in a college class obtain better course grades (e.g. quiz, exam, and final grades)?
C) are productive study strategies related to course grades?”

Using preparatory quizzes to aid study strategies study research questions

- Studies have shown that even when students do poorly on a pre-test, they do better than control groups on post-tests that cover the same material 
- Taking a practice test in the classroom has proven to be beneficial for learning
- The study quantifies studying behavior based on how often and how early students visit the online materials before a deadline 
- Study is founded on the testing effect-it is better to test oneself instead of studying and re-studying the material 
- Previous research has shown that distributed practice is more beneficial than cramming 


- “Hypothesis 1 First, we hypothesize that asking students to complete pre-quizzes will encourage students to manage their study time more effectively by engaging in more frequent study sessions and doing so earlier in advance of completing the weekly homework assignments than they otherwise would.
- Hypothesis 2 Second, we hypothesize that asking students to complete pre-quizzes will cause them to perform better on course assessments, including post-quizzes, midterm exams, and final course grade.
- Hypothesis 3 Third, we hypothesize that the level of engagement in productive study strategies (such as having more frequent study sessions and having them earlier before the deadline) is correlated with course assessments, including post-quizzes, midterm exams, and final course grade.” 


Using preparatory quizzes to aid study strategies study hypotheses

- 139 undergraduate students taking an intro Mechanical Engineering course 
- 42 students were in the control group (no pre-quizzes) and 97 students were in the treatment group (pre-quizzes)
- The majority of participants were men and mechanical engineering majors 
- The treatment and control group were the same in that they both had weekly problem sets for homework, in class post-quizzes, two midterms, and a cumulative final exam 
- Pre-quizzes were given out on blackboard and consisted of about 6-8 questions which required the student to compute a value 
- Most of the questions on the pre-quizzes were multiple choice and each student only had one opportunity to take it 
- Pre-quizzes match the material that is on the homework assignment 
- Pre-quizzes were easier and simpler than the post-quizzes
- Students were provided with an online version of the textbook 
- Docviewer is the program used to analyze how the students studied-it was pre-installed on Windows Surface tablets that each student was loaned in order to take the class 
- “DocViewer constructs a log file containing a time-stamped record of events including when the user opens or closes a document, changes pages within a document, or pans and zooms on a page”
- DocViewer recorded episodes in which a single page of the document is visible on the screen for at least 15 seconds 
- During the control quarter, DocViewer only dimmed after 10 minutes but in the treatment quarter, DocViewer dimmed after 2 minutes 
- They calculated the total time spent in the textbook and the number of unique visits to pages within the textbook-they also examined the total amount of time spent on each specific content area
- Lead time measurement-quantifying when the student read the material=average weighted time of reading prior to homework deadlines 
- Study session-”period of time during which a student interacts with documents in DocViewer such that there is no more than 1 hour of consecutive inactivity”
- Measures for academic performance-average grade on in-class quizzes for short term performance, grade on the final exam for long term performance, and course grade as achievement in the course 
- Procedure-3 hours of lecture a week and 1 hour of discussion a week; students assigned homework and readings; had in-class quizzes based on the homework assignments; the textbook corresponded with the pre-quizzes which encourages earlier readings of it 


Using preparatory quizzes to aid study strategies study methods

- Students in the treatment group visited the textbook more often 
- Students viewed homework problems and explanatory texts more in the treatment group 
- The treatment group had a significantly higher number of sessions 33.6 with SD=16.7 vs. 23.4 with SD=10.8
- Increased number of page visits and study sessions shows that students in the treatment group were more likely to distribute their studying 
- “The sum of the session times (or overall study time) was greater for the pre-quiz group than for the control group”
- “Students changed their reading behaviors in the presence of the pre-quizzes-distributing their time spent with Explanatory Text across more sessions between homework assignment deadlines”
- In the treatment condition, students significantly visited Explanatory Text and Worked Example pages earlier-this shows that students’ reading behavior changed due to the implementation of the pre-quizzes 
- Able to conclude that implementing pre-quizzes changes study strategies by encouraging distributed practice (consistent with hypothesis 1)
- Significant effect on average quiz grades but not on the final exam grade-pre-quizzes affected short term achievement but not necessarily long term achievement (which is somewhat consistent with hypothesis 2)
- Explanatory text lead time and homework problem lead time are significantly positively correlated with quiz grade in the treatment group 
- Final exam course grade “is significantly and positively correlated to all Lead Time measures in the control quarter, but only to Explanatory Text Lead Time and Homework Problem Lead Time in the treatment quarter. 
- There is a stronger relationship in general between lead time and academic performance in the control group-perhaps because pre-quizzes encourages both high and low achieving students to read earlier so the measure doesn’t pick up on the differences as easily
- Main findings-adding pre-quizzes 1) changed study behavior for the better (distributing and studying earlier instead of later) 2) encouraged an improvement in quiz grades 3) “productive study behavior correlated with course performance”
- This Study supports the pretesting principle 
- This Study supports implementing pre-quizzes in classrooms to encourage reading the textbook and doing so earlier on


Using preparatory quizzes to aid study strategies study results

Goal of the study is to focus on preparatory quizzes where they have to answer questions before they are exposed to the material 

University of Michigan - Ann Arbor

Studying by practice tests improves encoding into long term memory more than rereading passages (while there is no difference in short term understanding).
- using tests as learning events 
- question asking
- flashcards

Retrieval practice ( testing effect )

Gyllen, J. G., Stahovich, T. F., Mayer, R. E., Entezari, N., & Darvishzadeh, A. (2020). Priming productive study strategies with preparatory quizzes in an engineering course. Applied Cognitive Psychology.

Roediger, H. L., & Karpicke, J. D. (2006). Test-Enhanced Learning. ​Psychological Science, 17​(3), 249-255. (Page 1)

Test-Enhanced Learning (Roediger & Karpicke): Taking Memory Tests Improves Long-Term Retention

Previous studies analyzing the testing effect fail to do so in an educational context.  Roediger and Karpicke set out to discover this gap in the literature and prove whether or not testing promotes the learning process beyond restudying.  To analyze their hypothesis, the researchers conducted two separate experiments.  Their findings proved that, within educational contexts, testing is a crucial element on both the learning and long-term retention of educational materials than simply the re-studying of these materials.  The researchers suggest that the outcome of massed study leading to short-term benefit, while spaced studying or testing having a greater effect on long-term retention, may be a reflection of desirable difficulties.  Learning in spaced presentations facilitates deeper learning than massed presentations, over a given period of time. Within this study, the results suggest that the testing and the spacing effect introduced a desirable difficulty during learning for the subjects (Roediger & Karpicke, 2006, page 249).  

Test-Enhanced Learning

- Sixth-grade social studies class
- International political economics course at Washington University in St. Louis

Studying the Testing Effect in the Classroom

The practice of retrieving information from memory to fortify and maintain one's knowledge. 
- using tests as learning events
- question asking 
- flashcards

Retrieval Practice (Testing)

Results of previous studies done in differing classroom environments and different quiz formats showed that there should be benefits of testing if questions on the final exam were related in content to questions on quizzes throughout the semester. The two main goals of the study were to determine the effects of quizzing content that would be on a final exam compared to no previous quizzing, and to determine the effects of quizzes on the retention of information that are conceptually related to quizzed content but had not been on quizzes. In short, to see if testing is a desirable difficulty for learning specifically tested material and/or material related to quizzed material.

Multiple-choice testing as a desirable difficulty in the classroom

Students take a mock exam with similar material to test their knowledge.

Rated as high utility. Practice testing is broadly applicable to types of material, and ages and abilities of learners. Moreover, it is one of the least time-consuming habits and evidence shows that practice testing is very effective in boosting academic performance.


Practice Testing

Using preparatory quizzes to aid study strategies study

Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning?. Journal of Experimental Psychology: Applied, 15(3), 243.

This study looks at the pretesting effect in different contexts (compared to extra study time and memorizing questions)

The pretesting effect: Do unsuccessful retrieval attempts enhance learning? study

- The paper used Sans Forgetica to see how much the font can improve memory performance.
- Subjects read information either in Sans Forgetica or Arial.
   - Experiment 1: subjects rated the difficulty of reading the information presented.
   - Experiment 2 - 4: subjects attempted to recall information from readings.


Disfluent difficulties are not desirable difficulties: the (lack of) effect of Sans Forgetica on memory

- The article discusses desirable difficulties in vocabulary learning as:
   - Learning that creates difficulty at first but later retention and transfer of what is learned are benefited.
   - Difficulties that are related to intrinsic motivation, engagement, and understanding. 
- The article also touches upon evidence of benefits from having difficulties in learning and the consequences for second-language learning. 

Desirable Difficulties in Vocabulary Learning

The act of taking a test generally improves future performance on the subject compared to simply rereading it or not being exposed to it again.

Learn Before

Related

Learn After