- The TAs who ranked questions on Bloom’s taxonomy showed a high level of agreement. At least two of the three raters agreed on the identical Bloom’s level for 85% of the questions rated.
- There was also a high level of agreement among the three TAs who evaluated exam questions for the PES values. There is a strong association between the Weighted Bloom’s Indices and the PES values.


Results Regarding Exam Equivalence

Results Regarding Exam Impact on Course Grade
- If exam points represented as little as 55% of the total course points, total exam points still explained 89% of the total variation in the final grade. Across quarters in the study, the average number of exam points required to get a 1.5 varied little (the range was only nine points).
- Analysis of deviance indicated no statistically significant variation among quarters in how well total exam points predicted final grade.


Results Regarding Exam Impact on Course Grade

Across all six quarters of the study, there was a strong relationship between the Predicted Grade and actual grade for each student. These data support the claim by Freeman et al. (2007) that the Predicted Grade model is a robust index of student ability and preparedness across quarters and years. A significant heterogeneity in Predicted Grades across quarters was observed: of the 15 pairwise tests, only six were not significantly different, and these crossed levels of course structure.

Student Equivalence Across Quarters in Introductory Biology

There are five main results for the experiments evaluating the impact of increased course structure on performance in introductory biology:
- Exam equivalence
- Exam impact on course grade
- Student equivalence across quarters
- Evaluating the drop rate
- Changes in failure rate in Biology 180

University of Pennsylvania

Google

The authors hypothesized that highly structured course designs, which implement reading quizzes and/or extensive in-class active-learning activities and weekly practice exams, can lower failure rates in an introductory biology course, compared with low-structure course designs that are based on lecturing and a few high-risk assessments. The experimenters controlled for instructor effects by analyzing data from quarters when the same instructor taught the course, exam equivalence with new assessments called the Weighted Bloom’s Index and Predicted Exam Score, and student equivalence using a regression-based Predicted Grade. The authors also tested the hypothesis that points from reading quizzes, clicker questions, and other assignments in highly structured courses inflate. They found no evidence that points from active-learning exercises inflate grades or reduce the impact of exams on final grades. When controlled for variation in student ability, failure rates were lower in a moderately structured course design and were dramatically lower in a highly structured course design. This result supports the hypothesis that active-learning exercises can make students more skilled learners and help bridge the gap between poorly prepared students and their better-prepared peers.

Increased Course Structure Improves Performance in Introductory Biology

Course Background
•	This research focused on students in Biology 180, the first course in a three-quarter introductory biology sequence designed for undergraduates intending to major in biology or related disciplines at the University of Washington. This course is a five-unit course offered every quarter.
•	The data come from six quarters when one of the authors (S.F.) was the only instructor. The course included four 50-min class sessions and a 2-hour or 3-hous laboratory session each week. In every quarter there were 400 exam points possible; all exam questions were written, 

Student Demographics
•	During the study period, most students had to complete a chemistry prerequisite before registering for Biology 180. The majority of students were in their sophomore year.
•	In the most recent quarter analyzed here, the course’s demographic makeup was 61.1% female and 38.9% male; 46.7% Caucasian, 38.5% Asian-American, and 7.4% Under-represented minorities (African-American, Hispanic, Native American, or Pacific Islander), with 7.4% of students declining to declare their ethnicity.
•	In addition, 16.4% of the students were in the UW Educational Opportunity Program, meaning that they were identified as economically or educationally disadvantaged.
•	These demographic data are typical across the quarters and years of the study.


Increased Course Structure Improves Performance in Introductory Biology: Course Background and Student Demographics

Increased Course Structure Improves Performance in Introductory Biology: Course Design

To control for exam equivalence and difficulty, the study utilized two primary measurement methods:

* **Weighted Bloom's Index**: Summarizes the average Bloom's level of exam questions weighted by points possible. Levels 1–2 test lower-order cognitive skills, while levels 3–6 assess higher-order skills. Three experienced TAs rated the exams blindly.
* **Predicted Exam Score (PES)**: Establishes grading equivalence. Three experienced TAs (different from the Bloom's raters) predicted the average score for each part of every exam question, while remaining blind to the study's intent.

Learn Before

Related

Learn After