- The TAs who ranked questions on Bloom’s taxonomy showed a high level of agreement. At least two of the three raters agreed on the identical Bloom’s level for 85% of the questions rated.
- There was also a high level of agreement among the three TAs who evaluated exam questions for the PES values. There is a strong association between the Weighted Bloom’s Indices and the PES values.


Results Regarding Exam Equivalence

Results Regarding Exam Impact on Course Grade
- If exam points represented as little as 55% of the total course points, total exam points still explained 89% of the total variation in the final grade. Across quarters in the study, the average number of exam points required to get a 1.5 varied little (the range was only nine points).
- Analysis of deviance indicated no statistically significant variation among quarters in how well total exam points predicted final grade.


Results Regarding Exam Impact on Course Grade

Results Regarding Student Equivalence across Quarters
- Across all six quarters of the study, there was a strong relationship between the Predicted Grade and actual grade for each student.
- This data support the earlier claim by Freeman et al. (2007) that the Predicted Grade model is a robust index of student ability and preparedness across quarters and years.
- A significant heterogeneity in Predicted Grades across quarters is presented. This heterogeneity was distributed across quarters in the study: Of the 15 pairwise tests, only six were not significantly different, and these crossed levels of course structure.

Results Regarding Student Equivalence across Quarters

There are five results for the experiments.
- Results Regarding Exam Equivalence
- Results Regarding Exam Impact on Course Grade
- Results Regarding Student Equivalence across Quarters
- Results Regarding Evaluating the Drop Rate
- Results Regarding Changes in Failure Rate in Biology 180

University of Pennsylvania

The authors hypothesized that highly structured course designs, which implement reading quizzes and/or extensive in-class active-learning activities and weekly practice exams, can lower failure rates in an introductory biology course, compared with low-structure course designs that are based on lecturing and a few high-risk assessments. The experimenters controlled for instructor effects by analyzing data from quarters when the same instructor taught the course, exam equivalence with new assessments called the Weighted Bloom’s Index and Predicted Exam Score, and student equivalence using a regression-based Predicted Grade. The authors also tested the hypothesis that points from reading quizzes, clicker questions, and other assignments in highly structured courses inflate. They found no evidence that points from active-learning exercises inflate grades or reduce the impact of exams on final grades. When controlled for variation in student ability, failure rates were lower in a moderately structured course design and were dramatically lower in a highly structured course design. This result supports the hypothesis that active-learning exercises can make students more skilled learners and help bridge the gap between poorly prepared students and their better-prepared peers.

Increased Course Structure Improves Performance in Introductory Biology

Course Background
•	This research focused on students in Biology 180, the first course in a three-quarter introductory biology sequence designed for undergraduates intending to major in biology or related disciplines at the University of Washington. This course is a five-unit course offered every quarter.
•	The data come from six quarters when one of the authors (S.F.) was the only instructor. The course included four 50-min class sessions and a 2-hour or 3-hous laboratory session each week. In every quarter there were 400 exam points possible; all exam questions were written, 

Student Demographics
•	During the study period, most students had to complete a chemistry prerequisite before registering for Biology 180. The majority of students were in their sophomore year.
•	In the most recent quarter analyzed here, the course’s demographic makeup was 61.1% female and 38.9% male; 46.7% Caucasian, 38.5% Asian-American, and 7.4% Under-represented minorities (African-American, Hispanic, Native American, or Pacific Islander), with 7.4% of students declining to declare their ethnicity.
•	In addition, 16.4% of the students were in the UW Educational Opportunity Program, meaning that they were identified as economically or educationally disadvantaged.
•	These demographic data are typical across the quarters and years of the study.


Increased Course Structure Improves Performance in Introductory Biology: Course Background and Student Demographics

Increased Course Structure Improves Performance in Introductory Biology: Course Design

Weighted Bloom’s Index.
- Bloom’s taxonomy of learning identifies six levels of understanding on any topic and they are used to create a Weighted Bloom’s Index, which summarizes the average Bloom’s level of exam questions weighted by the points possible. Level 1 and 2 questions test lower-order cognitive skills. Level 3–6 questions assess higher-order cognitive skills. Specific Weighted Bloom’s Index values conform to each Bloom’s level
- A Weighted Bloom’s Index for every exam in the study is calculated by recruiting three experienced TAs to assign a Bloom’s level to every exam question given in each quarter. If two of the three raters agree, the experimenters assigned the majority rule rating.When ratings were sequential, the middle value is accepted. When ratings were nonsequential, the arithmetic average is accepted
- The raters were trained in “Blooming” exams by one of the authors (M.P.W.) and assessed questions presented in a common format and random order.
- The raters knew that they were assessing Biology 180 exam questions. But they were blind to the quarter and year, the study’s intent, and to the hypothesis being tested by the ratings. Point values for each question were omitted to avoid any bias introduced by high- vs. low-point-value questions

Predicted Exam Score
- Predicted Exam Score (PES) is calculated by recruiting three experienced Tas (different from the individuals who did the Bloom’s ratings) who were also experienced graders to predict the average number of points that students would receive on each part of each exam question in the study.
- The PES raters were blind to the study’s intent and the hypothesis being tested with the predicted-points data.
- After a norming session when they discuss their predictions for average points, each grader worked individually and not discussed. The arithmetic average of the three ratings submitted are computed to arrive at a single predicted-average-points value for each of the exam-question parts after the norming session.

Learn Before

Related

Learn After