Traditional exam review sessions, typically conducted orally and in class by the instructor, are intended to identify the most frequently missed or problematic question with the intent of helping students perform better on subsequent exams. The shortcoming of this instructor-led activity is that it tends to focus on issues with content or understanding rather than helping the individual student prevent or avoid similar mistakes on future exams. Here, we report that students who performed a more comprehensive out-of-class exam debrief after the first exam significantly improve their exam performance compared with students that did not conduct the exam debrief. We also identify the most common mistakes that students make on exams and the most frequent self-selected strategies to improve their learning. By having students focus on missed questions coupled with addressing deficiencies in their test preparation strategies and behaviors, they likely engage in more self-regulated learning to better prepare for exams and avoid repeating past mistakes.
teachers use a variety of strategies before and after exams to help students learn course material and perform better on exams. Interactive in-class activities, practice exams, group work, concept mapping, online quizzes, student role playing, and practice problems are all common strategies that may be used before an exam and that have been shown to be effective in promoting learning (3, 4, 6, 16, 21). After exams, instructors often take time during class to go over the most frequently missed or misunderstood questions, with the instructor often reexplaining the concept or demonstrating the correct way to solve the problem. This strategy may marginally improve student learning if the course content is cumulative and/or the students will encounter similar problems on future exams. Unfortunately, this teacher-centered postexam strategy is mostly passive for students and often fails to address the challenges of the individual learner, their specific (mis)understanding of the material, or any potential errors in their exam preparation. What would be most helpful is for a postexam activity to mirror the interactive activities that happen in the classroom (20).
Success or difficulty on exams is usually highly individualized (10). By the time students arrive in college classrooms, they have had an extensive array of experiences that have shaped themselves and their learning behaviors, all of which can impact test performance (8). In addition to previous preparatory-related course work, students often bring with them a set of study habits that, for better or worse, have served them well enough up to that point in their academic career. Some students learn early on in their student life that they need several weeks to prepare for an exam, but, typically, most do not study seriously until only a few days before an exam (9).
When it comes to what they study, some students diligently read and highlight the book, some copy “highlighted” or key phrases from the book into their notes, some make flash cards, and some recreate figures and drawings to help them learn materials (10, 15). Few students use active learning strategies when studying on their own, despite the fact that they have been exposed to them in class, or engage in self-testing or retrieval practice (11).
What is also clear from the literature is that no two students use the same set of learning strategies (9, 10). For that matter, given the variety of learning styles or preferences that have been shown to exist in health science students, no single strategy would work with any group of reasonably diverse college students (6). Furthermore, the majority of students lack the habits of a self-regulated learner (23). Self-regulation involves the selective use of specific activities and processes that are personally adapted to each learning task (23). The self-regulated learner sets goals, selects strategies to attain those goals, monitors their progress, and restructures their process if the goals are not being met. While the teacher has control over the classroom, its environment, and the suite and timing of interactive activities to be used, they often have no control over the activities or timing that students use to organize, study, and learn the content outside of class. While self-regulated learning is a skill that can be taught, few teachers effectively prepare students to learn on their own (23). Therefore, the goal of this study was to assess if an interactive exam debrief (ED) process was successful at improving students' exam performance on subsequent exams.
Participants and course.
The procedures used in this study were reviewed and approved by the Institutional Review Board at the University of Portland. Participants were recruited from two upper-division Human Anatomy courses taught to biology and nursing students by the same instructor. Enrollment in the Human Anatomy course requires a grade of at least a C in a 200-level Introductory Cell Biology and Genetics course. The lecture course, a 300-level 3-credit hour class taught for 3 contact hours/wk, is accompanied by a separate and distinct 1-credit 3-h laboratory taught and graded by a different instructor. The typical student is a second- or third-year student and ∼19–20 yr of age. Exams for both sections are identical. In this course, on average, students were exposed to at least one interactive learning activity per class period. Some of those activities included the following: think-pair-share, 1-min paper, progressive partner diagramming/drawing, web-based interactive quizzes (similar to clickers), flash card puzzles (see below for description), and problem-based learning.
The ED process.
After the first exam was returned to students, every student was invited to debrief their exam, typically during the professor's offer hours, at any time before the second exam. A scripted process was used to guide the student through the debrief process and is included in the appendix. In summary, the debrief process comprised five parts. In part 1, using their graded exam, students were instructed to identify any questions they missed and to determine, as best they are able, why they missed each question using several suggested reasons (i.e., misread the question, made a dumb mistake, or could narrow it down to two but selected the wrong answer) or the option of providing their own reason. In part 2, each student performed a cursory analysis to determine if, from their responses in part 1, a pattern emerged that would help them identify their most common mistakes. In part 3, students were asked to briefly describe the ways in which they prepared for the exam. Part 4 requested students to identify learning strategies (appendix, part 4) from a list that they believed, given their most common errors coupled with the ways in which they prepared for the exam, would be most beneficial to help their future learning and test performance. Topical categories included: time on task, attention to detail, active learning strategies, general study habits, and other. Part 5 asked students about specific requests they may have to the professor to help aid them in better learning the material. After completing the form, each student met with the professor for a short discussion of the students' “findings” from completing the ED process and could request additional assistance.
To ensure our samples were of equal distribution and independent, we conducted a χ2-test on those student populations across the A, B, C, and D/F grade populations for those who did and did not complete the ED. Using a Students' t-test, we compared the means on the first exam to determine if the first exam score was different between those students that completed the ED versus those that did not. We also calculated the mean difference between the first and second exam scores using the t-test for both groups. Significance was set at P < 0.05. Finally, we assessed the relative size of the effect based on standardized estimates of effect size according to Cohen's benchmarks (5). For the ED data, which were self-reported by students, descriptive statistics (numbers and percentages) were computed for each part of the ED. For part 1, the total number of missed questions was calculated and the percentage was calculated to identify the most common reasons for missing questions. For part 3, students' written responses on how they prepared for the exam, we tallied and organized into response thematic groupings (i.e., read the book). For part 4, student responses in each section were tallied to determine which learning strategies students believed would be the most productive for future studying. To determine if the debrief sessions were effective in improving student scores, we compared the mean difference between first and second exam scores for those students that participated in the ED with those that did not using an unpaired t-test (P < 0.05).
A total of 64 students were enrolled in two identical upper-division Human Anatomy courses taught back to back on the same mornings 2 days/wk. Fifty-two percent of the students opted into the ED process. No differences were seen in students' first exam scores for those that completed the ED and those that did not across four grade categories (A, B, C, and D/F grades) using a χ2-test. We also found no difference between the averages of the first exam scores for those two student populations (Table 1). However, a significant difference was observed in the mean increase in exam performance from the first exam to the second exam for those students that conducted the ED (Table 1). The calculated effect size was 0.48, demonstrating a moderate or medium effect size for the ED.
The results of the exam analysis showed that students miss test questions for a wide range of reasons (Fig. 1). Students cited two predominant reasons for missing questions, which accounted for almost two-thirds of the responses. They self-identified that they (the student) did not know the basic anatomic information or the relationships between the structures or that they could narrow the answer down to two choices but ultimately chose the wrong one. Making a silly mistake and having the correct answer before changing it also proved to be frequent reasons students missed questions on exams.
The most commonly cited study behaviors were reading the book, taking notes, and reviewing those materials. About 5% of the students indicated that they didn't read the book before the exam. About 50% of the students noted they reviewed the PowerPoint slides and filled out or answered questions on the study guide for each chapter. Thirty-three percent of the students made flash or note cards to study with. About 25% of the students took a more active approach by discussing material with one other student or in a study group. The least cited activities (<15%) were diagramming or drawing figures and taking online quizzes or tests (∼10%). We did not ask specifically about how much time students spent on these activities.
When asked the ways in which students would change their preparation methods to improve their exam score, 100% of the students identified strategies from the active learning category to help them succeed. Ninety-three percent identified time on task activities for the second most selected item. Paying more attention to detail and working on better general learning habits were identified by 80% of the students (data not shown).
Within the category of time on task, as shown in Fig. 2, the most common suggested improvements were to start serious studying sooner (85% of students), to study a little more every day (79% of students), and to manage time better with other classes and activities (61% of students). Only one student cited coming to class more or on time as a suggested improvement.
Figure 3, which shows the suggested improvements in the attention to detail category, had a similar number of responses for each of the three options, ranging from 40% to 52%. Trying to understand the big picture before focusing on the details was selected with the greatest frequency by 52% of students, whereas trying to understand what they read in the book and paying more attending to the figures in the book and PowerPoint slides were each cited ∼40% of the time.
In the active learning strategies category, as shown in Fig. 4, 73% of students identified the active strategy of drawing (in two categories) as the most cited corrective actions. Making concept maps and finding and taking online practice quizzes were each identified by ∼50% of the students as a way to improve with the remaining categories, each receiving between 25% and 35% of responses from students.
As shown in Fig. 5, the suggested improvements in the general habits category, the most popular response was “to not panic,” with almost 70% of students selecting this option. Thirty-three percent of the students selected asking for help when needed or to not waiting so long after class with this option. Getting more sleep or breakfast before the exam was selected by 30% of students.
Most students entering college carry with them study behaviors and temporal learning patterns (i.e., preparing for weeks vs. cramming) they developed in high school. As a result, most do not know how to individualize or self-regulate their learning to address discipline-specific demands. Despite the fact that the cognitive tasks in college multiply and diversify, students generally apply their similar study techniques across multiple disciplines until those techniques no longer produce adequate results. Many books have been written to help students study (7, 19), and many have identified techniques that may be effective (14, 17, 22). The truth is how each student learns is highly individualized, and very few of these published suggestions help students identify and optimize learning strategies that make sense for them and across various disciplines. The work presented in this study shows that when students identify self-directed strategies based on how and why they missed exam questions, they are more likely to improve their exam performance.
The first step for instructors in helping students develop more effective exam preparation strategies is trying to determine errors in student thinking and/or preparation. Having the professor provide the correct answer to a missed question may not immediately generate a concrete idea for a student about how to avoid or correct a future mistake. What is unique about this ED process is that going through an exam to categorize the frequency of missed questions allows the student to identify patterns of most common errors. We observe that the two most frequently cited reasons for missing questions were that they didn't know the basic information or they could narrow their choices down but did not select the right answer. Both of these common errors generally reflect a lack of preparation and/or depth of studying. Knowing how and why students miss questions provide a starting place for discussion about how to modify and adapt their study activities.
When this error analysis is coupled with the student's own description of how they studied, both student and professor can begin to see the direct causal link between preparation and performance. Interestingly, from the students' description of exam preparation, we observed a trend that the more passive the exam preparation strategy, the more frequently it is used. Not surprisingly, most of the students' preparatory activities were passive, i.e., read and review of course materials or practice, known as maintenance rehearsal. Some noted active strategies, such as writing, but this mostly consisted of rewriting of notes and completing study guide. Very few students engaged in elaborate rehearsal exemplified by drawing figures, diagrams, or concept maps, all known to be effective in improving understanding in anatomy (15). The students' descriptions of their, mostly passive, study habits are consistent with two reports. In a report from a psychology class (9), it was noted that the passive maintenance rehearsal activities received the greatest amount of student study time. In another study, almost 60% of the students indicated that they would go back and reread the chapter after they had read it the first time rather than use a new activity to build on their reading (11). Our results confirm what others have shown: that most students lack the metacognition understanding or a self-awareness of the student about his/her own capability in a particular learning area (18).
A key feature of this ED process is for students to self-select the changes in their study behaviors that they believe would be most helpful and to assist them in becoming more self-regulated learners. Not surprisingly, 100% of the students selected options from the active learning category. We demonstrate, model, and use these same interactive these activities in class to arm students with more effective study options when out of class. While we don't track students' use of these activities, it would be helpful to better understand what they believe is most critical in assisting their learning.
For example, the use of flash cards, generally a passive technique, remains a frequent option for students. While flash cards work for some, the strategy is typically only effective for knowledge-type questions. Because many students use this strategy, they are shown a different and more interactive flash card strategy. In class, they are handed a set of premade flashcards for one chapter. Rather than flip through them to read the knowledge component, students are instructed to lay them face up of the table and organize the cards into categories. For example, in the nervous system, the cards can be categorized as central versus peripheral or motor versus sensory. Students can also identify nervous system structures from a regional perspective (i.e., which structure directly borders another), making a physical map of the brain or nervous system using the cards. Students often find this technique a struggle at first but then realize it is not just about the definition written on the back of the card but the relationship between that structure and any other structures with which it connects, anatomically and/or functionally.
Time on task was the second most identified strategy (87% of the options), and the majority of the students indicated that they underestimated the amount of time it took to learn the material. Thus, it was not surprising that “start serious studying sooner” and “study a little more each day” were the most frequent choices in the time on task category. With each student enrolled in an average of five lecture courses each semester and two 3-h laboratories, managing their time is one of the most challenging aspects of the transition to college learning. Several papers have shown a positive correlation between time management and student grade point average (2, 13). Many students noted test anxiety as an issue in test performance, a behavior that has been inversely related to academic performance (24). Far too many students recognize too late in the process that they may not be fully prepared for the exam. Anxiety that can accompany poor preparation increases their stress level, which likely contributes to their poor academic performance. While text anxiety can be difficult to fully ameliorate, some aspects of test anxiety can be resolved with changes to time on task and the confidence that comes with studying sooner and feeling more prepared. We did not ask students how much time they spent studying before the first exam in part 3 of the ED, but it is well known that activities that promote deep learning takes more time than most students believe. Since the publication of How People Learn: Brain, Mind, Experience, and School (1), it has been quite well documented that real transferal of information and deep learning takes more than a few days. Their recommendations of shorter but intensive sessions over a period of weeks rather than longer sessions over a few days has proven to be a successful strategy for most students.
Students also recognized that they need to be more intentional and mindful, or metacognitive, in their study habits by identifying when is it best to read, what is the most effective way to take notes, or what are the most effective activities to use after reading. Science textbooks can be quite dry subject matter when reading and can bore students. No single textbook reading strategy has been shown to be highly effectively across multiple students. Some students shared they like to read the text before each lecture to make the lecture clearer, whereas others have noted that they like to read after each lecture because the lecture helps them better understand material from the book and key elements on which to focus. Regardless of when the reading is complete, having students identify strategies that work for them prepares them to be successful in the long run.
At the end of the debrief process during discussions with the professor, most students responded “no” with regard to the question “Is there more the professor can do to help?.” While a few asked for additional interactive strategies such as more online quizzes or electronic diagrams, the majority of students owned up to the fact that that they did not adequately prepare and misjudged the time it would take to be successful, hopefully the beginnings of becoming more self-regulated in the approach to learning.
The ED had a moderate yet significant and positive effect on student exam performance. While we cannot say for certain which of the study behaviors were most helpful to students, self-selection in determining how, what, and when to study and following through on those changes does make a difference. That said, we cannot discount that the motivation to complete the ED process and/or meeting with the professor alone may spur students to become more motivated to work harder. But, any activity like the ED that helps connect the professor and student outside of class and leads to better learning strategies while yielding material differences in student performance should be included as a helpful learning activity.
We also note that another effective strategy to improve performance is for students to correct their corrected exam to identify the right answer. We did not focus on identifying the correct answers in a direct way, although this happens through the debrief informally. We recognize that this can be a very effective strategy to improve performance, and students are encouraged to spend more time to correct their exam if they believe it will be helpful. Rather, we were interested in how and why students make mistakes and how visualizing those patterns may be more helpful in changing study habits.
In conclusion, the data from this study demonstrates that no strategies work all of the time, for all students, in all classes (7, 9, 10, 22). By actively involving students in the improvement process and having students self-select study strategies that will address their most common mistakes, their academic performance improves significantly. Hopefully, this process helps them to recognize that the responsibility for lifelong learning is within their hands.
No conflicts of interest, financial or otherwise, are declared by the author(s).
T.G.F. conception and design of research; T.G.F. performed experiments; T.G.F. and N.H. interpreted results of experiments; T.G.F. and N.H. drafted manuscript; T.G.F. and N.H. edited and revised manuscript; T.G.F. approved final version of manuscript; N.H. analyzed data; N.H. prepared figures.
One way to improve test performance is to outline and debrief your examination after the test. To do this, you should determine, as best you can, why you missed exam questions.
Go through your test carefully and identify each question you missed. Select the best reason, if you can, as to why you missed each question.
Is there a pattern that you see emerging? (it is okay if there isn't a pattern.) If so, what is it? (See Table 2 for possible reasons for missing a question.)
Briefly describe the ways (both time spent and activities) in which you prepared for the exam.
Based on the information, what changes do you think you could make in your studying that could help you prepare for the next test? Please check all that apply.
Time on Task
□ Manage my time better with other classes and activities.
□ Start serious studying sooner.
□ Study a little more every day.
□ Come to class more often or on time.
□ Come to class prepared or better prepared.
Attention to Detail
□ Try to understand what I read in the book.
□ Pay more attention to the figures in the book and Powerpoints.
□ Try to understand the big picture and not focus on details until I do have the big picture.
Active Learning Strategies
□ Try to recreate figures or tables from memory.
□ Write down answers to the reading questions.
□ Make concept or word maps (start with little ones and then combine and add complexity).
□ Try actively in class to solve the problems instead of just waiting for the answer.
□ Actively search for answers or more information from outside sources/the web/Google.
□ Find and take online practice quizzes.
□ Create my own compare/contrast questions or tables.
□ Draw figures or diagrams to better understand the relationship between structures.
□ Make vocabulary lists or flash cards.
General Study Habits
□ Ask for help when needed or not waiting so long after class.
□ Get more sleep/breakfast before the test.
□ Don't panic.
□ Anything not listed? Write it here.
In what ways can I help you with the material in this course? (Is there anything else that I need to know about you as a learner/student?).
- Copyright © 2016 The American Physiological Society