The flipped classroom was utilized in a two-semester, high-content science course that enrolled between 50 and 80 students at a small liberal arts college. With the flipped model, students watched ~20-min lectures 2 days/wk outside of class. These videos were recorded via screen capture and included a detailed note outline, PowerPoint slides, and review questions. The traditional format included the same materials, except that lectures were delivered in class each week and spanned the entire period. During the flipped course, the instructor reviewed common misconceptions and asked questions requiring higher-order thinking, and five graded case studies were performed each semester. To determine whether assessments included additional higher-order thinking skills in the flipped vs. traditional model, questions across course formats were compared via Blooms Taxonomy. Application-level questions that required prediction of an outcome in a new scenario comprised 38 ± 3 vs. 12 ± 1% of summative assessment questions (<0.01): flipped vs. traditional. Final letter grades in both formats of the course were compared with major GPA. Students in the flipped model performed better than their GPA predicted, as 85.5% earned a higher grade (vs. 42.2% in the traditional classroom) compared with their major GPA. These data demonstrate that assessments transitioned to more application-level compared with factual knowledge-based questions with this particular flipped model, and students performed better in their final letter grade compared with the traditional lecture format. Although the benefits to a flipped classroom are highlighted, student evaluations did suffer. More detailed studies comparing the traditional and flipped formats are warranted.
the options available for educators to approach teaching and learning have changed with recent advances in technology (11, 12). One noted method is the use of the flipped classroom that provides students with basic factual material through videos or other deliverables, whereas classroom time is devoted to performing activities that help to apply the material (flippedlearning.org). This is in contrast to a typical traditional classroom in which time outside of class is spent on activities that include application of material. By flipping the classroom the instructor is directly available to facilitate application of material, when the student would typically be struggling to master such fundamentals. The factual content that typically does not require intense dialogue to understand and conceptualize is placed on the student to master outside of class.
Although the prevalence of the flipped classroom has risen over the last 5 yr, our understanding of its role with regard to retention and student learning assessment is still being discovered (1, 14). The use of technology in the classroom should be approached with specific goals in mind, not merely because the technology is available. To this end, the following research question was addressed: Does the flipped classroom allow for more activities and assessments devoted to critical thinking? After teaching courses with both traditional and flipped modalities, it was certain that basic factual knowledge was mastered with each teaching mechanism, but the flipped classroom appeared to offer more time for application of material and, to this end, critical thinking to answer such applications. A second research question was investigated to determine whether technological supports, including student response systems and analytics related to flipped video viewing, would correlate to exam scores. Taken together, information regarding these questions will inform the usefulness of the flipped classroom with regard to critical thinking, technological supports, and student learning assessment.
All protocols were approved by the Institutional Review Board at Le Moyne College. Anatomy and Physiology is a two-semester, 200-level course sequence offered at a small liberal arts college with 1 yr of general biology and general chemistry as prerequisites. The class met for 150 min/wk along with a laboratory component. In 2007, the class had three 50-min lectures/wk, whereas 2011 and 2014 followed a 2 day/wk format. The course is typically carried out in a tiered lecture hall with the ability to move seats but not desks for collaborative work. In 2011 the course was offered in a flat, open, large room, with all desks and seats made mobile. Since only one section of the course was offered during the investigation, it was anticipated that the samples in all cohorts (2007, 2011, 2014) were random samples, thus avoiding a common challenge in teaching intervention studies (13). Anatomy and Physiology is typically one of the largest courses at the institution (50–80 students; “large enrollment” is certainly institution dependent), and the flipped classroom was initiated to find ways to increase one-on-one interactions and discussions with the students. To this end, various forms of technology were instituted while this particular course was taught. During the 2014–2015 academic year the course followed a flipped classroom model, with class time used for practicing application of material and review of the most difficult concepts. It was the third year of offering the flipped model for this material at the institution, and therefore, most students were aware of the format when enrolling in the course. The differences and similarities between the traditional and flipped models for this investigation are listed in Table 1.
The first day of the flipped class was used to explain the course structure and demonstrate to students how to access the materials, which were available through the institution’s course management system (Canvas; Instructure, Salt Lake City, UT). The videos were produced using screen capture through Echo360 (Reston, VA) and contained primarily PowerPoint slides, with the instructor moving the cursor to explain topics; the longest instructor-made video was 25 min in length. In addition, short YouTube animations were also utilized within the course content. A note outline, PowerPoint slides, and review questions were made available for each video along with eight multiple-choice questions via Lecture Tools (an Echo360 Active Learning) technology. Importantly, the Lecture Tools questions were used as formative assessment, and the questions were not repeated on exams; this was critical for the ranking of application questions on summative assessments. Lecture Tools is a student engagement technology that allows questions to be displayed to students; answers are delivered with any electronic device connected to the internet or by texting via a cell phone. During class discussion, the instructor shows the results for a particular question to the entire class; the percentage of respondents that picked each choice is listed in a histogram. By the end of the first week of classes, students accessed course content, watched an introductory video, and answered some basic Lecture Tools questions. Finally, during the first laboratory meeting for the course, students were assigned to semester-long teams for all collaborative work.
Teams were established by the instructor and were comprised of four to six students with various academic strengths and long-term career goals. In addition, teams were usually gender equal or in some cases predominantly female, as this has been shown to influence how women address problem-solving activities (3). However, in the instance of this course, there were no differences between gender-mixed groups and women-only groups with respect to final grades (3.29 ± 0.11 vs. 3.31 ± 0.13; means ± SE). After the teams were selected, the students chose a team name that was used for all announcements regarding the team members for the remainder of the semester. This format of teamwork was modified from Team Based Learning by Larry Michaelson (teambasedlearning.org). Students work together throughout the semester in both laboratory and lecture; at the end of the semester, peers rate each other in an evaluation to describe how much they have contributed to teamwork. For this course, teamwork included laboratory dissection, case studies, team assessment appeals, and team quizzes. After an individual multiple-choice quiz was taken, the Scantron forms were turned in, and each team picked up an Immediate Feedback Assessment Technique (Epstein Educational Enterprises, Cincinnati, OH) form from the instructor. The forms are similar to a foiled lottery “scratch off” ticket; the foil is covering choices A, B, C, D, and E. Under the foil of the correct answer for each question there is a star. Teams then completed the exact same quiz but revealed the correct answer (via the star) before moving on to a new question. Each additional attempt on the card reduced the number of points earned for each question. After completing a quiz, teams could appeal assessment questions. If the appeal was accepted, the points were returned to the individual quiz score. Although team exams were not employed per se, the class period following an exam was spent with teams going through the exam questions, followed by an appeals process for up to two assessment questions for each exam. Prior to the teams evaluating their answers on a particular exam, the instructor would go through each assessment question regarding its Blooming Biology rating (2). Therefore, students would know which questions were knowledge, comprehension, application, analysis, and synthesis as rated on the Blooming Biology scale.
The Blooming Biology Tool was used according to that described by Crowe et al. (2). With this scale, knowledge-ranked questions focused on rote memorization. A question addressing knowledge would consist of listing characteristics of a body system or defining a term properly. Comprehension ranked questions involve taking facts about a process and putting them in the proper organization or describing the development of a pathology with known facts about a body system and the accompanying disease. Application questions can only be classified as such if the student has not seen the particular question before the exam; students must be able to take new information and put the pieces together when answering these types of questions. Example questions at the knowledge, comprehension, and application level are listed in the appendix. For classifying the questions using the Blooming Biology Tool, three raters scored each assessment question independently and then met together to compare ratings. If discrepancies arose between raters, a solution was reached through discussion.
To quantify the level of critical thinking assessed with the flipped model compared with a traditional lecture, exams from 2014 (flipped) were rated as well as those from 2007 and 2011 (both traditionally taught with lectures). The two traditional cohorts were chosen for multiple reasons. First, the laboratory portion of the course was overhauled in 2010. Therefore, if 2007 and 2011 showed differences in assessment of critical thinking, it could be attributed to the laboratory alterations; this design would confirm and control for such a possibility. Second, changes to the general chemistry curriculum were instituted in 2010, and following in 2011 students could not advance to the second semester of general chemistry without earning a higher grade than previously allowed. Comparing 2007 and 2011 should control for differences associated with the change in chemistry prerequisite. Third, this is a built-in control to compare the ability of the instructor, as it may have changed over the time course of the study; the assessments could have transitioned to include higher-level critical thinking questions. Notably, no differences were found in the Blooming levels of questions from 2007 to 2011 (see Fig. 1 legend) along with comparable grades earned in each cohort (2.84 in 2007 vs. 2.89 in 2011).
The 2014 flipped model questions were analyzed since it was the only year of flipped classroom teaching that included composition of 100% multiple-choice questions in the assessments comparable with the traditional model. In both teaching models similar note packets, PowerPoint slides, and review questions were provided to students. Importantly, the same general objectives were covered in each format of the course; this is a key component of the study design. The traditional model spent virtually all of the class time with the instructor lecturing. However, ungraded Clicker questions were utilized in 2011 with the traditional model to make the class more interactive during lectures. In addition, some ungraded case studies were also deployed in the traditional model in 2011 when time allowed, although less class time overall was spent on case studies in this model compared with the flipped model. Five graded case studies were administered in each semester of the flipped model course. Therefore, the primary differences between models were video lectures, case studies, Lecture Tools, and ample class time to go over and apply the most difficult concepts.
Significance was chosen a priori to be P < 0.01. A two-way ANOVA was performed on Blooming Biology data and a t-test with Bonferroni correction used when an interaction was found. Correlations were run on data plotting Lecture Tools performance vs. exam scores and video viewing vs. exam scores. The f-test was utilized to determine whether the variance was different between assessment scores in 2011 (traditional) and 2014 (flipped). A χ2 test was performed on major GPA compared with course grade data with traditional instruction used as the expected outcome.
A key finding of this study is that a flipped classroom approach allowed for more time devoted to active learning and a transition of assessment questions to include additional higher-order thinking activities. As quantified with the Blooming Biology method (2), the summative assessment questions in the flipped classroom included a greater percentage that was at the application level compared with the traditional classroom (Fig. 1). In addition, the overall course letter grades were elevated (Fig. 2) along with the exam and quiz grade averages (Fig. 3) in the flipped design. More telling are the comparisons between major GPA and letter grade in the course (Table 2).
It was expected that the major GPA (including only biological sciences courses) would be a predictor for performance in future major courses. As an example, if a student has a 2.0 major GPA, then the expected average grade for their next major course would be a C, which is calculated as 2.0 on a 4.0 scale. The 4.0 scale for grades was as follows: A = 4.0, A− = 3.7, B+ = 3.3, B = 3.0, B− = 2.7, C+ = 2.3, C = 2.0, C− = 1.7, D = 1.0, and F = 0. For the traditional format, GPA did predict the grades in the course, as similar percentages of students performed lower/higher in the course compared with their major GPA. However, in the flipped course, 85.5% of students earned a higher letter grade than their GPA; suggesting that they earned a higher letter grade than expected based on their past performance. The flipped course data for this comparison may also be underestimated since all four of the students who earned a grade equal to their GPA had a 4.0 major GPA, creating a ceiling effect for these strongest performing students (i.e., they earned an A in the course but are listed in the equal row). When comparing the cohorts of students in these two models, the higher overall GPA for the flipped model cohort (Table 3) should not be ignored. However, students with an existing higher GPA do have a reduced window to earn an even higher grade in the course. Therefore, the data are presented in multiple ways (Figs. 2 and 3 and Tables 2 and 3) for the reader to make their own interpretation.
Another performance aspect of the flipped classroom was the reduced number of low-scoring students with respect to final grades. Both 2007 and 2011 with the traditional model had students earning D’s and F’s (2 D’s and 1 F each year). With the flipped classroom there were no F’s/withdrawals and only one D. This is noteworthy since the flipped course was larger in enrollment compared with 2007 and 2011. Part of the reduced failures could be explained with the difference in final grade calculation or by the elevated overall GPA in the flipped model cohort (Table 3). The team component accounted for 15% of the final grade, Lecture Tools 15%, and summative assessments 45%, thus allowing students additional ways to support their final grade beyond exam scores in the flipped model. The traditional model included summative assessments as 75% of the final grade. In all courses, the laboratory component was 25% of the final letter grade (Table 1). Two cohorts of low-performing students emerged from the flipped data: 1) lowest exam scores and low team and Lecture Tools scores; and 2) low exam scores, higher team, and Lecture Tools scores. Those students who were active in class drastically improved their final grades in the course compared with the lowest-scoring group, who only marginally improved their final grade in the course with respect to exam scores. This suggests that while the additional activities could raise the students overall grade, it also encouraged the student to attend class and interact with peers, possibly increasing their understanding of the material. At the very least, it should have prevented students from falling behind in the coursework before an exam. This cohort of students is small, but these data highlight the need for additional studies in the area of the flipped classroom and student performance, particularly those students struggling with the material.
To directly compare how students performed on summative assessments without the confounding possibility of teamwork and Lecture Tools raising the final grade, the average scores (%) for summative assessments are plotted in Fig. 3. The mean was higher in the flipped cohort (P < 0.01, t-test) despite there being substantially more questions at the application level. In addition, the variance was lower (P < 0.01, f-test) for the flipped classroom assessments compared with the traditional model. In the performance analyses for this investigation, 2011 and 2014 were compared for the fall semester since both cohorts were comprised of primarily Biological Sciences majors. In 2007 many nonnatural science majors were in the course, and therefore that year is excluded. The spring semester was not analyzed since a scheduling conflict prevented some students from enrolling in 2015. Therefore, two cohorts of students that included 142 students were compared (see Table 3 for additional demographics). Importantly, the same instructor taught both courses, and the print materials provided were similar. That stated, the flipped model supplied additional supports, including team quizzes and online lectures with accompanying Lecture Tools questions. In addition, the overall GPA for the traditionally taught students was lower compared with students in the flipped model. Therefore, it is possible that students in the flipped model were more likely to perform better in the course. To control for this concern, the expected grade in traditional and flipped courses (major GPA compared with course grade) is presented in Table 2.
Unexpectedly, there was no correlation between completing lectures online and exam scores or correctly answering Lecture Tools and exam scores (data not shown). The latter was likely due to the course layout. The original goal was to have students watch the video outside of class and answer the Lecture Tools questions at that time. To ease student anxiety about Lecture Tools questions, polling was left open until students were in class and had met with their peers for 10 min. Although this increased peer discussion about the material, it likely inflated student scores with Lecture Tools and possibly prevented full student preparation for each class day. Therefore, if students attended and interacted with their classmates, they could receive ~100% on these assignments. These data are in line with a report discussing “inappropriate” use of formative assessment questions by students in a medical physiology course (7). Regarding the video viewing, most students did view the lectures, but often times this was the week before the exam. In a survey deployed to the class, 76.7% of students identified as viewing the videos on time. In a separate question, 50.7% reported watching each video once, 35.6% twice, and 16.4% more than three times before an exam. If a more detailed ability to follow student’s viewing habits existed, it would be useful to see whether the videos were viewed on time for each day in which they were discussed in class and how the survey data correspond to actual viewing.
The data from this study demonstrate that the flipped classroom allows for a transition of assessment questions and activities focused on critical thinking while also elevating final and predicted letter grades in a high-content, larger enrollment (50–80) science course compared with the traditional method. In addition, fewer students who enrolled in the flipped course withdrew or failed. This could be attributed to the students, not the course format, but the elevated course grade compared with major GPA implies that it is the flipped model that carries more weight in the comparison. These data are in line with a recent investigation demonstrating enhanced student performance when active learning components are employed (4). Overall, with the described flipped model, students were assessed at a deeper level of thinking yet performed better, which suggests that different learning occurred.
Although the positive data generated from the flipped format paint a rosy picture, the reality is that preparing and/or converting a course to the flipped model takes faculty and instructional technology time as well as student buy-in and administrative support. The data presented with the flipped model represent the third year the course was taught with this method. During these 3 yr, new perspectives and insight were gleaned regarding this pedagogy. Briefly, the major points that stood out during this time were as follows. 1) The online video lectures are best received when they are ≤25 min; this time frame maintains student engagement, and therefore, only the most important information is included. 2) What is considered to be the most important video content involves contemplation and discussion with other colleagues; organizing how/when additional material will be applied with case studies and Lecture Tools must fit into this equation. 3) More responsibility is placed on the students to watch the videos and recognize areas in which they are struggling; this requires watching the online lectures and performing the Lecture Tools questions. 4) Some students, especially those who have mastered a way to memorize large bodies of material, will be frustrated with this new concept of teaching; therefore, student evaluations may suffer. 5) Students need ample practice with higher-level assessment questions, whereas exam questions should be new (i.e., students should not merely memorize an application question but be able to work through information and apply it to a particular exam question). 6) Offering students practice questions requires generating more formative assessments and, therefore, more class time devoted to this activity to help students succeed. The flipped model allows for this additional time requirement. Clearly, each instructor will approach a flipped classroom in their own way; the items above list those that stood out for this particular experience.
Regarding the overall content of this course: the online videos included only the most important depth of concepts, whereas the overall learning objectives, or breadth of material, were similar in each cohort. To this end, all courses (2007, 2011, and 2014) were provided with the same detailed note outline and review sheets. Despite these similarities, the flipped classroom focused on applying material through graded case studies and additional Lecture Tools questions, whereas the traditional classroom focused on lectures, including more examples of various applications of the material. Therefore, the traditional students were told how the basic content applied to physiology, whereas the flipped students carried out the case studies through active learning.
Based on student evaluations for this investigation, it appears that students prefer to be told what to memorize as opposed to learning the material through activities. In 2007, (traditional) students answered the question “Overall, rate this course,” as 1.8/5, with 1 being the best course possible. The same question scored 2.7/5 in 2014 (flipped). The data from 2011 are not included since various factors, including a temporary classroom with technical challenges, may have influenced the survey that semester. The question “Take this instructor again,” scored a 1.2/5 in 2007 (traditional) vs. 2.2/5 in 2014 (flipped). These data are certainly instructor and institution specific, but they highlight the importance of administrator support when transitioning to a flipped classroom. At some institutions these student surveys hold considerable weight with promotion and tenure decisions. Having fellow colleagues and supervisors observe faculty who are using active learning pedagogies is recommended for junior faculty who may be considering such a transition.
Although the flipped classroom can take additional time to prepare, the data presented in this article suggest that efforts are worth it for different learning to occur. Particularly in the field of physiology, it is important for students to put facts together to answer “bigger picture” questions about homeostasis (8). Once students begin to grasp this concept, they are more accepting of a teaching and learning format that is different to them. Importantly, when application questions were administered to students in the traditionally taught format, it was rare for the question to be answered correctly. Students had little experience putting the pieces together, so (looking back) it was not surprising that they had trouble with this type of higher-order thinking. The flipped classroom not only offers time to practice and learn how to answer application questions, it allows the instructor time to cover analysis and synthesis questions while supporting students to get these questions correct.
Taken altogether, it is argued that the time and effort spent to overhaul a course from traditional to flipped are a useful investment for the benefit of student learning. The type of course should certainly come into play with this decision, but for content-heavy courses that require building one topic on the next, it can be a fruitful experience for both the instructor and student. The additional time to help students with the most difficult concepts can be rewarding as an instructor, as is watching teams work together to answer case studies. This does require more preparation by the students; instituting forms of accountability to document this preparation are key to the flipped format. To this end, Lecture Tools questions (8/class day) were used as a way to keep students on track with the material. Although there were no correlations between Lecture Tools correct answers and exam performance, these activities were still viewed favorably by the students through informal feedback. In fact, students often requested more Lecture Tools questions as a way to prepare for exams. Since Lecture Tools questions were never used directly as exam assessments, the questions mainly helped build student confidence with the material. It was also Lecture Tools questions that were used as a springboard to discuss some of the most difficult concepts. Therefore, daily formative assessment questions were useful in helping to support the overall format of the course, which relies heavily on student preparation. This observation is in line with other findings that preparation is key to student success in the flipped classroom (5).
These formative assessment questions and their corresponding interactions support a recent report demonstrating that it is active learning, not the flipped classroom, that influences student learning gains and attitudes (6). In the reported model, the instructor acted as a guide to learning the material, with an emphasis on active learning, and there were similar unit exam scores compared with a flipped model with active learning. The work of Jensen et al. (6) points to the benefits of restructuring a course to include active learning. Their cohort included nonscience majors and only those who attended all semester long and passed the course. Therefore, more information about ways to support struggling students and how to incorporate and make time for active learning in a course is an important aspect to consider when designing curricula.
Although the flipped classroom offers its own challenges, multiple lines of evidence now support that the benefits outweigh the risks (1, 9, 10, 14). Increased higher-order thinking activities and assessments along with more time to cover the most difficult concepts make it a rewarding endeavor for both students and instructors. For high-content science courses, hopefully more opportunities will open for faculty to try this pedagogy in their own hands to see for themselves how this format can be transformative for all involved.
This work was supported by the Echo360 Newcomer’s Grant: “The Flipped Classroom at a Small, Liberal Arts College: In Support of the Culture?”
No conflicts of interest, financial or otherwise, are declared by the author.
L.R.D. conception and design of research; L.R.D. performed experiments; L.R.D. analyzed data; L.R.D. interpreted results of experiments; L.R.D. prepared figures; L.R.D. drafted manuscript; L.R.D. edited and revised manuscript; L.R.D. approved final version of manuscript.
I thank Mark Ramsden and Royce Robertson for logistical and technical requirements for the flipped classroom and Natasha Farrell for help with acquiring GPA data. Brooke Binion, Ashley Brown, Allen Bidwell, and Terri Anne Willman aided in “Blooming” the assessment questions.
Below are sample questions rated as knowledge, comprehension, and application levels.
Which enzyme is important for breakdown of proteins in the stomach?
Ninety-nine percent of organic solute reabsorption occurs in the
A. Proximal convoluted tubule.
B. Distal convoluted tubule.
C. Loop of Henle.
D. Collecting duct.
E. Renal papilla.
In the loop of Henle, the following are true:
A. Water is secreted into the descending limb to contribute to the tubular fluid.
B. The filtrate in the descending limb becomes less concentrated (or more dilute).
C. Sodium and chloride ions are reabsorbed at the ascending limb.
D. The ascending limb is very permeable to water.
E. Blood flows through the tube that makes up the Loop of Henle.
This concept was discussed previously in class. If a person has extremely low proteins in the diet, what would happen at Bowman’s capsule?
A. A higher quantity of filtrate is produced.
B. The person would be overhydrated (too much water in the blood).
C. An elevated capsular colloid osmotic pressure would occur, acting to “pull” more fluid into the capsule.
D. All of the above choices.
E. A and B only.
The next three questions include distractors and were discussed as concepts in class. Distractors elevated the question to comprehension level even if the question focused on facts.
The following are true of cholecystokinin:
A. Stimulated by the presence of lipids in the duodenum.
B. Increases release of pancreatic enzymes.
C. Increases bile release.
D. All of the above choices.
E. B and C only.
If the chyme entering your small intestine is basic (assume that this is a typically healthy person taking no prescriptions or over-the-counter drugs):
A. Your stomach did not do enough mixing.
B. It will signal to reduce stomach contractions.
C. More mucous will be produced at the duodenum.
D. All of the above choices.
E. B and C only.
If water reabsorption did not occur properly in the digestive track:
A. You would need to reduce your overall water consumption to avoid water intoxication.
B. Diarrhea would occur.
C. Constipation would occur.
D. All of the above choices.
E. A and C only.
If a person were doping with whole red blood cells, what would happen to their filtrate?
A. More filtrate would be produced.
B. Less filtrate would be produced.
C. Filtrate production would not be changed.
If sodium channels were not operating in the ascending limb, what would happen to filtrate over time?
A. A higher quantity of filtrate would end up in the collecting ducts and the renal pelvis.
B. There would be dehydration due to excessive fluid loss.
C. There would be an elevated capsular colloid osmotic pressure, acting to “pull” more fluid into the capsule.
D. A and C only.
E. A and B only.
If a person has heart failure resulting in lung congestion, which would be better to prescribe to reduce the workload of the heart?
These questions are a compilation. They do not represent one person’s work.
- Copyright © 2016 the American Physiological Society