The flipped classroom is a relatively new approach to undergraduate teaching in science. This approach repurposes class time to focus on application and discussion; the acquisition of basic concepts and principles is done on the students' own time before class. While current flipped classroom research has focused on student preferences and comparative learning outcomes, there remains a lack of understanding regarding its impact on students' approaches to learning. Focusing on a new flipped classroom-based course for basic medical sciences students, the purpose of the present study was to evaluate students' adjustments to the flipped classroom, their time on task compared with traditional lectures, and their deep and active learning strategies. Students in this course worked through interactive online learning modules before in-class sessions. Class time focused on knowledge application of online learning module content through active learning methods. Students completed surveys and optional prequiz questions throughout the term to provide data regarding their learning approaches. Our results showed that the majority of students completed their prework in one sitting just before class. Students reported performing less multitasking behavior in the flipped classroom compared with lecture-based courses. Students valued opportunities for peer-peer and peer-instructor interactions and also valued having multiple modes of assessment. Overall, this work suggests that there is the potential for greater educational gains from the flipped classroom than the modest improvements in grades previously demonstrated in the literature; in this implementation of the flipped classroom, students reported that they developed independent learning strategies, spent more time on task, and engaged in deep and active learning.
- flipped classroom
- deep learning
- student engagement
- time on task
undergraduate teachers continuously struggle to design courses that can support the development of deep and active learning strategies in their students. According to educational psychologist Noel Entwistle, undergraduate students need to be supported to move away from surface learning approaches characterized by the mere memorization of content for the purpose of scoring well on examinations toward deeper learning strategies, strategies characterized by a drive to understand underlying principles and concepts by grappling meaningfully with content (8, 15). While many physiology courses inherently include activities that support active learning such as inquiry-based laboratories (17), active learning-based review sessions (9), and hands-on manipulation of models (25), many basic science instructors continue to struggle with providing a coherent and consistent curriculum that encourages and supports deep and active learning. A relatively new pedagogical approach that is purported to support deep and active learning in basic medical science disciplines is the flipped classroom (26). In this instructional model, students are required to participate in preclass preparatory work, which may include the use of prerecorded lectures, readings, or online modules. Class time is then repurposed to focus on problem solving, application, synthesis, and collaborative learning.
Proponents of the flipped classroom suggest that it can enhance teacher-student interactions, facilitate deep learning through active learning activities in the classroom, help students understand their own learning styles and preferences, and foster student engagement (26). While aspects of the flipped classroom seem promising in this regard, little research has, in fact, been done to demonstrate this. For the most part, ironically, research has focused on its impact on examination scores, a somewhat surface learning-based gain. Other research has explored student preferences for the flipped classroom and found that the method is generally well received by students but commonly, when introduced, faces some initial resistance (23).
Focusing on education outcomes, as measured by examination scores, may not always be sufficient for studying process-based changes such as a course design's ability to foster deep and active learning, encourage time on task, and foster engagement. To date, the impact of the flipped classroom on these aspects of student learning behaviors has not been studied.
Another issue that is ripe for exploration is the impact of the flipped classroom on multitasking behaviors. The majority of undergraduate students in today's classrooms are members of the Millennial generation, a population that is often characterized as technologically proficient and accustomed to multitasking in academic settings (14, 26). However, research demonstrates that students are not truly multitasking and are instead quickly switching from one task to another, a practice that may be detrimental to learning (14). While proponents of the flipped classroom suggest that the active learning strategies used by this method should appeal to Millennials and serve their typical learning strategies (26), it is also possible that it provides them with even more opportunities for off-task multitasking. Using a mixed-methods design, the purpose of this research was to explore the impact of a flipped classroom design on undergraduate student approaches to learning. This report will highlight student-reported strategies for adjusting to the flipped classroom design, their perceived time on task in the flipped classroom environment, and their perceived use of deep and active learning strategies.
MATERIALS AND METHODS
Medical Sciences 4200 was offered as a new course in September 2013 and was an elective for students in their fourth year of a nonthesis-based honors specialization basic medical sciences bachelor's degree. This course covers principles of physiology, biochemistry, and immunology in the context of the role of inflammation in normal and pathological processes. The course had a flipped design from its inception (Fig. 1). Online modules (OLMs) were completed by students before class attendance and served as the information delivery modality. OLMs were created by the instructor (except for one module created by a guest lecturer) and covered all course topics. There were 11 OLMs in total. Modules were created using Adobe Captivate (Adobe Systems, San Jose, CA) and were essentially voice-over slides with opportunities for interactivity. Examples of interactive elements include formative quizzes, user-controlled “buttons,” and videos. Approximately 25% of slides in a given OLM had interactive elements. The majority of the OLM were ∼50 slides in length and had a playtime of 1 h. Students were able to pause, rewind, and review slides at their leisure. Students could complete prequizzes to assess their knowledge of the material either before or after OLM completion. The prequizzes consisted of multiple-choice, true/false, and short-answer questions. Before class attendance, students were required to complete an online quiz administered through the learning management system that tested their knowledge of the OLM. The regular quiz consisted of 10 multiple-choice or true/false questions. The regular quizzes contributed to 5% of the students' total marks. As an incentive for students to complete the formative components of the prequiz, students were informed that some of the regular prequiz questions could be derived from the prequiz questions (usually fewer than 3 questions/quiz). The regular quiz tested mostly lower domains of Bloom's taxonomy (such as remembering and understanding) as in-class sessions were used for application and synthesis of content. Due to limitations in our learning management system at the time of course delivery, we were unable to track module completion by the students. Therefore, the online quizzes also served to ensure that students accessed and completed OLM material. In-class sessions occurred once a week and were 120 min in duration. Each session had a maximum of 25 participants and focused on the application of OLM content; there was little to no new content delivered during the in-class sessions. The in-class activities varied week by week and included debate, scientific literature analysis, case studies, guest speakers, group discussion, and group projects. Students were assigned to groups for the group project on the first day of class. Briefly, students were asked to reflect on their skills (“I am best at writing,” “I am best at seeing the big picture,” “I am best at explaining my thoughts clearly,” and “I am best at designing”) and were organized into groups such that every member of the group had a different strength that they could bring to the team. Simply put, the students were randomly numbered to form their groups, and each group consisted of three to four members. The groups were permanent, although students had ample opportunity to work with other peers on formative in-class activities. The group project was twofold: the first part was an oral presentation addressing common medical myths, worth 5% of their mark, and the second part was a capstone project worth 10% of their mark wherein students had to create an online learning resource for their peers that discussed the epidemiology, biochemistry, and pathophysiology of an inflammatory disease. Students also completed a peer evaluation that contributed to 5% of their total mark. Each student evaluated every other group member, and the average score for each group member was tallied.
Students in this course were assessed through a variety of means (Fig. 1). The mean grade for students in the course was 85% in the fall term and 84% in the winter term. This is similar to this cohort's performance in a mandatory lecture-based course (Medical Sciences 4930) in which the average for both the fall and winter terms was 85%. Due to the selective and competitive nature of the program, students tend to be highly motivated and very high achievers.
This research was approved by the Office of Research Ethics at Western University, London, ON, Canada (HSREB no. 104196). Participants included all students enrolled in Medical Sciences 4200 during the fall or winter terms of 2013/2014 who provided written consent. All sections of the course were subjected to the same study.
On the first day of class, students were provided with a letter of information outlining the purposes of the study and were told that the study was for course evaluation as Medical Sciences 4200 was a newly implemented course. Throughout the term, students completed surveys and short-answer questions regarding their experiences in the course (see Study Design below). On the final day of class, students were debriefed and informed that the study was carried out for education scholarship. They were provided with a new letter of information and a consent form, and those that gave written consent were included in the study. Data analysis began after the submission of the final grades, and student responses were deidentified before analysis. All of the 54 students (24 men and 30 women) eligible to enroll in the study consented to participate.
This study used a mixed-methods and thematic approach to explore the learning strategies used by students in a flipped classroom course. Data collection was carried out through surveys and weekly reflective questions. Participation in the reflective activities was optional, and the amount of time required to complete all reflective components was quite minimal. Three surveys were administered throughout the year: one survey on the first day of class, the second survey before the midterm exam, and the final survey on the last day of class. All surveys were administered during in-class sessions, and each took ∼15 min to complete. Surveys consisted of multiple-choice, 5-point Likert scale, and short-answer questions that asked students about various aspects of the flipped classroom, such as time on task and strategies for OLM completion. The weekly reflective questions were administered through the prequizzes on the learning management system. The prequizzes were a formative assessment and were optional for students to complete. They were not graded. Briefly, they contained 10 questions regarding course material followed by 2 reflective questions that were either a multiple-choice or short-answer format. While the majority of students accessed the prequizzes to complete the formative questions (90–100% of students, depending on the week), a small percentage (15–20%) of students opted to not complete the reflective questions, particularly the short-answer reflective questions. The weekly reflective questions were optional; therefore, some of the n values for figures vary slightly. As most students generally wrote a few point-form responses for the short-answer questions, it is unlikely that any given prequiz reflection took >5 min to complete. The prequizzes were formative assessments that allowed students to evaluate their understanding of the material. They consisted of 10 questions that related to OLM material content and 2 questions that asked students about their experience in the course. The two questions that were asked regarding the student experience in the course were based on the instructor's interactions with the students and emerging themes observed during in-class sessions. The responses were not anonymous at the time that they were submitted but were deidentified during data analysis. The ongoing collection of student responses throughout the term was purposefully tied to the content questions so as to get a real-time picture of the student experience in the course.
After completion of the course and consent, the surveys and short-answer questions were analyzed. Student responses for all multiple-choice and short-answer questions were deidentified. Multiple-choice responses were evaluated for frequency. Where appropriate, a two-tailed Fisher's exact test was performed using GraphPad Prism.
Short-answer questions were analyzed using an inductive coding process consistent with content analysis (27). Data were coded focusing on the explicit or semantic meaning of the data (4). The aim was to offer a description of self-reported student learning behaviors and perceptions of Medical Sciences 4200. Coding is a systematic approach to analyzing textual data in qualitative studies. Codes are tags that are used to ascribe meaning to textual data such as words, paragraphs, or sentences (2). To begin coding the short-answer questions, an open coding approach was taken whereby the data were highlighted, examined, and compared with evaluate emerging themes (19). After the initial analysis, four different types of codes were created. Attribute codes were used to denote simple dichotomous answers (example: yes or no codes). Descriptive codes summarized the basic topic of a passage, such as a short phrase (example code: understanding). Process codes were developed to describe actions taken by the participants (example: writing notes). Similar to the approach by Billings-Gagliardi and Mazor (3), if one student's response included multiple themes, each was counted; however, if a student used one theme multiple times in a response, it was only coded once.
Coding analysis was performed using the program Atlas.ti (ATLAS.ti, Berlin, Germany). S. McLean developed the codebook and highlighted and evaluated the codes. S. M. Attardi acted as a second coder and applied the codes from the codebook to the short-answer questions independently. Fleiss' κ analysis was performed using the Coding Analysis Toolkit to evaluate interrater reliability between S. McLean and S. M. Attardi (10). The Fleiss' κ score for all codes except for the attribute codes was 0.79 (if including the attribute scores, the Fleiss' κ score was 0.80). According to Landis and Koch (16), a Fleiss' κ value of 0.61–0.8 indicates strong agreement between the raters, particularly because S. McLean and S. M. Attardi were working independently; therefore, the codebook was reliable (16).
In our analysis of the full data set, three main themes emerged: adjustment of learning behaviors, multitasking behaviors, and strategies for deep and active learning. The data in the results are both quantitative (from multiple-choice and Likert scale questions) as well as qualitative (from short-answer responses and coding).
Adjustment of Learning Behaviors
We sought to examine students' approaches to learning in the flipped classroom and assess how their learning strategies changed compared with their approaches in lecture-based courses. Students responded to weekly prequiz questions to share insights about their access of the OLM and whether their prework completion strategy changed after the in-class experience. For the most part, students realized that they had to take a different approach to learning in the flipped environment. Many students realized this early on and wrote about their learning experiences with insight and self-awareness.
Access of OLMs.
In a prequiz question, students were asked whether they completed the OLM at home, at the school/library, or in a public place. The vast majority of students in Medical Sciences 4200 reported that they completed the OLM either at home (78%) or at the library (20%). Students were also asked how often and when they complete the OLM before class. Most students (70%) appeared to use a “just in time” learning approach and completed the module the night before class (Fig. 2). Furthermore, the majority of students (80%) only viewed it once, and, encouragingly, only one student reported rarely completing the modules before class.
Interestingly, students' approaches to flipped classroom prework appear to mirror what they were accustomed to doing in their lecture-based courses: they completed the “lecture” all at once from start to end in one sitting. As the content was user controlled, students could opt to “chunk” the OLM into smaller sections if desired but were not explicitly told to do so. When asked whether they planned on completing the OLM all at once or bit by bit, 74% of students indicated that they would complete the OLM all at once. The data suggest that students' use of the OLM mimics what they were used to doing with their lecture-based courses: watching the whole lecture from start to end. Some students gave a rationale for their approach, and the themes of time management (13% of respondents) and focus/attention (11% of respondents) emerged from coding analysis of the data. For example, one student stated that they planned on completing the OLM all at once because “All at once allows me to stay focused and absorb the big picture better.” Other students suggested that their strategy would vary week by week depending on the demands of their other coursework.
Change of strategy.
After their first in-class session, students were asked in a short-answer prequiz question whether their experience in class would change their approach to completing the OLM. Half of the respondents suggested that they would change their approach. Twenty-four percent of all respondents offered a rationale related to time management in planning their approach to future OLMs (Table 1). Some of the proposed strategies included leaving more time to complete the OLM or starting the OLM earlier in the week. Students also brought up deep learning themes such as evaluating their own understanding of the topic. As one student explained:
Yes. After the first class session, I realized that I had paid attention to the online learning module but hadn't really absorbed the information. In many other courses I've taken, things are reiterated over and over again, so learning this way is sufficient. However, realizing this was a significant wake-up call for me, as I feel like it shows me that I need to be able to discuss and apply information to prove that I've really learned it. I definitely plan on concentrating on the material in Module 2 and learning it thoroughly so I am able to discuss it in class.
The majority of students agreed with the statement “Medical Sciences 4200 helped to improve my independent learning skills,” with the mode value being 2 (agree).
We sought to examine students' multitasking behavior in the flipped classroom. To address this question, students were asked to report their perceived time on task in the flipped environment and to report on their multitasking behavior in the flipped classroom and their favorite lecture-based course.
Students' time on task in the flipped environment.
To begin to examine students' multitasking behaviors, they were first asked to report their perceived amount of time on task and were given options for explaining the reason for their time on task. In a multiple-choice prequiz question, students were asked to compare their time on task in Medical Sciences 4200 to their favorite lecture-based course. Over half of all students stated that they spent more time on task because they spent more time working on material for this course. Only 10% of students stated that they spent less time on task. Overall, students indicated that they committed the same or more time to learning in Medical Sciences 4200.
Students' multitasking behavior during content acquisition.
To evaluate students' multitasking behaviors in the flipped classroom environment compared with a lecture-based environment, students were asked to reflect about their multitasking activities during class for their favorite lecture-based course and during study sessions for that same course. They were also asked to reflect on their multitasking behavior in Medical Sciences 4200 during in-class sessions and during completion of OLM. Data analysis compared student responses about multitasking during lecture time and during OLM viewing. This is an important point because in both pedagogical approaches this is the “content acquisition” phase of learning. Students were asked to indicate if they engaged in any of the following behaviors during lecture/OLM viewing: checking social media, checking e-mail, surfing the web, and texting/talking to friends. As shown in Fig. 3, during lecture attendance, students reported more multitasking activities than during completion of OLMs. A Fisher's exact test was performed to compare the frequency of each multitasking activity between Medical Sciences 4200 and the students' favorite lecture-based course. Students reported significantly fewer multitasking activities in Medical Sciences 4200 compared with their favorite lecture-based course (P ≤ 0.01).
Students' multitasking behavior during content application.
Supporters of the flipped classroom approach state that in-class active learning activities enable students to grapple with complex concepts and application that are frequently reserved for study sessions outside of class in many lecture-based courses. In light of this framework, students were also asked to report their multitasking activities during in-class sessions of Medical Sciences 4200 and study sessions of their lecture-based course. Students engaged in significantly fewer multitasking activities (P ≤0.001) during in-class sessions of Medical Sciences 4200 compared with study sessions for their favorite lecture-based course (Fig. 4). This suggests that students spend less time multitasking during both content acquisition and application in the flipped classroom compared with lecture-based courses.
Deep and Active Learning
We sought to examine students' engagement with deep and active learning in the flipped classroom. To address this question, students were asked to outline their strategy for completing the OLM, to provide feedback on assessments in this course, and were offered the opportunity to further provide feedback about the course and learning experience.
Students emphasized time management and understanding for completing prework.
In a weekly prequiz short-answer question, students were asked to outline their strategy for completing the weekly OLM. Students wrote self-reflectively about their approach and highlighted self-directed learning strategies. Several important themes were identified from their responses by coding analysis: note taking, reviewing, and deep learning approaches (Table 2). Fifty-seven percent of respondents outlined their approach for taking notes for the OLMs, and nearly half (49%) of all respondents stated that they wrote their notes in pen onto the provided study guide notes. In contrast, <10% of students who reported their strategy indicated that they took notes by typing. This is an interesting point, as students had the option to either print out their notes and add to them or they could type directly on the notes, as they were provided in a digital format. Since the OLMs were self-paced, handwriting notes may allow students to reframe the information and process it at their own pace. Students also commented on the ability to self-pace with the OLM; using OLMs or online videos allows the user to review the material at their leisure and also go at their own pace, affordances not offered in traditional lecture courses. Sixteen percent of respondents mentioned pausing the OLM, and 35% mentioned reviewing the material by either rereading or relistening to the content before class (Table 2). Encouragingly, many respondents identified metacognitive practices as shaping their learning approaches. Almost one-third of respondents mentioned ensuring that they set aside enough time to complete the OLM, and 22% mentioned either assessing their understanding or writing down questions so that they could ask in class and clarify their understanding. Several students observed that the prework of the OLM paid off in greater learning in class:
It was hard to keep up with the OLM during my busiest weeks but when I did properly prepare I found class really useful (I almost didn't have to study outside of class if the discussion and interactive learning was good).
Overall, students' strategies to completing class prework were more mature and purposeful than expected.
Modes of assessment.
To further evaluate the implementation of a flipped classroom course, students were asked to provide their opinions on aspects of the course design. Students completed Likert scale questions on the final day of class survey asking them about the OLM design, assessments in the class, and their independent learning skills. Overall, students reported most positively about the embedding of practice questions into the learning object design (OLM; Fig. 5). These questions allowed students to assess their knowledge before the in-class application component of this course. The questions were embedded throughout each OLM and were one example of interactivity in the OLM (Fig. 1).
In Medical Sciences 4200, students' learning was assessed using multiple methods (Fig. 1) such as a collaborative group capstone project, oral presentations, and written assignments. Since students were accustomed to multiple-choice exams as the typical mode of assessment in their other courses, it was important to evaluate whether students preferred the multiple assessment modalities. Figure 5 shows that students scored the multiple assessments method positively, with the majority of the students agreeing with the statement “I liked that there were multiple ways for me to be assessed…” The utilization of multiple forms of assessment is commensurate with the flipped classroom design objective of providing many opportunities to apply and synthesize knowledge.
Students value interactivity, creativity, and application in the flipped classroom.
Providing a forum for students to provide feedback on their educational experiences is important when introducing a new pedagogical approach. To ensure that students could fully share their classroom experience, students were asked on the final day of class survey if there was anything else that they wanted to share about their experiences. This question was not mandatory, and 39 of a possible 55 students offered insights into their experiences. Students provided both positive and negative feedback but gave greater than three times as many positive comments as negative comments about their experiences (Table 3). As previously described, the short-answer data were analyzed by coding, and three general themes of responses emerged: in-class experience, OLMs, and assessments (Table 3). Overall, students emphasized their enjoyment of the course as a whole and, in particular, the in-class experience. Several students highlighted the interactive and collaborative nature of the course (both with classmates and with the instructor) as being particularly important in their enjoyment of the class:
I really enjoyed this class and the intimate setting. You made it fun and I felt comfortable to approach you if I need help or had questions.
Many students (23%) responded positively to the opportunities to apply their knowledge in class.
I really enjoyed this class. Typically, I don't get that much out of just listening to a professor lecture. I liked how I could do my own independent learning at home and then come to class and apply it. Using the OLM allowed for me to make more complete notes because I could pause it when needed. Thanks.
While many students emphasized the positive aspects of this course, some also had negative feedback. This negative feedback mostly focused on issues with workload: in terms of the volume of content delivered in the OLM (15%) and the number of assignments (13%).
Overall, when given the opportunity, students readily reflected and shared their experiences, and most respondents spoke positively about their overall experience of the course.
This work sought to illuminate students' learning approaches to the flipped environment by focusing on their adaptation to the flipped environment, their time on task, and their use of deep and active learning strategies. By understanding students' approaches to learning, instructors can better aid students' transitions to the flipped learning environment. Ultimately, we argue that the investment of time and effort in the design of a flipped classroom course should result in deeper and more self-directed learning on the part of students. To this end, this study contributes evidence about how students adapt their learning behaviors to the flipped classroom environment.
One of the major challenges to the flipped classroom approach is the completion of prework. Instructors need to ensure that students are completing prework so that they can apply content in class; students need to ensure they manage their time to complete prework and have a sufficient understanding so as to contribute in class. The majority of students in this study completed the prework using a just in time learning approach. This result is consistent with a previous study (21) that showed that students access online material just before learning sessions and major examinations. Whether this just in time learning approach is purposeful and is used to help students retain knowledge for class for the following day or is by accident and is due to poor time management strategies should be further investigated. Flexibility of content acquisition may be a double-edged sword in the flipped classroom: for students who are able to manage their time, this could be one potential benefit of this pedagogical approach; for students who have poor time management skills, this may further enable procrastination. Interestingly, students' approaches to flipped classroom prework appear to mirror what they were accustomed to doing in their lecture-based courses: they completed the lecture all at once from start to end in one sitting. As the content was user controlled, students could opt to chunk the OLM into smaller sections if desired but were not explicitly told to do so. Mayer (18) has suggested that chunking information is a useful strategy for students when the material is complex. Chunking can be helpful pedagogically if the material is parsed into meaningful units and students combine chunking with distributed practice (6). Furthermore, this chunking is most useful when combined with distributed practice and can be used effectively in courses that rely on e-learning (6). While distributed practice was successfully implemented in Medical Sciences 4200 by engaging students in a capstone project that relied on their applying material from earlier coursework to a final project, most students did not take advantage of it at the OLM level. Future exploration of the pros and cons of encouraging students to take advantage of this function of OLMs are, however, likely warranted.
In terms of the location of module completion, the vast majority of students reported that they completed the OLM at home or at the library. Since the students were completing the prework at home, this suggests that they were not using their mobile devices for content acquisition. Chen and Denoyelles' work (5) showed that nearly all undergraduates own mobile devices (91%), but only 58% of those who own a mobile device use it for academic purposes. The benefits (flexibility and time management) of making flipped classroom content available on mobile devices should be carefully weighed against the potential technical issues as well as the fact that students tend to view the content in their own homes.
This study offers credible evidence that a well-designed flipped classroom approach may support better student learning by supporting them to spend concentrated time on learning tasks and by reducing multitasking behaviors. Erroneously, students today are reported to be habitual multitaskers who are able to process information from a variety of different media streams simultaneously. However, evidence strongly demonstrates that multitasking has a negative effect on learning (1). Since most iterations of the flipped classroom rely heavily on technology and student time management, one could hypothesize that the flipped classroom may encourage and enable multitasking behavior. In our study, this was not the case; students reported that they engaged in fewer multitasking activities during both content acquisition and content application phases in the flipped classroom compared with their lecture-based courses. These findings are encouraging in light of the literature that informs our understanding of how multitasking impedes the learning process. According to Fried (11), 65% of students bring their laptops to class and spend >20% of their time multitasking. Moreover, Fried (11) also found that the greater the amount of time students spent multitasking on their laptops, the lower their performance. Work by Ophir and colleagues (24) has also illustrated that heavy media multitaskers have a different approach to information processing and have a difficult time filtering irrelevant stimuli compared with individuals who multitask less. The flipped classroom design, as suggested by this study, may be an effective way to adjust students' multitasking activity: since students are required to be prepared and participate in class, they have to purposefully engage with the material and be on task both during content acquisition and application.
The most significant contribution presented here is students' reported deep and active learning in the flipped environment. Deep learning involves an approach whereby the learner's intention is to extract meaning, look for patterns, and monitor one's own understanding of the material (7). Students who use a surface approach to learning simply attempt to complete the task and see the information as unrelated (7). In this study, we found that students reported using strategies aimed at deep learning in both their prework completion and in their classroom participation. Nearly 25% of respondents stated that they wanted to ensure that they understood the material, and 25% of students mentioned assessing their understanding by completing the prequiz either before or after the module (Table 2). Some respondents also reported enjoying the application component of the course when describing the assessments (Table 3). Students also used a deep learning approach when completing their course notes. For example, many students reported handwriting their course notes during OLM completion. Mueller and Oppenheimer (20) demonstrated that handwriting notes allowed students to reframe information, resulting in superior conceptual learning compared with students who used laptops to take notes. They argued that students who used laptops tended to transcribe material and process it more shallowly (20). Since the OLMs were self-paced, handwriting notes may allow students to process it at their own pace, leading to further deep learning and understanding. The overall goal of this course was to promote deep and active learning in students; their completion of their notes by handwriting them was an unexpected gain toward this goal.
This study illustrated that the most positive aspects of the flipped classroom, as reported by students, was the purposeful interaction with peers and the application of content during class time. Indeed, Jensen and colleagues (13) have argued that the most crucial aspects of the flipped classroom are the active learning and constructivist approach that occur in the classroom. Their work compared a flipped versus a nonflipped version of a course by simply repurposing the instructor's role (13). In the flipped version, the instructor used active learning strategies to aid in content application; in the nonflipped version, the instructor used active learning strategies to aid in content acquisition (13). They showed that it mattered little whether the instructor was involved in the content acquisition or the content application phase: what mattered was the active learning and peer-based learning that occurred (13). Based on the work of Jensen and colleagues and the results of this study, students will benefit most from the flipped classroom design when instructors purposefully create active learning in the classroom where students can engage meaningfully with the instructor, with the content, and with one another.
This study also illustrated the benefits of using a variety of assessment methods and the importance of formative assessments. Previous literature evaluating flipped classroom learning gains simply studied students' performance on examinations (23); this approach undercuts the principles of flipped classroom learning. Traditional multiple-choice examinations capture some aspects of student learning. As such, multiple styles of evaluation were used to bring in variety to the assessment. While certain types of multiple-choice question design can promote and evaluate deep learning, such as assertion-reason types of multiple-choice questions (28), this study used a variety of assessment formats to fully capture student learning in a flipped environment. Using multiple assessment styles could support the development of a greater range of skills, such as communication and collaborative problem-solving skills, and may create more opportunities for constructive feedback than a strictly multiple-choice exam-based approach (12). In this study, the majority of students responded positively to the creativity and diversity of assessments. Students also appreciated the opportunity to self-assess their learning through the use of ungraded quizzes in the OLMs. Furthermore, it is also important to keep in mind the workload of the students: the primary critique of this course design had to do with the number of assignments and perceived workload. Therefore, while having many small summative assessments can be helpful for student learning, this should be balanced with a realistic expectation of student workload.
While this work revealed many important aspects of the flipped classroom on student learning, there were some limitations. One limitation of this study is that students' actual total time engaged with the OLM prework was not evaluated. It may be important to evaluate whether students who spend more time on OLM perform better in the flipped environment or whether students who struggle with course work spend more time on OLMs. Furthermore, it would be very useful to evaluate whether students who report using deep learning strategies actually perform better in the flipped environment. Since a number of students self-reported writing notes by hand, and handwriting notes has been associated with deeper learning (20), it would be interesting to see if students who use these strategies perform better in the flipped classroom. These questions are an important future area of study.
One possible limitation to this study is the design of data collection. The data for this study were collected while students were actually working through specific content related to the course rather than as a simple anonymous survey at the end of the course. As such, student responses to prequiz reflective questions were not collected anonymously as they were tied to our learning management system; the responses were deidentified during analysis. While one may argue responses may have differed slightly if the responses were completely anonymous, it is also possible that solely relying on an anonymous survey at the end of term may have introduced bias based on student performance in the course.
Another limitation is related to the variability in flipped classroom designs. Specifically, it should be noted that the implementation and in-class activities of the flipped classroom vary greatly (23). Therefore, student gains from this course may not be directly applicable to other flipped classroom designs that do not emphasize active learning in the classroom. Based on this work, instructors should think carefully about in-class activities that best illustrate concepts presented during course prework and should allow students many opportunities to assess their own learning. While this study offers further insight into ways that a flipped classroom design can be used to promote deep and active student learning strategies, it should not be interpreted as evidence of superiority over other methods; this was not the intent of the study, nor would our methods support such conclusions.
Overall, this study showed several novel outcomes for implementation of the flipped classroom. Students reported that they gained independent learning skills and used time management and deep learning strategies when completing OLM prework. Students also reported spending less time multitasking when engaged with the flipped classroom format compared with their favorite lecture-based course. As increasing access to technology may allow even more opportunity for students to engage in off-task activities, this is an important potential benefit to flipped classroom implementation. Students reported spending more time on this flipped course compared with their other courses, and the majority of students completed OLM prework 24 h before class time. Since students used a just in time learning approach for completing the OLM prework, future iterations of this course will use a just in time teaching approach to briefly address confusing material from the OLM during class time (22). The marriage of the flipped classroom with the just in time teaching approach may be a useful union to help students clarify confusing material and develop deep learning strategies. By facilitating student reflection about their misconceptions before class, class time can be used more purposefully and foster students' ability to gauge their own learning. Finally, instructors interested in implementing a flipped classroom should consider implementing formative assessments in class prework; students value the opportunity to assess their knowledge in a safe environment. Instructors should also consider multiple methods of student assessment, as this may better capture the performance of a student in a flipped environment, and students generally prefer having multiple assessment formats. Since students are interacting more meaningfully with content in the flipped classroom, this may also suggest that students would experience greater long-term learning gains in a flipped environment than in lecture-based courses. Future work to evaluate this hypothesis would add significantly to the flipped classroom literature.
No conflicts of interest, financial or otherwise, are declared by the author(s).
Author contributions: S.M. and M.G. conception and design of research; S.M. performed experiments; S.M., S.M.A., and L.F. analyzed data; S.M. and S.M.A. interpreted results of experiments; S.M. prepared figures; S.M. drafted manuscript; S.M., S.M.A., L.F., and M.G. edited and revised manuscript; S.M., S.M.A., L.F., and M.G. approved final version of manuscript.
S. McLean thanks Jay Loftus (Strategic Technology Commons, Western University) and Tim Wilson (Department of Anatomy and Cell Biology, Western University) for feedback and suggestions as well as Anita Woods (Department of Physiology and Pharmacology, Western University) for help with statistical analysis.
- Copyright © 2016 The American Physiological Society