| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
RESEARCH-ARTICLE
Department of Physiology, The University of Melbourne, Melbourne, Victoria, Australia
Address for reprint requests and other correspondence: R. E. Kemm, Dept. of Physiology, The Univ. of Melbourne, Melbourne, Victoria 3010, Australia (e-mail: r.kemm{at}unimelb.edu.au).
Abstract
We investigated whether an active learning approach, facilitated by a personal response system, would lead to improved student engagement and learning outcomes in large-group physiology lectures for undergraduate science students. We focused on encouraging students' active learning in lectures, whereas previous studies have made more use of audience response technology during lectures for formative or summative assessment. Students voluntarily answered questions posed during lectures with their personal response system (clickers), with individual answers automatically collated for immediate histogram display. This feedback then dictated the focus of followup discussions in the lecture. Student and instructor attitudes were surveyed through voluntary interviews with student responses correlated with their degree of clicker participation and individual exam results. Active lectures were found to increase both student motivation and engagement. Students who participated in answering questions achieved better results than students who chose not to. Students with the lowest scores in a prerequisite course (previous semester physiology exam marks of < 60%) showed significantly better outcomes from the use of clickers than both middle-achieving (60–75%) and high-achieving (>75%) entry students. Significant improvement was evident in both mid- and end-semester exam results compared with student cohorts from preceding years, although this could also be influenced by many other factors. Increased student engagement and the immediate feedback obtained during lectures were advantages commonly noted by lecturing staff.
Key words: audience response system; higher education; large-group teaching
NUMEROUS PUBLICATIONS have demonstrated the effectiveness of active learning in small group sessions, particularly in the enhancement of student learning and performance (10–12, 17). However, promoting active learning in large classes is more challenging, so many instructors end up delivering information with minimal student interactions within a traditional lecture format. As such, students may focus on information and favor memory retention rather than thinking, understanding, and solving problems.
Questions posed to students in lectures may stimulate active learning, but only low response rates are possible within large audiences. Engaging small groups within a lecture may be a more effective means of promoting active learning. However, without significant time conveying information back to the lecturer it is difficult to accurately gauge student engagement. Students can also display color-coded cards in response to a lecturer's question (9). However, when they can view each other's responses, active learning and critical thinking may be inhibited as they may have less confidence in their own answer or can simply copy the most popular response.
In contrast, electronic response systems may provide accurate and rapid feedback on students' understanding or misconceptions during lectures. Electronic response systems allow students to give an immediate response to a question posed by the instructor. Question styles can include multiple-choice quizzes and text or numeric responses, and individual responses remain anonymous to the audience. Students' responses can be promptly graphically displayed in a PowerPoint presentation, or responses can be withheld until further discussions ensue with revised answers sought for comparison with previous responses. Collective responses allow students to relate their performance to the rest of the class.
Electronic audience response systems have been employed in many disciplines (2, 5–8, 13, 15). A reports (5) of a 2-yr institution-wide adoption of electronic response systems in lectures indicated that increased student interactivity is the main reason for successful implementation. Electronic response systems have been shown to dramatically improve lecture attendance (2, 6), and improve quiz performance both initially (6, 15) and up to 1 mo after lectures (15). Improved learning outcomes may be related to factors such as increased engagement and thinking within an anonymous environment. Furthermore, immediate feedback can serve as a motivating factor in focusing student attention (8) and allowing opportunities for review and reflection.
The main aim of this study was to introduce active learning in large-group lectures facilitated by an electronic audience response system during a single-semester physiology course of an undergraduate science course and to evaluate both student engagement with the course matter and learning outcomes. Our novel analytical approach was to investigate examination results for different cohorts of students based on their performance in the prerequisite physiology course. This enabled us to compare similar cohorts of clicker participators and nonclicker participators. This had the important advantage that it could be approved ethically, as no student was potentially disadvantaged. We also planned to trial some higher-order question sequences rather than the simple quizzes used more commonly in previous studies.
METHODS
Approval was obtained for this project by the Human ethics approval was obtained for this project by the Human Research Ethics Committee of the Melbourne Research and Innovation Office of The University of Melbourne (Project No. 0604877).
The Personal Response System
Various electronic response systems were evaluated to find the one most suitable for our needs, and the personal response system (PRS) GCTO CALComp (Fig. 1) was selected. It permitted a diversity of answer styles, was cost effective with a bulk purchase, and allowed tracking of individual student responses as each unit had a unique serial number.
|
This study was conducted with a cohort of 175 undergraduate science students undertaking the second-year course 536-211 Physiology: Control of Body Function in semester 2 of 2006. Student background data was collected using PRS units in the first lecture of the semester, which was attended by 132 students with 118–128 students responding to the various survey questions. The demographic information is shown in Table 1. The "typical" student in this course was <20 yr old, born in Australia, with English as their primary spoken language, attended secondary school in the Melbourne Metropolitan area, and was in paid or unpaid employment from 6 to 20 h/wk. Most students spent 1–10 h/wk studying or completing homework outside normal class time and read textbooks relevant to their studies every 2–3 days or weekly. All students had successfully completed the semester 1 prerequisite physiology course 536-201.
|
In introducing the PRS to students in this course, no mention was made of the potential for improvements in their exam performance that may come with its use. Students were only told that, based on our previous experience with students using colored cards in lectures, the introduction of the PRS system could make lectures more active and interesting.
Instructors were asked to insert PRS questions throughout their traditional lecture format (thus, in an interrupted mode). They were encouraged to pose questions before the topic to establish students' prior knowledge and draw students' attention to their own limitations. If necessary, adjustments could also be made in the remainder of the lecture. It was also suggested that instructors pose PRS questions after they covered a topic to check students' understanding of the material. Instructors were advised to allow students to discuss their answers with their nearby peers; however, students always responded individually. Answers to questions were provided immediately after each question was asked during the lecture (as a histogram of selected responses) and made available electronically to students after the lecture. Lecturers were encouraged to discuss misconceptions indicated by the incorrect answers selected by students.
The lack of expertise with active learning and the PRS system meant that lecturers set mostly simple multiple-choice questions (MCQs) to investigate the progress of student knowledge and understanding, such as the question shown in Fig. 2. This example also shows the class view of a question with a histogram of student responses together with the control bar for the PRS, which the lecturer uses for timing the questions and adjusting the display of student feedback. A few questions were also included to investigate the use of authentic scenario-based situations by exploring students' prior opinions before further discussions in the lecture, such as the dilemas shown in Fig. 3. Students engaged in peer-to-peer discussion before answering both these examples. We also trialed a few complex series of questions that dealt with difficult concepts, since this was seen as potentially the most advanced use of the PRS to engage students in thoughtful consideration of difficult issues. Understanding of the concept of lengthening (eccentric or pliometric) muscle contractions is not intuitive for science, medical, or physiotherapy students but is an essential component in the understanding of the control of whole body movement, sports and exercise performance, and even muscle damage processes. Figure 4 shows selected slides from a series that is used to illustrate, test, and consolidate this concept while the PRS questions are used throughout the sequence as described.
|
|
|
Contact time for this course was three 50-min lectures/wk for 12 wk and ten 2-h computer-assisted learning (CAL) sessions. Assessment consisted of CAL activities based on pre-CAL questions, task sheets, attendance, participation, and workbook (10%); one midsemester exam composed of 30 MCQs (15%); and an end-semester exam with 30 MCQs, 5 short-answer questions (SAQs), and 1 long-answer question (LAQ) (75%). PRS questions in lectures did not contribute toward assessment for this course. Audio recordings from each lecture (ilecture) were made available to students via the student learning management system Blackboard.
Administeration of PRS Units and Monitorization of Student Attendance and Participation
Students were assigned PRS units, which were leased to them for a small nonrefundable fee that was used for ongoing maintenance costs and replacement of batteries. Students signed a contract to return the unit at the end of semester, which also required them not to loan their PRS to another student or use any other student's PRS unit during lectures. Student attendance at lectures was monitored through entries on an attendance sheet or by recorded participation with their assigned PRS unit. Students were informed that records of attendance and participation would only be used for research purposes.
Student Evaluation Questionnaire
At the end of the semester, an evaluation questionnaire was distributed to all students undertaking the course to evaluate positive and negative impacts of PRS. The questionnaire consisted of rating-type questions with a five-point Likert scale (where 5 was "strongly agree," 3 was "neither agree nor disagree," and 1 was "strongly disagree") and also contained open-ended questions for student comments. The questionnaire was voluntary and could be completed anonymously, although the importance of the survey information for the improvement of student learning experiences was emphasized. Voluntary entry of student numbers allowed responses to be matched to attendance, participation, and examination results.
Survey participation rate was high (84%), and only 13 of 147 respondents submitted the questionnaire anonymously. Student numbers were replaced with a research identification number by an independent research assistant, and the questionnaire data were then transcribed. Frequencies of each Likert rating (as a percentage) and average ratings ± SE were calculated. Student written comments for open-ended question were grouped thematically, and the frequency of similar themed comments was determined. Only thematic responses of >7% (
10 students) are reported.
Student Interviews
Many students across a wide range of achievement at the 536-211 midsemester exam (75–100%, 65–75%, 50–65%, and <50%) were randomly selected for interviews with a total of nine students volunteering (75–100%, n = 4; 65–75%, n = 2; 50–65%, n = 2; and <50%, n = 1). Interview questions were scripted and consistent in the different interviews. Student interviews were conducted by an independent research assistant, audiorecorded, and then transcribed. Student interviews focused on past experiences with questions in lectures and voting in response to questions, the impact of questions and their voting responses, thoughts on assessment and PRS, perceived advantages or disadvantages of questions and PRS, and whether the use of PRS would be relevant to other courses they were undertaking.
Quality of Teaching Questionnaire
The standard University of Melbourne quality of teaching (QoT) questionnaire was distributed to students, and data from 2006 and 2005 classes were compared.
Analysis of Student Examination Performance
Examination results for the 2006 cohort were compared with those who undertook the same course in 2005 or 2004. Apart from the introduction of questions and PRS in 2006 and minor unavoidable changes in teaching staff, the course matter, delivery, and assessment criteria were identical in these years other than the fact that audiorecordings of lectures were made available in 2006. Past exams were made available to students, as they were in previous years.
The 2006 cohort was divided into two approximately equal groups based on the calculated level (total number of questions attempted) of use (participation), with 52% being the boundary participation score between the groups. These groups were further subdivided based on prior achievements in the prerequisite (semester 1) course 536-201 examination. These groups represented the basis for comparisons of PRS participation, prior (entry) achievement level, and final 536-211 examination overall performance.
Instructor Cohort and PRS Preparation and Support
Ten instructors, all with several years of experience in teaching physiology, including three authors of this study and a guest lecturer from overseas, delivered the 36 lectures in this course. Before the semester, a seminar was held by the authors to provide instruction on the use of the PRS for active learning in lectures, available question types, and approaches to questioning. Pedagogical discussion focused on potential benefits and possible drawbacks of using the PRS. Seminar attendance and use of PRS in lectures was voluntary.
Before commencing their lecture series, each instructor was also provided with one-on-one training on how to set up questions using PRS software, run the software, and display and review students' responses. One author attended the first lecture of each series to provide assistance with the setup, trouble shooting, and advice. This support system served as an important scaffold in building the ability and confidence of instructors in the use of the PRS.
Instructor Interviews
Six instructors were interviewed both before and after their lecture series in this course. The three authors and the overseas guest lecturer were excluded. Initial interviews established prior instructor experience with active learning and PRS, whether the use of PRS was planned, and, if so, the exact plans for its use. Subsequent interviews determined how the PRS was actually used in lectures and instructors' perceptions of the impact of the PRS on their students. Instructor interviews were conducted by an independent research assistant, audiorecorded, and transcribed.
Statistics
Rank correlations, Kendall's coefficient (
) and Spearman's coefficient (r) were calculated, as they are the most appropriate for analyzing Likert-type survey data that have some ordinal nature but have ranges that may not be fixed or viewed similarly by the respondents to surveys (3). However, only Kendall's correlation coefficients ( values) are reported here for survey data as they provide a distribution-free test of the dependence between two variables as appropriate for this study and our similar analyses (4).
Nonparametric Kendall's () rank correlation was used for correlating participation assessment with other factors, since the marks represent a continuous distribution and the inherent scale was different from other examination marks.
Independent t-tests were used for comparisons of 2004, 2005, and 2006 examination results, 2005 with 2006 QoT data, and for PRS participation with achievements of various cohorts of students. Effect sizes, comparing the difference in means with weighted standard deviations, were calculated with Cohen's d values. A d value of 0.2 indicates a small effect, 0.5 indicates a medium effect, and 0.8 indicates a large effect.
RESULTS
Student Attendance and Participation
Average attendance over the semester for the 33 lectures using the PRS was 60% ± 2.4. The average student participation rate for the lectures employing PRS was identical at 85% ± 0.8 before the midsemester exam and over the whole semester.
Student Evaluation Questionnaire
Student questionnaire responses (Table 2) showed they felt "more engaged" (83%), "intellectually stimulated" (85%), and "motivated to think" (89%, rating 4.2) through the use of the PRS in lectures. The majority (86%) of students thought "carefully and seriously" about the questions posed. Answering PRS questions (voting) was also perceived to be enjoyable (rating 4.0) and to improve understanding (4.0). These views were verified by the majority disagreeing (82%) with the contention that "PRS was a waste of time." Interestingly, only 19% of students agreed that voting was a factor in their lecture attendance. Most students (63%) preferred that the voting was not assessed, but 51% of the students would have "answered questions more seriously" if it was assessed. Only 48% of students agreed with a scheme where assessment was based on PRS participation.
|
= 0.60, P < 0.01, n = 132), with intellectually stimulated students also enjoying the voting process ( = 0.64, P < 0.01, n = 132). Engagement and motivation to think both correlated positively with improved understanding of physiology ( = 0.58, P < 0.01, n = 132; and = 0.54, P < 0.01, n = 129, respectively), as did enjoyment of voting ( = 0.57, P < 0.01, n = 127) and the view that voting in the lectures improved problem-solving, critical-thinking, and analytical skills ( = 0.46, P < 0.01, n = 132). Students who agreed that voting improved their problem-solving, critical-thinking, and analytical skills or that PRS contributed effectively to their overall learning were also more engaged/interested in lectures ( = 0.40, P < 0.01, n = 132; and = 0.52, P < 0.01, n = 110, respectively). Summary of Written Responses to Open-Ended Questions in the Student Evaluation Questionnaire
Table 3 shows the top themed responses to open-ended questions regarding attendance and participation in the student evaluation questionnaire. There was a diversity of reasons given for lecture absence, including lectures being too early (9 AM) (16%), illness (9%), the availability of audiorecordings of lectures (9%), employment (8%), and time periods that students had one or more simultaneous classes for different subjects (7%). Fifty percent of students stated that the reason they did not participate in all voting questions was that they forgot to bring their PRS clickers. Among the most common written responses were indications that the voting made the students think (21%) or actively learn (14%), highlighting that the use of the PRS had a positive student impact. In addition, there was strong perception that voting assisted in the understanding of lecture material, reinforced or consolidated lecture material, provided feedback, and highlighted key course areas. Student responses highlighted that the negative features of PRS centered predominantly upon wasted time, with comments on technical glitches, setup delays, or excess time allowed for students to submit answers. Suggested improvements included a focus on either more exam-type questions or past exam questions (9%) or more questions based on concepts (7%).
|
Responses to the individual student interviews (n = 9) provided further insight into the students' likes and dislikes about voting in lectures and the use of the PRS. In general, students spoke positively about voting in lectures and the use of the PRS, with most enjoying answering questions and feeling more engaged in lectures where questions were asked. Six students said that the questions posed during the lectures encouraged them to study after the lectures, with one student remarking as follows:
It made me look forward to study. Normally I put off going over stuff again. It made me get into it faster because I started thinking about the kind of questions they could ask me.
Six students remarked that questions were most beneficial if they were immediately after a topic, suggesting that they preferred questions that reaffirmed their learning. Five students wanted more think-pair-share questions or appreciated these questions, suggesting that they valued peer interaction in lectures. Three students remarked that PRS questions should be more directly related to exam questions. Six students preferred that PRS questions were not assessed because of pressure to perform well and the possibilities for cheating (e.g., consulting written materials).
Some students suggested that lecturers should pose more challenging questions, although one student remarked that the questions were challenging enough. There was almost unanimous support for the use of the PRS for other courses that students were undertaking at the university.
QoT Questionnaire
Only two responses of the QoT questionnaire differed between the 2005 and 2006 student cohorts where the PRS was used for the first time (Table 4). Ratings for the statement "The course was well taught" significantly (P < 0.01) increased from 3.9 ± 0.1 to 4.2 ± 0.0 with a large size effect (d = 0.89), whereas those for the statement "I received helpful feedback on how I was going in this course" decreased significantly with a medium size effect (P < 0.05) from 3.7 ± 0.1 in 2005 to 3.4 ± 0.1 (d = 0.52).
|
Figure 5 shows a comparison of average examination results for student cohorts from 2004, 2005, and 2006 (with the PRS). A significant (P < 0.01) improvement was observed in both midsemester examination results in 2006 (65%) compared with either 2004 (56%, d = 0.84) or 2005 (59%, d = 0.65). A significant (P < 0.01) improvement was also observed in the end-semester examination result in 2006 (72%, d = 0.7) compared with that of the 2004 cohort (66%, d = 0.78) or 2005 cohort (66%, d = 0.71). There were no differences in performance for either exam between the 2004 and 2005 cohorts.
|
Correlations were found between both mid- and end-semester examination results and average lecture attendance rates ( = 0.70. P < 0.01, n = 132; and = 0.26, P < 0.01, n = 134, respectively). Mid- and end-semester examination results also correlated with participation rates in PRS voting ( = 0.16, P < 0.01, n = 131; and = 0.21, P < 0.01, n = 132, respectively).
Student Examination Results and Prior Examination Performance in Physiology
Figures 6
–8 show comparisons of examination performance for groups of students with high (>52%) and low (<52%) PRS participation scores. The final examination scores for these groups in the prerequisite semester 1 course (536-201) were not significantly different (Fig. 6), indicating that the groups were well mixed for this criterion. However, examination scores for the final 536-211 exam were significantly increased for the high versus low PRS participators with a large size effect (+5.6 marks, P < 0.01, d = 0.81), as were scores for all components of this exam: MCQs (+6.5 marks, P < 0.05, d = 0.74), SAQs (+4.5 marks, P < 0.01, d = 0.62), and LAQs (+7.2 marks, P < 0.05, d = 0.54).
|
|
|
Figure 8 shows comparisons for groups defined by further division of the entry (536-201 exam): high achievers (>75%), middle achievers (60–75%), and low achievers (<60%). Figure 8A shows that whereas the two high entry achiever groups varied in their entry scores (+3.1 marks, P < 0.01, d = 1.20), a larger increase was evident for the high PRS participation group in their 536-211 exam (+8.4 marks, P < 0.05, d = 1.21) and MCQ scores (+14.1 marks, P < 0.01, d = 1.55). The two groups of middle entry achievers showed no significant differences in entry scores (Fig. 8B) and no differences in their 536-211 exam or any of its components. Figure 8C shows the two groups of low entry achievers with the same entry scores and significant increases in their marks for the high PRS participators in the 536-211 exam (+5.4 marks, P < 0.05, d = 0.86) and MCQ scores (+6.6 marks, P < 0.05, d = 0.48).
Instructor Perspectives
Prelecture instructor interviews. All 10 instructors of the course in 2006 planned to and then implemented voting with the PRS in their lectures. Four of the six interviewed instructors were positive in anticipation of the use of the PRS, whereas two instructors used the PRS simply because it was provided. Three instructors had rarely posed questions in their lectures previously.
All six instructors predicted that students would be more engaged with the use of the PRS. Five instructors planned to use MCQs, whereas one lecturer had not decided on question types. While three instructors hoped they would not have to remove content to include PRS questions, two instructors anticipated they would, with one instructor commenting that this "may not be a bad thing," allowing "time to concentrate on the main concepts." Interestingly, one instructor anticipated "using the questions actually to replace previous slides where I just make statements and talk to those rather than asking them and then explaining."
Negative perceptions of the use of the PRS included doubt whether it would improve educational outcomes, potential technical problems, loss of time to cover lecture material, possible negative impact on QoT scores, the increase in workload associated with writing questions, and that students may not take voting seriously.
Postlecture instructor interviews. Instructor comments on the impact of PRS voting were generally positive and included the following:
[PRS questions] took time away from [the lecture] and time away from the focus, and there was a bit more chit chat during that and getting [students] going, so you have to grab them back again and put them into focus, so there is a bit of disruption.
All interviews mentioned the benefit to both students and lecturers of gauging the level of students' understanding. Comments included the following:
Half of the instructors did not remove content but chose to cover content more quickly, with some areas not covered in as much detail as usual. One instructor had more time available in their lectures, without sacrificing core content, by using a creative style of questioning. "It allowed me to do a system...whereby I asked them a question which I knew they would not be able to answer, then teach them the information that would allow them to answer that in a more systematic way and then ask them the same question again...and then get them to reflect on what they had learnt." This instructor felt that they had "only just scraped the surface of how to use [the PRS] in lectures and how to get the best out of it."
Prior concerns regarding the technical aspects associated with the introduction of new technology were confirmed for one instructor, although this instructor did not avail themselves of all support mechanisms offered. Another instructor with the same anticipation found these concerns did not eventuate. Educational benefit was "still a question mark" for one instructor, whereas another instructors with concerns about workload required to prepare questions dismissed this concern, stating that while they did take some time to prepare, "I did a good job designing the questions that were appropriate for the different things I wanted, so that concern was dismissed. It still does take time to design a question..., but presumably they will be used in subsequent years, so that is ok."
DISCUSSION
This study evaluated the implementation of a PRS in physiology lectures undertaken by undergraduate science students with the anticipation that this would promote student-centered active learning and student engagement in lectures and may have a positive impact on student learning outcomes.
On average, students improved their examination results (compared with previous years) and rated the use of the PRS in lectures highly. Interestingly, the benefits in examination performance among those who participated most frequently with the PRS in lectures were dependent on prior achievements in related physiology courses. The university-set QoT survey statement "the course was well taught" significantly improved, and instructor feedback was also largely positive. However, the answers to the questions on "critical-thinking and analytical skills" together with those on "helpful feedback" are not easy to interpret, since it is difficult to know what students really understand by such questions, and further investigation with objective measures would be required. Evaluation questionnaires established that the overwhelming majority (89%) of student respondents thought that the use of the PRS motivated them to think, and this was one of the most positive findings of this study. Written responses confirmed this view, with students commenting that the impact of PRS was in making them think (21%) or actively learn (14%). Furthermore, 83% of students agreed they were more engaged/interested in lectures where PRS voting occurred, and this was endorsed by responses in student interviews. These findings are a strong endorsement for the positive role of PRS use in promoting student engagement during lectures.
Student perceptions also highlighted the role of the PRS in contributing to their learning (78%) and understanding (82%) of physiology. Furthermore, if examination marks are used as an indicator of learning, students undertaking the course in 2006 achieved significantly higher scores for both the mid- and end-semester exams compared with students undertaking the same course in the previous 2 yr (2004 and 2005). This finding is in agreement with previous reports (6, 15, 16) of significant improvement in exam performance associated with the use of electronic response systems. Some have argued that such simple comparisons of test scores are not worthwhile and rarely show significant differences. One report (14) has suggested that such comparisons are "pseudoscience" and are not reliable. However, data that support improved examination performance through curriculum innovation are still viewed positively by universities. Nevertheless, it is clear that in all of these studies the lack of ability to control all independent variables means that differences between examination results may not be due solely to the use of the PRS. We need to assume that the student cohorts (2004–2006) are comparable in terms of ability and prior knowledge and account for unavoidable changes to teaching staff or content, which is difficult. A controlled experiment with some students using the PRS and others not using it within the same lecture would be impractical in our learning environment and would not have been approved by our Human Ethics Committee, as our anticipated outcomes suggested benefits to PRS participants.
As well as noting improvements in average exam performances within the cohort, it was clear that students who used the PRS to answer questions achieved higher examination results than students who did not participate, with participation rates positively correlated to both mid- and end-semester examination scores. It is possible that that PRS questions enabled students to practice the MCQ type that would be in the final exam but not the SAQ and LAQ types required for another exam. This may suggest that more complex learning was also taking place for students engaged in active lectures. All students in all years had the same access to past exams for practicing answers. Additionally, significantly higher results were achieved in the final examination by students with higher levels of participation (boundary participation rate: 52%) than those with low levels of PRS participation. These are important associations, particularly as the high and low participation groups had similar achievement levels on entry to the course based on their examination scores in the prerequisite semester 1 course (536-201). Improved exam performance was more likely due to PRS participation than to prior knowledge and understanding. The average PRS participation rate of those attending lectures was 85%, which is particularly remarkable considering that 50% of students claimed that they sometimes forgot to bring their PRS unit to lectures.
Furthermore, when these groups (high and low PRS participation) are then divided into two approximately equal size cohorts, based on their achievement levels (boundary score of 65%) in the semester 1 course (536-201), improved examination performance was evident in both of the high PRS participation groups. Clearly, the improved examination outcomes are independent of the prior knowledge and understanding of the students, suggesting improved performance from this innovative curriculum change across the whole student cohort. To explore this assertion, we further subdivided the student group to compare the influence of PRS participation in three cohorts with prior exam scores (536-201) of >75% (high achievers), 60–75% (middle achievers), and <60% (low achievers). Low achievers with high PRS participation showed significantly higher final 536-211 examination marks, including higher outcomes for the SAQ component, an assessment format that tested their understanding of concepts. The findings of similar improvements in exam performance in the high achievers with high PRS participation were confounded by significant differences in their entry 536-201 exam scores. While this makes it hard to exclusively attribute this improved outcome to PRS use, it may highlight a trend toward higher engagement with additional learning tools for students with high performance levels. While it is interesting that there was no improvement evident in the middle achievers, there is no clear understanding for this outcome. This diversity of findings provides a useful foundation for future investigations of practices with the PRS.
It is difficult to determine in previous reports (2, 6) of improved lecture attendance with the use of audience response systems whether the improvements were due to the inclusion of questions and voting or due to the assessment of the responses. In the present study, although lecture attendance and PRS use were voluntary, students (19% of written responses) commented that PRS voting encouraged them to attend more physiology lectures than they normally would have. This is an encouraging result considering the diversity of reasons that students have for not attending all their lectures. It was not possible to quantitatively establish whether actual lecture attendances were improved, as attendance records were not kept for iterations of the course before 2006.
As a possible encouragement of greater attendance and participation rates by future student cohorts, a small assessment component will be trialed based on effective participation with the PRS. This trial is encouraged by the students who indicated that they would have taken the subject more seriously if it was assessed (51%) and those in favor of future PRS assessment (21%). However, this assessment component will not be based on getting the correct answer so as not to prevent the students from attempting questions, exploring the concepts, or finding out how to solve problems.
Draper and Brown (5) explored an issue of emerging importance: contingent teaching, that is, making the teaching depend on the actions of students rather than being a fixed sequence predetermined by the teacher. This promising pedagogical approach to teaching arose in the postlecture instructor interviews. In our study, the use of the PRS assisted some instructors to focus the direction and pace of the lecture more closely to the pace of student understanding. Interestingly, one lecturer was surprised that s/he was underestimating the students' knowledge and that this was "valuable information that you don't usually get." Beatty (1) also described this experience by stating that "Many instructors are quite shocked by how incorrect their expectations are of students' comprehension."
Previous reports (1, 8) have noted that some students meet electronic response system-based instruction with apprehension or discomfort, particularly those that are accustomed to doing well with traditional style lectures, whereas others are resentful out of laziness since being asked to engage requires effort. This was not a general observation of the present study and, in fact, was a perception offered by only one instructor. It has been recommended elsewhere (1) that instructors discuss the role of the new technologies with students with the aim of enlisting them as "collaborators in their own education." This is a view with which we are in strong agreement. Communication, with both students and the body of instructors, is clearly a key element in the successful implementation of such a program.
There is no doubt that there are some hurdles–some perceived, some actual—that must be addressed in the implementation of technologies such as the PRS in lecture classes. Inclusion of PRS questions will occupy some time during a lecture, and a common perception is that, as a result, there is less time that can be spent covering content. However, this inevitability has actually been a positive factor as it has driven an evaluation of the factual content level of lectures leading to prioritization of content or a refocusing of the lectures toward concepts and applications of knowledge. This has led us to consider having students prepare for lectures by prereading basic (factual) information so that they can be more engaged in an active learning process focusing on the application of concepts and problem solving.
There is also an increase in workload involved with setting up the hardware, mastering the software, creating and setting questions to be posed in the lectures, running the system during the class, and, despite best efforts, the possibility of technical malfunctions. Indeed, in-class delays with setup or technical glitches can be of irritation to the student cohort (12%) but could be minimized through adequate communication, training, and support systems. It has been suggested that that the best way to assist instructors to adjust to their new roles is to provide mentoring and support by instructors experienced in the use of electronic response system-based instruction (1), and it is clear in the present study that training seminars and the one-on-one support provided for each instructor served as important scaffolds in building the instructors' ability and confidence in the use of the PRS in lectures.
We focused on encouraging students' active learning in lectures, whereas previous studies have made more use of audience response technology during lectures for formative or summative assessment. The present study suggests that a number of future improvements can be made to increase student preparation and engagement with active lectures. Encouraging more students to undertake prereading is vital for students like ours, with diverse cultural, learning approaches, and knowledge backgrounds. If they are similarly prepared for the intellectual challenges of the new style of lectures, they should be able to also engage confidently and effectively in peer-to-peer discussion. Finally, we need to ensure that future assessment is constructively aligned with these new higher-order learning objectives if the students are not to regress to the rote learning encouraged by traditional style examinations.
GRANTS
This work was conducted with financial support obtained from the Learning and Teaching Fund, the University of Melbourne, 2006.
Acknowledgments
This project was inspired by Dr. Dee Silverthorn, a world leader in physiology teaching who uses an active learning approach with personal response systems. We also thank Alan Lambell and Chris Bramich (Department of Physiology) for administrative assistance.
Received for publication November 28, 2007. Accepted for publication January 5, 2009.
REFERENCES
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
t, change in time. B: the model is progressed to a model muscle that lifts weights (loads) tethered at one end (the muscle shortens; contraction is concentric) (muscles A–C). Students are introduced to concepts of load, force, velocity, and the important relationships that derive from them: "length-tension" and "force-velocity" relationships. 
