| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
TEACHING WITH TECHNOLOGY
1 Higher Education Development Unit, Centre for Learning and Teaching Support, Monash University, Victoria, 3800
2 Department of Physiology
3 BioMedical Multimedia Unit, University of Melbourne, Victoria 3010
4 Teaching and Learning Development Support, James Cook University, Queensland, Australia 4811
Abstract
We describe the design and development of a highly interactive model-building program to assist students from a diverse range of academic backgrounds to understand the baroreceptor reflex. Our approach is to have students work in small groups to construct their own simple model of such a control system. This model then provides the basis for a structural framework for students to add further complexity without losing overall perspective and allows exploration of deeper issues. Our program is suitable for many disciplines and student backgrounds and provides a visual representation of a difficult concept, providing a basis to ground further knowledge. Audit trail data have been analyzed to identify and resolve areas of student difficulty, and extensive surveys and observations on students’ use of the program over three years in several courses have been used to test and improve its effectiveness.
Key words: baroreceptor reflex; model building; negative feedback; formative evaluation
This article reports on the development and subsequent iterative evaluation of a multimedia module [first described in Weaver et al. (9)] in which students are asked to construct and explore their own model of a complex biological control system and then use that model for further investigations. The long-term continuous formative evaluation and adjustment have now led to the international distribution of the program on CD-ROM (7).
In the development process, an important starting point was our awareness that tertiary-level (university) students with disparate entry levels of academic experience and achievement need to be provided with the tools to deal with complex control systems usually found in the biological sciences. This is especially the case in health sciences, where classes include both undergraduate and graduate students with a broad range of backgrounds (arts, law, commerce, physical and biological sciences, and engineering).
In biology, the complexity of operation of basic control systems often means that we must deal with what are classified as "ill-defined systems" with nonlinear and discontinuous behavior. Complex biological systems are difficult to describe satisfactorily with mathematical functions and are also influenced by other variables that are poorly understood.
We have chosen blood pressure control as a control system suitable for our modeling approach, in particular the reflexes that stabilize human blood pressure during changes in posture. The baroreceptor reflex is an excellent example of a cardiovascular reflex and also involves understanding concepts of negative feedback and baseline tone (2). This is an example of an ill-defined system that has been the subject of extensive modeling attempts using mathematical approaches, but we have found that most of our students in their early years do not use such mathematical models effectively and seem to be lost in their complexity. The computer models tried varied from simple Windkessel models of circulatory hemodynamics (without feedback) to more complex models like those of Guyton’s of integrated systemic behavior. Academic staff reported difficulty in engaging students without intensive tutoring, and we did not find a product that allowed students to extend their thinking into considering some "What if?" scenarios.
Our new approach is to introduce a qualitative conceptual framework that facilitates the students’ building of the "big picture" of a complex system before going into the details. Such an approach has also been taken to bring teaching into alignment with practices in new curricula that increasingly rely on problem-based learning, problem solving, student-centered learning, and self-paced learning.
This project is a direct response to student and staff difficulties with the teaching of biological feedback control concepts in an engaging manner and arose from a need to replace practical classes in which students investigated the control of blood pressure in anesthetized animals. These classes were very successful in assisting students to develop manipulative skills and experimental techniques and ways of investigating control systems by interrupting control pathways. However, despite having one tutor for every eight students, these classes were not so successful in assisting them to understand the overall operation of blood pressure control. Later, in line with a worldwide move to reduce animal experimentation, these experimental laboratory classes were replaced first by class demonstrations and then by a video of the experiment shown to a class in a lecture theater. The latter, rather passive sessions were of limited success. Students learned effectively neither about experimental design nor analysis of the control system.
PROGRAM DESIGN TARGETING THE CURRICULUM NEED
The first stage in the development of this tutorial was to identify the need in our curriculum. We analyzed essay answers to an examination question set for target students to identify the major problems with this topic. In consultation with the examiner, each exam response was analyzed against a checklist of requirements, and statistical data were collected to ascertain where problems occurred. We found that students experienced difficulty understanding the concept of the model/circuit and displayed problems with the signaling sequence of reflex control (i.e., detection of imbalance must occur before reflex correction can take place). Compilation of this data also allowed us to evaluate future cognitive outcomes of implementation of this tutorial.
Not only is this examination analysis an essential step in deciding that the students’ difficulties justify the investment of time and effort in a multimedia solution, but it is also a useful guide to the development of the program. It also provides a basis for future comparative analysis of the extent of misconceptions after such a program is incorporated into a curriculum.
Pedagogical Issues
We take a constructivist approach by allowing students to build their own model systems, within the bounds of the physical constraints that govern the behavior of the system, which has proved successful in similar situations at other institutions (6). The computer is a key component of this approach when it is used as a cognitive tool (3). The pedagogical principles and interface design issues considered in development of computer programs involving students’ modeling of secretory cell functions have been described previously (5, 10), and many of these principles have been followed in this project. We have also chosen to present the program in a collaborative learning environment, with two to three students per computer. We have found this most effective as judged by the busy computer laboratory, with fingers pointing at the screen and lively and often noisy interactions between students. The success of this collaborative environment, also judged by extensive student surveys, has been reported (4, 8).
We take a basic approach to understanding the behavior of a control system by having students develop the logical arrangement of system components. Students are challenged to build and explore the operation of an on-screen model of the blood pressure control system by making logical assemblies of blood pressure detectors (receptors), signaling circuit components (neurons), and output mechanisms (in heart or blood vessels).
We use the computer program effectively as an expert system that assists students’ logical assemblies and testing of their model by providing construction tools, simple animations of their model operating, feedback screens indicating success or failure of their model, and hints on possible changes they might make. Students may find deficiencies in their model or a need to adjust misconceptions, suggest modifications in the expert model for future implementations, or exhibit misunderstandings in common with other students. On completion of the model, students are given tasks with a strong element of reflection in their use of the tutorial as they try "What if?" scenarios and elaborate on the knowledge into new areas. At the end of the class, if required, tutors work through the model with any students who failed to complete the exercise and conduct a discussion on the questions on the paper-based task sheet.
The essential feature of the program is to present the information in a simple qualitative manner that is used to give a global view of the control system. This conceptual framework can then be used by students to add specific details of the mechanisms or indeed to better understand a mathematical simulation of the system at a later time, such as would be appropriate for engineering students. Teaching programs are always based on either explicit or implicit assumptions about learners and learning and the desired cognitive outcomes. Some issues we have tried to address are described in Ref. 9.
Learning Outcomes of This Program
The major aim of the tutorial is for students to understand how the central nervous system controls blood pressure to the brain when we change posture, e.g., when we stand up from a lying position.
The specific learning objectives are to understand the mechanisms used by the body in:
INITIAL IMPLEMENTATION
Model Building as a Series of Cognitive Steps
The program provides students with a background of an on-screen schematic representation of the structures involved in the neural control of blood pressure by the baroreceptor reflex. On this one screen, students create and position components of an electrical (neural) circuit to build a control system that will allow the maintenance of blood supply to the brain with a change of posture. They select from various components consisting of receptors, afferent neurons, interneurons, and efferent neurons (Fig. 1).
|
|
Our educational approach is to provide feedback at every stage, and this is in the form of a simple animation of the system working (i.e., various numbers of electrical impulses moving around the connected components of the system and releasing a cloud of neurotransmitter at the terminals), followed by textual feedback. The textual feedback is always in the format of a positive statement (to reinforce correct aspects of the preceding action), followed by a statement about what is not yet correct, and a hint about what to consider next (Fig. 3).
|
The program was written in SuperCard on a Macintosh Computer and later translated to Macromedia Director for cross-platform delivery.
Initial Formative Evaluation
The first draft of the program was tested on a cohort of medical students working in pairs in a scheduled class. Students were observed by the program developers and tutors and completed a written questionnaire at the end of the session. At the same time, electronic audit trails of student progress through the model construction were collected and matched to questionnaire responses. Audit trails tracked student viewing of feedback panels, so that we could map exactly which path was taken to reach the complete model. Importantly, this was only a very selective collection of data (not every button click was stored) to limit the amount of information collected to a manageable and interpretable amount. Logging the amount of time spent on any individual screen was soon discovered to be useless information, as it could not tell us whether students were engaged in fruitful discussion or irrelevant chat.
Students took some time to familiarize themselves with the program (and only some students read instructions screens and feedback panels) but were generally able to complete the given tasks in the time provided. It was apparent that some students quickly became frustrated when they felt they were unable to move on and became less likely to read further instructions or textual feedback. The level of sophistication of the model and lack of familiarity with the symbols used were such that some students were overwhelmed by so much new information. At this stage, small groups of students were led through the exercise, but such a didactic approach meant that students missed out on the chance to build their own understanding of the concepts and did not have the opportunity to explore different scenarios.
Questionnaire analysis.
The questionnaire responses generally reflected the students’ enjoyment in the interactive task of constructing their own model, although most were challenged by the task. The questionnaire also asked students for suggestions for improving the tutorial, and many took this opportunity to make some excellent suggestions. Several of these were incorporated into the later tutorial design.
More specifically, students enjoyed the animated sequences, with typical comments being that they appreciated the visual representation of the circuit but had difficulty understanding some of the textual feedback. Typically, they asked for more directional hints ("Tell me what to do!"). The most consistently reported difficulty related to students’ feeling confident about getting started. This problem was perceived as an early cognitive overload and was dealt with in subsequent modifications to the program. Importantly, most students reported that they used the textual feedback in constructing the model, even if most of them still experienced difficulties along the way. Many reported difficulty in understanding the accepted symbols used in representing components of the nervous system, since this was the first time they had had exposure to these in their studies.
Audit trail analysis.
Analysis of the electronic audit trails allowed us to collate information from the numbers of groups/pairs of students that received any particular feedback screen more than three times. This was used to indicate difficulty in moving past a particular stage of the model-building exercise. Repeated viewing of the same textual feedback does not necessarily indicate that no new attempt has been made (e.g., in some cases positioning an element in any of 3 different sites will give the same feedback response). This was a deliberate choice, helpful for identifying a common problem arising in different audit trails; e.g., placement of component gives the same feedback in similar situations.
The data generated by audit trail analysis and by scoring of student questionnaires (both cognitive and affective) after the prototype testing were used to redesign some aspects of the software. In particular, it was possible to identify crucial sites of confusion in the model construction process, where repeated viewings of feedback panels significantly above background levels could be detected in frequency distribution plots of viewing practices. (see Fig. 4)
|
Second, it emerged that efforts to ground the model-building exercise in anatomical reality by using specific structural components distracted the students from obtaining an overall grasp of the feedback circuit architecture. Subsequently, some stages of the model were simplified, and more detailed anatomy of "central processors" was made available as hyperlinks in the program for those students who had an inquiring interest. This practice of progressive unmasking of system complexity on demand accommodates a broader range of student backgrounds.
Additional Modifications
Audit data also allowed identification of feedback panels that required editing to remove ambiguities, and in some instances the stages of model construction were simplified. A case study was introduced to put the model into context, being the example of a modification of the control of blood pressure following a prolonged period of bed rest. A key was also introduced beside the practice screen to remind students of the color coding of the different types of neurons that could be created. Feedback screens were reworded if audit trails had identified that students had difficulties understanding the information provided.
FINAL REVISED VERSION OF THE PROGRAM AND ITS EVALUATION
The revised program was then tested with a new cohort of second-year medical students and also with students in second-year biomedical science and second-year science undergraduate courses.
Students completed two-hour scheduled computer-aided learning (CAL) sessions, mostly working in groups of two or three per computer (up to 40 students per class), with one of the authors (Weaver) and tutoring staff to provide in-class support. A paper-based task sheet was provided for students to record answers to questions extending their understanding of the physiological concepts involved, including questions asking them to use their model to predict outcomes in different situations (e.g., carotid occlusion, hemorrhage). Attendance was high, as students were assessed on attendance and participation in collaboration with fellow students.
Medical Student Evaluation
Since the initial rounds of evaluation, major changes have been made to the whole medical course, moving from a traditional structure to one that includes significant problem-based learning elements.
Evaluation of this package in the new medical course consisted of class observations together with collation of student questionnaires focusing on the major changes incorporated into the program since the previous evaluation round. Generally, we found that, where students had used the practice screen for its intended use, they did not experience the frustration associated with the large cognitive load of getting started and reported enjoying the model-building task and the interactive animations. Students now spent relatively little time mastering the program and concentrated more on the physiology of the exercise. Student perceptions of their own understanding of the topic increased significantly from a pre-CAL rating of 2.85 ± 0.94 (1 = understood topic not at all, 5 = understood very well) to a post-CAL rating of 3.80 ± 0.71 (P < 0.0001; n = 46).
However, a large proportion of the students did not realise that they could use the practice screen for this purpose, so further modifications were introduced to direct students to this resource. Also, we found that viewing of the cartoon case study was low, as students often did not follow the link to this in the program, so the case study was highlighted in the Table of Contents, and these screens were moved to the top of the menu to simulate the "trigger" used in problem-based learning. Students also reported a lack of closure with the program, so a summary screen was included.
As frustration with using the program decreased, students’ investigation of the physiological concepts increased, and responses on the questionnaires revealed difficulties in understanding some deep concepts that we had not identified previously. The major one of these related to inhibitory mechanisms, particularly to the effects of decreased levels of neural inhibition. Appropriate explanatory material and animations were added to the program.
Biomedical Science and Science Student Evaluation
Both the science and the biomedical science courses have a more traditional structure and different content compared with the medical (PBL) course, and the CAL classes are scheduled for all students in two-hour sessions. The three courses have different average entry scores for student acceptance. Thus they are analyzed separately. After the aforementioned modifications were introduced, a further round of similar evaluation was conducted, which also included matched pre-CAL and post-CAL tests. The tests consisted of three short multiple-choice questions, with the same test being repeated at the end of the session. Full answers and explanations were supplied to students the week following their class once all evaluation sessions were completed.
The aim of this was to determine the level of understanding of key physiological concepts before and after this class and so determine whether we had been successful in improving this understanding. We recognized that merely conducting the test would focus attention on these areas. Ethical considerations did not allow us to conduct this trial without giving all students access to the teaching material, so it is impossible to determine whether any improvement in test scores resulted from the teaching intervention or from merely considering the topic for an equivalent length of time.
At the start of the class, the purpose of the evaluation exercise was explained to all students. Questionnaires and tests were both anonymous, but the tests were color coded and number coded to allow matching of results. The test was deliberately brief, as students were also asked to complete the questionnaire, and we were trying to avoid taking too much of their valuable study time. Completing these evaluation tasks still took a total of 20 minutes from a two-hour class.
Observations and questionnaire.
In both courses, student progress with the model-building exercise was observed to have improved, with very little evidence of the previously recorded difficulties. The task was still demanding, but students enjoyed the challenge, and the level of discussion and questioning of tutors were very high. Student use of the practice screen had increased, and this was reflected in a much decreased level of frustration in getting started with construction of the model. This observation was confirmed by responses to the questionnaire indicating that the practice screen was invaluable not only as an introductory exercise to the tools but also as an introduction to the concepts of negative feedback and neuronal circuits. Student viewing of the case study increased, and nearly all students spent some time considering the major questions posed before moving on to the rest of the tutorial. During discussion at the end of the class, many students referred back to the case study and used what they had understood from the tutorial to explain the case. Even when the case study was not specifically mentioned, students still tried to explain why prolonged bed rest could cause fainting on suddenly standing. They enjoyed discussing examples that they could relate to "real-life" situations.
Overall, the students enjoyed the hands-on nature of the model-building exercise immensely and gave the program a very positive rating. They judged their own understanding as having increased significantly from 2.23 ± 1.06 (biomedical science) and 2.38 ± 0.92 (science) to 3.72 ± 0.77 (P < 0.0001; n = 71) and 3.55 ± 0.75 (P < 0.0001; n = 104), respectively, after using the program.
Pre- and post-CAL tests.
The tests consisted of three short multiple-choice questions (see APPENDIX 1), graded in difficulty and requiring understanding at increasing levels of complexity. The starting level of understanding was different for the two courses, but each group exhibited a similar significant improvement in understanding (see Table 1).
|
We were surprised that performance on question 3 was lower than expected, particularly in the post-CAL results, given that it is very similar to a question which students had been answering on their task sheet. It was observed that a few groups of students did not know the definition of hemorrhage; some said it was a bruise, and others used the word clot. The question on the task sheet refers to blood loss, so this different term used may have led students to believe that this was a completely different question. Most students completed the test without discussion, as instructed, so it is impossible to conclude where they had difficulty.
From observations only, we tend to believe that the program itself was responsible for the improved performance on questions 2 and 3, because we did not observe students referring back to the questions during the class or discussing these topics more than in similar sessions without tests, even though they had been informed that they would be undertaking the same test at the end of the session. Even when specific reference was made in discussions with tutors to points that had been included as options on the multiple-choice questions, no students commented on this or seemed to recognize these from the test.
However, we do recognize the limitations of pre- and posttests as a summative evaluative tool and realize that further evidence is needed before firm conclusions can be reached on the effect of the program in meeting the learning objectives. This would require more extensive assessment of pre- and post-CAL understanding of the topic by more focused and extensive questioning.
OVERVIEW
The program has now been extensively evaluated with groups of second-year students in medical, biomedical science and science undergraduate courses at the University of Melbourne. The program has also been distributed nationally and internationally to over 12 selected universities, which have trialed the program with their students and have provided useful feedback that has been used to modify the final program.
Our approach to helping students learn about control systems by fostering the development of conceptual understanding has been successful, judging by the improvement in student performance not only in the short posttest but also in the number and depth of advanced physiology questions occurring in discussion both during and after the class. Although the students’ learning outcomes could not be assessed in an exam situation or compared with previous similar cohorts of students, we were able to gather information from a variety of sources (students’ self-reporting of their level of understanding, results from the pre- and posttest, observations of discussion between students, and feedback from tutors in the class). Although each of these sources of information has its own limitations, together, all the data do indicate an improvement in the students’ understanding of the key concepts, as outlined in the learning objectives.
The goal was to establish a level of qualitative fundamental understanding that could be built upon later by including quantitative transfer functions. Thus our approach is usable in a wide variety of contexts, including those in which mathematical constructs are inaccessible or unknown.
The emphasis is to encourage student hypothesis formulation and visual testing as the continuity between control system input and output components is represented graphically. This approach lends itself to the description of control systems for which real components can be identified (i.e., receptors, neurons, blood vessels) rather than systems in which the components are conceptual. The use of paper task sheets provided the opportunity for students to both test and generalize their newfound understanding and has proved useful in provoking hypothesis generation and discussion among groups of students. This opportunity was exercised in most classes, and students liked to test their ideas extensively with the tutors. This approach is being extended to programs involving other control systems, including acid-base balance and thermoregulation.
The audit trail data proved invaluable in undertaking this evaluation and may be used in this manner only if the feedback panels have been formulated in advance to identify and respond to a prioritized hierarchy of student errors. Tracking the characteristics of the feedback panels most frequently displayed in response to student error provides useful data about student understanding. Indiscriminate keystroke logging is unlikely to be helpful in the evaluation process, because the volume of data produced is unmanageable and the common patterns of student error cannot be recognized. Because the audit trail code saved data automatically to the hard drive of the local computer, it was not considered appropriate to include this programming feature in the commercially-released final version of the tutorial.
APPENDIX
Pre- and Posttest
Circle your answers for the following questions.
Note: There may be more than one answer!
Acknowledgments
Address for reprint requests and other correspondence: D. Weaver, Higher Education Development Unit, PO Box 91, Monash Univ., Victoria 3800, Australia (E-mail: debbi.weaver{at}celts.monash.edu.au).
Received for publication April 3, 2002. Accepted for publication September 10, 2002.
REFERENCES
This article has been cited by other articles:
|
|
 |
|
  S. A. Azer A multimedia CD-ROM tool to improve student understanding of bile salts and bilirubin metabolism: evaluation of its use in a medical hybrid PBL course Advan Physiol Educ, March 1, 2005; 29(1): 40 - 50. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
