| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
TEACHING WITH TECHNOLOGY
ana Kukolja Taradi
imir Radi
a PokrajacDepartment of Physiology and Immunology, University of Zagreb Medical School, Zagreb, Croatia
Address for reprint requests and other correspondence: Sunana Kukolja Taradi, Dept. of Physiology and Immunology, Univ. of Zagreb Medical School, 10,000 Zagreb, 
alata 3, Croatia (E-mail: skukolja{at}mef.hr)
Abstract
World Wide Web (Web)-based learning (WBL), problem-based learning (PBL), and collaborative learning are at present the most powerful educational options in higher education. A blended (hybrid) course combines traditional face-to-face and WBL approaches in an educational environment that is nonspecific as to time and place. To provide educational services for an undergraduate second-year elective course in acid-base physiology, a rich, student-centered educational Web-environment designed to support PBL was created by using Web Course Tools courseware. The course is designed to require students to work in small collaborative groups using problem solving activities to develop topic understanding. The aim of the study was to identify the impact of the blended WBL-PBL-collaborative learning environment on student learning outcomes. Student test scores and satisfaction survey results from a blended WBL-PBL-based test group (n = 37) were compared with a control group whose instructional opportunities were from a traditional in-class PBL model (n = 84). WBL students scored significantly (t = 3.3952; P = 0.0009) better on the final acid-base physiology examination and expressed a positive attitude to the new learning environment in the satisfaction survey. Expressed in terms of a difference effect, the mean of the treated group (WBL) is at the 76th percentile of the untreated (face-to-face) group, which stands for a "medium" effect size. Thus student progress in the blended WBL-PBL collaborative environment was positively affected by the use of technology.
Key words: computer-assisted learning; distance learning; distributed learning; hybrid course; online education; information technology; medical education; Web-based learning; problem-based learning
THE EXPONENTIAL GROWTH OF knowledge is one reason that strategies, such as problem-based learning (PBL), are becoming popular, and technology support can be so valuable (1). Understanding physiology (or any other discipline) requires students to have the facts and the ability to use those facts to solve problems (22). In PBL, students are put in an active learning environment by giving them problems and training them to identify what they need to learn to solve those problems (8). Consequently, today a major trend in higher medical education is the move to PBL and collaborative learning environments supported by information technologies.
PBL and collaborative learning educational methods can occur without technology, and technology can be used in a non-PBL-collaborative environment (18). However, an innovative online learning environment has a substantial impact on student learning and problem solving (25). Communication and collaboration tools like online forums, chat, and e-mail facilitate discussion and learning among student peers and faculty (17). Therefore, The World Federation for Medical Education (WFME) advocates the introduction and integration of computers into the medical curriculum (34), but the real power of computers to improve education will only be realized when students actively use them as cognitive tools rather than passively perceive them as tutors or repositories of information (14).
Independently, each of the mentioned methods [PBL, World Wide Web (Web)-based learning (WBL), and collaborative learning] offers an opportunity for moving beyond content acquisition to develop skills and dispositions needed for lifelong learning. The debate is whether differences in media used make a difference in student learning outcomes. Because all three methods support active learning in a rich educational environment, it is reasonable to expect that combining them together could help students to achieve meaningful learning (21), resulting in a positive, cumulative impact on their progress. By combining PBL with collaborative Web-based distributed education, instructional designers can create active, vibrant learning environments that enhance student learning (12, 25). The underlying presupposition is that a well-designed program must be based on sound pedagogical principles (41).
The aim of the present study was to identify the impact of a blended WBL-PBL collaborative learning environment on student learning outcomes by comparing student scores on the summative acid-base physiology examinations in a blended WBL-PBL collaborative environment with scores of the summative acid-base physiology examinations of students enrolled in traditional PBL classes. A null hypothesis was developed for testing at the two-tailed a priori level of 
= 0.05, stating that there are no significant differences in the mean achievement scores between student learning in a blended WBL-PBL collaborative learning environment and student learning in a traditional face-to-face PBL course.
MATERIAL AND METHODS
Participants and Computer Accessibility
The population of the study consisted of 121 second-year medical students enrolled in an elective course of acid-base physiology at the University of Zagreb Medical School. The study examines the performance of two student populations: 1) two generations (2002–2003) of WBL students (n = 37) who studied in a blended WBL-PBL course in which a significant portion of the learning activities have been moved online, and 2) six generations (1996–2001) of face-to-face students (n = 84) enrolled in a conventional version of the PBL class. The two student populations did not differ significantly in the ratio of men to women (Table 1).
|
All WBL students (n = 37) had home computers, and 34 students had Internet access from home. The University of Zagreb Medical School provides computer facilities for classroom use and general student access. Computer labs and public Internet workstations with unrestricted Internet access are open to all students, staff, and faculty.
Traditional PBL Course of Acid-Base Physiology
The elective course of human acid-base physiology was created at the University of Zagreb Medical School in 1991 as a face-to-face course designed to include mixed traditional teaching methods (including PBL) to achieve the learning outcomes in knowledge, skills, and attitudes (30). The idea was to provide an overview of this difficult subject material in conjunction with the PBL scenarios.
The traditional face-to-face PBL class (on average 10–15 students) had two formal meetings per week during 5 wk (one of which was the visit at the Clinical Hospital for Lung Diseases to meet real-life patients) with each session lasting 3 h (30 credit hours) during which the tutor was available. Students were provided with written teaching materials and were taught in a traditional classroom PBL environment without using computer technologies.
Development and Organization of Blended Course of Acid-Base Physiology
In January 2000, a faculty team sponsored by the Croatian Academic Research Network started a pilot project on creating the first blended course in which a significant portion of the learning activities had been moved online and time traditionally spent in the classroom is reduced but not eliminated. The goal was to join the best features of in-class teaching with the best features of online learning to promote active independent learning and reduce class seat time. Using computer-based technologies, we redesigned the traditional PBL course with new online learning activities, self-testing exercises, simulations, and online group collaborations. It took 
18 mo to develop and create the "interactive module of acid-base balance in humans" (16). Since spring of 2002, the blended WBL course has been offered by the University of Zagreb Medical School as an elective course to second-year medical students.
Course delivery (software).
A rich, interactive educational Web-environment supporting PBL was created by using the commercially available Web Course Tools (WebCT) software package (http://www.webct.com). It is a tool that meets the criteria for good course-authoring software (28).
The WBL environment is logically organized into 10 topics (e-tutorials) presented through a series of 78 Web pages, illustrated with 62 original drawings and photographic images, 3 interactive Flash animations, 26 formative self-tests and quizzes, 4 specialized calculators (scientific calculator, pH calculator, anion-gap calculator, and square equation calculator). Each topic is presented by goals and objectives, content notes, assignments, quizzes, and links to outside Web resources.
Quizzes and exams.
WebCT supports multiple forms of quizzes and exams. Using these tools, we created several test types: a diagnostic pretest, 26 formative self-tests, and quizzes with immediate feedback, as well as a final summative test. All 26 formative tests are not assigned to each student group. On average, one student is offered 10 self-assessments (1 formative assessment per 1 e-tutorial). Students submitted their answers electronically and received instant feedback on their performance. In addition to multiple-choice questions, true/false, matching, calculated, short answer, and written paragraph questions were used.
At the end of the course, both groups were given similar final summative tests. The main difference was the delivery of the test: face-to-face students were offered paper-and-pencil tests, which were evaluated electronically, whereas WBL students had to complete online proctored tests. Both types of tests were "open book" problem-based tests: students were confronted with a problem description based on real cases (medical findings of blood gases data of patients with acid-base disturbance). The test contained 36 questions with a maximum score of 36. The test examined both factual recall and higher-order thinking, including integration of knowledge and problem-solving ability. The final exam was proctored by the instructor because of the need to determine student identity. The test was timed and automatically graded by the WebCT system. Students’ proctored test scores of the two WBL-PBL generations were used as the dependent variable, as an indicator of their academic achievement (learning effectiveness) in Web-based classes.
Administrative issues.
WebCT offers a helpful administrative tool for student progress tracking. It has the ability to record first and last access to the course by individual students, as well as the number of times he/she accessed site’s instructional materials, the number of messages read, and items posted.
Course Goal, Learning Objectives, Learning Method, and Activities
Because acid-base homeostasis is one of the most difficult parts of physiology for medical students to master, medical teachers are seeking ways to help their students with this topic (31). The overall purpose of the course (in both environments) is to help students acquire a deeper understanding of the human "acid-base balance" concept. After attending this course, the student will be able to understand complex homeostatic mechanisms that maintain the concentration of H+ ions in human body fluids in a narrow range of 35–45 nM, define the normal acid-base status in humans, differentiate among metabolic, respiratory and complex acid-base disorders, discriminate between the cause of disturbance and the compensatory process, assess and interpret arterial blood gases of real patients by an effective use of acid-base nomograms, and make positive diagnosis of acid-base disorders.
Use of a WBL environment was the main difference between the two groups. The amount of material covered in the WBL course, and the depth with which it is covered, was in general equal that of a classroom-based PBL course. The material presented was not overly comprehensive. The concept was to motivate students for active participation through which they could find the details themselves and come to a clear understanding of the topic. In designing the WBL acid-base balance course, we aimed to provide a variety of learning experiences that would encourage students to take a deeper approach to learning. A group-learning plan that was self-paced but bound to a strict schedule was created. The lectures were integrated with static (photographs, pictures) and interactive services (Flash interactive animations, calculators) and self-assessment features with immediate feedback.
In the WBL-PBL course, there were three face-to-face meetings: the first and the last meetings were organized in the computer labs, and one meeting (in the middle of the course) was a visit at the Clinical Hospital for Lung Diseases where students had the opportunity to meet real-life patients with acid-base disorders.
On the first day, students were instructed how to use the rich interactive educational Web-environment and were placed in randomly assigned groups consisting of 4 to 6 students and a trained facilitator. Also, students took an online pretest to determine the level of their understanding. A paper survey was administered to all students to investigate their familiarity and attitudes toward Web-based technology. Because of the need to determine student identity, the final proctored exam took place in the computer lab as well. Other formal meetings (twice a week during 5 wk) were flexible; students had the choice to come to the campus computer lab or to participate from a remote networked computer. A typical online tutorial progressed as follows. The brainstorm phase started with individual preparation by studying the online materials, followed by a discussion in small teams via synchronous electronic communication tools (chat and whiteboard). Students were given multiple tasks (both individual and group based) with deadlines. They had to share their work, give each other feedback, and complete a larger group project by participating in a 90-min synchronous session (chat and whiteboard) with team members and the instructor. Synchronous online meetings were scheduled similarly as in traditional classes—twice a week during 5 wk. Each team had an assigned private chat space for discussion during the synchronous 2-h meeting. A separate public chat space was assigned for the whole class for discussions, help activities, reflection, sharing information, and socializing. Chat room sessions are recorded for the benefit of students not able to participate during the allotted time. We also made great use of the whiteboard tool, which allows groups of users to dynamically share graphics, draw images, enter text, or annotate images in real time (i.e., synchronously). Because the whiteboard efficiently pulls all the collaboration technologies together, students feel almost as if they are communicating and interacting face-to-face. Between the formal meetings, students had the opportunity to communicate asynchronously through forum discussions and e-mail.
Learning Effectiveness and Satisfaction
Learning effectiveness was measured in terms of student learning outcomes and satisfaction. We defined learning outcomes as intended learning resulting from a teaching process. Student learning outcomes assessment measures whether the learning outcomes, the objectives faculty have set, are being met. For this purpose, the scores of the final summative acid-base tests of all students (n = 121) in both groups (face-to-face students vs. WBL students) were analyzed.
Information about student satisfaction was gained by surveys. Questionnaires were designed to gather general information about what students think and feel about the courses and their relationship with the courses.
General physiology course.
It should be noted that for enrollment into the elective course, students in both groups were self-selected. To eliminate the objection that students using information technology resources may be the most motivated students who would perform better on the final examination with or without using this resource, we also analyzed their learning outcomes on the final multiple-choice-question, paper-and-pencil test examination of the general physiology course. For this purpose, the grades of the final general physiology examination of both student groups were analyzed. The general physiology course is an obligatory undergraduate traditional (non-PBL) 170 credit hours course. The general physiology course and the elective acid-base physiology course were entirely separate courses. Usually, the elective course (traditional as well as blended) starts a few weeks after the beginning of the general physiology course.
Statistical Analysis
We used two different approaches to assess the magnitude of the intervention’s effect. First, we assessed the statistical significance of the effects. Descriptive statistics are reported as means ± SE. Analyses of data were performed by using Student’s t-tests. Differences were considered significant for P values of 
0.05. Second, we assessed the relative size of the effect based on standardized estimates of effect size according to Cohen’s benchmarks, which defined d as the difference between the means divided by SD of either group (7).
RESULTS
The study results were divided into three groups: 1) results from student scores of the summative final examinations, 2) results from the descriptive analysis of the WebCT electronic administration data, and 3) results from the analysis of the survey questionnaires.
Final Examination Scores: WBL Students vs. Face-to-Face Students
Results of final examination of a total of 121 students were analyzed. The study population consisted of two groups of second-year medical students: face-to-face students (n = 84) and WBL students (n = 37). The WBL students’ proctored final online test scores were used as an indicator of their academic achievement (learning effectiveness) in Web-based classes. The control in this study was traditional PBL students’ final paper-and-pencil test scores. WBL students achieved significantly higher scores. There was a significant difference (t = 3.3952; P = 0.0009) between WBL students and face-to-face students regarding the mean score of the examination (Table 1). The achievement scores of the WBL students ranged from 18 to 33 of 36 possible points, whereas the scores of the face-to-face students’ ranged from 7 to 32. The calculated effect size (d = 0.721) indicates that the mean of the treated group (WBL) is at the 76th percentile of the untreated (face-to-face) group, which stands for a "medium" effect size (Table 1).
On the other hand, there was no statistically significant difference between the two groups regarding the mean grade of the final general physiology examination (Table 1).
Descriptive Statistic Results of the WebCT Electronic Administration Data
Student usage of the site was generally high, but varied widely between individual students. A WBL student accessed the WebCT course environment on the average 491 ± 30 times, ranging from a minimum of 170 times to a maximum of 857 times. A WBL student spent on average 1,620 min (36 school hours) in the virtual environment. Students who achieved a high score in the final summative acid-base physiology examination completed on average two times as many formative assessment items and achieved a higher score for formative tests than students who did poorly.
Analysis of the Survey Questionnaires
Anonymous postcourse questionnaires were completed by students of both groups to elicit subject rankings. Student feedback about both types of PBL courses was positive. There was a statistically significant difference between the two groups regarding their overall satisfaction with the course (P < 0.0001) (Table 1). At the same time students in both groups had a very positive impression of the teacher, and there was no statistically significant difference between the two groups regarding their satisfaction with the quality of the tutor (Table 1). Furthermore, students in both groups liked that the teacher supported students’ learning and gave guidance and hints, but did not aim to give information. According to students, the teacher tried to make them think for themselves and enhance their critical thinking. The majority of students reported that the course material and assessment items available were useful to their learning. All students agreed that accessibility of educational materials, access to information, self-assessment, feedback, and communication improved in a blended WBL-PBL collaborative learning environment (Table 2).
|
DISCUSSION
Today there is an intense debate in the literature as to whether delivery media alone influence learning outcomes (24). Including our investigation, the majority of research on WBL focuses on its effectiveness compared with traditional classroom learning. According to a number of studies, there is a "no difference effect" in performance between students enrolled in the two environments (29). There are studies, including our data presented in this paper, which acknowledge that differences in media used can make a difference in student outcomes (12). Congruent with our findings, students in blended courses achieve better grades than students in traditional face-to-face courses or totally online courses (13). On the other hand, some other researchers found negative effects of e-learning (32). The practical significance of an effect is determined by its relative costs and benefits. For e-learning, they are difficult to quantify due to the large number of variables involved. In education, however, if it could be shown that making a relatively inexpensive change would raise academic achievement by an effect size of even as little as 0.1, then this could be a very significant improvement (10).
By combining PBL with collaborative Web-based distributed education, instructional designers can create active, vibrant learning environments that enhance student learning (9). The underlying presupposition is that a well-designed program must be based on sound pedagogical principles (41). Because learning involves two types of interaction critical in many types of learning, interaction with content and interpersonal interaction, we made efforts to facilitate both types of interaction throughout our course. Our "interactive module of acid-base balance" uses the Web as a source of knowledge exploration and provides exploratory problems to guide students to think and explore. Extensive research has shown that educational outcomes improve when students take a deep approach to learning in which the emphasis is on understanding subject material rather than rote learning (5). Virtual environments encourage students to explore a topic beyond the boundaries of given material, thus supporting the proactive and exploratory nature of learning that allows the student to become self-reliant (26). Also, design interface can have a large influence on how motivating a Web course program is (6). Our experience is in accord with these statements. In an attempt to explain this finding, it is interesting to point out that the main problem we faced when using online communication (chat) was how to move students from seeing the communication tool as an essentially social environment to seeing and using it as an educational tool as well. Thus the tutor faced an important and difficult role: to get students to focus on meaningful discussions and tasks by giving them guidance and hints. In our opinion, paradoxically, WBL students were sometimes more frustrated than face-to-face students, perhaps because, in the traditional environment, the tutor could work with them, empathize, and solve the problem before they reached that level of frustration. In other words, the tutor was more active and more helpful to them, whereas in a virtual environment, it was easier for the tutor to be "invisible," to sit back and let the students do their job. The frustration didn’t develop only from the technology. Rather, it developed as a consequence of the inability to ask questions of the professor in a face-to-face environment. It is likely that this led, paradoxically, to more involvement between and among peers. That this collaboration manifests itself in better tests scores is consistent with the findings of the collaborative learning literature (11). In addition, use of the collaborative interactive whiteboard tool can increase student engagement during the learning process (3). We believe that as much of the effectiveness differences can be referred to student collaboration as to the technology, itself.
Students rated overall satisfaction with the hybrid course as better than with the traditional course (Table 1). Better results in terms of student performance and satisfaction achieved through virtual interaction are associated with greater motivation and enjoyment by students and tutors (23). As described in a conceptual model developed by Sternberg (33), motivation drives metacognition, which, in turn, stimulates the development of thinking and learning skills (specifically, inquiry learning and application of skills; two scales derived from WBL activities). Thus increases in thinking and learning processes would result in increases in teacher-scored measures of student achievement.
By having students record progress toward their goals, the motivation to become an independent learner can be enhanced (20). In this sense, formative self-assessment with automated marking and immediate feedback can have an important impact on student motivation. The improvement discussed in this paper relates to students using Web applications that can be embedded into the assessment activities so that more interesting and challenging problems can be presented to students in an online format. Compared with the traditional PBL course, our blended WBL-PBL course was enriched with 26 self-tests and quizzes with immediate feedback. The use of immediate feedback, making students aware of what they do not know, increases understanding of the material (15).
In summary, there is no one way to teach any class, face-to-face, online, or blended. Student usage of computer-supported systems is far from being well understood, and, consequently, the evolution of technology and pedagogy for these systems will continue for some time before agreed understanding and standards will have emerged (27). Generally, blended learning environments are new educational media that can inspire great challenges and advantages for both teachers and students. They provide an opportunity to rethink our teaching and achieve learning goals that previously may have been beyond our reach (2). Thus the way in which online activities are designed and delivered can add value and make a positive difference.
GRANTS
The study was supported by the Croatian Academic Research Network.
Received for publication June 18, 2004. Accepted for publication November 22, 2004.
References
K. Interactive module of acid base balance in humans [Online]. [accessed at: http://webct.carnet.hr:8900/] (9 Jun 2004).
i F.
Are we sour? [in Croatian]. Zagreb, Croatia: University of Zagreb Medical School,1991
.
This article has been cited by other articles:
|
|
 |
|
  S. Martino and T. Odle New Instructional Technology Radiol. Technol., September 1, 2008; 80(1): 67 - 74. [Full Text] [PDF]
|
|
|
|
 |
|
 M. E. Simmons, X. B. Wu, S. L. Knight, and R. R. Lopez Assessing the Influence of Field- and GIS-based Inquiry on Student Attitude and Conceptual Knowledge in an Undergraduate Ecology Lab CBE Life Sci Educ, September 1, 2008; 7(3): 338 - 345. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
