The aim of this study was to determine whether problem-based writing with peer review (PW-PR) improves undergraduate student performance on physiology exams. Didactic lectures were replaced with assignments to give students practice explaining their reasoning while solving qualitative problems, thus transferring the responsibility for abstraction and generalization to the students. Performance on exam items about concepts taught using PW-PR was compared with performance on concepts taught using didactic lectures followed by group work. Calibrated Peer Review™, a Web-delivered program, was used to collect student essays and to manage anonymous peer review after students “passed” three calibration peer reviews. Results show that the students had difficulty relating concepts. Relationship errors were categorized as 1) problems recognizing levels of organization, 2) problems with cause/effect, and 3) overgeneralizations. For example, some described cells as molecules; others thought that vesicles transport materials through the extracellular fluid. With PW-PR, class discussion was used to confront and resolve such difficulties. Both multiple-choice and essay exam results were better with PW-PR instead of lecture.
- active learning
- elementary physiology
- writing to learn
Delivery of factual information is ineffective as a mode of instruction for some students. A broad body of science education research highlights this fact (4). Recent studies about college science teaching and learning support the view that active involvement by students improves learning (13, 15, 16, 19–21, 25). Even students who learn effectively from lectures and textbooks may do so because they reflect on the material presented, what it means, and how it fits or does not fit with their previous understanding. Good learners confront their difficulties and actively persist in trying to resolve conflicts (13). This degree of self-regulation for learning may not be common among students taking general education undergraduate physiology. Some students seem to approach physiology as a collection of isolated vocabulary and facts to be memorized. This paper shows that problem-based writing with peer review (PW-PR) may help undergraduate nonscience majors understand relationships in their physiology knowledge structure.
Writing about physiology problems forces students to explain their reasoning, thus revealing the cognitive structure of each student’s emerging understanding. Writing assignments show the instructor how students think about the concepts taught. A quick review of student essays can improve faculty judgment of beginning students’ factual knowledge and their ability to apply their knowledge, an important challenge for physiology instructors (22). At the same time, students detect and discriminate patterns as they work to find the language needed to organize and express their understanding. They are alerted to what must be noticed to learn from situations and to apply that learning to new contexts (10). According to Ausubel (2), both the substance and the organization of the content are principal factors influencing meaningful and lasting learning. On the basis of these theories and with the assumption that writing to solve physiology problems promotes active involvement, this study was designed to address the following question: Does peer review of writing assignments about conceptually challenging physiology problems (PW-PR) improve performance?
Results show that, when undergraduate nonscience majors write about problem-based learning assignments followed by anonymous peer review (PW-PR), they perform better than with didactic lectures followed by group work. Physiology problems about unifying principles were presented to students by use of a new, Web-delivered program for peer review of writing, Calibrated Peer Review™ (5). Problem-based writing is defined here as “working on an unsettling, puzzling, unsolved issue that needs to be resolved” (11). Students submitted essays about each problem, and then calibration essays were provided to give students practice as peer reviewers and to determine whether they were qualified to evaluate peer essays. After “calibration,” each student peer reviewed three anonymous essays from other students. The Web-delivered tools used to deliver these assignments can be viewed at http://cpr.molsci.ucla.edu for details. With PW-PR, students performed better on multiple choice test items and equal or better on essay test items.
This study was designed to replace a more traditional approach to instruction with PW-PR, not to supplement traditional didactic lectures. As a control for the experimental instruction, the instructional approach was alternated throughout the semester between experimental and control treatments. The author instructed both the experimental and control portions of the course. Results were obtained by comparing student performance on exams after PW-PR experimental instruction with their performance on another group of physiology principles taught using a more traditional instructional approach as described below. The independent variable in this study was the instructional approach: PW-PR assignments vs. didactic lecture/group work; student achievement was measured as the dependent variable by using both multiple choice and essay exam questions.
The experimental instruction used PW-PR with a new tool called Calibrated Peer Review. In addition to the writing prompts for six problems about unifying principles of physiology (see appendix), each assignment was designed by the author with assignment goals, source materials, guiding questions, and three sample responses to help students “ calibrate” their ability to peer review. Instead of didactic lectures, students learned about the concepts involved in the problems by use of resource materials such as visual aids, Internet links to PubMed research abstracts, and textbook assignments from a human physiology textbook (23).
Calibrated Peer Review was developed by Orville Chapman, a UCLA chemist, to structure assignments and manage the anonymous peer review process (5). The network tools make it possible for the instructor to view written student responses to identify issues for class discussions while the system manages the peer review process and generates student reports in a way that challenges the traditional top-down evaluation of science students’ written work by an instructor. Although PW-PR could be done as a paper-and-pencil task, the Calibrated Peer Review tools provide several unique advantages. First, Calibrated Peer Review employs texts that have been calibrated by the instructor to train students to evaluate peer responses in a process modeled after the procedures used to calibrate instruments with “standardized” trials. This aspect of Calibrated Peer Review provides the instructor with an opportunity to confront students with alternative perspectives or with examples of common learning difficulties. Second, the anonymity structured into Calibrated Peer Review is an important feature. Demands placed on students with language or writing difficulties authentically mirror the work environment.
The control “traditional” instruction consisted of didactic lectures followed by group work on physiology textbook problems that require Bloom’s higher-order thinking (Ref. 24, introduction). To control for the frequency of assessment, six multiple-choice quizzes were used to provide students with feedback on concepts taught using traditional instruction, just as peer review feedback on each writing assignment indicated progress on the PW-PR problems.
Major principles of physiology were addressed using either experimental or control instruction with an effort made by the researcher to avoid overlap and to assign topics of comparable complexity to each treatment group, as shown in Table 1. The concepts taught using PW-PR were not presented in lecture. Instead, the problem was presented first, and then students used the textbook and resources to figure out how to use physiology concepts to respond to the problem in short essay format, with the instructor serving as an additional resource as needed. The essays were graded through peer review, with scores generated electronically on the basis of the peer reviews. Individual feedback on the essays came from other students, not the instructor, but problems and conflicting ideas were discussed in class. Errors on quizzes after traditional instruction were also discussed.
Essay and multiple-choice exam items were matched to the topics taught using either experimental or control instruction before the items were graded. Two premed postbaccalaureate students confirmed the validity of the questions by independently reviewing each exam item. Exam questions on course content that could not be categorized exclusively according to the topics shown in Table 1 were not counted for this study. Results were blind scored and tabulated by a student assistant who did not know which questions corresponded to the experimental or which to the control instruction.
Biology 310, Human Physiology, is an introductory survey course that can fill the upper-division general education science requirement for nonscience majors. The prerequisite is one semester of college biology. The present study took place in the spring of 2000. Of 40 students enrolled that semester, five students were excluded from the study because they did not complete the assignments. Four of those excluded failed due to nonparticipation, whereas the fifth student was auditing the course. Over one-half of the students were there to fulfill degree requirements: 34% were kinesiology majors, 11% were premed (including nursing and dental), and 8% were social work majors. The remaining students were psychology (16%), speech, elementary education, anthropology, and undeclared majors. The students were sophomores (21%), juniors (29%), seniors (37%), and graduate students (13%). Of the students in the study, 37% were from families that primarily speak a language other than English (mainly Spanish), and 14% were themselves more fluent in a language other than English. Many students had difficulty expressing themselves in essays, and some used a language center and writing services that were available to help all students improve their writing.
Data Collection and Analysis
All multiple-choice items were selected from those provided by the textbook publisher, and a main source for essay questions was the problems at the end of each textbook chapter (23). Some essay questions were instructor-developed items about topics addressed through lecture or the problem-based assignments. One-half of the essay items were simplified versions of either PW-PR assignments or assigned practice problems from the textbook, and the other half were new problems that required application of basic physiology concepts to arrive at a solution. Reported results are based on subscores from items on the midterm and final exams, with items involving overlap between the two treatments not counted. Only multiple-choice and essay questions that assess learning of topics taught exclusively by the experimental or the traditional approach were counted. Scores for each subset of items matched to the instructional approach were expressed as a percentage of the highest possible subset score. The mean percentage ± SE for each subset score was calculated. A Student’s paired t-test was used to compare individual student’s results on each subset of test items for topics taught using PW-PR vs. traditional instruction.
Essay responses to PW-PR problems were analyzed individually to identify conceptual difficulties and then together to find patterns in the kinds of conceptual difficulties detected. Emerging patterns were tested against subsequent essays to find ideas that were consistent with the conceptual problems apparent in the numerous essays reviewed.
To find out whether students thought PW-PR was a good way to understand important concepts in physiology, students were asked to describe two important concepts that were not assessed on the essay portion of the final exam and then to describe what they did that worked best to help them understand those concepts.
Table 2 presents the mean percentage scores for each subset of test items for topics taught using traditional instruction vs. PW-PR. For multiple-choice items, subscores were higher for topics taught using PW-PR compared with traditional instruction on both the midterm and final exams (P < 0.001, paired t-test, n = 35). Results on multiple-choice items were disappointing, with a mean subset score of 69.1 ± 1.9% for topics taught using PW-PR compared with 63.0 ± 2.2% for topics taught using traditional instruction on the midterm, with some improvement in the final exam mean scores of 76.9 ± 2.1% for topics taught using PW-PR compared with 65.1 ± 3.1% for topics taught using traditional instruction. Similarly, performance on the midterm essay questions was higher for topics taught using PW-PR compared with traditional instruction (P < 0.001, paired t-test, n = 35), with a mean essay subset score of 82.7 ± 2.1% for topics taught using PW-PR vs. 75.0 ± 2.6% for topics taught using traditional instruction. Results for essays on the final exam showed a mean of 81.2 ± 3.0% for topics taught using PW-PR compared with 76.9 ± 2.7% for topics taught using traditional instruction, although the difference in these means was not significant (P > 0.05, paired t-test, n = 35).
Because students from families that primarily speak a language other then English might be at a disadvantage for learning through PW-PR, their essay scores were analyzed. The mean cumulative essay score of 81.8 ± 4.5% for topics taught using PW-PR vs. 76.6 ± 3.1% for topics taught using traditional instruction was significantly different for students from families that primarily speak a language other than English (P < 0.05, paired t-test, n = 13). In comparison, for the subset of students from primarily English-speaking families, the mean cumulative essay score of 82.2 ± 3.0% for topics taught using PW-PR was not significantly different from the mean score of 77.5 ± 3.1% for topics taught using traditional instruction (P > 0.05, paired t-test, n = 22).
Subset multiple-choice and essay scores for individual students were plotted against the final course grade to show whether top-scoring or low-scoring students performed better on one type of test item after either type of instruction. Regression analyses of scatter plots of subset scores vs. final course percentage grades are shown in Fig. 1. In general, low-scoring students, average students, and top-scoring students benefited equally from instruction with PW-PR on both the midterm exam essay and multiple-choice subset scores. Surprisingly, on the final exam, low-scoring students performed better on the subset of multiple-choice questions for content taught with PW-PR, but they did not show a similar benefit in their performance on essay questions about topics taught with PW-PR.
Essays written by students in response to the first PW-PR problem revealed surprising difficulties in their thinking about vesicles, membrane-bound compartments within cells. At least 5 of the 35 students in this study thought that vesicles store and transport molecules to and from different cells, i.e., that vesicles transport materials through the extracellular fluid, as exemplified by the following quotes from their essays (emphasis mine):
“Cells are responsible for bringing nutrients to other parts of the body through vesicles.”
“Through the fluid cells can communicate and send things to each other through pouches called vesicles.”
“Cells also use vesicles to transport proteins to other parts of the body.”
“Cells are surrounded by extracellular fluid in which vesicles store and transport molecules to and from different cells.”
“Within organ systems vesicles are used for storage and transport.”
Errors in the written essays generated by students in response to the six PW-PR problems could often be traced to erroneous relationships between concepts. Relationship errors were grouped and are exemplified by quotes from student essays as follows:
Levels of organization.
Difficulty assigning components to an appropriate level of organization (atom, molecule, organelle, cell, tissue, organ, system) occurred in nearly one-half of the essay responses:
“Cells are molecules that are separated from the external environment and each other by membranes.”
Students sometimes invented an inappropriate cause and effect relationship for two ideas used in conjunction:
“[Steroid] receptors are located inside the cell membrane because it is easier for [hormones] to bind on the receptor in the nucleus.”
Overgeneralization of a relationship.
Students used principles that work in one situation to make a generalization that is not always valid:
“IGF-1 cannot enter the cell and needs testosterone to work.”
Only rarely were the problems identified in student essay responses due to incorrect use of vocabulary:
“IGF-1 is a messenger hormone. The second messenger breaks down ATP into cyclic AMP.”
“Next, the process of transcription occurs in which amino acids transcribe into proteins based on a specific genetic code.”
These and other erroneous quotes were used to generate class discussions during the peer review process for the experimental instruction.
Even though errors were apparent in the PW-PR essay responses, students perceived the experimental instruction to be an effective tool for learning physiology. When asked to describe how they learned an important concept in physiology, most students listed more than one activity: 39.3% of students listed PW-PR, 28.6% listed some form of visualization (textbook diagrams or video images), 25% listed graphic organizers like matrices or concept maps, 21.4% listed case studies or personal experience, 21.4% listed talk with a study group, 10.7% described an experiment or simulation (even in this nonlab class), and 7.1% provided a mathematical equation.
The main finding of this study is that PW-PR improved undergraduate nonscience majors’ performance on physiology tests. Paired t-test comparisons of means show that performance was better on objective multiple-choice and equal or better on essay test items for concepts taught using PW-PR. This result suggests that both measures were valid and that students really learned better with the experimental instruction. The results of this study support pedagogical theories about writing (2, 3, 10) and active learning (1, 3, 11, 14, 15, 19, 21) as effective organizational and integrative tools for the purpose of meaningful learning. In fact, using the 90, 80, 70, and 60% cut-offs for A, B, C, and D grades, results on assessments after experimental instruction were generally one letter grade higher than results on content taught using traditional didactic lecture with group work.
Discussion of conflicting ideas from student essays during the peer review process was more engaging than discussion of errors on six multiple-choice quizzes. The essays and subsequent discussions sometimes revealed surprising wrong ideas such as the view that vesicles transport materials through the extracellular fluid. Whether this erroneous view of transport is generalized beyond the student population addressed here remains to be determined. One student pointed out that this view of transport vesicles may come from the media, because a television commercial for Zoloft shows an animation where the antidepressant alters the number of neurotransmitter bubbles leaving a cell and flowing across the synaptic cleft. The idea that vesicles transport materials outside of cells may be quite generalized. Having students write about problem-based assignments reveals such mistakes so that the instructor can find strategies to help students confront and correct these difficulties.
Traditional didactic lectures provide content with clarity and organization designed to help students avoid mistakes. But analysis of student essays shows that some students have ideas that differ from the instructor’s understanding of how concepts are related, and these differences should be addressed to enhance student learning. McDermott (13) reports that one way to obtain “the necessary intellectual commitment from students is to generate a conceptual conflict and to require them to resolve it.” The body of science education research that has accumulated over the past 40 years supports this view (9, 12, 14, 17). It may be that PW-PR promotes reflective learning by unveiling underlying conceptual difficulties and causing students to confront erroneous ideas that hinder performance. During this study, the instructor used the problems identified in student essays as interesting issues for class discussion. Discussion helped the instructor find what challenged the learners. Participants in this study struggled with relationships that involved levels of organization, causes/effects, and other logical relationships where the extent and limits must be recognized as students apply or generalize new science knowledge. More work is needed to find out how prevalent such relationship difficulties are among undergraduate nonscience majors learning physiology and whether frequent writing with peer review reduces the incidence of these problems. This report represents a step toward research to find instruction that works to expose and help students confront and resolve their conceptual problems learning physiology.
Another finding is that students in this study did not require didactic lectures as a source of information to learn about the principles of physiology. The textbook, visual aids, and research abstracts were used in place of didactic lectures to provide the information needed by students to address problems and explain their reasoning. The suggestion from a study by Richardson and Birge (21) that problem-based learning should be used to supplement rather than replace didactic lecture should be reexamined. Richardson and Birge found that students who tackled problems in a supplemental course scored nearly a letter grade higher than students who did not supplement their learning from didactic lectures with group work on physiology problems. Those results are comparable to results in the present study in a course with strikingly similar undergraduate students. Characteristics of the student population may be an important variable. The experimental instruction used here may be most beneficial for students who are less proficient at questioning their own comprehension, confronting their difficulties, and persisting until they resolve them. Writing about problem-based assignments leaves the learner in control of time, selection of resources, and their degree of engagement, factors that may have helped the subset 37% of students learning physiology in English as their second language who performed significantly better on essays about topics taught using PW-PR. Some students may need the multiple opportunities for learning that occur 1) while discussing problems, 2) during writing, 3) while reading what others write, and 4) through self-assessment. It could be argued that the use of PW-PR deserves serious consideration if it is more effective for students who are motivated but not yet ready for careers in medical sciences.
Although all students showed benefit on the midterm, the fact that low-scoring students in the present study failed to show the same benefit as higher-scoring students in their performance on final exam essay questions about topics taught using PW-PR compared with traditional instruction presents some concerns (Fig. 1D). The essays produced by the low-scoring students provide a rich source of data that suggest a reason for the low scores: complex or integrative problems may place low-scoring students at a disadvantage. The fourth and sixth PW-PR assignments were written to be challenging because the students performed quite well in the first three assignments. (See problems 4 and 6 in the appendix.) These two complex problems were designed to expose students to scientific advances that are raising difficult questions, with the added goal of revealing the value of scientific contributions to society. The students who were unable to combine and synthesize reports from recent research publications may have lagged in their development of reasoning (9). On the other hand, assignments that were integrative also required review and application of previous course content. Confusion detected in some low-scoring student essays is not expected to motivate them to become lifelong learners of physiology. Indeed, such results may reinforce a belief that physiology is too difficult for some. Future studies should address how to provide an organizational framework to assist low-scoring students in combining information from a variety of resources to alleviate some of their confusion. Research is needed to find out whether simple problems are more effective at improving conceptual performance, as measured by essay exams, than are complex problems with several variables such as the fourth and sixth problems used in this study.
Views on how students learn might be expected to move us away from the information delivery mode of instruction that is the norm at most colleges. Learning in physiology is commonly assessed through written essays, a practice that encourages students to select unifying concepts and to organize and integrate ideas across levels and in a variety of systems as they study. However, dismal student essay responses are not unusual. If we accept Ausubel’s perspective (2), poor essays may be due to our failure to “arrange practice sessions.” Essay assessment without practice is unfair, and the experimental instruction investigated in this report may increase opportunities for underrepresented groups in science by providing practice to make assessment more equitable. Currently, Calibrated Peer Review is available for use by faculty and students free of charge at http://cpr.molsci.ucla.edu. The assignments used here can be shared, or faculty may write new assignments to optimize instruction and maximize learning.
The students in this study described PW-PR as a good tool for learning. Other collaborative and investigative tools for college science teaching were also used for both experimental and control instruction in this study. The cooperative group activities (8), concept maps (18, 23), case studies or problem-based activities (1, 11, 15), and on-line simulations or visualizations (6, 7) that were used in this study are gaining recognition as effective modes of instruction. However, when asked in an open question what worked best to help them learn important concepts in physiology, PW-PR was listed as an effective tool more often than any other tool.
From the results of this study, we can conclude that PW-PR improves performance in physiology regardless of whether performance is measured by multiple-choice or essay exam items, even when used to replace rather than supplement didactic lectures. With peer review of writing about problems designed to engage undergraduate nonscience majors to think critically about physiology, student performance was equal to or better than performance after traditional instruction. The favorable results may be a product of the work students complete when writing about their thinking, or perhaps students did better because PW-PR made it possible for them to confront and resolve difficulties they encountered relating concepts. Whether assignments that unveil such difficulties also work to improve achievement for science majors, medical students, or other student populations remains to be determined. Measures of whether this experimental approach to instruction improves attitudes or motivation to become lifelong learners or supporters of research in physiology are also needed.
Writing Prompts Used for PW-PR Assignments
Problem 1 (Cellular transport and epithelial function):
The first half of this course on Human Physiology deals with equilibrium, energy, pressures, and flows. You must understand the compartments in your body before you can answer the key question, “How does your body control the movement of ’stuff’ so that it goes where it’s needed? ” Write a paragraph describing various walls and compartments at different levels of organization within the human body. Use the figures in your textbook to help you organize your paragraph. For full credit, demonstrate your understanding by using these terms: cell membrane, cell junctions, epithelia, extracellular fluid, organ system, phospholipid, and vesicles.
Problem 2 (Resistance and bulk flows):
Students in a nursing program were assigned the task to model a health disability for one day. Two students decided to model asthma. Both pinched their noses shut with a nose clip. One decided to breathe only through a straw from McDonald’s all day. That straw was narrow and about 15 inches long (38 cm long by 0.7 cm in diameter). The other breathed through a 6 foot long vacuum cleaner hose (2 m long by 3 cm diameter).
Compare and contrast each model with the effects of asthma on ventilation (similarities, differences).
Compare and contrast each model with the physiological consequences of asthma (similarities, differences).
Will these students be able to practice each model of asthma for an entire day? Explain.
Use these terms in your answer: bulk flow, dead space, partial pressure gradient, bronchoconstriction, airway resistance, hypercapnia, hypoxia.
Problem 3 (Endocrine function 1):
Would you permit your spouse to use testosterone cream? To answer such questions in the future as new discoveries are made, you should be able to evaluate the knowledge claims presented in research reports. A knowledge claim is a conclusion supported by valid data. Identify the main knowledge claim from a testosterone research study by Urban et al., AJP 269: 820–6 (1995). Compare and contrast the peptide hormones and steroid hormones discussed. Explain the role of cell membranes in determining the location and mechanism of the peptide and steroid hormone receptors. Analyze this primary research article to arrive at valid conclusions and apply the conclusion along with information from your textbook in recommending whether you would permit the use of testosterone cream by your spouse. (State whether your spouse is male or female.)
Problem 4 (Endocrine function 2):
Explain to social workers the significance of the glucocorticoid system for their work by evaluating the biomedical research evidence relevant to the following claim: “For the health of children, social relationships may be more important than socioeconomic status.” Decide whether biomedical research supports this value claim, and cite studies with knowledge claims to justify your view. Qualify the validity of the studies by describing some limitations and suggest what investigators should do to make their knowledge claim more applicable. Then, explain the role of the glucocorticoid system. Be sure to describe the role of the hypothalamic-pituitary-adrenal axis in response to stress. Compare and contrast the data from primary research paper abstracts. Describe a hormone response to a variety of stress stimuli that has been reported in several studies. Describe other hormone responses to stress stimuli that are reported in some studies, but not others. Describe relationships between these hormone responses. Conclude with a therapy you might develop for a child, citing studies to justify your recommendations. Finally, identify unresolved issues that should be addressed in future research studies to confirm or refute your suggested therapy.
Problem 5 (Excitatory and inhibitory postsynaptic potentials and lateral inhibition):
Trace the signal transduction pathways and the steps from reception of the external light signal in the eye through transmission of the information, to interpretation of the brightness of the light. Your explanation should show why the brightness of a wall looks different when viewed through an opaque tube compared with what you see without the tube. To accomplish this, you will need to describe what happens to the signal from reception to transmission through the ganglion cells and then to the optic nerve.
Problem 6 (Integrative physiology: genes associated with heart failure):
At what age will you die? What do you need to know about the traits in your family to make decisions that will lead to a long, quality life? Listed here are nine hereditary trait variants that, when defective, have been linked to a genetic risk of heart failure in some families. Is there a high risk of heart failure in your family and, if so, how should that condition be treated? The mechanism for the hereditary cause may be an important factor in deciding how to prevent or treat heart disease.
Review functions of each of these molecules. Discuss how defective protein function resulting from a defective gene might lead to heart failure in each case. Your job will be easier if you group together traits with similar functional consequences.
epithelial sodium channel
myosin heavy chain
voltage-gated potassium channel
Write in paragraph form your interpretation of how these defective traits might contribute to heart failure. Support and justify each interpretation using citations from your textbook or the resource abstracts. Then recommend treatments and/or future research that you would support with tax dollars to improve the value and quality of life for families with these hereditary defects.
Thanks are due to Terence Gan and Miyoung Kim for validating questions and for assistance with data collection, to Drs. Arlene Russell and Barbara L. Gonzalez for discussions about the benefits of Calibrated Peer Review™, to Tim Su for managing access to the Calibrated Peer Review server, and to Dr. Kathryn Dickson for helpful suggestions.
Address for reprint requests and other correspondence: N. J. Pelaez, Dept. of Biological Science, College of Natural Science and Mathematics, California State University, Fullerton, 800 N. State College Bl., Fullerton, CA 92834-6850 (E-mail:.)
- © 2002 American Physiological Society