| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
TEACHING IN THE LABORATORY
Department of Biology, Swarthmore College, Swarthmore, Pennsylvania
Address for reprint requests and other correspondence: S. M. Hiebert, Dept. of Biology, Swarthmore College, Swarthmore, PA 19081-1390 (e-mail: shieber1{at}swarthmore.edu)
 |
Abstract |
|---|
 TOP Abstract IntroductionREFERENCES |
|---|
Key words: experimental design; multiple hypotheses; teaching
|
Introduction |
|---|
|
TOPAbstractIntroductionREFERENCES |
|---|
Why are the outcomes and interpretations crucial components of an effective protocol? In his often-cited paper on the subject of strong inference, Platt (10) outlined a series of steps that should be followed systematically in scientific investigation:
Platt was not the first to propose many of the ideas in his 1964 paper, but he gave voice to them in a colorful way that brought them to the attention of, and subsequently influenced, researchers in a variety of fields (3). Although some have argued with some of Platt's points (3), it is generally accepted that the practices of identifying multiple hypotheses (when available) and planning data collection with statistical analysis and interpretation in mind will result in a higher quality of research, even though the words "strong inference" are not always applied to this method (14). Investigations planned in this way are characterized by efficient use of time, money, and animals (4–6, 14) because they reduce the chances of drawing incorrect conclusions (4, 5, 14), increase the statistical power derived from a given sample size (4, 6, 11), and ensure that appropriate conclusions can logically be drawn from the data (14). Good strong-inference protocols are an important element of successful grant writing in experimental sciences. Why, then, would we not want to make this tool a part of every student's scientific training? We do not need to wait until graduate school to introduce it (14). My purpose is to encourage instructors to incorporate strong-inference protocol writing into the undergraduate curriculum as early as possible.
A Strong-Inference Protocol for a Physiology Experiment
The strong-inference protocol is a concise document that summarizes why and how an experiment will be done, describes how the data will be analyzed and interpreted, and lists potential difficulties in experimental design or in drawing logical conclusions from the data. The general form of the strong-inference protocol is applicable to any experiment. Below, a general description of each section of the protocol is provided. In the Appendix, more specific content is described for the case of a student-designed experiment described in the companion article "Are chicken embryos endotherms or ectotherms?" (7). In this experiment, students investigated the thermoregulatory mode of chicken embryos by measuring respiration rates (
O2 ) at different ambient temperatures and comparing their results with the signature relations between temperature and metabolism in endotherms and ectotherms.
Sections of the Protocol
Introduction and background.
In one or two paragraphs, this section explains why the study is interesting and how it relates to existing knowledge. Students are asked to cite one to five published papers in this section. Unlike the introduction to a scientific paper, it is not meant to be an exhaustive review. This section should also explain how the data that will be collected relate to the physiological variables of interest and to the hypotheses being addressed.
Methods.
This section describes the experimental procedures, including an explanation of controls, a timetable of measurements, and salient details of the protocol that are not found in the laboratory manual because they are features of the students' own experimental design. I encourage students to omit standard details of the procedure such as how to use the equipment by referring instead to the appropriate sections of the laboratory manual. In my experience, students do not respond favorably to tasks that seem like busywork; for this reason, requiring them to copy procedures that are already printed in the manual tends to disengage students from the intellectually important features of the strong-inference protocol.
Data analysis.
In a few sentences, students are asked to name the statistical test that they will use and exactly which groups of measurements will be used in the analysis. A statement such as "We will use a t-test to analyze our results" is inadequate because it does not provide evidence that the student understands how the test will be applied.
Outcomes and interpretations.
This section contains one entry for each possible outcome of the experiment, followed by a brief description of what the student could conclude if this outcome occurred. One of the greatest conceptual difficulties students have with this section is distinguishing between an outcome and an interpretation. Outcomes are summaries of results, whereas interpretations represent the way in which the results address the question that the experiment was designed to answer. Students are encouraged to mention how probable they consider a particular outcome or interpretation and to provide evidence to support their assessment from the literature or from a priori reasoning. They may also briefly suggest further tests that might resolve ambiguities or further corroborate the result.
Caveats.
This section should include a brief description of untested assumptions and identify a study's shortcomings. Although students are often hesitant to expose any weaknesses in their design, the ability to identify caveats indicates clarity of thought. Few experiments are perfect.
Classroom Implementation
In the laboratory section of my intermediate-level Animal Physiology course, students participate in two 3-wk modules, in each of which the 12 members of the laboratory section design, as a group, an experiment they will perform together to answer a physiological question that I pose. These two 3-wk modules are followed by a 7-wk independent project, in which students work in small groups to address a question of their own choosing. At the conclusion of each module or project, students are required to write a laboratory report in the style of a scientific paper.
In the first year that I incorporated the strong-inference protocol into the laboratory for this course, I discovered that students needed guided practice before they could produce an effective strong-inference protocol on their own. This was especially evident in the Data analysis and Outcomes and interpretations sections, even after we had discussed the experimental design together in depth (but without specific reference to how the different elements of the design would be represented in the strong-inference protocol). A method that produced much better results in subsequent years was to design the experiment in the first module as a class and then talk very specifically about how this experiment would be addressed in each section of the protocol. In this discussion, we considered wording that might be appropriate for each section. The significantly higher scores that students earned on the protocol for the first module in the next 4 yr (17.7 ± 0.3 compared with 15.2 ± 0.2 out of a total possible score of 20 in the first year) are consistent with this interpretation (one-way ANOVA, F4,111 = 13.2, P < 0.0001).
For the second module, we design the experiment together but I leave to students the task of figuring out the essential content of each section of the protocol as it applies to this experiment. For the independent project, students design the experiment and write the protocol themselves after discussing their ideas with the instructor. Protocols are graded on an increasingly stringent rubric as the semester progresses, such that a particular kind of mistake on the first protocol might elicit a comment but not a point deduction, whereas the same error on a later protocol would incur a point penalty. Thus even a slightly better score later in the semester reflects a substantial improvement. Even with this rising scale of expectations and with progressively less instruction, protocol scores (out of a total of 20 possible points) increase by an average of 0.5 points with each successive protocol (repeated-measures ANOVA, F2,179 = 26.3, P < 0.0001). This progression suggests that students can learn and are later able to apply the general form of the strong-inference protocol to specific experiments, with the proviso that they benefit greatly in the early stages from seeing a specific example on which to model their own protocols.
Aside from technical questions on how to use the statistical software, students in the Animal Physiology course almost never ask questions about data analysis and logical interpretation after they have written their strong-inference protocol. Rather, our discussions regarding interpretation focus almost exclusively on the larger questions of why chicken embryos might be ectothermic and what implications ectothermy has for the life and energy economy of the embryo and the incubating parent. The strong-inference protocol provides a basic outline of the contents of each section of their laboratory report, beginning with the introduction and continuing with the methods, results, and the first paragraph or so of the discussion. Having the protocol in hand as a road map for the laboratory report appears to have the added advantage of reducing the anxiety associated with having to produce the entire laboratory report at once. If, after they collect their data, students forget the reasoning behind the experiment or need to be reminded of how they will use their data, they have only to consult their own protocol.
Adding a data analysis and an outcomes and interpretations section to student protocols means that instructors will need to read and provide feedback on a slightly longer document. However, in my opinion, the ultimate benefit to students weighed against the relatively small increase in an instructor's reading time gives this section one of the best per-word pedagogical values in the laboratory classroom.
Appendix: Specific Content for the Chicken Embryo Metabolism Experiment
Introduction and background.
A chicken embryo will eventually grow into an endothermic adult and must necessarily start life as an ectothermic zygote, but the point at which the transition takes place is intuitively ambiguous and could be argued either way from first principles. To put the methods into context, this section must explain how endotherms and ectotherms differ in their metabolic responses to environmental temperature and must state, in general terms, how this difference will be used in the experiment to answer the question of whether chicken embryos are ectotherms or endotherms (see Fig. 1 in Ref. 7). Specifically, two features of the relation between temperature and metabolic rate differ between endotherms and ectotherms: the shape of the curve and the range of absolute values for O2 (an index of metabolic rate), which is substantially higher overall in endotherms than in ectotherms. Finally, students need to explain how what they will measure (O2) is related to metabolic rate, the physiological function that they are investigating in this experiment (see Ref. 7).
Methods.
This section should include details that could vary from experiment to experiment, such as the temperatures at which measurements will be made, where and for how long eggs will be placed to equilibrate to the temperature treatments, the volume of the test chamber (respirometer), and the duration of each measurement period.
Data analysis.
The content of this section will, of course, vary with the experimental design that students choose. It might read as follows: "A two-sample t-test will be used to compare O2 between experimental and control eggs after a 90-min equilibration at 23 and 38°C, respectively."
Outcomes and interpretations.
For an experiment in which O2 is measured at 23 and 38°C, there are three possible outcomes.
OUTCOME 1.
O2 for eggs maintained at 38°C is significantly greater than that for eggs incubated at 23°C.
These results suggest that the embryos are ectothermic, because O2 is directly proportional to ambient temperature. A similar outcome could be obtained if all three of the following conditions are true: 1) the embryos are endothermic, 2) 38°C lies above the thermal neutral zone, and 3) 23°C lies within or just below the thermal neutral zone (see Fig. 4 in Ref. 7). However, this interpretation is considered highly unlikely because 38°C is the normal incubation temperature provided by the hen. If the embryos were endothermic with a body temperature set point of <38°C (condition 2), incubation by the hen would cause the developing chicks to expend energy to cool themselves to the body temperature set point. Because this would consume energy otherwise available to fuel development, an incubation temperature greater than the body temperature set point is unlikely to be favored by natural selection.
OUTCOME 2.
O2 of eggs maintained at 38°C is significantly less than the O2 of eggs incubated at 23°C.
These results would suggest that the embryos are endothermic, because O2 is inversely proportional to ambient temperature (see Fig. 1 in Ref. 7).
OUTCOME 3.
O2 of eggs maintained at 38°C is not significantly different from the O2 of eggs incubated at 23°C.
Because preliminary measurements of egg cooling show that the equilibration times we chose are adequate to cool an ectothermic embryo to 23°C, it is unlikely that this result could be due to a failure of the embryos inside the eggs to cool to 23°C if they are ectothermic. A second possibility is that the embryo is endothermic and that 23 and 38°C both lie within the thermal neutral zone (Fig. 3 in Ref. 7), but this is considered unlikely because the thermal neutral zones of similarly sized endotherms (
16 g on incubation day 16) typically span a much smaller range of ambient temperatures (see Fig. 1 in Ref. 7). A third possibility is that a 14°C temperature difference is insufficient to provoke a detectable change in O2. This is also considered unlikely because the typical Q10 values for aerobic metabolism predict that a two- to threefold increase in metabolic rate should accompany a change in body temperature of 10°C (1) in an ectotherm. A fourth possibility is that there is so much individual variation in O2 that sample sizes are too small to detect a statistically significant effect. In this case, the experiment could be repeated with additional eggs. A fifth possibility is that the embryos are only partially endothermic. In this scenario, the embryos have some capacity to slow the rate of cooling but not enough to cause an increase in metabolic rate (7). Further experiments in which younger, prethermoregulatory embryos are tested or in which embryos are subjected to more prolonged bouts of cooling could address this possibility (13).
Caveats.
First, the conclusions apply only to embryos of the age actually tested in the experiment. Second, we must assume that all embryos are healthy and at the same developmental age and that the vendor shipped embryos at the age stipulated in the order. (On one occasion, we received embryos that were 6 days old rather than 16 days old, a fact discovered when a student accidentally dropped and broke one of the eggs.) Third, the choice of temperatures reflects the reasoning that if the chick were an endotherm, the normal incubation temperature of 38°C would likely lie within the thermal neutral zone and the lower temperature (23°C ) would likely be less than the lower critical temperature, which is the lower limit of the thermal neutral zone. If this assumption is incorrect, an endothermic embryo could show a positive relation between temperature and O2 that might be interpreted as an ectothermic response. However, the range of values measured for O2, typically much higher in endotherms than in ectotherms, should prevent such a misinterpretation (see Fig. 1 in Ref. 7).
|
Acknowledgments |
|---|
Received for publication May 13, 2006. Accepted for publication October 13, 2006.
|
REFERENCES |
|---|
|
TOPAbstractIntroductionREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  S. M. Hiebert Teaching simple experimental design to undergraduates: do your students understand the basics? Advan Physiol Educ, March 1, 2007; 31(1): 82 - 92. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. M. Hiebert and J. Noveral Are chicken embryos endotherms or ectotherms? A laboratory exercise integrating concepts in thermoregulation and metabolism Advan Physiol Educ, March 1, 2007; 31(1): 97 - 109. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
