|
|
||||||||
HOW WE TEACH
Educational Research Unit, School of Biomedical Sciences, The University of Queensland, Brisbane, Queensland, Australia
Address for reprint requests and other correspondence: R. Moni, Educational Research Unit, School of Biomedical Sciences, The Univ. of Queensland, St. Lucia, 4072 Brisbane, Queensland, Australia (e-mail: r.moni{at}uq.edu.au)
| Abstract |
|---|
|
|
|---|
Key words: media; journalism; nonprofessional audience; scientific literacy; survey
| Introduction |
|---|
|
|
|---|
The School of Biomedical Sciences at The University of Queensland strongly advocates the acquisition of effective communication skills by their graduates. The final-year course BIOM3006 (Human Physiology and Pharmacology in Disease) has learning goals that specifically align with the development of 1) communication skills appropriate for entry into a science-based workplace, 2) the ability to chose and use the most effective strategies in communicating to the community, and 3) appreciation of the value of science to our culture.
However, when faced with marking scientific reports, many of our faculty members have claimed that "the students can't write." This frustration is frequently echoed by employers who value effective communication skills of their employees in the workplace. A recent analysis of essays in BIOM3006 from previous years revealed that most students had a grasp on the mechanics of writing, e.g., grammar, punctuation, and sentence construction. Thus we hypothesised that the key challenge lay more with students lacking formal training in writing and, specifically, in genres of writing intended to clearly inform nonprofessional readers of technical concepts. We then designed a novel assessment task: to introduce BIOM3006 students to writing for a media context, where the required level of understanding includes "the scientific terms and constructs sufficient to read a daily newspaper or magazine" (15). An Opinion Editorial (Op-Ed) and a subsequent Peer Review were developed as a two-part assessment task in physiology. The Op-Ed was selected because it requires clear and concise writing of complex biomedical research to inform readers without relevant technical education: broadly, "nonprofessional audiences." The Peer Review (in which each student formally reviews and grades one Op-Ed from the same cohort) was included to enhance opportunities for students to reflect on the effectiveness and quality of their own writing and the writing of their peers.
This paper reports the findings of an action learning project to address three research questions:
| CONTEXT AND RESEARCH METHODS |
|---|
|
|
|---|
The Media Role model of science communication to nonprofessional audiences.
For the purpose of this study, we identified the professional scientific community, science students, the media, and nonprofessional audiences as the key groups most relevant to our role as educators (Fig. 1A). The formal academic curriculum links faculty members (who are professional scientists) to students. The curriculum, characterized by dense and rich scientific content, is delivered through frequent and reciprocal communicative interactions of teaching and learning. Contrasting this, in the community, direct communication between scientists and nonprofessional audiences (e.g., via public forums) is relatively infrequent and usually is not dense in technical content. Nonprofessional audiences learn the most about science via the mass media, e.g., via Op-Ed newspaper articles (2). The media act as community gatekeepers of new scientific content by interpreting, paraphrasing, reporting, and commenting on the outputs of the professional science community.
|
Framework for the action learning project.
Kolb's experiential learning cycle (10) was adopted to frame the learning interactions between faculty members and students' active learning. For BIOM3006 (Fig. 2), this cycle describes how incremental improvements to writing tasks can be made. In this study, faculty members reflected on the features of effective biomedical writing and then designed explicit learning and teaching activities around the Op-Ed and Peer Review. These were implemented in lectures. Students worked individually to 1) write an Op-Ed and then 2) write a Peer Review of another student's Op-Ed. Students' understandings and perceptions of the task were obtained from the students in pretask and posttask surveys. The findings from 2004 generated the next (2005) cycle of experience, reflection, and action as part of the ongoing improvement of the course. In the following sections, each element of the study is described.
|
Support for students.
Students were assisted in several ways to complete the writing tasks. First, a 2-h lecture was delivered by an academic colleague researching the role of "junk DNA." Second, an experienced journalist from the School of Journalism (The University of Queensland) delivered a 2-h lecture on how the professional media communicate science to the public and on the guidelines for writing an Op-Ed piece. Third, the assessment rubrics for each task were explicitly taught to students by the second author. Fourth, articles on junk DNA from recent media releases and popular science journals and a transcript from a recent television special on junk DNA (the Australian Broadcast Commission program "Catalyst") were made available on the course website. These served as exemplars of writing for the students. Finally, after students completed the Op-Eds, they were required to review the Op-Ed of one other student and then write a half-page report (5% of total course).
Assessment.
The Op-Ed submissions (10% of total course) were marked by the authors using comprehensive assessment rubrics that explicitly linked criteria to standards and grades (Table 1). Students then used the same rubric to mark one Op-Ed randomly assigned from their cohort and wrote a 150-word review (Peer Review, representing 5% of total course), which was graded by faculty members using another rubric (Table 2). After all students received their final mark and feedback, top-ranked students were also given feedback by Prof. Mattick and two university journalists. The Op-Eds of two students were published in the university's corporate newsletter, UQ News.
|
|
Prewriting task survey: your initial ideas about science communication.
The survey consisted of eight items presented in the RESULTS. Respondents were asked to circle one number on a Likert rating scale (13) from 1 to 5, where 1 is very strongly disagree, 2 is strongly disagree, 3 is neither agree nor disagree, 4 is strongly agree, and 5 is very strongly agree. The results of the survey were subsequently discussed by an experienced professional journalist in a lecture where students were explicitly taught how to write an Op-Ed.
Postwriting task survey: Op-Ed and your ideas about science communication.
After completing the Op-Ed, students were again surveyed to determine whether they considered the task to be useful and whether they perceived it had developed their skills and ideas about science communication. This survey consisted of six items presented in the RESULTS. Items were designed to measure 1) to what extent did writing the Op-Ed give the students a deeper appreciation of the role of media and the difficulty in communicating science to the public, 2) if writing the Op-Ed was challenging and valuable to the student, and 3) if the students perceived any changes in their own communication skills. The survey form also allowed for additional comments by students.
Analysis of survey results.
Survey analysis was framed around recent guidelines of the American Physiology Society (3). Descriptive statistics of each item of the survey data were calculated using SPSS version 10.0 (Macintosh). These included median, mode, arithmetic mean, and SD. It should be noted that means and SDs are included here as location parameters to report patterns (5). Because survey data are ordinal and often had skewed distributions, the nonparametric Mann-Whitney U-test was used to report two-tailed P values (7). Occasionally, to compare the statistical significance of opinions at the extremes of the Likert scale (i.e., combined 1 and 2 as disagreeing versus combined 4 and 5as agreeing), Fisher's exact test was used to report two-tailed P values. This modified
2-test of significance was appropriate for the small number of data sometimes observed in survey response categories. The internal consistency of all survey items was measured using Cronbach's
-coefficient using SPSS version 10.0 (Macintosh). An
-value of 0.8 or higher indicates that the survey items reliably measure related themes or constructs.
Analysis of the readability of Op-Eds written by students.
Readability refers to the ease of understanding that is dependent on how written texts are constructed (8). Algorithms such as the Flesch reading ease scale have been used since the 1920s as reliable and valid methods to measure the readability of diverse printed media. For example, Roberts et al. (19) demonstrated improved readability of articles submitted for publication to a medical journal after peer review and editing.
In this study, the Flesch reading ease scale, the related Flesch-Kincaid scale (representing readability as a U. S. grade-school level), and the complexity of paragraphs, sentences, and words were measured to compare Op-Eds written by students and professional journalists. The fine-grained analysis of texts thus afforded complemented the more global nature of criteria in the assessment rubric viz. the content, genre requirements, and quality of writing (Table 1).
Many Op-Eds written by students were available as Microsoft Word documents (76%, n = 177). These were analyzed to measure Flesch reading ease, the Flesch-Kincaid U. S. grade level (F-KGL), percentage of text written in the passive voice, and the numbers of paragraphs, sentences per paragraph, words per sentence, and characters per word.
Descriptive statistics (means and SDs) and linear regression analyses were used to compare Op-Eds written by BIOM3006 students with Op-Eds written by professional journalists. For the latter, 10 articles on current issues in biomedical sciences were selected from two broadsheet newspapers: The Australian and The Courier Mail.
| RESULTS |
|---|
|
|
|---|
-coefficient for this survey was 0.9828, indicating very high reliability of survey items in measuring related themes. Student responses from this survey are recorded against each item of the survey listed below.
Item 1. Nonprofessional audiences get most of their information about current science: A. from reading books; B. from open public forums (e.g., public talks); or C. through mass media.
There was strong agreement that nonprofessional audiences received most of their information about current science from the mass media (95% rated 4 or 5, median = 5; Fig. 3A). Conversely, reading books and public forums (e.g., public talks) were both considered to be little used by the public (62% rated 1 or 2, median = 2).
|
Item 3. The general level of "science literacy" (understanding science) in the community is very low.
Most students agreed that the level of science literacy (understanding) was very low (56% rated 4 or 5, median = 4; Fig. 3C); 12% disagreed with the statement (rating 1 or 2), whereas 32% neither agreed nor disagreed (rating 3).
Item 4. As an early career scientist, it is very important that you are able to communicate what you know to your nonprofessional peers.
Students strongly agreed that it was very important to be able to communicate their knowledge of science to their nonprofessional peers (88% rated 4 or 5, median = 4; Fig. 3D).
Item 5. Employers expect that graduates will have above-average levels of communication skills.
Approximately the same percentage of students perceived that employers expected above-average communication skills from science graduates (84% rated 4 or 5, median = 4; Fig. 3D).
Item 6. You rate your own ability to communicate scientific facts to nonprofessional peers as being very high.
A substantial number of students were confident in their ability to communicate scientific facts to their peers (43% rated 4 or 5); 13% of students were not confident, whereas a large number neither agreed nor disagreed with the statement (45% rated 3, median = 3).
Items 7 and 8. You rate your oral/written communication skills as being very high.
Approximately 42% of students rated their oral or writing skills as being high (rated 4 or 5, both median values = 3; Fig. 3E). There were no significant differences among median values for overall communication (item 6 above), oral, or written skills.
Op-Ed Assessment
Op-Ed marks from faculty members.
The grades (from a maximum of 10) ranged from 3 (n = 4) to 10 (n = 15) with a mean score (SD) of 7.4 (1.6) (n = 232). A mean score of 74% was high compared with other assessment tasks (e.g., the end of semester exam mean = 63%). One of the top-ranked Op-Eds (by David Wood) is presented in full (Table 3).
|
Analysis of Op-Eds Written by Students
The surface features of Op-Eds written by student and professional journalists were very similar across five measurements (Table 4). There were only two differences. First, Op-Eds written by students were longer than those by professionals (median values of 753 and 408 words, respectively). This reflected the target word count in the assessment. Second, most students used the passive voice, whereas professional journalists did not. In writing the Op-Ed, students had not been given any instructions regarding the use of passive sentences. Linear regressions compared overall student grade with each of the seven "readability statistics." Three weak correlations were found between the final grade and surface features of student Op-Eds. Students receiving higher marks 1) used fewer passive sentences (r2 = 0.4784), 2) wrote more paragraphs (r2 = 0.4925), and 3) wrote more words (r2 = 0.3365).
|
Postwriting Task Survey: Op-Ed and Your Ideas About Science Communication
One hundred thirty postwriting task surveys were obtained from 230 students, a response rate of 57%. Cronbach's
-coefficient was 0.9726, indicating very high reliability of survey items in measuring related themes. Additional comments were submitted by 26 students. The majority of these comments (70%) were extremely positive, such as "the best thing I have done at Uni" and "do it again next year." Student responses from this survey are recorded against each item of the survey listed below.
Item 1. After completing this exercise, I have a deeper appreciation of how the media present information to the public.
Student responses indicated that writing the Op-Ed gave them a deeper appreciation of the role of the media in communicating science to the public (69% rated 4 or 5, median = 4; Fig. 4A).
|
Items 3 and 4. I found this exercise a challenging/valuable learning experience.
Clearly, most students found writing the Op-Ed to be both challenging (80% rated 4 or 5, median = 4) and valuable (71% rated 4 or 5, median = 4; Fig. 4B).
Item 5. You rate your own ability to communicate scientific facts to the public as being very high.
The distribution of student responses on the 1–5 Likert scale was almost identical as for item 6 of the prewriting task survey (45% rated 4 or 5, median = 3). So, after students completed the Op-Ed, there was no significant change in the way the students perceived their ability to communicate scientific facts to the public.
Item 6. You rate your written communication skills as being very high.
On completion of the assessment task, many students rated their written communication skills as being higher after completing the Op-Ed (54% rated 4 or 5, median = 4; Fig. 4C). However, the differences in pre- and postwriting task survey responses were not quite significant (P = 0.0507 by Fisher's exact test).
Course Evaluation by Students
One hundred thirty course evaluations were returned from 230 students, a response rate of 57%. The overall course score was 4.2 (from a maximum of 5), the same as the previous year. Although few comments were offered about the Op-Ed writing task (as 1 of 6 assessment tasks through the semester), the following student comments were of interest:
| DISCUSSION |
|---|
|
|
|---|
Broadly, this study is a practical application of the goals proposed in the charter for global science (2), viz. to share knowledge and engage the community in democratic dialogue. A more specific but long-term goal in our school is to foster in undergraduates their "civil scientific literacy." This aligns with the recommendation from researchers in science education that the understanding of science should go beyond the learning of concepts and knowing how science operates, to emphasize science as a social process (4). Civil scientific literacy has been described as the cornerstone of informed public policy (23) and interpreted in more practical terms as being able to understand science as written about in a daily newspaper (15).
Our Media Role model was useful in framing the Op-Ed and Peer Review as a novel writing and assessment task for final-year physiology students. This model describes how our undergraduates can better understand the dominant role of the media in communicating science to nonprofessional audiences. Furthermore, it positions communication skills in the context of learning physiology and is, therefore, distinct from vocational "media training" for selected scientific spokespeople and from specialist courses in science communication or journalism. The model helped frame our three research questions.
Can Writing an Op-Ed and Peer Review in Undergraduate Physiology Classes Increase Student Awareness of the Importance of Media in Science Communication?
For the first time, the opinions of our final-year physiology students about how they might communicate their technical expertise to nonprofessional audiences were surveyed. In our prewriting task survey, most BIOM3006 students recognized that the mass media play a dominant role in communicating new scientific findings to nonprofessional audiences. After completing the Op-Ed, most students felt they had gained a deeper appreciation for the role of the media in communicating science to the public.
Most students believed that the general level of "scientific literacy" (understanding science) was very low. However, many did not, possibly due to multiple and often conflicting definitions of scientific literacy (12). Our surveys did not probe how students conceptualized this construct beyond "understanding science."
Not surprisingly, given that many university students in Australia work to pay for their tuition, students recognized that effective communication skills are valued by employers. Competent expression in literary form is one of the skills most highly prized by employers (11). Indeed, human resource managers tend to employ science graduates not because of their knowledge of the content of a field of study but rather because of the processes that they have learned in mastering the content (20).
Can Writing These Same Tasks Be Used to Demonstrate the Difficulties in Communicating to the Public?
Most students felt that writing the Op-Ed was a challenging but valuable task. Furthermore, most students reported that communicating science to the public can be difficult. There was little shift in their own perception of their communication skills and, specifically, their writing skills. This was not surprising, given that the Op-Ed task was only one element of the course and an even smaller component of the undergraduate degree program. Currently, we are developing a program-wide approach to systematically improve technical writing. When this is implemented, it is expected that students will become more confident in their abilities to communicate science to wider audiences. While most students (
90%) recognized that communication to the public was important and that employers put value on communication skills, only 45% of the students thought that their level of communication skills was high. Furthermore, many students agreed that they were not being skilled adequately in this area throughout their undergraduate degree: an obvious point of concern for faculty members. This study highlighted embedded problems regarding the teaching of scientific literacy. It was recognized by students and faculty members that writing a publishable-quality Op-Ed requires not only knowledge of the genre conventions but also considerable pragmatic competencies. In short, the task was not trivial.
Can the Writing Skills of BIOM3006 Students Be Developed Through Completing an Op-Ed and Peer Review in Physiology?
Despite the concerns summarized above, most students demonstrated that they were able to write an Op-Ed to a high standard. Students were able to write an Op-Ed with features that matched those of professional journalists. This was reflected in close similarity of text features measured in Op-Eds written by students and professionals (Table 4). However, the main difference was that students often used passive voice. This probably reflects the style of formal scientific writing traditionally required in most academic tasks, despite recommendations to use active voice (e.g., Ref. 16) and caveats regarding the use of "distorted passive verbs" (e.g., Ref. 24). While the readability of texts results from complex interactions between the writer and reader (6), the measurement of surface features of text in this study pinpointed some of the ways that students were able to reproduce the Op-Ed genre. The Op-Ed was a discriminating assessment task across the large student cohort. While most students achieved a high grade (mean 74%), there was a wide distribution of grades. That students who received higher grades used fewer passive verbs and wrote more paragraphs and words could reflect their more sophisticated linguistic competencies.
Students enjoyed reading and marking each other's writing. Peer assessment is a powerful tool to engage students actively in both the course and assessment process (1, 21). When peer reviewing written work, learning occurs at three levels: 1) during writing, 2) while reading the opinions of others, and 3) through the subsequent process of self-assessment/reflection (18), thereby fostering deeper learning. Not only were most BIOM3006 students motivated by the opportunity to review each other's writing, but they also achieved high marks in both the review and Op-Ed.
Other Outcomes
This study was cross disciplinary in nature. To be effective, the Op-Ed and Peer Review writing tasks required the collaboration of faculty members from journalism and biomedical sciences who jointly refined and delivered the teaching and assessment tasks. Students responded positively to this team approach. This intervention was very valuable in forcing both faculty members and students to reflect on what scientific literacy means for physiology students. It further strengthened the resolve of faculty members to reevaluate the nexus between what the students know relative to what they need to know and how best to do this.
Future Directions: the Next Iteration
The 2004 iteration of this action learning project has provided faculty with student opinions about effective scientific communication to nonprofessional audiences and evidence of the challenges in teaching this to physiology undergraduates. In 2005, the study is being refined (Fig. 2) to empirically evaluate whether authentic nonprofessional audiences agree that physiology students explicitly taught how to write of an Op-Ed are more effective writers to generalist readers.
| Acknowledgments |
|---|
| Footnotes |
|---|
Received for publication November 18, 2005. Accepted for publication February 5, 2006.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
R. W. Moni, D. H. Hryciw, P. Poronnik, and K. B. Moni Using explicit teaching to improve how bioscience students write to the lay public Advan Physiol Educ, June 1, 2007; 31(2): 167 - 175. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |