| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
ILLUMINATIONS
University of Calgary Health Sciences Centre, Calgary, Alberta, Canada T2N 4N1
Address for reprint requests and other correspondence: P. K. Rangachari, Univ. of Calgary Health Sciences Centre, O’Brien Centre for B.HSc Programme, 3330 Hospital Blvd., Calgary, Alberta, Canada T2N 4N1 (E-mail: pkrangac{at}ucalgary.ca)
 |
Abstract |
|---|
 TOP Abstract REFERENCES |
|---|
Key words: active learning; consequential validity; assessments
ORAL EXAMINATIONS have been used for centuries to assess student performance (2). In recent years, their low psychometric properties (limited reliability and validity) have made them suspect (4, 6). These criticisms are quite justified if the objectives are to produce relatively homogeneous products for diverse trades or professions. However, these same characteristics may be quite powerful if one wants to promote diversity of student performance. The capacity of the student to engage with teachers in a discussion may prove a powerful incentive not only to learn, but also for individual students to go beyond mere competence and demonstrate flair.
I have used a variation of an oral exam for engaging students in their own learning. Most of the students who were assessed were taking an upper-level Inquiry course in an Arts and Sciences Program. An earlier version of the course has been described in detail in this journal (5). The course entitled "Discovery: The Context of Biomedical Research" used a problem-based format to explore the interactions between investments, individuals, and ideals. Students were given an option to explore topics of particular relevance to them. In the earlier version of the course, these explorations were submitted as written essays. In later years, I gave students the option of choosing either a written report or the targeted oral. A smaller number of students who opted for the oral exam were taking a fourth-year course in pharmaco-epidemiology. Here too, the course was run in a problem-based format and students were provided opportunities to explore topics of specific interest. Once again, options were given for submitting either written reports or participating in an oral exam.
The students were told that the objectives of this exercise were for me to gauge their comprehension of the specific area that they had chosen to explore. They were asked to submit an abstract that stated briefly the objectives of the Inquiry along with a brief description of the area of research and the boundaries that were being placed on that inquiry. In addition, key references had to be provided. Hard copies were required if the references stemmed from sources that were either web based or from sources not readily available.
The examinations took place in a room equipped with a one-way mirror. The examinee and I sat in an interview room and the other students witnessed each exam from behind the mirror. Each student began with a brief summary (2–3 minutes) of what they had read and stated specifically the issues they felt comfortable in being examined on. The tenor was conversational. There was no set order and questions were built on answers given. After 20 minutes, the student summarized their impression of what they had accomplished within that time frame. Prompt feedback was given. Since each student selected topics that were of particular interest to them, all students got to learn something about a variety of different topics. Thus a wider range of subjects was eventually discussed than were covered in the course itself.
Student performances were assessed on their abilities to provide clear, cogent summaries of the material they had read, their awareness of the conceptual basis of the methods used, their ability to critically assess the information they had gathered and their abilities to suggest avenues for further exploration. The final mark was a composite.
The student assessment of this exercise has been very positive. They were asked to evaluate the success of this exercise on meeting its objectives using a 10-point scale (10 being highest). The results of data collected from 23 students were as follows: 10 (19), 9 (2), 8 (1), and 5 (1). Their comments were generally positive. Some of their comments are given below:
This procedure was used with third- and fourth-year undergraduate students. These students were close to the ideal students for any University. They were highly motivated, dedicated, seriously interested in learning and not given to needless carping about grades. Whether such an approach would be widely transferable is difficult to say. Nevertheless, it should be possible to use this with graduate students who should be interested in their chosen discipline.
Clearly this process took time. I examined five to six students per day (each student being given 25 minutes). Furthermore, the topics chosen ranged widely from the neural basis of romantic love, an assessment of Skinner’s contributions to psychology, Oliver Lodge and the investigation of the paranormal, the failure of the Nazis to develop nuclear weapons, placebos, drug development, and migration of peoples across the Pacific. I had to read the abstracts and references to ensure that I could question them intelligently. Ultimately, I probably learned as much or even more than they did.
Oral examinations have the weight of tradition behind them. In the clinical setting, they have some appeal because they appear to mimic to a certain extent real encounters. Thus there is the possibility that they measure certain components that cannot be assessed by written examinations (4). In some British schools, short orals or vivas have been used in the past to test candidates who were borderline on written examinations (7). This may be problematic given their low psychometric properties. The relative strengths and weaknesses have been well discussed (4). In addition to the low reliability, the issue of feasibility and cost is a serious one. The subjective element can be reduced by use of carefully designed rubrics and measures to improve reliability can be put in place (4, 8). On the other hand, whether the subjectivity of such procedures is intrinsically problematic is difficult to say since objective measures can be unreliable and subjective measures can yield reproducible results (6).
My argument here is somewhat different. I believe that the procedure itself has intrinsic heuristic value. Thus to exclude such an assessment based only on its relatively poor psychometric properties may deprive students of a rich educational experience and may do them a disservice. In an interesting commentary, Spike and Jolly (9) discuss several aspects of oral examinations. They ask whether the "relentless drive to achieve benchmark reliability blinded us to the impact these assessments have on learners?" This raises the issue of consequential validity, which refers to the impact on examinees from the assessment procedures, since these have profound impacts not only on what students learn and retain but also on how they view and value the educational process itself (1). If orals provide students an incentive to explore topics, give them a chance to interact one-on-one with examiners and get excited about learning, is it fair to deprive them of that opportunity?
|
REFERENCES |
|---|
|
TOPAbstractREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  P. K. Rangachari Promoting self-directed learning using a menu of assessment options: the investment model Advan Physiol Educ, December 1, 2006; 30(4): 181 - 194. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
