This article reports on the development and evaluation of a Web-based application that provides instruction and hands-on practice in managing electrolyte and acid-base disorders. Our teaching approach, which focuses on concepts rather than details, encourages quantitative analysis and a logical problem-solving approach. Identifying any dangers to the patient is a vital first step. Concepts such as an “appropriate response” to a given perturbation and the need for electroneutrality in body fluids are used repeatedly. Our Electrolyte Workshop was developed using Flash and followed an iterative design process. Two case-based tutorials were built in this first phase, with one tutorial including an interactive treatment simulation. Users select from a menu of therapies and see the impact of their choices on the patient. Appropriate text messages are displayed, and changes in body compartment sizes, brain size, and plasma sodium concentrations are illustrated via Flash animation. Challenges encountered included a shortage of skilled Flash developers, budgetary constraints, and challenges in communication between the authors and the developers. The application was evaluated via user testing by residents and specialists in internal medicine. Satisfaction was measured with a questionnaire based on the System Usability Scale. The mean System Usability Scale score was 78.4 ± 13.8, indicating a good level of usability. Participants rated the content as being scientifically sound; they liked the teaching approach and felt that concepts were conveyed clearly. They indicated that the application held their interest, that it increased their understanding of hyponatremia, and that they would recommend this learning resource to others.
- clinical problem solving
- software development
- System Usability Scale
electrolyte and acid-base disorders are clinical problems that are common and may be life threatening. This area is highly integrative and quantitative, and it is one that students and clinicians find particularly difficult to master (9).
Medical experts solve most clinical problems using pattern recognition, drawing on a large domain-specific database of schemata or “illness scripts” (15, 22, 41). When an unusual or complex situation is encountered, however, the expert can draw on extensive relevant basic science knowledge and apply it to the problem (38). This is often required in disciplines such as anesthesiology, nephrology, and intensive care medicine, where much of the clinical reasoning involves the application of physiology (37). Electrolyte and acid-base disorders are typical examples common to these disciplines where an understanding of physiology is central to correct diagnosis and treatment.
As teachers, our challenge is to help students and clinicians develop an expertise in clinical problem solving that can be effectively applied when they encounter related, but different, problems. This transfer of expertise is difficult to achieve (10, 14, 39). It can be facilitated by active learning and “deliberate practice” with carefully selected examples (12, 13, 40, 42). This helps to develop a fund of domain-specific knowledge and facilitates the abstraction of underlying concepts and the transfer of clinical reasoning ability from one problem to another.
Increasingly, medical educators are engaging learners using animations to illustrate dynamic processes and simulations to provide the opportunity to interact with clinical problems. Well-known examples include Chopra’s operating room simulator and “Harvey,” the cardiology patient simulator (7, 21, 44). A receptive atmosphere now exists for the increased use of simulations following large studies describing preventable injuries to patients as a result of medical error (4, 20, 27, 28). Some authors view the use of simulations as an “ethical imperative” and argue that harm, or exposure to the possibility of harm, to patients in the course of training or resulting from trainees’ lack of experience can only be justified once approaches that do not put patients at risk have been maximized (62). Additional advantages of simulations include the ability to provide exposure to uncommon conditions and a variety of clinical presentations. Errors can be allowed, and even encouraged, as they provide valuable learning opportunities. Trainees can then have their first encounters with real patients having already attained higher levels of confidence and proficiency.
Developing good simulations and other e-learning materials can be resource intensive, and it is therefore important to ensure that the time and money invested is justified by the educational impact. The usability of computer interfaces has a major impact on the effectiveness of e-learning but is an aspect that has not been sufficiently emphasized in the medical education literature (26, 33, 45). The concept derives from the field of human-computer interaction and describes how easy a technology interface is to use. The terms “fitness for purpose” and “quality of use” (2) are also used to describe usability. The International Organization for Standardization, in ISO 9241-11 (25), defines usability as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”
High usability of e-learning materials is an essential element in ensuring maximum educational impact (3, 5), especially when dealing with complex subject matter (49, 50). Poor design can impair learning, as the user has to struggle with challenging content as well as with the technology interface. Reducing such an extraneous cognitive load can lead to large gains in learning efficiency (49); thus, optimizing the usability of learning resources seems essential. A recent review of internet-based medical education (58) reported that learners are more likely to be engaged when the technology is easy to use.
The routine evaluation of usability is well established in the software development industry. An iterative design approach is followed and involves the creation of prototypes, testing them, and making improvements based on the test results (32). This cycle is repeated until performance and usability goals are met, and only then will the application be shipped to the marketplace. This approach is seldom used in the development of e-learning resources, especially in the area of medical education (45).
The two main types of evaluation techniques are empirical user testing, which involves typical end users using the application, and usability inspection, which involves experts evaluating the application against established metrics or design principles (5). Evaluations are conducted in a wide range of settings, from sophisticated usability laboratories (34) through to informal settings using paper prototypes and think-aloud protocols (47). Many tools are available to assist with the collection, analysis, and reporting of usability data (24). These range from self-administered questionnaires (6) to specialized software for recording and analyzing user testing sessions. Selecting which measures of usability to use is difficult. Some are subjective and others objective, all have their own cost and time requirements, and all examine a particular aspect of usability. Among the objective measures are parameters such as successful task completion, completion times, and error rates, whereas subjective measures include aspects such as satisfaction, perceived workload, fun, aesthetics, and flow (23). Nielsen and colleagues (35, 36, 54) have popularized simpler and less expensive methods, pointing out that any usability testing is better than no testing at all, and demonstrating that four to five users are sufficient for each cycle of testing.
We have developed a Web-based multimedia resource to help students and clinicians acquire expertise in the diagnosis and treatment of electrolyte and acid-base disorders. Colleagues in disciplines such as internal medicine, pediatrics, nephrology, endocrinology, emergency medicine, and intensive care medicine are included in our target user population. Our “Electrolyte Workshop” provides instruction and allows for hands-on practice in treating electrolyte disorders via a highly interactive simulation. This e-learning application can be freely accessed at http://www.learnphysiology.org/sim1/. An evaluation of the Electrolyte Workshop was conducted with a group of residents and specialists in internal medicine to determine its level of usability. User satisfaction was measured via a questionnaire based on the widely used and validated System Usability Scale (SUS) (6). The SUS is a low-cost option that is easily administered and scored and is therefore an especially useful tool in resource-constrained environments.
In this article, we briefly describe our general teaching approach and report on the development of our Electrolyte Workshop. We discuss the challenges encountered, outline the resources required, and highlight the lessons learned. Recommendations are made for managing the development of similar projects. We then report on our evaluation of user satisfaction involving a group of specialists and postgraduate trainees in internal medicine.
METHODS AND RESULTS
Ethical approval for the project was granted by the Health Research Ethics Committee of Stellenbosch University (approval no. N08/05/158).
The Underlying Teaching Approach
Our teaching approach is based on an understanding that learning in this area is most effective when it is built around the relevant basic sciences (9, 37, 38, 59, 60). Building sound mental models based on physiological principles is likely to aid knowledge retention and retrieval, especially when novel or complex problems are encountered and when pattern recognition might not be effective (38). We emphasize an understanding of integrative whole body physiology, deductive reasoning, a focus on concepts rather than details, and quantitative analysis. We emphasize principles of control and point out to our students that if one recognizes the function of a metabolic process, it is often possible to deduce the nature of its likely controls (46). For example, if one views buffering as a way to protect enzymes, receptors, and transporters from an acid load, then the importance of “directing” protons to the bicarbonate buffer system becomes clear. Students are then better able to deduce that this process can be optimized by effective removal of the resultant CO2 through hyperventilation and the provision of an adequate circulation.
We use clinical and laboratory data from real cases in our teaching. Each case starts with the identification of a key abnormality, usually the most abnormal electrolyte or acid-base parameter. This is usually self-evident and often the reason for the consultation. When there is more than one major abnormality present, any one can be selected as the starting point and the analysis followed through to the end. Thereafter, the next abnormality is dealt with in a similar way.
The first step is always to identify and address immediate threats for the patient and to anticipate dangers that may develop later. These often arise as unintended consequences of therapy. The primary imperative is always to “save the patient!”
We continue with the diagnostic workup once urgent therapeutic issues have been addressed. Patient data are analyzed and interpreted focusing on the actual versus expected (appropriate) response to the particular perturbation. We repeatedly emphasize the concept of “an appropriate response” and the ability to recognize it in a given situation. This requires an understanding of the relevant physiology. Students are asked what an appropriate response would be and then what clinical or laboratory data are required to identify whether their patient is responding appropriately, for example, “How should your patient’s kidneys respond to hypernatremia?” and “What data would tell you that antidiuretic hormone is acting on the kidneys?”
A problem-solving process that is simple and logical is encouraged. Basic principles such as mass balance and the need for electroneutrality in body fluid compartments are used repeatedly. Other frequently used concepts are those of “driving forces and permeability,” which are the elements that determine whether water or solutes will move across cell membranes. Through this line of inquiry and educational approach the student arrives at a functional diagnosis, e.g., “fast sodium absorption disease” in a patient with hypokalemia and hypertension. This is followed by making a structural or anatomic diagnosis, e.g., “overactive epithelial sodium channel disease,” and is sometimes followed by assigning a specific diagnostic label, e.g., “Liddle’s syndrome.” We emphasize the systematic analytic process and not the arrival at the correct diagnostic label. To discourage students from taking shortcuts and jumping to possible final diagnoses too quickly (i.e., guessing!), we often ask them to interpret a set of clinical and laboratory data, specifying that “a diagnostic label is not required.”
A quantitative analytic approach is always promoted. For example, “In this 60-kg female with a plasma sodium concentration of 130 mmol/l where we’ve estimated the extracellular fluid volume to be contracted by ∼10%, what would be the magnitude of her sodium deficit?” followed by “What volume of 0.9% saline would be required to correct a sodium deficit of 230 mmol?” Since exact answers are seldom required at the bedside, students are urged to round off numbers and develop their skills at estimating rather than resorting to using a calculator. Another example comes from a case of cholera (61) where severe extracellular fluid contraction completely masked metabolic acidosis. An important teaching point was that one has to consider the content, and not just the concentration, of plasma bicarbonate when assessing acid-base status.
Our teaching approach provided the background for the development of the multimedia, Web-based learning resource that is discussed below. Here, we describe our e-learning application, the development process involved, and the challenges encountered and suggest some recommendations for managing similar projects.
Development of the Electrolyte Workshop
Description of the application.
Flash from Adobe Systems (www.adobe.com) was used for the development of our Electrolyte Workshop, a case-based multimedia application illustrating and simulating the pathophysiology, diagnosis, and treatment of a variety of electrolyte and acid-base disorders. Flash can provide an engaging user experience by producing interactive content that can include pictures, sound, and video. It is well suited to creating rich content for the Web because its files are relatively small.
The Electrolyte Workshop (Fig. 1) consists of two main sections, each currently containing one case-based tutorial on hyponatremia. Eventually we plan to cover all the common electrolyte and acid-base disorders, with several examples of each. In the WalkThru section, the case consists of a series of 14 slides that presents the clinical problem of acute hyponatremia related to ingesting the drug Ecstasy. Through words and pictures, we demonstrate how an expert would interpret the patient data and embark on treatment. Flash animation is used to illustrate and emphasize important changes in body compartment sizes, brain size, blood pressure, and plasma sodium concentrations (PNa). The pace at which information is presented is controlled by the user as s/he navigates from one slide to the next. One of the interesting aspects highlighted in this case is that of hidden dangers related to stomach contents. Water ingested shortly before admission, and present in the lumen of the gastrointestinal tract, still has to be absorbed and may further aggravate the severe hyponatremia.
The HandsOn section is also self-paced but more interactive. The case is that of chronic hyponatremia due to Addison’s disease. It starts off with five “lead-in” slides that describe the clinical problem and highlight the key issues such as the adrenal hormone deficiencies and the contacted extracellular fluid volume. Users are then provided with a “treatment console” (Fig. 2, A and B), a highly interactive simulation where they are able to select from a menu of therapies (and dosages) and apply their treatment. The main issue here is the danger of too rapid correction of the chronically low PNa, which may result in serious neurological damage. The available therapies include a selection of intravenous fluids, from water through to 3% saline. It also includes drugs (a vasopressin analog and cortisol in this case) and sodium or potassium salts. More than one treatment can be administered; this happens sequentially and not simultaneously, so that feedback can be given after each step via on-screen text and animations. The animations illustrate the effects of treatment choices by showing changes in body fluid compartment volumes, brain size, blood pressure, and PNa. The text messages indicate the success of the interventions applied, for example, “Your patient developed osmotic demyelination and died from serious neurological damage. This was caused by a too-rapid rise in PNa! Try again?” After successful completion of the treatment simulation, the case concludes with a final slide of “take home messages.”
The two cases were chosen as they provide striking examples of acute and chronic hyponatremia, respectively. Additional examples will be added later so that so that students encounter the same important principles in a variety of contexts. In both sections we use the teaching approach described above and try to foster a better grasp of the underlying physiology principles (Table 1). The quantitative aspect is stressed consistently to reinforce the importance of accuracy in the assessment of the disorders and in prescribing treatment. The HandsOn simulation provides practice to help users develop a better feel for treating these conditions and, in particular, the ability to use intravenous fluids correctly and confidently.
An informal, conversational style was deliberately chosen for the presentation of the cases. Users are addressed directly using words such as “you” and “your.” Mayer and colleagues (29) have shown that this “personalization effect” can improve the engagement of users and promote active learning. As undergraduate and postgraduate health science students are the main target audience, we created a trendy young character called Suzie (Fig. 1, bottom left) as the patient in the case scenarios, also with the intention of increasing students’ engagement through the compelling need to save her from harm!
To provide support, especially for novice users, there is a glossary (Fig. 3), which can be accessed from the main navigation menu or via text hyperlinks in the cases. Terms that might be unfamiliar or require further explanation are underlined to indicate a hyperlink to the glossary.
The development process.
We contracted a team of Flash developers with extensive experience in Web design and animation but with no background in the biomedical sciences. After initial discussions, we constructed “wireframes” or “storyboards” to communicate our ideas for the two case scenarios. Microsoft PowerPoint was used for this purpose.
The iterative development process that followed is shown in Figs. 2 and 4. From each set of PowerPoint slides, the developers created static screenshots (.jpeg files) to reflect the different screens or slides in the case. We reviewed these and compiled a list of changes, which were then implemented by the developers. Two iterations were required during this phase. The application was designed to fit into an area of 800 × 600 pixels to accommodate users with smaller computer screens. A grayscale version was produced initially, and color was added once all changes had been agreed upon.
After three cycles of development and reviewing and revising the application, the fully animated, interactive version was completed.
Several challenges were encountered during the development process (Table 2). The shortage of skilled Flash developers has been mentioned. A limited budget was another constraint (see below for a discussion of the costs involved). With respect to the actual development of the simulation, the main challenge was the difficulty in communicating to the developers exactly what was required. Their feedback was that our PowerPoint wireframes were not detailed or accurate enough. As this was a novel project for all concerned, it was difficult to fix all the specifications for the work at the start. Different expectations of the number of iterations that would be allowed became apparent, as did differences in the list of features that were included in the original costing. These issues were amicably resolved through discussion.
For future projects, the developers have requested that detailed and accurate screen-by-screen wireframes be provided up front. This should include all content and all algorithms and formulae with relevant text messages needed for user feedback in the interactive parts of the application. It should be clearly stated which parameters need to be tracked by the application. Finally, it must be specified at the start which elements need to be editable by the client. This would establish the full extent of the development required. Not only would this be essential for accurate cost assessment, but it provides the basis for a written agreement from which the project can be managed and eventually signed off.
There is a shortage of skilled Flash developers in South Africa, and more than a year was spent finding a suitable team who were willing and able to execute the project within the available budget. We focused on finding developers in the greater Cape Town area so that face-to-face meetings could be held as required.
The development team comprised of one programmer and one designer. The work was completed over a period of 7 mo. Development costs amounted to approximately $5,300 (R40,000; $1 = R7.60), a greatly reduced rate offered because of the academic, not for profit nature of the project and because the interest of the developers was piqued by the project. The usual rate for commercial Flash development in South Africa is $60–85/h (R450–650/h), and a fixed-price estimate for a project such as this would usually be around $12,500 (R95,000).
Evaluation of the Electrolyte Workshop
Participants and testing procedures.
User testing was conducted at two South African academic departments of medicine with 10 residents and 6 specialists in internal medicine, nephrology, and endocrinology. This group is typical of our target user population because their disciplines involve the management of electrolyte and acid-base disorders. Although five users have often been reported to be sufficient to test usability (35), we engaged a larger number of participants to improve the usability error detection rate (16, 53) and to allow us to consider the influence of participants’ knowledge and experience. The application was presented via two 15-in. laptop computers, each equipped with a mouse. Participants received written instructions that included information about the purpose of the application and of their involvement, which was to help us improve the application. Their tasks were to view the home page, familiarize themselves with the different sections of the application, and then work through the cases in the WalkThru and HandsOn sections, trying different options in the treatment simulation. They were also asked to look at the glossary. No time limits were set, and participants worked at their own pace.
Participants each completed a two-page questionnaire at the end of their session. The first part consisted of the 10-item SUS developed by Brooke (6). SUS yields a single number representing a composite measure of the overall usability of the system being studied. The scores have a potential range of 0 to 100, with a score of 70 or greater regarded as acceptable (1). SUS is widely used by usability professionals and is reliable, freely distributed, easy to administer, and easy to score (1, 53). It can be used to provide a point estimate measure of usability and customer satisfaction, compare different tasks within the same interface, compare different versions of a system, and compare competing systems or interface technologies (1).
We used SUS with minor adaptations (Table 3), replacing all occurrences of the word “system” with “application” and changing the word “cumbersome” in item 8 to “cumbersome/clumsy.” Other authors have recommended replacing “cumbersome” with “awkward” to improve understanding, especially when the survey involves non-native English speakers, as was the case with some of our participants (1, 17).
The next section of the questionnaire (Table 4) included 11 statements about various aspects of the application, such as the soundness of the content, the ease of navigation, and whether participants would recommend the resource to others. Participants indicated their level of agreement on a five-point Likert scale. This was followed by questions on the treatment simulation (Table 4) and a question on the length of each case (Table 5). Participants were also asked about the suitability of the application as a learning resource for groups ranging from specialists to medical and nursing students (Table 6).
Participants were then asked to rate their own level of computer literacy (Table 7). Finally, they were asked to comment on things they liked about the application, anything they did not like or which could be improved, and for final suggestions or comments.
Results of the evaluation.
The results of the SUS are shown in Table 3. The mean score was 78.4 ± 13.8 (range: 45–100). There were no differences between senior (specialists, n = 6) and more junior (residents, n = 10) colleagues. Mean scores were 82.1 ± 10.5 and 76.3 ± 15.6 (P = 0.477) in these two groups, respectively.
On analysis of individual questionnaire items, senior clinicians expressed a greater degree of confidence in using the application (P = 0.037), but there were no other differences between the two groups. User satisfaction with various aspects of the Electrolyte Workshop is shown in Table 4. Participants rated the content as being scientifically sound (15 of 16 participants agreed); they liked the clinical detective story approach (14 of 16 participants), the emphasis on key concepts (14 of 16 participants), and felt that these concepts were conveyed clearly (14 of 16 participants). They indicated that the application held their interest (14 of 16 participants), that it increased their understanding of the topic (14 of 16 participants), and that they would recommend this learning resource to others (15 of 16 participants).
A few participants felt that the glossary was not useful (5 of 16 participants) and that navigation was difficult (3 of 16 participants). The treatment simulation was experienced as realistic by 9 of 16 participants (5 participants were neutral) and increased 8 participants’ confidence for managing similar cases (6 participants were neutral). It was considered difficult to use by two participants and too far removed from the real world to be useful by one participant (6 participants were neutral).
Participants considered the application suitable for residents, subspecialty trainees, and specialists. It was also considered suitable for medical students but not for nursing students. Half of the participants considered it suitable for renal and intensive care unit nurses. Of the three participants who indicated that their level of computer literacy was weak, none found the application difficult to use.
Comments from the open-ended questions are shown in Table 8. Participants liked the interactive nature of the application, the real-life cases, and the overall design. Most of the negative comments related to the treatment simulation. These included being limited to apply only one treatment at a time, struggling with the slider control, inadequate guidance and feedback from the application, and an inability to navigate back to the lead-in slides once in the treatment simulation.
A multimedia application was successfully developed to provide instruction in the area of electrolyte and acid-base disorders. It is a reusable learning object that is sharable and that can easily be incorporated into learning management systems. Two case scenarios were built in this first phase to explore the feasibility and optimal design of the application.
While learning from authentic, complex, and ill-defined problems is encouraged by constructivist approaches and facilitates the transfer of expertise, novices may be overwhelmed and demotivated if problems are too difficult (52). The WalkThru section of the application was therefore designed to facilitate learning using worked-out examples (43, 51). This allows students to move from focusing on finding solutions to appreciating the underlying principles or rules. Offering “step-by-step” guidance in this section promoted the development of expertise by “making the thinking of experts visible,” in line with the cognitive apprenticeship model (8). This model involves a focus on teaching the processes, the cognitive and metacognitive skills, by which experts solve complex problems.
Deliberate practice in a specific domain is important in the acquisition of expertise (11), and the HandsOn section was designed around this principle. The treatment console is an example of a deterministic simulation, where a given action by the user in a particular situation always produces the same result. This predictability helps novices build confidence in their ability to apply treatment correctly and accurately. In the future, we intend to cater for more experienced users by including probabilistic simulations to model the uncertainty and unpredictability that is always part of managing real patients (19).
Although the initial development included only two case scenarios, eventually multiple examples of each type of disorder will be presented, so that students encounter key concepts and the same step-by-step physiology-based approach in a variety of contexts. Sound physiological principles should provide the framework around which their new knowledge and schemata are built. This facilitates knowledge retrieval and application and also improves the accuracy of the nonanalytic (pattern recognition) components of the clinical reasoning process (60). The provision of a range of cases will also cater for users with different levels of expertise, with more advanced users free to skip some of the simpler WalkThru cases and tackle the more challenging HandsOn cases directly.
Regarding our choice of development platform, we decided on Flash to create a resource that was visually appealing and interactive, increasing the likelihood that students would be engaged and motivated to use it. It also needed to be delivered via the internet so as to be accessible to a wide audience. Learning objects created with Flash can be used with a variety of learning management systems. They can be accessed via any Web browser using the free Adobe Flash Player plug-in, avoiding the problems of cross-browser and cross-platform incompatibility. They will run on almost all personal computers as well as an increasing number of mobile phones and other devices (30). The delivery of rich content with relatively small file sizes is possible because Flash uses vector graphics, which are represented by mathematical formulae, rather than bitmap graphics with their larger file sizes.
Mastering Flash involves a very steep learning curve, however, and thus it is not a realistic option for most educators. Skilled developers are in short supply and are expensive. Since starting the project, we have become aware of many tools that allow nonexperts to build e-learning courses without special programming skills. These range from simple PowerPoint-to-Flash converters through to high-end applications with prebuilt templates, interactions, quizzes, branching logic, and the like. The eLearning Guild (www.elearningguild.com) has compiled a useful report on authoring and development tools (55) that could serve as a starting point for readers interested in developing their own materials. In the future, we plan to use Flash developers more selectively, thus reducing our dependency on expensive, scarce skills. For a project like this one, for example, we could use PowerPoint to develop many of the slides and the simpler animations ourselves. The more complex animations and the treatment simulation would then be the only parts we would need to “outsource,” and these components would then simply be inserted on the appropriate slide of the case. The entire PowerPoint presentation could then easily be converted to Flash using a variety of applications such as Articulate (www.articulate.com), iSpring (www.ispringfree.com), Adobe Presenter (www.adobe.com/products/presenter), and others.
Those e-learning applications that have to be custom built by independent contractors need to produce the desired end product within time and budgetary restrictions and ideally allow for easy expansion and maintenance. Unfortunately, this is the case with only a minority of software development projects. Research by the Standish Group (48) has found that the most common outcome is that projects are late, over budget, or have less than the required features. Many projects are never completed or used, and only around one-third are delivered on time, within budget, and with the required features. For small projects such as ours, we offer some recommendations to increase the chances of a successful outcome: the use of wireframes and prototyping, following an iterative development process, and drawing up a formal written agreement.
Wireframes and prototypes.
Wireframes and prototypes are used to represent the structure and functionality of a website or desktop application (57) and are constructed early on in the development process, before any artwork or coding is undertaken. They provide a basis for communication between clients and developers, helping to define the functionality of each page (or screen) and the positioning of elements such as navigation menus and search fields. It is important to reach agreement on the user interface and functionality right at the beginning of the project. Once these specifications have been defined, an accurate cost assessment can be done. These critical early steps help to reduce the risks and costs of the software development process. Simple wireframes can be created using paper prototyping (47) or easy-to-use software such as Microsoft PowerPoint, Balsamiq Mockups (www.balsamiq.com), MockupScreens (www.mockupscreens.com), or Pencil Project (www.evolus.vn/pencil), an add-on for the Firefox browser. Higher-fidelity tools allow for the creation of prototypes with a richer user interface, more interactivity, and even basic conditional logic. They are usually more expensive, with a steeper learning curve, which may be daunting for nontechnical users. Software programs in this category include Flash and Flex (www.adobe.com), Axure (www.axure.com), and Irise (www.irise.com).
Iterative development process.
An iterative development process usually involves the following: 1) identifying the basic requirements; 2) developing a first version of the application, typically a basic prototype; 3) review(s) by the client and preferably end users; and 4) revision by the developers based on the feedback received. There may be several iterations of steps 3 and 4 until there is agreement on the user interface and functionality. The code is then written to transform the prototype into a dynamic, interactive application.
A written agreement concluded at the beginning of the project helps with managing the project and preventing disputes. Essential elements include software specifications, timelines and payments, ownership of the software, warranties, and dispute resolution (18). The developer should be required to fix software errors at no charge for a specified period of time. It is advisable to break down a sizable project into discrete parts and link payments to the completion of each part. This also makes it easier to monitor progress and avoids the danger of getting an unsatisfactory product at the end. The intellectual property rights to the software usually reside with the developer. However, many different options can be negotiated, ranging from sole ownership by the client to ownership by the developer with the client merely having a license to use the software. Finally, provision must be made for resolving disputes, preferably through mediation or arbitration.
Overall, our Electrolyte Workshop was positively received by the participants in our initial evaluation, with the SUS score of 78.4 indicating a good level of usability. The additional questionnaire items confirmed the satisfaction of participants with the case-based approach and overall design of the application. They considered it useful, thought that it improved their understanding of the topic, and would recommend the resource to others. It was considered to be a suitable learning resource for residents and specialists, our target audience, and also for medical students. Of some concern was the data on the treatment simulation in the HandsOn case. Only 9 of 16 participants found it realistic, and only half felt that it increased their confidence for managing similar problems. Difficulties with the selection and application of treatments as well as inadequate guidance and feedback were highlighted as issues in this interactive part of the application.
In our Electrolyte Workshop, we have the foundation of a multimedia resource that has the potential to offer a rich, immersive learning experience and assist students and colleagues to acquire expertise in the area of electrolyte and acid-base disorders. User testing with the aid of a standardized questionnaire indicated that we achieved a good level of usability. Further evaluation should include objective measures of usability and an assessment of gains in knowledge. The development of e-learning materials of high quality requires a multidisciplinary team that includes content experts, instructional designers, and developers. Implementing good project management, with clarification of roles and expectations, is important in ensuring a successful outcome. Finally, using an iterative development approach with the routine testing of usability is an essential aspect in realizing the full educational potential of the electronic medium.
This work was supported by grants from the South African Universities Health Sciences Information Technology Consortium and Stellenbosch University’s Fund for Innovation and Research into Learning and Teaching.
No conflicts of interest, financial or otherwise, are declared by the author(s).
The authors thank the team at PixelProject (www.pixelproject.com) for generously providing Flash expertise and Martin Schreiber for a critical review of the final drafts of the manuscript.
- Copyright © 2011 the American Physiological Society