Michelle Lui, B.Sc. M.Sc.BMC; Jodie Jenkinson, B.A., M.Sc.BMC; Jana Bajcar, B.Sc.Phm., M.Sc.Phm., Ed.D., FCSHP; Micheline Piquette-Miller, BSc, Ph.D.; Linda Wilson-Pauwels, AOCA, B.Sc.AAM, M.Ed., Ed.D.

Figure 1. Screen capture of multiple-choice interactivity as seen in the interactive version of I. Introduction to pharmacogenetics section.

Figure 2A.Example of a sequence of events while testing the interactive version.

Figure 2B. A sample sequence of events in the interactive version of III. CYP2D6 case study.

Figure 2C. A sample sequence of events in the non-interactive version of I. Introduction to pharmacogenetics.

Figure 3. Comparison of mean scores in participants who viewed prototype -interactivity or +interactivity versions.

Figure 4 . Evaluation of learning experience, clarity and level of difficulty on a 10-point Likert scale.

Recent discoveries in pharmacogenetics suggest that the field will radically alter the way in which medicine is discovered, developed, and used (Brock et al. 2002). However, visual materials used in support of education of pharmacogenetic principles are currently few in number and often lacking in instructional, as well as aesthetic quality. This is symptomatic of the current drought of continuing education initiatives dedicated to translating novel scientific knowledge for use by health-care professionals in Canada. Large gaps remain between best evidence and practice (Davis et al. 2003). However, online courses allow swift delivery of information to clinicians with little geography and time constraints (Qureshi 2002). Thus, online courses can facilitate communication of health research findings, and aid in attaining research’s ultimate goal of yielding positive health outcomes in practice.

Animations possess a presentation and clarification function not available in still images. They are especially useful in presenting complex and dynamic conceptual information, such as systems that are impacted by simultaneous influences, changes over time, and systems not visible to the naked eye (Weiss 2002, Tversky et al. 2002).

On the other hand, conflicting studies suggest that animations may be less effective learning tools than still pictures, especially in instances involving novice learners. In these cases, animations are thought to be more difficult to perceive and understand (Lowe 1999 and ChanLin 2001). Elements of motion in animation might require more extensive efforts in constructing connections between textual and graphical information in integrating knowledge (ChanLin 2001).

With the advent of computer-assisted learning, the role of animation has become increasingly prominent in multimodal learning. Further research is required to examine the necessary conditions for the specific and effective use of animations and interactivity in education (Lewalter 2003). The role of animations in learning and the mechanisms behind cognitive processing of animations are unclear thus far and constitute fertile ground for research.

Interactivity in Multimedia Learning

Interactivity’s important role in online learning (Schittek et al. 2001) as well as its useful role in knowledge acquisition and the development of cognitive skills (Sims 1997) are well-documented.

Some experts generalize interactivity in web-based learning into two major categories: social interactivity and instructional interactivity (Gilbert and Moore 1998). Social interactivity encompasses body language and greetings/socializing with examples such as videoconferencing, electronic bulletin boards and moderated discussions. Instructional interactivity includes activities such as questioning, confirmation of learning, and controlling navigation (Gilbert and Moore 1998).

Narayanan and Hegarty (2002) utilized pop-up multiple-choice questions where students were asked to predict various behaviours of a system before viewing any animations. The researchers proposed that the questions supported the construction of a dynamic mental model by encouraging mental stimulation. Other elements of interactivity included clicking on labels and revealing other corresponding components or seeing explanatory text. This model produced better learning outcomes than conventional presentations.

Schittek et al. (2001) reviewed computer-assisted learning and also determined that interaction between users and content is an important aspect. Increased levels of interactivity in web sites have been shown to increase web users’ perceived satisfaction, effectiveness and efficiency of learning, value and overall attitude of their experience (Teo et al. 2003).
Although interactivity’s effect in education has not yet been thoroughly studied, it is hypothesized that interactivity can help overcome the difficulties in perception and comprehension of animations by encouraging an active (rather than strictly passive) viewing of animations (Tversky et al. 2002).

Objectives

The main objective of this research project was to develop a prototype website for a visually-based online course in pharmacogenetics targeted at community pharmacists as a continuing professional development module. 2-D biomedical animations incorporating multiple-choice questions as a form of interactivity were developed and their effectiveness tested against traditional animations with the same informational content.

A major goal of this research was to determine whether instructional interactivity, specifically, multiple-choice question interactivity, promotes learning when it is incorporated into animations depicting complex and dynamic concepts such as those in pharmacogenetics. It is expected that the integration of multiple-choice questions that encourage reflective thinking will increase the educative value of instructional animations.

Methodology

Content Development

The prototype course is comprised of three sections, each building upon the knowledge presented in the previous section. The first section, I. Introduction to pharmacogenetics introduces terms and concepts by giving an overview of basic concepts in pharmacogenetics. The second section, II. From genes to proteins, teaches fundamentals of DNA. This section integrates knowledge gained from section I by illustrating the interaction between concepts from both sections. The third section III. CYP2D6 case study, takes a widely taught example in pharmacogenetics and integrates the concepts from the previous two sections in an applied perspective for a practicing pharmacist.

Each section is comprised of a 2-D animation approximately 5 minutes in length. The storyboards were developed from content information drawn from relevant literature and textbooks and reviewed by an expert panel.
Animated sequences were developed using the following software applications: Flash MX™ (Macromedia, 2004) with graphics created in Flash MX™ (Macromedia 2004), Photoshop CS™ (Adobe Systems 2004), Illustrator CS™ (Adobe Systems 2004), Maxon Cinema 4D™ (Maxon Computer 2003), and Swift 3D™ (Electric Rain 2004). Narration was recorded in Peak DV 3™ (BIAS 2003).

Animations were adapted in 2 ways: 1) A linear narrative (non-interactive) where the animation played non-stop from start to finish; and 2) An interactive narrative where 4-5 multiple-choice questions were inserted between each scene relating to content from the previous scene. Questions were based solely upon presented content so as to prevent any additional learning through the questions themselves. Since later sections relied on knowledge from previous sections, the complexity of the questions increased and required mastery of earlier concepts. Answering incorrectly resulted in the user being taken back to view the previous scene and progressing to the next section only upon answering all questions correctly (Figures 1, 2). As a result, a model was developed where learners viewed content that presented progressively complex levels of learning while simultaneously encountering progressively complex multiple-choice questions throughout.

Multiple-choice questions were chosen as the mode of instructional interactivity in this module since they are an established form of interactivity that can be easily tracked and have its effects isolated.

Evaluation

Preliminary testing was conducted after the prototype was complete. Testing consisted of a questionnaire, pre- and post-test, and questions evaluating participants’ learning experience. A comment box was placed on each page of the content pages so that participants could pause the lesson and submit immediate feedback before continuing.

I. Participants
Fourteen practicing pharmacists, with no specialized knowledge of pharmacogenetics, were gathered from a convenience sample in the Greater Toronto Area. Informed consent was obtained from all participants prior to the evaluation.

II. Procedure
The participants were randomly assigned into two groups. One group (n=8) viewed the non-interactive prototype program while the second group (n=6) used the interactive version. Participants were asked to log into the websites at their leisure, but to finish the session in one sitting.

Upon logging in, participants were prompted to complete the questionnaire and pre-test. The questionnaire inquired of their background in pharmacy, pharmacy education and background in genetics and pharmacogenetics. The pre-test consisted of five multiple-choice questions that tested pre-existing knowledge of pharmacogenetics. Upon their completion, the main menu was made available to participants. The main menu consisted of the three available sections and led to the teaching portion of the program.

After viewing all three sections, the participants completed a post-test by answering questions regarding the material covered. The post-test consisted of five multiple-choice questions and were different from the pre-test questions.

Immediately after, participants were prompted to complete a final evaluation consisting of six questions relating to the user’s learning experience with this prototype module. Three of these questions asked participants to rate their learning experience as well as the clarity and level of difficulty of the information presented on a scale of 1 to 10. Participants who viewed the interactive version were asked to answer four additional questions about their experience with multiple-choice question interactivity.

III. Data analysis
The pre- and post-test data were analyzed by single factor and two-factor without replication ANOVA (ANalysis Of VAriance). Feedback gathered from the comments box and the final evaluation was analyzed qualitatively. Average score was calculated on questions that asked users to rate a point on a scale of 1 to 10.

Results

Participants’ Background

Approximately 64.29% had not taken an undergraduate course in genetics and none worked with genetic information on a daily basis. However, 42.86% had taken a course or attended a seminar about pharmacogenetics or pharmacogenomics. When asked about participants’ current knowledge about pharmacogenetics or pharmacogenomics, 71.43% indicated that they were ‘not knowledgeable’, and 28.57% indicated that they were ‘somewhat knowledgeable’.

Knowledge Acquisition

Knowledge acquisition was assessed by comparing pre- and post-test scores (Figure 3). A preliminary comparison was made with a two-way without replication ANOVA, with time test taken (pre- vs. post-test) as between-subject measure and version of prototype program viewed (non-interactive or interactive) as between-subject measure. The analysis revealed a significant effect of time test taken, F(1, 13)=9.66, p<0.05, but did not detect a significant difference in scores between non-interactive or interactive versions. Upon further analysis using single factor ANOVA, a significant difference was found in pre- and post-test scores only in participants who viewed the non-interactive version, F(1, 14)=6.16, p<0.05 and not in participants who viewed the interactive version, F(1, 10)=2.96, p>0.05.

Summary of Learner Assessments

Participants’ learning experience with the prototype module was positive overall (Figure 4). Visual learners especially appreciated this alternative mode of learning. Predictably, responses on the level of difficulty of information presented varied depending on participants’ background in genetics and/or pharmacogenetics. Those with previous knowledge of the material found the module to be “basic information” or “a good overview.” On the flip side, participants who had never encountered genetics in their schooling found the information to be somewhat difficult. Overall, the level of difficulty was scored close to the mid-way point, at 5.4/10 (Figure 4). When asked about the clarity of the information, those who had a background in genetics found it easy to understand while those who did not had more difficulty. The clarity of information scored an average of 7.6/10 (Figure 4).

Among participants who tested the interactive version and answered additional questions about their experience, reviews of multiple-choice interactivity were mixed. Some participants liked them and felt more attentive as a result. Others thought they were distracting and were better left to the end of each section. However, most participants found they reflected more on the information when asked to answer multiple-choice questions while viewing a section, and that it did assist their attentiveness. When asked whether participants would prefer learning about pharmacogenetics with sections that did not stop to ask questions in between, 66.67% answered “no”, while 33.33% said “yes”.

General comments about the prototype included enthusiasm about the animation and audio. Recommendations included increased font size, displaying bullet points as paragraphs are read (instead of displaying entire paragraphs), and adding objectives at the beginning of each section. The section III. CYP2D6 case study was particularly well-received since participants found it most closely related to their work as pharmacists.

Discussion

One of the purposes of this pilot study was to determine whether multiple-choice interactivity incorporated into an animated online program in Pharmacogenetics would enhance learning outcomes. The research data did not demonstrate a statistical difference between the test scores of those who viewed the non-interactive presentation and those who viewed the interactive presentation. While overall learning outcome did increase between both groups, only those who viewed the linear, animated program scored significantly higher in their post-test compared to their pre-test. It was expected that the multiple-choice questions parsed between animated sections would provoke the learner to contemplate what they had just seen and as a result facilitate more complex modes of learning, such as reflection and abstract thinking.

While learners who tested the interactive version perceived themselves to reflect on the information more, the results from the pre- and post-test scores did not demonstrate this. Several reasons exist which might explain this incongruence. Most obviously, the small sample size may have skewed results. This is typical of pilot studies and to be expected. Another reason may be that the type of question may have been inappropriate for testing the cognitive benefits of increased processing and reflection. Perhaps problem-solving type questions could have been asked instead of straight recall questions. Another potential reason is that the design of the study failed to control for an extra-motivated non-interactive group. It is possible that these participants gave more thought to the content than if they were not in an evaluating situation (i.e. simply watching the module as a course).

The feedback given in the qualitative assessments was highly variable. While some participants preferred to answer questions interspersed throughout the sections, others preferred having them at the end. Some wanted more textual information while others appreciated the paucity of statistical figures on display. This suggests that learner preference should be considered carefully and more choices should be given to users to satisfy their individual learning needs. With advanced multimedia technology at our fingertips, personalization of learning environments can be realized. More thought should be given to this concept when developing a multimedia module in the future.

Feedback was uniform on one matter, however. The high level of excitement and enthusiasm among participants of this pilot study offers encouragement for the use of animated content in professional development courses. While research is still providing mixed messages on the efficacy of animations in education, such feedback suggests that this learning method is well-received. Animations have the potential to help learners build coherent, high-quality mental models of complex change processes, especially since they can depict situational dynamics explicitly. Additionally, interactive animations provide opportunities for learners to deal with available information selectively and so avoid excessive processing demands (Lowe 2004).

Future directions for research as a result of this study might include efforts to tweak its design to increase research design power. Including a larger sample size as well as more discerning test questions and interactivity elements might lead to clearer results as to the efficacy of interactive animations in education. Examples of alternative questions, such as case-based questions that test applied knowledge rather than memorized facts, would allow for more sensitive testing. Alternative types of interactivity, such as chatrooms or puzzles might also reveal the circumstances in which interactivity best assists online learning.

Possible extensions of this research would explore the ability of online teaching modules to provide a personalized learning experience that accounts for an individual’s learning preferences. The highly diverse set of participant feedback comments on the module design indicates a desire among the participants to engage learning in a manner best-suited to their learning abilities. Determining whether the provision (and viability) of a module with content and design tailored for the individual as determined from user input increases learning outcomes would be an intriguing study. Indeed, it is entirely possible that interactivity itself may be a mode of learning that suits a particular individual well or not. Visual learners and auditory learners might soon be joined by interactive learners.

References

Adobe Systems. Adobe Illustrator CS™ Version 11.0. 2004. San Jose, CA: Adobe Systems, Inc.

Adobe Systems. Adobe Photoshop CS™ Version 8.0. 2004. San Jose, CA: Adobe Systems, Inc.

BIAS, Peak™ Version 3. 2003. Petaluma, CA: BIAS Inc.

Brock, T.P., C.M. Faulkner, D.M. Williams, and S.R. Smith. 2002.

Continuing-education programs in pharmacogenomics for pharmacists. American Society of Health-System Pharmacists 59(8): 722-725.

ChanLin, L. 2001. Formats and prior knowledge on learning in a computer-based lesson. Journal of Computer Assisted Learning 17: 409-419.

Clark, J.M., and A. Paivio. 1991. Dual coding theory and education. Educational Psychology Review 3:149-210.
http://www.thecounter.com/stats/2004/May/res.php, January 01, 2005, 11:00pm.

Davis, D., M. Evans, A. Jadad, L. Perrier, D. Rath, D. Ryan, G. Sibbald, S. Straus, S. Rappolt, M. Wowk, and M. Zwarenstein. 2003. The case for knowledge translation: shortening the journey from evidence to effect. British Medical Journal 327:33-35.

De Muth, J.E. 1991. A national pharmacy survey on the availability of hardware required for continuing education delivery systems. American Journal of Pharmaceutical Education 55: 36-39.

Electric Rain. Swift 3D™Version 4. 2004. Boulder, CO: Electric Rain, Inc.

Gilbert, L., and D.R. Moore. 1998. Building interactivity into web courses: Tools for social and instructional interaction. Educational Technology 38(3): 29-35.

Hegarty, M. 2004. Dynamic visualizations and learning: Getting to the difficult questions. Learning & Instruction 14(3), 343-351.

Hollander, S.M., and D. Lanier. 1995. Orientation to the Internet for primary care health professionals. Bulletin of the Medical Library Association 83(1): 96-98.

Klein, J.D., and C. Koroghlanian. 2004. The effect of audio and animation in multimedia instruction. Journal of Educational Multimedia and Hypermedia 13(1): 23-46.

Maxon Computer. 2003. Cinema 4D™ Version R8. Newbury Park, CA: Maxon Computer GmbH.

Lewalter, D. 2003. Cognitive strategies for learning from static and dynamic visuals. Learning and Instruction 13: 177-189.

Lowe, R. 1999. Extracting information from an animation during complex visual processing. European Journal of the Psychology of Education 14: 255-244

Macromedia. Macromedia FlashMX™ 2004. 2004. San Francisco, CA: Macromedia Inc.

Mayer, R.E. 2003. The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction 13: 125-139.

Mayer, R.E., and R. B. Anderson. 1992. The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology 84(4): 444-452.

Moreno, R., R.E. Mayer, H.A. Spires, and J.C. Lester. 2001. The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction (19): 177-213.

Moss, S.H., P.H. Redfern, and K.N. Brown. 1998. Opportunities for the use of computer-based media in pharmaceutical training and education. Pharmaceutical Science & Technology Today 1(2): 54-61.

Narayanan, N.H., and M. Hegarty. 2002. Multimedia design for communication of dynamic information. International Journal of Human-Computer Studies 57: 279-315.

Qureshi, N., R. Hapgood, and S. Armstrong. 2002. Continuous medical education approaches for clinical genetics: A postal survey of general practitioners. Journal of Medical Genetics 39: e69 <www.jmedgenet.com/cgi/content/full/39/11/e69>.

Schittek, M., N. Mattheos, H.C. Lyon, and R. Attström. 2001. Computer assisted learning: A review. European Journal of Dental Education 5: 93-100.

Sim, R. 1997. Interactivity: A forgotten art? Computers in Human Behavior 13(2): 157-180.

Teo, H., L. Oh, C. Lei, and K. Wei. 2003. An empirical study of the effects of interactivity on web user attitude. International Journal of Human-Computer Studies 58: 281-305.

Tversky, B., J. B. Morrison, and M. Betrancourt. 2002. Animation: Can it facilitate? International Journal of Human-Computer 57: 247-262.

Weiss, R.E., D.S. Knowlton, and G.R. Morrison. 2002. Principles for using animation in computer-based instruction: Theoretical heuristics for effective design. Computers in Human Behavior 18: 465-477.

Wilson V., and L. Bagley. 1999. Learning at a distance: The case of the community pharmacist. International Journal of Lifelong Education 18(5): 355-369.

Acknowledgements

This research was generously funded in part by the CIHR Health Research Communications Award and the Vesalius Trust Vesalian Scholar Award.

Authors

Michelle Lui, B.Sc., M.Sc.BMC, is a Medical New Media Developer at InViVo Communications Inc. At the time of writing, she was a second year student in the Master's of Science in Biomedical Communications (MScBMC) Program, Institute of Medical Science (IMS), Faculty of Medicine at the University of Toronto (UofT). Prior to entering the program, she completed a Bachelors of Science degree at UofT specializing in Pharmacology. Michelle’s skills and dedication to health sciences knowledge translation was honored by the Canadian Institute of Health Research (CIHR) with the Health Research Communications Award. Her interests include knowledge translation of complex and conceptual scientific material and interactive instructional design.
Email: michelle.lui@utoronto.ca

Jodie Jenkinson, B.A., M.Sc.BMC, is an Assistant Professor specializing in instructional design in BMC in the Institute of Communication and Culture (ICC) University of Toronto Mississauga (UTM). She is the author and designer of The Prostate Centre, a CD-ROM and Web site and co-author of the BreastMatters Web site, both educational programs designed for patients of The University Health Network in Toronto. Jenkinson has a cross-appointment to the Department of Surgery. Her research is in interface design and data visualization.

Jana Bajcar, B.Sc.Phm., M.Sc.Phm, Ed.D., FCSHP, is an Associate Professor in the Leslie Dan Faculty of Pharmacy & Department of Family and Community Medicine, Faculty of Medicine at UofT. She is an academic pharmacist with a doctoral degree in health education. Her research deals with reduction of preventable medication-related problems through innovations in patient and clinician medication education.

Micheline Piquette-Miller, BSc, PhD, is an Associate Professor and Director of the Divison of Pharmaceutical Sciences at UofT. She currently serves as Associate Editor of Clin. Pharmacol. Therap. and is President of the Can. Soc. Clin. Pharmacol. She has served as Chair of Pharmacogenetics & Molecular Pharmacology and Board of Directors for the American Society of Clinical Pharmacology and Therapeutics (ASCPT). Her research focuses on molecular regulation of drug transport and multidrug resistance.

Linda Wilson-Pauwels, AOCA, B.Sc.AAM, M.Ed., Ed.D., is Director of BMC and Director of the MScBMC program. Dr. Wilson-Pauwels is a tenured Professor and is cross-appointed to the Department of Surgery. She is a visual communicator with advanced degrees in higher education. Her interest is in information transfer to target audiences, development of visual organizational tools, and visual cues and their role in directing attention.