| Project Summary | Objectives:
The goal of this project is to integrate multimedia into course-based computerized comprehensive examinations in the Undergraduate Medical Program, Faculty of Medicine, particularly by incorporating dynamic stimuli such as audio, video, and animation. The following objectives will assist in achieving this goal:
- Develop a valid model for integrating multimedia that will facilitate performance assessment of problem solving and related cognitive constructs by allowing the presentation of tasks more like those actually encountered in academic and professional settings.
- Pilot studies in the effect of different item formats in order to compare the performance and evaluation results.
- Develop multimedia assessment items (exam questions) that will aim to cultivate an understanding of the diagnostic process, and promote a sound base of knowledge integrated with problem solving.
- Implement model for integrating multimedia assessment.
- Increase research component regarding multimedia and computers in assessment.
Rationale (why):
With the increase in the number of medical students and the transition to geographically distributed sites of program delivery, the role of information technology in the learning environment greatly increases. The implementation of information technology, however, is not a mere replication of the existing program in a different media. It requires adaptations in program delivery and features. Incorporating multimedia into exam questions improves the fidelity of questions and supports enhanced problem-solving. Medicine is a discipline where including multimedia in test items can greatly enhance the validity of the assessment. Problem-solving skills in medicine require the practitioner to be able to interpret an assortment of results (sounds, images, physical exam findings) in the medical setting to arrive at a diagnosis, prognosis, or treatment plan. Multimedia can present test items in a manner more consistent similar to how a practitioner would receive medical information. By requiring the student to interpret the multimedia stimulus, the student is tested at a higher level of problem solving and encourages learning at a deeper level. Many areas within medical education are highly visual. For example, microscopic images, electrocardiograms, and radiographs all require high levels of visual interpretation. Through the use of multimedia test items, problem solving and related cognitive constructs can be measured more effectively and present the student with tasks more like those actually encountered in academic and work settings.
Methods (how):
The development of the project will reflect a team approach. Coordination of the project and educational expertise will be provided through the Division of Educational Support and Development (DESD) and the Student Assessment Committee (SAC) in the Faculty of Medicine (FoMD). The process will involve three phases:
- Phase I (-June 2006): The initial phase will focus on developing a valid model for integrating multimediaassessment into the FoMD computerized course-based comprehensive examinations. The Student Assessment Committee (SAC) will create a working group to direct the model development phase. The working group will research and evaluate the literature available regarding integrating multimedia assessment items into a medical school curriculum examination. The working group will also organize, conduct, and summarize several focus groups sessions with Faculty of Medicine students, faculty, and staff. Emphasis will be placed on grounding the model in accepted test theory. The Project and Assistant Project Coordinator will oversee this phase of the project with the help of two medical students.
- Phase II (June -October 2006): The second phase will focus on the development of multimedia assessment items. By the end of phase II, the project will have developed multimedia assessment items for year 1 of the curriculum. During this phase, students will classify existing assessment items and help identify which items lend themselves to multimedia enhancement. The student graphic designer and programmer will work in collaboration to develop and customize the multimedia assessment items. Student and faculty involvement will continue by conducting focus and workgroup sessions regarding the design and development of the items. Medical experts will be used to target content and provide consultation in the design process and user interface functionality. Educational experts will be used to incorporate sound pedagogical learning principles.
- Phase III: The last phase will focus on piloting of multimedia items. The piloting will compare the performance and evaluation results of randomly assigned first year volunteer groups on equivalent-sets of formative assessment items. One set of items will be administered via computer and will use audio, video, animation and high-resolution images; the other set, delivered in paper and pencil format, will contain the verbal descriptions of the multimedia features, as currently done in the existing assessment format. Order of set-administration will be randomized. At this point, final adjustments will be made and evaluation will begin.
|
|---|
| Year 1: Team Members | Justin M. Bonzo, Educational Support and Development, Faculty of Medicine
George Pachev, Director, Student Assessment
Santiago Toro-Posada, Manager, Evaluation and Student Assessment
Leanne Picketts, Examination Coordinator
Brian Powell, Coordinator, Learning Projects
Emily Ling, Faculty Student Assessment Consultant
Helen Nadel, Faculty Student Assessment Consultant
Duncan Etches, Faculty Student Assessment Consultant
Jasmine Lam, Medical Undergraduate Society Student Representative
Jory Simpson, Medical Undergraduate Society Student Representative
Blaine Martens, Student, Pharmacology
Mike Hicketts, Student, Pharmacology
Chad Trytten, Student, Computer Science
|
|---|