An Embedded Systematic Program Evaluation Process for Pharaceutical Sciences

The Faculty of Pharmaceutical Sciences is implementing a new competency-based curriculum and simultaneously developing a comprehensive process to monitor and improve the quality of its educational outcomes. The proposed evaluation process integrates the collection of evaluation data during the routine conduct of specific program activities (e.g., student recruitment, admission, instruction etc.) with the aim of creating and fostering a culture of continuous program improvement in our Faculty.

Rationale: Questions about a program’s quality/success cannot be answered without relevant outcomes data. Refinements to educational programs should be guided by evidence derived from evaluations of program accomplishments and shortcomings. To be successful, such a program evaluation process needs to be comprehensive, systematic, non-threatening, simple, and, to the greatest extent possible, embedded in daily routine. The Faculty of Pharmaceutical Sciences intends to develop and operationalize such a program evaluation model.

Specific objectives:

1. To develop a comprehensive evaluation process that identifies critical program elements (those program activities that could impact the quality of program outcomes), identify data to be collected as indicators of program accomplishments and the process by which this evidence should be gathered (when, how and how often and by whom) for the B.Sc. (Pharm). Program (Year 2004-05).
2. To Develop a prototype evaluation tool/instrument for each program element and have it refined by key stakeholders (e.g., students, instructors, high school career counsellors, etc.) to ensure effectiveness and efficiency (to the greatest extent possible, data collection will be embedded as part of usual and customary activities and use the latest technological advances in data capture and automation) (Year 2004-05).
3. To develop necessary policies and procedures to guide data collection and evaluation judgements. (Year 2004-05).
4. To pilot-test the program evaluation tools for each program element, i.e., gather evaluation data and compare with expectations (specific standards) and judge whether the tools permit valid inferences about the outcomes of that specific program element (e.g., instruction) (Year 2005-06).
5. To refine program evaluation tools and evaluation process as needed (Year 2005-06).
6. To implement a field-test of the full evaluation process using the B.Sc. (Pharm.) program (Year 2006-07).

Methods: In order to ensure development of quality evaluation tools for each program element, we will engage in the following process: 1) instrument development through consulting the literature, key stakeholders, determining current practice and tools, potential practice and tools, as well as locating, evaluation and pilot-testing existing software that will allow us to automate evaluation, 2) obtain feedback from key stakeholders who will pilot test tools for effectiveness and efficiency, 3) modify tools based on pilot test dada, 4) implement full field test and refine entire evaluation process before adoption by faculty.

David Fielding, Faculty of Pharmaceutical Sciences
Ingrid Price, Faculty of Pharmaceutical Sciences
Michael Marshall, Executive Director, ARES, Faculty of Education

David Fielding, Faculty of Pharmaceutical Sciences
Ingrid Price, Faculty of Pharmaceutical Sciences
Michael Marshall, Executive Director, ARES, Faculty of Education