Checkpoints help PharmD students self-assess learning

New curriculum emphasizes formative assessment opportunities

In the Faculty of Pharmaceutical Sciences, discussions of assessment and how to evaluate student learning have been circling for years. When the Faculty began planning a new degree program, it became an opportune time to revamp the nature of assessment.

After more than three years of planning, the Entry-to-Practice (E2P) Doctor of Pharmacy curriculum launched at UBC in 2015, phasing out the Bachelor of Science in Pharmacy degree. This six-year program, known as the PharmD program, aligned with a nation-wide shift to more practice-based, patient-focused pharmacist education programs that target nationally-defined educational outcomes.

Within this program, one of the new features of the assessment system is an emphasis on formative assessment, or assessment for learning. This is paired with a commitment for valid and reliable summative assessment, or assessment of learning.

As the first few courses were implemented in the new program, the emphasis on formative assessment led faculty to create weekly quizzes known as Checkpoints. These Checkpoints gave students an opportunity to test their understanding of course material and look at their learning progress. With the rollout of the new program, and with support from the Teaching and Learning Enhancement Fund (TLEF), Checkpoints have become a permanent feature in PharmD courses.

A new outlook

“Assessment drives learning, and the trick is to steer it in the right direction, to drive the learning,” explains George Pachev. Pachev, the Director of Educational Assessment in the Faculty of Pharmaceutical Sciences, was brought aboard for the development and implementation of the new assessment program.

The Bachelor of Science in Pharmacy program, which ends in 2018, was a more traditionally structured program and consisted of courses taught by individual instructors. Instructors, as the content experts, designed their own courses and assessment considerations often came at the end of the design process. This made assessment highly fractionalized.

“Those responsible for the courses were responsible for their assessments in the course,” Pachev explains. “There was no look at students’ progress across courses.”

But in the PharmD curriculum, a practice-based program that takes a more holistic look at patients, it became important to look at the bigger picture of student learning.

“At one time our program was centred on the drug,” explains Simon Albon, Professor of Teaching and Director of the Office of Educational Support and Development in the Faculty. “Now it’s focused on the patient and how the drugs can help the patient, their quality of life, and improved health outcomes.”

Instead of focusing on disciplines, the Faculty began to focus on competencies. They looked at what milestones of development students should reach at the end of each term and year.

Led by Pachev and his team, the Faculty integrated a curriculum-wide program of assessment. Pachev explains that assessment programs have two functions. The first is accountability—to ensure that the university is producing capable pharmacists. The second is for student learning.

“[The assessment program] is there to support students’ learning,” Pachev says. “Formative assessment has been increasingly more [of the] focus of professional programs in particular and university programs in general in their attempts to implement different ways through which students can learn from assessment.”

Assessment in the Faculty is now a collaborative effort between course teams and the assessment team. Together, they create a blueprint to figure out what learning objectives they want students to attain, and how to reach them through different forms of assessment.

The Checkpoints

Checkpoints are formative assessments in the form of weekly online quizzes delivered through Canvas, UBC’s online learning platform. They provide study guidance as well as opportunities for practice and self-assessment.

Within the PharmD curriculum, many courses are subdivided into modules. Checkpoints are implemented into every course and every module. Depending on the course, some Checkpoints are assessed at a small percentage of the course grade, while others are marked for completion or participation—but their main purpose is to help students.

“They’re meant for students to check in on their own learning, and to help them be accountable in their learning,” explains Natalie LeBlanc, a Postdoctoral Research Fellow in the Faculty. LeBlanc was brought on as the project lead after the team received a Large TLEF grant in 2016 to support the development of the formative assessment program.

The Checkpoints consist largely of multiple choice questions. According to students, one of the most useful parts of Checkpoints is the feedback component. Each answer includes references to textbook pages, PowerPoint slides, or other resources for students to review the answer or learn more.

To support the creation of these questions, the project team turned to a unique type of expert—senior students in the Bachelor of Science in Pharmacy program, who could draw on their knowledge of the material and their experience learning it.

Learners as teachers

“[I was] trying to think about what questions I had at the time I was learning [the material] to target the right things to be testing the students on,” explains Elizabeth Ratcliffe. Ratcliffe was one of the students hired to help create Checkpoint questions. She worked closely with LeBlanc and the professor leading the module to figure out how to write a good question, and what information to include.

LeBlanc says the students’ ability to draw on their own learning experiences is one of the biggest benefits of using student-generated questions. “They’re very much in that state of learning, and they know what approach works for them, what types of questions work for them as a student.”

Sometimes, the students would also have to reflect on the gaps in their own knowledge and do additional research. Ratcliffe says this helped enhance her own learning.

“I found having to do my own research to make sure I had a full understanding, that I wasn’t going to write a bad question, meant that I came out with so much more knowledge than I went in with,” Ratcliffe shares.

This process, LeBlanc says, gave the students a unique learning opportunity. “Having students think back at the gaps in their own learning gives them a drive to want to learn and want to share that knowledge,” she says. “There’s a pivot that they do where they begin thinking less as students and more as teachers.”

The impact

At the end of each module, students complete an evaluation survey that includes specific questions about the Checkpoints. Students are asked to rate if the Checkpoints were a good opportunity to practice the material, if they helped them identify areas of strength and weaknesses, and if they helped direct and support their learning. According to the team, the results have been very positive.

Albon says, “Students really appreciate [the Checkpoints] for the reasons they were originally intended—it gives me practice, it helps me keep up, it helps me self-assess my knowledge and identify gaps.”

LeBlanc hopes the Checkpoints will also help students become more aware of their own learning. “When you’re the instructor and you’re teaching the content, there’s only so much that you can do for your students,” she says. “They have to become accountable for their own learning, and I think the Checkpoints help them do that.”

What’s next

After the first cohort of PharmD students graduate in 2019, the team will be looking at how formative assessment relates to performance in practicums, whether the program produces the types of graduates they are hoping, and how this cohort differs from previous graduates.

Now in its second and final year of funding from the TLEF, the team is working to ensure the sustainability of their materials and processes, so that they can continue to be used each year without external funding support.

Through this process, the team also hopes to move away from opinion-based decisions about curriculum, and towards the scholarship of teaching and learning—generating context-specific evidence for making curriculum decisions. Based on the students’ experience as question writers and content creators, the team will be exploring the impact of having learners as teachers and how this impacts the learning experience.

As the Faculty is settling into the third year of the PharmD curriculum and all the changes are steadying, the team hopes more instructors will begin to think about assessment early in the planning and design process.

“This project is instrumental in changing the whole culture of assessment within the Faculty,” Pachev says. He adds, “I’ve found that everyone has been very dedicated to teaching in the best way.”

Resources 

Stiggins, R. (2005). From formative assessment to assessment for learning: A path to success in standards-based schools. The Phi Delta Kappan, 87(4). http://www.jstor.org/stable/20441998

Black, P. (2015). Formative assessment – an optimistic but incomplete vision. Assessment in Education: Principles, Policy & Practice, 22(1). http://dx.doi.org/10.1080/0969594X.2014.999643

TLEF Showcase posters, May 2017 & May 2018: https://tlef.ubc.ca/funded-proposals/entry/11/

 –

Feature image photo credit: Justin Ohata/UBC Faculty of Pharmaceutical Sciences