UBC tool innovates peer feedback process

ComPAIR, a cross-disciplinary teaching and learning tool, is offering a new approach to peer feedback. Developed at UBC, this open-source online tool offers an innovative extension of traditional peer feedback tools by tapping into students’ innate ability to compare. By presenting pairs of assignments side-by-side, students have a reference point when reflecting on strengths and weaknesses in their peers’ work and their own.

The tool

To use ComPAIR, instructors first design a question or assignment for their course. Using the intuitive tool, instructors can customize the assignment details, timeline and criteria to best suit desired learning goals. After students submit their assignment online, ComPAIR generates pairs of peer answers displayed next to each other. For each pair, students select the answer they think best meets instructor-set criteria and offer feedback. Instructors can also ask students to then revisit, revise or comment on their own original answer, using the new level of understanding they gained from assessing their peers’ work to reflect on their own.

“One of the reasons why comparisons are so useful for novices is that they’re easy to do,” explains James Charbonneau, an Instructor in the Department of Physics & Astronomy and Associate Director of Science One and the Coordinated Science Program.

The process is based on a psychological principle called Thurstone’s Law of Comparative Judgment, which states that novices are better at comparing two things rather than judging one item in isolation.

“If you ask students to grade one of their peers’ assignments…it’s a really hard thing to do,” Charbonneau says. “But if you give students two assignments and ask ‘Which one is better?’ then that’s a lot easier.”

Mark MacLean, Professor of Teaching and Undergraduate Chair in the Department of Mathematics, adds, “We’re helping students scaffold from their novice states of understanding and making judgments to a more expert state of thinking.”


Asked to “provide feedback” just on the first square below, students might note 9 boxes, a square and letters.

 

A B C
D E F
G H I

A d G
b E H

 

C

 

f

 

i

But once they see that square in comparison with the second box, they can recognize new qualities: letters are all caps, oriented horizontally, all in the same font, all centered in boxes of the same size and shape. Comparisons change what students observe in individual work and reveal qualities that can otherwise be overlooked.

Figures above modified from: Bransford, J., & Schwartz, D. (1999). Rethinking Transfer: A Simple Proposal with Multiple Implications. Review of Research in Education, 24, 61-100. http://www.jstor.org/stable/1167267


The development

After receiving funding in 2014 from the Teaching and Learning Enhancement Fund, the project team began developing the tool through a participatory design process in which the development team and end users came together to collaborate on the design. The team included developers, educational specialists from the Centre for Teaching, Learning and Technology, and faculty members from Physics, Math and English.

“It was interesting to have a team of people who were coming from such disparate teaching spaces,” says MacLean. “These three departments are further apart than you might think in our traditional approaches to teaching.”

To accommodate all three departments, the team created a flexible design that can be used across disciplines, class sizes and assignment types. ComPAIR accepts a range of assignment formats, including text answers, image files, multimedia, scientific and mathematical notation, as well as alphabetical notation for several European and Asian languages. It is also fully integrated with UBC’s online learning platform Canvas.

“It was really designed to allow cross-disciplinary usefulness,” says Tiffany Potter, Professor of Teaching and Associate Head, Curriculum and Planning in the Department of English Language and Literatures. “Because of that participatory design process, we ended up with something that was really widely valuable and flexible for different kinds of teaching.”

English students gain hands-on practice

Potter piloted the tool in English 110: Approaches to Literature, a course with around 180 students. The week prior to an in-class essay, students drafted and submitted short critical premises on ComPAIR. They then provided feedback on three pairs of their peers’ answers. Using the tool’s capacity to allow comparison on multiple axes, Potter asked students to evaluate both the content and articulation of ideas. In tutorials, teaching assistants reviewed strong answers and common challenges before leading an essay development exercise that developed still further the deep thinking already in progress in the ComPAIR assignment.

Potter structured the assignments to give her students hands-on practice followed by peer assessment and self-assessment before a higher stakes assignment. “The way I use it, it’s not a standalone online exercise,” Potter explains. “It’s an online exercise explicitly in dialogue with the work they’re going to do in person in their tutorial.”

Each assignment was worth 4 per cent, with the student’s answer and feedback equally weighted. Potter says, “The learning from this exercise is not only from the critical process of answering, but also from the critical process of assessing and offering feedback to their peers. Those are both essential intellectual processes that are beneficial in this assignment and so they need to be equally weighted.”

Physics students explore different viewpoints

Physics 333: Climate and Energy is an online course with around 80 students. The course includes a ‘Big Picture’ assignment—an open-ended and vague question where students use their problem solving and collaboration skills.

Charbonneau wanted to give his students an opportunity to see different approaches and viewpoints in the assignments, and to explore the grey areas in physics. Using ComPAIR, students submitted 3-6 page written and calculated solutions and completed three pairs of comparisons followed by a self-evaluation.

Students were given two external rubrics: the first had guidelines on how to assess their peers’ work, and the second detailed how the instructor would determine their overall assignment grade. Students were never graded on the content itself—rather, they were assessed on the quality of their feedback and their contributions to the process.

“I don’t think I could teach my Physics 333 course without [ComPAIR],” Charbonneau says. “I think that without this tool, the students would be missing some key elements of the course learning goals.”

Math students illustrate their understanding

MacLean first used ComPAIR in MATH 317: Calculus IV, a third-year lecture with around 275 students. At the end of two major units in the course, students completed a ComPAIR assignment for participation points. The assignments were a chance for students to explore the relationship between mathematical machinery and the geometry it is intended to explore. “I wanted to have students demonstrate how well they understood [a mathematical tool] and how they could apply it to a situation,” MacLean says.

Students were asked to show how a certain tool would help them answer a particular mathematical question. Rather than a written or calculated solution, MacLean asked students to draw a series of figures to visually illustrate their understanding of the concept. Students completed three comparisons and a self-evaluation.

The response

The project team has conducted surveys, focus groups and interviews with students across disciplines. Among the findings, students reported that their critical thinking and ability to offer feedback had strengthened, and that they felt more confident reflecting on their own work.

Charbonneau believes it is the opportunity to see their peers’ work that students most appreciate. “It’s very rare you get to see somebody else’s assignment outside of your friend group,” Charbonneau says. “There’s a lot of value in seeing how someone structured their paper or calculation.”

Another key element that students appreciate is the anonymity of feedback.

“One of the challenges of doing in-class peer feedback is that it’s stressful to tell a classmate their work is imperfect. There’s a social risk in that, and there’s a social anxiety around it,” Potter says, noting that this can often lead to students providing only surface-level edits on spelling, rather than on deeper analytical work.

With ComPAIR, Potter has noticed that students are offering higher quality, in-depth feedback. “Our students want to help each other,” Potter says. “They want to do well, and they want to learn from each other.”

What’s next

ComPAIR has grown substantially—close to 6,000 students at UBC have now used the tool in more than 60 courses. A ComPAIR working group has been formed, with developers and faculty members from across the university.

This UBC-created tool is also gaining worldwide attention. It is now being used in more than a dozen institutions across Canada, the United States, Australia and the United Kingdom. ComPAIR also recently received a silver medal at the 2019 IMS Global Learning Impact Awards.

“There’s great potential for it to become simultaneously something that supports students learning, but also helps us understand the trajectory of that learning,” MacLean says.

Potter adds, “We’re really excited about the possibility that ComPAIR can be a learning tool that has impact well beyond UBC.”

To learn more about ComPAIR, visit the demo site and the resources below.

Resources