| Title | Math Exam Resources Wiki Review and Development |
|---|---|
| Faculty/College/Unit | Science |
| Status | Completed |
| Duration | 1 Year |
| Initiation | 04/01/2014 |
| Completion | 11/19/2015 |
| Project Summary | The Math Exam Resources (MER) wiki, hosted on the UBC wiki, is a free online learning resource for UBC students taking large multi-section math courses, particularly first year calculus. The resource consists of over 1500 reviewed hints and solutions to more than 30 previous department exams, with all hints and solutions written and curated by math graduate students on a voluntary basis. We proposed to develop and execute a study to measure how the MER wiki is used by students and how it impacts their learning, in particular mathematical understanding and problem-solving skills. This aim of the study was to help evaluate the benefits of sustaining the MER wiki as an online learning resource. As a second objective, we proposed an expansion to the scope and reach of the MER wiki to develop its potential and to increase its flexibility of use. |
| Funding Details | |
| Year 1: Project Year | Year 1 |
| Year 1: Funding Year | 2014/2015 |
| Year 1: Project Type | Large TLEF |
| Year 1: Principal Investigator | Eric Cytrynbaum |
| Year 1: Funded Amount | 12,508 |
| Year 1: Team Members | Eric Cytrynbaum, Mathematics, Faculty of Science |
| Project Report | Report-2014-FL-Cytrynbaum-WEB.pdf |
| Project Outcomes | Products & achievements: The Math Exam Resources (MER) wiki (http://wiki.ubc.ca/Science:Math_Exam_Resources); web framework to address shortcomings and suggestions from students (www.math-education-resources.com); report of study findings (https://github.com/MathEducationResources/FL-TLEF_Report/blob/master/report.md). Intended outcomes/themes:
Evaluation approach: The student improvement project goal had data collected via an online survey and cognitive interviews with students. A link was distributed through the Mathematics department to students enrolled in undergraduate mathematics courses. The data was collected anonymously. Students were asked to provide a contact email address (kept separately from their response data) so that they could be offered a complimentary version of a mobile app for the MER wiki as a gift for completing the survey. The survey had a question indicating if students would be willing to be contacted for a follow-up interview and a random selection of 15 people who agreed to the interviews were chosen. The interviews were conducted with various members of the project team with one interview participant and 2 to 3 project team members per interview. The raw data for the online surveys was collected and analyzed for various tests of statistical significance. The analysis of these data and the full implications of these results are ongoing. The transcripts of the interviews were analyzed and summarized into broad categories of impact on learning and suggested wiki improvements. The final report which summarizes the conclusions from statistical analysis of data and inferences from interviews is available at https://github.com/MathEducationResources/FL-TLEF_Report/blob/master/report.md. The main metric for project sustainability was a pilot project for departmental TA support on the MER wiki. The evaluation involved communicating with the TA in charge of the project and documenting their experiences and suggestions. Findings: Develop long-term sustainability model - This goal could not be fully achieved. The number of volunteer contributors could not be increased significantly and faculty involvement was extremely limited. In fact, since the current contributors all graduated, there is no ongoing wiki development. However, we were able to pilot a sustainability model with graduate and undergraduate teaching assistants producing and editing solution content as well as a model where students contribute content as a review activity in their course. This sustainability plan was discussed with the head of the Department of Mathematics as well as the faculty responsible for TA assignments. The implementation of this sustainability plan is a choice entirely up to the Department of Mathematics. We hope that the final report underscores the positive impact of the MER wiki on student learning and helps the Department of Mathematics sees the value in continuing the work on MER. Develop new web platform to improve interactivity and impact - This goal was achieved to a major extent. We were able to automatically mirror the content of MER to the new platform (located at www.math-education-resources.com), increase the number of interactive features, and drastically simplify the process for contributors. Some of the new interactive features include multiple choice questions, an easier voting system, and a more streamlined topic search functionality mainly provide aesthetic improvements. However, the improvements of the content creation system cannot be overstated in terms of how much they simplify the overall content contribution process. Using the old wiki, contributors needed to have a fairly sophisticated knowledge of the raw code involved, even for simple tasks like adding or reviewing solutions. This has all been automated through the use of custom graphical user interfaces. The overall goal of long term sustainability previously mentioned may not have been accomplished to a standard we would have liked in the lifetime of the FL funding, but the implementation of our new platform has made long term sustainability a much more attainable goal. As with the content of MER, all work on the new platform is open source under an open license. Some features that could improve the interactive features of the website even further have been suggested but have not yet been implemented due to time constraints. Training in project management skills and data analysis - This goal was achieved to full extent. The roles of contributing were clearly understood by all team members throughout the project. An extensive final report has been created that summarizes the student responses to the survey and interview questions, and highlights MER’s impact on students. We gathered plenty of new data and the final report gave plenty of opportunity to practice data analysis skills. Dissemination: Papers:
Posters:
Presentations:
After thoroughly completing the analysis of the online survey data and interview responses we plan on producing a manuscript or series of manuscripts about the results and implications. Sustainability: The resources and web application developed with this grant will be online and useful to learners at UBC and elsewhere for years to come. As such, it will continue to impact thousands of math-taking undergraduate students. This remains true even in the absence of new contributors, since the current version has over 1500 solutions which will continue to be accessible to future cohorts of students regardless of system updates. However, should the developed sustainability model become implemented by the UBC Mathematics Department, this would ensure that the online material gets updated and extended regularly, improving the service to current and future students. Our report outlining the positive impact of the MER on student learning could inspire similar online learning platforms at UBC or other institutions of higher education. Further, MER has also proven the potential of the UBC Wiki and could be a role model for future innovation on this UBC platform. |