At inspirED 2022, we had the opportunity to speak with Professor Lambert Schuwirth, one of the pioneers in introducing and implementing programmatic assessment. Our Content Specialist, Dan Hasan and Dr. Schuwirth had a witty and insightful conversation on how programmatic assessment can provide holistic, data-grounded ways to evaluate students, while stimulating lifelong learning. Below are the highlights of this conversation.
And you can watch the full conversation here.
There has been a major shift in the role of assessment in higher education, from assessment of learning to assessment for learning. That is, assessment should provide rich information that guides and fosters the learning process, instead of labeling students as competent or incompetent. To generate and process information, assessment for learning relies on formative assessment approaches such as feedback.
However, Lorrie Shepard in her article, The role of assessment in a learning culture, proposes that assessment for learning does not suffice to measure student achievement:
“Only after successful completion of a module can students progress to the next.”
This means that summative assessment is still essential to the evaluation process.
Programmatic assessment then emerges as an ideal way to harmonize both traditional and modern assessment approaches. The ultimate goal of programmatic assessment is to provide the whole picture of a student’s “competency” via a combination of assessment methods.
This novel approach to assessment seems to be promising, and appealing. But can it be applied widely at institutions, or is it just another hype in education?
“The idea of programmatic assessment emerges from medical studies, because it mimics so much the process when the doctor diagnoses the illness.”
Dr. Schuwirth summarized the idea behind programmatic assessment with an analogy of medical education.
To generate accurate diagnosis, the doctor has to rely on different channels like blood tests, physical check-ups, conversations with patients, and so on. Similarly, triangulation is the key to assessing students’ competencies. By triangulating assessment formats across a longitudinal basis, we can generate rich information to make “high-stakes decisions” regarding the student’s competency.
As we keep talking about competency, what is it exactly? Competency is defined as the ability to complete a task by integrating knowledge, skills, attitudes, and professionalism. How competency is divided and measured varies according to institutions. Competency-based education (CBE) is then developed to help students master the core competencies required for their chosen fields of study, while allowing faculties to evaluate how well their learners accumulate the desired skills.
According to Katie Larsen McClarty and Matthew N. Gaertner, author of the article ‘Measuring Mastery: Best Practices for Assessment in Competency-Based Education”:
“A CBE model is workable only insofar as its measures of learning yield trustworthy data about students’ prospects for future success.”
A major challenge facing competency-based education (CBE) is to ensure the effective use of robust assessment methodologies that result in useful performance-relevant information (PRI) by which to provide feedback on learning and make decisions about learner progression.
Programmatic assessment; therefore, is the perfect evaluation methodology for the CBE model, due to its reliance on abundant learning data and methods to collect these data.
The Competency-Based Medical Education (CBME) Core Components Framework highlights programmatic assessment as one of the 5 essential components for competency-based teaching. In essence, programmatic assessment must ensure the following elements:
The full description of the Core Components Framework can be found here.
To further understand the idea behind programmatic assessment, Dan raised the question: “What makes programmatic assessment different from traditional approaches?”
The main difference, according to Dr. Schuwirth, lies in how each approach generates and makes use of the information. While the traditional approach is inherently reductionist, producing rich information from tests then reducing them to a score; programmatic assessment aims to retain the complexity of information.
“Learning is a complex interactive process, you can’t apply a reductionist approach. In Dutch, we usually say – If you disassemble a bike, then reassemble it, you will have a bike. But if you dissect a frog, and glue it back together, it is not a frog anymore. So it is the interaction that counts.” – Dr. Schuwirth commented.
Learning is indeed interactive and complex, and it requires a combination of different methods to accurately measure and gauge how far students have progressed over the course of learning.
Student learning and decision-making functions are the focal points of programmatic assessment.
Instead of assigning grades as incentives to motivate students, meaningful and continuous feedback is provided to help students reflect, improve and become better learners. As stated by Dr. Schuwirth:
“What we want to instill on our students is that they continue to learn, and have the agency and ability to assess, collect information, self-assess, reflect, and to set goals for future learning after graduation.”
Rich information is then needed to activate lifelong learning. Instead of connecting pass/fail decisions from one single assessment moment, multiple assessment approaches should take place over a longitudinal basis. Eventually, the information generated from this assessment combination becomes sufficient to reach summative decisions.
The main focus is the student’s learning. Instead of ‘jumping through summative assessment hoops’, assessments should promote learning. This is done by disconnecting the pass/fail decision from one single assessment moment. Instead, the student is provided with meaningful and rich feedback to help them develop themselves. Ultimately, multiple moments of assessment over a longer period of time lead to a pass/fail decision. This decision is made to decide if the student has met the desired learning outcomes. As such, there is no decision moment based on a single assessment; the decision is made based on rich information on the student’s learning trajectory.
The conversation then switched to the question on the role of modern technology in adopting programmatic assessment with Dan asking, “Our students today have all sorts of affordances. Do the Internet, and digital and social infrastructures necessitate programmatic assessment more than traditional assessment?”
“I would say absolutely! Our students have affordances that are unprecedented.”
And what are those affordances? Students can now be part of multiple communities, have access to different channels and modes to communicate and create knowledge. Most importantly, modern technology exposes students to endless sources of information, and knowledge. This then led to a shift in the imbalanced relationship between teachers and students, as now students also become the knowledge creators. And pedagogical technology has quite a major role to play in this shift in education.
“What relationship do you think educational technology should have with the development of programmatic assessment at institutions?”
Edtech can play an enormous role in optimizing the educational processes, as it can help reduce the manual work of administrative tasks, survey forms, logistics, and learning management systems. When integrating edtech into institutions, it is important to make sure there are opportunities for co-construction and co-design. Most importantly, pedagogical technology should be utilized to enhance, augment, and strengthen interactions and engagement throughout the learning process.
Below you can find articles that provide more background to programmatic assessment, mostly written by Cees van der Leuten and Lambert Schuwirth, two key advocates of this approach.
Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles | A consensus statement on the 12 principles underlying programmatic assessment.
Programmatic assessment: From assessment of learning to assessment for learning | An introduction to programmatic assessment, its definition, principles, and benefits.
Twelve tips for programmatic assessment | 12 tips on how to implement programmatic assessment.
Discover how to maintain continuous student engagement at every stage of your course using Interactive Document
FeedbackFruits announces partnerships with many institutions worldwide over the past 4 months
An overview of the state of competency-based education (CBE) in higher education around the world
An insightful recap of our annual conference discussing innovative and sustainable strategies for student-centered learning.