The pandemic has generated chaos in the world of higher education, as institutions were prompted to move online in order to hinder the COVID spread.
Multiple challenges arised, how does one go from a physical classroom to a multi panel Zoom window, how do you balance real-time teaching with asynchronous learning and how do you support faculty and students along the way? #AcademicTwitter has been the new home for educators searching for tips, and many universities and organizations have gathered resources to kickstart online, blended, hybrid courses. Edtech vendors have lowered financial commitments, and in some cases have decided to offer their technology free of charge like we did.
It is during this changing time that we see a complete transformation in every aspect of course design, especially assessment. Institutions kept seeking and experimenting with novel, alternative ways to evaluate students such as formative assessment, Pass or No pass. Interestingly, it takes a pandemic to realize that higher education needs to transform, and institutions need to move away from emergency planning and evolve strategic, long-term strategies, they work towards “the higher education we deserve”.
How has assessment changed during the emergency online period, what were the challenges and opportunities regarding this transition? These questions will be answered in this article.
Once physical lectures are replaced by online conferences and weblectures the next item on the agenda to be addressed is student assessment. Several academic leaders moved quickly and decided that a beneficial way to continue to assess if students met their learning objectives in this time is to disregard the rigorous normality pretensions and shift all assessment to result in a Pass or No Pass option.
MIT was one of the first schools to take this decision, quickly followed by CUNY, Northwestern, Cornell and others. Thanks to Laura Gibbs we found a comprehensive article that provides a list with institutions that have moved towards a Pass or No Pass system or other alternatives.
Pass or No Pass allows instructors to focus on learning and growth, providing students the opportunity to improve their knowledge by trying and making mistakes without the feeling that their final GPA suffers. Nonetheless, in order to get to the Pass or No Pass system one or multiple assessments need to be taken by students to see how they meet with the constructive alignment principle where assessment of a course should match its learning objectives and learning activities. Current common e-assessment options offer faculty a wide range of valid activities to assess students.
During these times faculty reconsiders the assessment type they introduce to their “newly” thought-out course. To assess if its assessment type could be used we have to look at the validity and trustworthiness of the assessment type.
Validity: quite simply indicates if the assessment type covers the learning objectives of the course. In order to assess the validity of the assessment. Reliability, transparency and feasibility are key components to think of in the case of validity.
Trustworthiness: in order to assess the trustworthiness of an assessment type one should think primarily of cheat prevention but also of inclusiveness & accessibility and privacy.
The assessment types are all valid in the online space, however, are they all trustworthy?
1. Essays, reports and project presentations remain a valid and trustworthy type of assessment as students will still have the same type of resources to use. However, plagiarism can occur, and it can happen more often as there is less interaction with faculty, thus the institution needs to have plagiarism check technology in place, such as TurnItIn.
2. Oral examinations can now move to online 1-on-1 sessions via web conferencing technology like Zoom or Webex. This is under the assumption that all students have internet access these days, however, bandwidth may play an important role for students. Thus, faculty should take into account how easy to operational this methodology is for students.
3. Exams are less likely to occur in online courses - be it open book, closed book or digital. These types of evaluations have to take into account the available resources of the institution but most importantly of students. Proctoring is a viable solution for the trustworthiness of these assessment types allowing faculty to keep the current examinations. Tools like Proctorio, ProctorU, Exam.net lock the device of the students and the technology flags faculty as students try to cheat.
Discussion around the use of proctoring software is sparking up at every institution or online conference. Though these softwares are useful, educational leaders identified some key elements to consider: Institution resources; Student hardware resources to cope with this technology; Implementation and training.
Read more on different strategies to assess student performance in online learning:
“The purpose of assessment is not to gather data and return “results,” but to illuminate ways to strengthen the curriculum, instruction, and student learning” (Pearsons, 2006)."
The toolbox to assess learning outcomes is well filled with methods as described above. Yet each individual method of assessment has limitations. In higher education, not a single assessment method can do it all. As a substitute, faculty could move to more small and diverse data points to evaluate student learning outcomes, which is called programmatic assessment.
“The idea of programmatic assessment emerges from medical studies, because it mimics so much the process when the doctor diagnoses the illness.”
Dr. Lambert Schuwirth of Flinders University summarized the idea behind programmatic assessment with an analogy of medical education.
To generate accurate diagnosis, the doctor has to rely on different channels like blood tests, physical check-ups, conversations with patients, and so on. Similarly, triangulation is the key to assessing students’ competencies. By triangulating assessment formats across a longitudinal basis, we can generate rich information to make “high-stakes decisions” regarding the student’s competency.
As we keep talking about competency, what is it exactly? Competency is defined as the ability to complete a task by integrating knowledge, skills, attitudes, and professionalism. How competency is divided and measured varies according to institutions.
Below you can find articles that provide more background to programmatic assessment, mostly written by Cees van der Leuten and Lambert Schuwirth, two key advocates of this approach.
Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles | A consensus statement on the 12 principles underlying programmatic assessment.
Programmatic assessment: From assessment of learning to assessment for learning | An introduction to programmatic assessment, its definition, principles, and benefits.
Twelve tips for programmatic assessment | 12 tips on how to implement programmatic assessment.
Discover how to maintain continuous student engagement at every stage of your course using Interactive Document
FeedbackFruits announces partnerships with many institutions worldwide over the past 4 months
An overview of the state of competency-based education (CBE) in higher education around the world
An insightful recap of our annual conference discussing innovative and sustainable strategies for student-centered learning.