Online Assessment: challenges and opportunities for distance education [Updated 2022]

Pass or No Pass: An alternative to traditional assessment

Once physical lectures are replaced by online conferences and weblectures the next item on the agenda to be addressed is student assessment. Several academic leaders moved quickly and decided that a beneficial way to continue to assess if students met their learning objectives in this time is to disregard the rigorous normality pretensions and shift all assessment to result in a Pass or No Pass option.

MIT was one of the first schools to take this decision, quickly followed by CUNY, Northwestern, Cornell and others. Thanks to Laura Gibbs we found a comprehensive article that provides a list with institutions that have moved towards a Pass or No Pass system or other alternatives.

Pass or No Pass allows instructors to focus on learning and growth, providing students the opportunity to improve their knowledge by trying and making mistakes without the feeling that their final GPA suffers. Nonetheless, in order to get to the Pass or No Pass system one or multiple assessments need to be taken by students to see how they meet with the constructive alignment principle where assessment of a course should match its learning objectives and learning activities. Current common e-assessment options offer faculty a wide range of valid activities to assess students.

Problems with Online Assessment, Validity & Trustworthiness

During these times faculty reconsiders the assessment type they introduce to their “newly” thought-out course. To assess if its assessment type could be used we have to look at the validity and trustworthiness of the assessment type.

Validity: quite simply indicates if the assessment type covers the learning objectives of the course. In order to assess the validity of the assessment. Reliability, transparency and feasibility are key components to think of in the case of validity.

Trustworthiness: in order to assess the trustworthiness of an assessment type one should think primarily of cheat prevention but also of inclusiveness & accessibility and privacy.

The assessment types are all valid in the online space, however, are they all trustworthy?

1. Essays, reports and project presentations remain a valid and trustworthy type of assessment as students will still have the same type of resources to use. However, plagiarism can occur, and it can happen more often as there is less interaction with faculty, thus the institution needs to have plagiarism check technology in place, such as TurnItIn.

2. Oral examinations can now move to online 1-on-1 sessions via web conferencing technology like Zoom or Webex. This is under the assumption that all students have internet access these days, however, bandwidth may play an important role for students. Thus, faculty should take into account how easy to operational this methodology is for students.

3. Exams are less likely to occur in online courses - be it open book, closed book or digital. These types of evaluations have to take into account the available resources of the institution but most importantly of students. Proctoring is a viable solution for the trustworthiness of these assessment types allowing faculty to keep the current examinations. Tools like Proctorio, ProctorU, Exam.net lock the device of the students and the technology flags faculty as students try to cheat.

Discussion around the use of proctoring software is sparking up at every institution or online conference. Though these softwares are useful, educational leaders identified some key elements to consider: Institution resources; Student hardware resources to cope with this technology; Implementation and training.

Read more on different strategies to assess student performance in online learning:

4 strategies for effective assessment in online/hybrid learning

4 ways to assess student performance with FeedbackFruits

Potentially Moving to Programmatic Assessment

“The purpose of assessment is not to gather data and return “results,” but to illuminate ways to strengthen the curriculum, instruction, and student learning” (Pearsons, 2006)."

The toolbox to assess learning outcomes is well filled with methods as described above. Yet each individual method of assessment has limitations. In higher education, not a single assessment method can do it all. As a substitute, faculty could move to more small and diverse data points to evaluate student learning outcomes, which is called programmatic assessment.

“The idea of programmatic assessment emerges from medical studies, because it mimics so much the process when the doctor diagnoses the illness.”

Dr. Lambert Schuwirth of Flinders University summarized the idea behind programmatic assessment with an analogy of medical education.

To generate accurate diagnosis, the doctor has to rely on different channels like blood tests, physical check-ups, conversations with patients, and so on. Similarly, triangulation is the key to assessing students’ competencies. By triangulating assessment formats across a longitudinal basis, we can generate rich information to make “high-stakes decisions” regarding the student’s competency.

As we keep talking about competency, what is it exactly? Competency is defined as the ability to complete a task by integrating knowledge, skills, attitudes, and professionalism. How competency is divided and measured varies according to institutions.

Below you can find articles that provide more background to programmatic assessment, mostly written by Cees van der Leuten and Lambert Schuwirth, two key advocates of this approach.

Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles | A consensus statement on the 12 principles underlying programmatic assessment.  

Programmatic assessment: From assessment of learning to assessment for learning | An introduction to programmatic assessment, its definition, principles, and benefits.

Twelve tips for programmatic assessment | 12 tips on how to implement programmatic assessment.

Further resources on assessment in distance learning

Assessment is one of the key determinants of successful teaching and learning, that's why we have put together several resources on this topic to help you with your assessment planning, whether in face-to-face, online, hybrid, or blended setting. Find below all the go-to resources (blogs, checklists, ebooks) to elevate your course design: 

- 5 essential tips to digitalize authentic assessment

- 4 strategies for effective assessment in online/hybrid learning

- Online Summative and Formative assessment [Myths and Strategies]

We also recommend checking out our ebook on quality course design in online, hybrid, blended, and hyflex setting – Quality online teaching and learning: Achieving the higher education we deserve. In this you will find the 5 essential steps to design meaningful learning experiences in any course settings, along with 2 handy frameworks to help you utilize technology in implementing the flipped classroom and team-based learning.