Measuring learning outcomes: benefits and challenges

Understanding Learning Outcomes and Their Importance

What are learning outcomes?

Learning outcomes provide a clear, shared understanding of what students are expected to know, do, or value upon completing a course or program. They are not just about compliance or accountability; they fundamentally shape the learning experience by ensuring that all instructional activities, assessments, and curriculum align with the institution’s educational goals.

Recent trends in higher education have increasingly shifted from traditional credit-hour-based systems to outcome-based approaches, which emphasize the actual competencies and skills students gain (Prøitz et al., 2017). This shift aligns with the growing emphasis on competency-based education (CBE), where the focus is on students mastering specific skills and abilities, rather than merely spending a set amount of time in class.

By clearly defining these outcomes, institutions can better demonstrate their effectiveness, improve accreditation processes, and respond more dynamically to the evolving needs of students and employers.

Well-defined learning outcomes and CBE

CBE and learning outcomes are closely linked because CBE frameworks are designed around the mastery of specific competencies, effectively represented by clear, measurable learning outcomes.

Read more: A beginner's guide to competency-based education

 EDUCAUSE emphasizes that CBE models focus on the mastery of competencies, which closely align with measurable and clearly defined learning outcomes that guide both assessment and student progress (EDUCAUSE 2020, 2021). Here is how CBE and learning outcomes are closely correlated. 

• Measurability: CBE requires students to demonstrate mastery of competencies. Measurable learning outcomes provide clear criteria for what constitutes mastery.
• Clarity and flexibility: Learning outcomes in CBE are clearly defined, allowing students to understand expectations. Flexibility in demonstrating outcomes aligns with personalized learning paths.
• Relevance and applicability: CBE emphasizes applying skills in real-world contexts, aligning with learning outcomes that focus on practical and applicable knowledge and skills.
• Ongoing feedback and formative assessment: CBE involves regular, formative assessments to gauge student progress toward learning outcomes, providing timely feedback for improvement.
• Student-centered goals: Learning outcomes in CBE empower students to take ownership of their learning by setting and achieving personal educational goals.
• Career readiness and competency development: Learning outcomes are crafted to meet industry and career standards, ensuring that CBE programs prepare students for workforce demands.

The importance of learning outcomes for meeting various regulatory and quality assurance standards

To satisfy regulatory and quality assurance standards, institutions must demonstrate that educational objectives are being met and programs remain relevant and effective. As highlighted by the Higher Education Quality Council of Ontario (HEQCO), "outcome-based approaches" are becoming increasingly crucial for showcasing accountability and success in educational programs (HEQCO, 2015). These standards demand a focus on clearly defined, measurable student achievements, which align instructional activities, assessments, and curricula with institutional goals and accrediting bodies' requirements.

1. Accreditation: Ensuring educational quality and accountability

Accrediting bodies, such as the Middle States Commission on Higher Education (MSCHE), require institutions to establish measurable learning outcomes that align with their mission and objectives. These outcomes provide a framework for student achievement, enabling accrediting bodies to assess the overall quality of the institution’s education (MSCHE, 2021). 

2. RSI compliance: Facilitating effective online learning

With the increasing prevalence of online and distance education, ensuring quality through Regular and Substantive Interaction (RSI) is more important than ever. For courses to comply with RSI standards, they must include frequent and meaningful interactions between instructors and students. Learning outcomes are crucial in guiding these interactions, making sure they are purposeful and aligned with the course objectives (U.S. Department of Education, 2022). 

3. Institutional effectiveness: Promoting continuous improvement

Measurable learning outcomes are central to institutional effectiveness - an institution's ability to fulfill its mission and achieve its goals. They provide key indicators of student success and program quality, allowing institutions to regularly assess and refine their educational offerings. By consistently evaluating these outcomes, institutions can gather data to identify areas for improvement and implement data-driven changes that enhance overall educational quality (MSCHE, 2021). 

Challenges in Measuring Learning Outcomes

From defining clear and measurable outcomes to ensuring consistency in assessment across various courses and programs, institutions often struggle to find the right balance. This section explores these challenges and offers strategies that higher education institutions can adopt to navigate these complexities and optimize their approach to assessing learning outcomes effectively.

Common challenges in measuring learning outcomes

While defining learning outcomes is critical in shaping effective educational experiences, accurately measuring these outcomes presents several challenges. These challenges can impact the validity and reliability of assessments, as well as the ability to use results for meaningful improvement. Understanding these common obstacles is essential for educators and institutions to develop strategies that not only address these barriers but also enhance the assessment of learning outcomes.

• Ambiguity in outcome definitions: Learning outcomes that are not clearly defined lead to inconsistencies in assessment and make it difficult to evaluate student progress effectively (Biggs, J., & Tang, C. (2011).
• Diverse learning experiences: Students have different learning paths and styles, making it challenging to use a single assessment method to capture all forms of learning.
• Overemphasis on quantitative measure: Boud, D., & Falchikov, N. (2007) argue that relying heavily on quantitative assessments like exams may fail to capture critical skills such as creativity and problem-solving, requiring a balance with qualitative measures.
• Resource constraints: Effective assessment requires significant resources, which can be a challenge for large institutions or programs with limited budgets.
• Resistance to change: Faculty and staff may resist implementing new assessment methods due to comfort with traditional approaches or concerns about increased workload.

Effectively measuring student learning outcomes

In response to the challenges of traditional assessments, effective measurement of learning outcomes requires a holistic approach that moves beyond relying solely on exams and quantitative methods. Incorporating strategic alignment, diverse evaluation techniques, and continuous feedback mechanisms is essential for capturing a comprehensive understanding of student learning. This approach not only provides insights into academic performance but also helps in evaluating critical skills such as creativity, problem-solving, and real-world application. 

UNESCO advocates for such a holistic method, emphasizing the need for inclusive assessment strategies to ensure meaningful learning outcomes across diverse contexts​ (UNESCO).

1. Strategic alignment:

The first step in measuring learning outcomes effectively is ensuring that all assessments align with the educational objectives set out at the beginning of the course. When outcomes are clearly defined, educators can design assessments that target specific competencies and skills. This alignment ensures that every evaluation is meaningful, accurately reflecting whether students are meeting the expected standards.

2. Diverse evaluation techniques:

No single assessment method can capture the full range of student learning. A combination of traditional, formative, competency-based, and portfolio assessments provides a well-rounded view of student progress.

3. Continuous feedback mechanisms: 

In addition to diverse assessment techniques, the implementation of continuous feedback mechanisms is key to effective measurement. Providing regular feedback helps students understand their progress and identify areas for improvement. This not only improves learning outcomes but also fosters a growth mindset by encouraging students to engage actively in their learning journey.

Getting faculty buy-in and achieving consensus

Successfully implementing effective methods for measuring learning outcomes often hinges on securing faculty buy-in. Faculty members play a crucial role in designing and assessing learning, so gaining their support is essential for making any assessment framework functional and impactful. FeedbackFruits recognizes the challenges faculty may face when adopting new systems, and offers solutions specifically designed to ease the transition and streamline assessment processes.

Several common concerns arise when faculty are introduced to new frameworks:

Increased workload: Faculty may worry that incorporating various assessment methods, such as formative assessments or portfolio-based evaluations, will increase their workload. However, FeedbackFruits offers a solution through its Competency-Based Assessment (CBA) system, specifically with its competency rubric libraries. These libraries enable institutions to create, store, and share rubrics that track student competency development. By providing faculty with pre-defined, standardized rubrics, the system simplifies the grading process, minimizes manual effort, and ensures consistent evaluations across different courses, helping streamline the entire assessment process.

Fear of losing control over evaluation: Faculty often value their autonomy in creating and managing assessments. Involving faculty in the application of features tools like the rubric libraries ensures that they retain control over the learning outcomes being assessed. Allowing instructors to define the specific competencies and skills within these rubrics helps them see the tool as a means to enhance their teaching, not replace it.

Data Transparency and Use: Concerns about how assessment data will be collected, analyzed, and used can also cause hesitation. Addressing these concerns through transparency and collaboration is key. Faculty should be fully involved in decision-making regarding what data is collected and how it is used to inform improvements. Showing how automated feedback and evaluation tools can save time while providing actionable insights about student progress will help ease concerns about data usage.

References

1. Association of American Colleges and Universities (AAC&U). (2007). College learning for the new global century: A report from the National Leadership Council for Liberal Education and America’s Promise (LEAP). Washington, D.C.: AAC&U.
2. Prøitz, T. S., Havnes, A., & Briggs, M. (2017). Learning outcomes: A critical review of the use and implementation of learning outcomes in higher education. Teaching in Higher Education, 22(6), 1-17.
3. EDUCAUSE. (2020). Competency-based education: Technology challenges and opportunities. EDUCAUSE Review.
4. EDUCAUSE. (2021). Investing in quality competency-based education. EDUCAUSE Review.
5. Higher Education Quality Council of Ontario (HEQCO). (2015). The language of learning outcomes: Definitions and assessments.
6. Middle States Commission on Higher Education (MSCHE). (2021). Standards for accreditation and requirements of affiliation.
7. U.S. Department of Education. (2020). The Education Department’s 2022 letter to WCET.
8. FeedbackFruits. (n.d.). A beginner's guide to competency-based education. FeedbackFruits. https://feedbackfruits.com/blog/a-beginners-guide-to-competency-based-education
9. Educational Technology. (n.d.). Bloom’s taxonomy: The ultimate guide. Educational Technology. https://educationaltechnology.net/blooms-taxonomy/‍
10. Stanford Teaching Commons. (n.d.). Creating learning outcomes. Stanford University. https://teachingcommons.stanford.edu/teaching-guides/foundations-course-design/course-planning/creating-learning-outcomes‍
11. Cornell University Center for Teaching Innovation. (n.d.). Learning outcomes. https://provost.cornell.edu/assessment/learning-outcomes‍
12. Queen’s University Faculty of Health Sciences. (n.d.). Developing effective learning outcomes guide. https://healthsci.queensu.ca/source/Developing%20Effective%20Learning%20Outcomes%20Guide.pdf‍
13. FeedbackFruits. (n.d.). Generating holistic assessment with portfolio at Leiden University. FeedbackFruits. https://feedbackfruits.com/use-case/generating-holistic-assessment-with-portfolio-at-leiden-university‍
14. FeedbackFruits. (n.d.). Foster lifelong learning with peer feedback. FeedbackFruits. https://feedbackfruits.com/blog/foster-lifelong-learning-with-peer-feedback‍
15. Biggs, J., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). Open University Press.
16. Boud, D., & Falchikov, N. (2007). Rethinking assessment in higher education: Learning for the longer term. Routledge. https://doi.org/10.4324/9780203964309‍
17. UNESCO. (2021). Assessment for improved learning outcomes. UNESCO. https://www.unesco.org/en/learning-assessments