Leverage AI and pedagogy technology for self-assessment for learner engagement at ESSEC Business School

Effective assessment at large scale

As part of ESSEC's 2020-2024 strategic plan (RISE strategy), the institution introduced five mandatory Small Private Online Courses (SPOCs) for their 6,000 pre-experience students, covering key areas such as responsible leadership, AI for business, climate change, diversity and inclusion, and entrepreneurial thinking. These SPOCs were designed to provide a tailored educational experience for a specific community, in contrast with the traditional large-scale online courses.

However, the large scale of these SPOCs, with up to 600 students enrolled in a single course, presented a unique challenge for the ESSEC team. Navigating the complexities of assessment and engagement in this context became a critical priority, especially as the rise of generative AI threatened to undermine the integrity of traditional assessment methods.

As Marion remarked: 

“Sometimes have about 600 students in the same model. This is our first huge challenge with an SPOC and with assessments of an SPOC.”

Rethinking Assessment – From Summative to Formative Evaluation

According to Nadia:

“In our original SPOCs,  our assessment strategy was based on summative assessments, relying on automatically graded activities like multiple-choice questions, for example. Also, large online classes make it unrealistic for one single grader to grade hundreds of essays.”

Recognizing the limitations of traditional summative assessments in the face of generative AI, the ESSEC team decided to shift their focus towards formative evaluation, with a particular emphasis on self-assessment. Nadia emphasized: 

“So, this challenge led us to adapt by focusing on formative assessment and the concept of significant learning. And this way, we aim not only to assess students but also to engage them in a deeper and more meaningful learning process. And to do that, we had to create meaningful learning experiences.”

This strategic decision was informed by the principles of Fink's taxonomy of significant learning, which highlights the importance of developing skills for independent learning, fostering personal connections and values, and promoting self-awareness and understanding of others. The ESSEC team decided to focus on three of these learning dimensions, which are: “learning how to learn, caring, and human dimension.” 

Based on the Fink theoretical model, formative assessment was prioritized with the following goals:

• Reduce the misuse of generative AI by students, as the focus would be on personal reflection and engagement rather than simply achieving high grades.
• Encourage deeper learning and the development of critical soft skills, such as self-awareness, empathy, and collaboration.
• Empower students to take ownership of their learning journey and become more actively engaged in the educational process.

Implementing Self-Assessment – The ESSEC Approach

To put their new assessment strategy into practice, the ESSEC team leveraged the capabilities of the FeedbackFruits Teaching & Learning system, which they found to be designed with a strong pedagogical foundation, rather than simply adapting a technical tool to educational needs.

The key elements of ESSEC's self-assessment approach within their SPOCs include:

• Formative Quizzes with Detailed Feedback: Students were presented with formative quizzes that provided detailed feedback, regardless of whether their answers were correct or not. The goal was to facilitate learning, rather than simply testing knowledge.
• Self-Assessment Activities: Immediately following the formative quizzes, students engaged in self-assessment activities, where they reflected on their own skills and understanding of the course material. These activities were supported by the FeedbackFruits Self Assessment on Skills solution.
• Engagement-Focused Grading: Rather than relying on traditional summative assessments, the ESSEC team placed a greater emphasis on an "engagement score," which rewarded students for their active participation and meaningful self-reflection, rather than just their performance on graded assignments.
• Personalized Feedback and Guidance: To further support student engagement and learning, the ESSEC team leveraged the FeedbackFruits Automated Feedback Coach, which provided personalized suggestions and guidance to students based on their self-assessment responses.

Additionally, the ESSEC team introduced a unique "ValueBot" character powered by AI, which appeared before each assessment activity to provide students with clear instructions and encouragement. This playful and engaging approach helped to foster a positive mindset towards evaluation, positioning it as an opportunity for growth and learning rather than a source of stress or anxiety.

Evaluating the Impact: Insights from the "Diversity & Inclusion in the Workplace" SPOC

To assess the effectiveness of their self-assessment approach, the ESSEC team closely analyzed the results from their "Diversity & Inclusion in the Workplace" SPOC. By comparing the introductory self-assessment with the final self-assessment, they were able to gain valuable insights into the impact of their new assessment strategy.

The key findings from their analysis include:

• Consistent Engagement: The results showed that students maintained a high level of engagement throughout the course, with only a small percentage of responses being too short or inadequate.
• Improved Writing Quality: Approximately 30% of the responses demonstrated a high quality of writing, with well-structured sentences and minimal grammatical errors.
• Opportunity for Deeper Reflection: While the team was initially concerned about the relatively low percentage of responses that showed deep personal reflection (around 4%), they recognized that the criteria they had set for this metric may have been too stringent. Upon closer examination, the team found that the student responses were quite thoughtful and insightful, showcasing their ability to apply the course content to their own experiences and future aspirations.

“So, what we can conclude of these results anyway, these results suggest that students took the time to craft responses that were long enough, and well-structured, but maybe there is a need to work with a teacher to encourage them to ask questions that would encourage deeper personal reflection from the students.”

The ESSEC team was particularly encouraged by the quality and depth of the student responses, which included reflections on personal biases, plans to prioritize diversity and inclusion in future leadership roles, and a commitment to continuously questioning one's behavior and its impact on others.

Lessons Learned and Future Directions

The ESSEC team's experience with implementing self-assessment in their SPOCs has produced valuable insights and lessons that can inform the ongoing evolution of their assessment strategies:

Trusting Learners and Fostering Autonomy: By placing a greater emphasis on self-assessment and reducing the weight of traditional summative evaluations, the ESSEC team has found that students are more willing to take responsibility for their learning and engage more deeply with the course material. This shift in approach has helped to foster a sense of autonomy and ownership among the learners, which is crucial for promoting meaningful and lasting learning outcomes.

Iterating and Refining the Assessment Process: The ESSEC team acknowledges that their self-assessment approach is still a work in progress, and they are committed to continuously iterating and refining their methods based on feedback and ongoing evaluation. This includes revisiting the criteria used to assess the depth of personal reflection, as well as exploring ways to further streamline the analysis of student responses, potentially through the use of more advanced natural language processing tools.

Leveraging the Pedagogical Foundations of EdTech Tools: The ESSEC team's experience with FeedbackFruits has highlighted the importance of selecting educational technology tools that are designed with a strong pedagogical foundation, rather than simply adapting technical solutions to educational needs. By prioritizing tools that are grounded in sound educational principles, institutions can better align their assessment strategies with their broader learning objectives and create more impactful learning experiences for their students. As Nadia remarked: 

“It was the first time I encountered a tool that has this powerful foundational approach. And at this time, I knew that we could truly create something impactful with it.”

Embracing the potential of Self-Assessment

The ESSEC team's journey in fostering learner engagement through self-assessment serves as a powerful example of how institutions can proactively address the challenges posed by generative AI in online education. By shifting their focus towards formative evaluation and empowering students to take ownership of their learning, ESSEC has been able to create a more engaging and meaningful educational experience, while also mitigating the risks associated with the misuse of AI tools.