Evaluating AI and implementing guardrails (IDEA) with EDHEC

The challenge: Moving beyond the "Happy Path"

When EDHEC began teaching prompt engineering, something predictable happened: students created AI assistants, tested them with friendly, well-structured questions, got the answers they expected, and declared success. Claudia Carrone describes this as the "happy path", the ideal scenario where everything works as intended, inputs are clean, and nothing unexpected is thrown at the system. It is the version of the tool that looks great in a demo. But it is rarely the version that matters in practice.

That was the easy part. In professional settings, AI systems face ambiguous requests, edge cases, and adversarial users that lead to operational and reputational risks. A customer service assistant that works beautifully in a demo can produce dangerous outputs under real conditions. A research assistant that performs well on familiar questions may confidently hallucinate when pushed into unfamiliar territory.

Students learned quickly how to build AI tools, but had yet to develop the mindset to critique and analyse them.

Two distinct challenges had to be addressed:

• The first was what Claudia Carrone calls the "happy path" problem. Left to their own devices, students tested their assistants in ways that made them look good. They avoided edge cases, phrased prompts helpfully, and steered away from anything that might expose a flaw. The result was a false sense of confidence, and a missed opportunity to develop the kind of critical thinking that actually matters in professional settings. The gap was not one of attitude but of awareness: pushing an AI assistant to its limits was simply not something that had crossed their minds. Developing this thinking requires taking distance from what they produce to execute testing and provide feedback.
  
  
• The second challenge was scaling testing and feedback to large cohorts. Even at 30 students evaluating across multiple criteria, the volume of qualitative feedback was not manageable manually by a single professor. And the course was already designed to grow. Building an AI assistant is a technical skill, but judging whether it is reliable, safe, and fit for purpose requires something different entirely: the kind of critical thinking, domain awareness, and precise communication that business graduates will need when recommending whether an AI tool should be adopted, modified, or rejected.
  ‍

Listen to Claudia Carrone telling us more about the "Happy Path"

‍

Why this approach with AI is uncommon

Most AI courses ask students to produce something: a tool, an output, a working prototype. Assessment focuses on what was created, and the student is positioned throughout as a builder, someone whose job is to make the AI do something useful.

What EDHEC did was different: students were positioned as evaluators, with the AI assistant as the object of study rather than just the means of production, and the evaluation itself, not the product, was where the learning lived.

This approach is becoming more and more important as AI is evolving. It requires an instructor who understands that building and evaluating are different cognitive activities, and that the second is harder to teach but more valuable to develop. It also requires a platform that can hold the structure necessary to make peer evaluation rigorous rather than superficial. That is where FeedbackFruits came in.

‍

The learning activity: AI Assistant Peer Review

The activity runs across several weeks of the 12-hour course and is structured in three phases.

Phase 1: Build

‍Students create a custom AI assistant around a use case of their choosing. The brief is open: personal, academic, or professional applications are all valid. What is required is clarity. Each student must define who the assistant is for, what it is supposed to do, and what risks it might carry, forcing them to think about purpose and responsibility before prompts and parameters.

Phase 2: Evaluate‍

Each student evaluates two peers' assistants using the FeedbackFruits Peer Review activity, working through a structured rubric across three criteria: interaction and user experience, accuracy and reliability, and safety and risk management.

‍

That last criterion is where the most significant learning happens. Students are explicitly instructed to try to make the assistant fail or behave outside of its intended purpose. If it is designed to explain machine learning concepts, ask it for a dinner recipe. Ask it if it wants to go out to dinner tonight. Try to get it to engage in personal conversation it was never meant to have. The goal is to find the point where the assistant stops behaving as intended, to document what happens, and to suggest appropriate guardrails.

‍

Phase 3: Iterate‍

Students receive peer feedback and produce a Version 2, along with a written reflection on what they changed and why, and what they chose not to act on. This is not a second round of peer evaluation, the final version is reviewed and graded by the professor alone. Engaging with feedback, even to disagree with it, is itself part of the assessment.

Part of what students submit in their final version is an appendix in which they explicitly address the feedback they received: what they took on board, how they integrated it, and why. This makes the improvement process visible and accountable, the professor can see not just the final product, but the thinking that shaped it.

This structure has a direct impact on quality. The final submissions have shown to be significantly stronger than they would be without the peer feedback process. Students who have been pushed to find edge cases, confronted with failure scenarios they had not anticipated, and challenged to think seriously about guardrails arrive at their final product at a much higher level. For professors frustrated by flat, interchangeable AI-generated submissions where it feels like the tool did the thinking, this process raises the bar. The professor's expectation when receiving a final GPT is that it should have strong, tested guardrails. And because students have been through a rigorous peer evaluation cycle, that standard is achievable.

‍

‍

‍

"The peer feedback is not an evaluation mechanism. It is a learning mechanism. I want them to understand why making AI reliable is genuinely hard and how that slows AI adoption across businesses."
— Michelle Sisto, Associate Professor & Director, EDHEC AI Centre

‍

How it looks in practice

These are excerpts from two student groups' final submissions.

Example 1‍

A team building a music discovery tool used peer feedback to sharpen both the user experience and the guardrails:

"We updated the recommendation instructions to emphasize adjacent but non-mainstream and underground artists. While the GPT was giving accurate suggestions before, peers noted that many of them were still familiar or too mainstream."

"I strengthened the domain guardrails by explicitly adding 'movies' to the list of prohibited topics. This was in response to feedback that the GPT occasionally drifted into other types of entertainment."

‍

‍

‍

Example 2‍

A team building a CV and career coaching tool went a step further: after revising based on feedback, they ran a structured round of testing to validate the changes actually worked:

"We conducted further testing which included: testing both options using a CV and by filling up information manually... trying to push back the assistant by not accepting and hard contradicting its input. The assistant behaved in a self-correcting way."

That last detail matters. Closing the loop between feedback received, changes made, and changes validated is precisely the kind of professional rigour the course is designed to build.

‍

Listen to Claudia Carrone explain how students tested their peers' work.

‍

Template: Peer review of student-created AI assistants
‍

Claudia Carrone contributed a template to the FeedbackFruits Learning Design Community (LDC). This activity guides students through a structured peer review of custom AI assistants, developing critical evaluation skills around design, reliability, and safety.

How it works

• Students read activity guidelines and learning objectives
• They interact with a peer-created AI Assistant and submit a summary of their observations
• Students evaluate two peer AI Assistants using a structured scale rating across three criteria: interaction and user experience, accuracy and reliability, and safety and risk management, giving clear and constructive feedback
• A final feedback-on-feedback step asks students to reflect on the usefulness of the comments they received
  ‍

View the template →

‍

Feedback as the assessment

Students are not graded on the numerical scores they assign on the platform. They are graded on the quality of the written feedback they gave.

This is intentional. Giving specific, actionable feedback is one of the most important skills a future manager can develop, and one of the hardest to teach. By making feedback quality the assessment criterion, the professor creates a direct incentive for students to think carefully, observe closely, and communicate constructively and precisely.

That accountability runs both ways. As part of their final submission, students include an appendix in which they explicitly address the feedback they received: what they took on board, how they integrated it into their Version 2, and why, in writing. This makes the improvement process visible and traceable. The professor can see not just the final product, but the reasoning behind every decision. And for students, articulating how someone else's observations changed their thinking turns feedback from a formality into a genuine learning moment.

‍

‍

The results are visible. Students who receive specific feedback are able to act on it and improve. Those with generic feedback understand its low value. Students who invest in specific, well-observed feedback see their peers improve. The lesson of quality feedback lands because they live it.

"Students realised that if you don't give actionable, specific feedback, it is useless to the designer. That learning objective was met because they experienced it directly."
— Michelle Sisto, Associate Professor & Director, EDHEC AI Centre

‍

How FeedbackFruits made it possible

Students each providing written feedback on two assistants across three criteria generates more than 360 individual qualitative responses per iteration. Managing that manually, while maintaining anonymity, tracking submissions, and grading at the same time, is not feasible. At 800 students, it is not conceivable.

FeedbackFruits handled the infrastructure: automatic peer assignment, consistent submission structure, reliable anonymity, and a centralised view for grade review and adjustment. The structured rubric also raised the quality of the feedback itself. Guided criteria with space for qualitative comment under each produced responses that were more focused and more actionable than an open text box ever would.

"It is not [just] that FeedbackFruits saved me time. It allowed a rich learning experience that I simply would not have done otherwise."
— Michelle Sisto, Associate Professor & Director, EDHEC AI Centre

‍

Impact: What changed for students

Students became more rigorous critical thinkers, faster. Students are consistently more critical when evaluating someone else's assistant than their own. They find edge cases they never thought to test, notice unclear instructions that would not have occurred to the creator, and push into scenarios the original designer never anticipated. That rigour then feeds back into their own work, often before they have received any formal feedback.

Students developed a realistic picture of AI limitations. Watching someone else's assistant fail under pressure, and then discovering their own has similar vulnerabilities, gives students something more valuable than confidence: a grounded understanding of what AI can and cannot reliably do. In broader institutional surveys, nearly 20% of students reported that AI was beginning to affect their self-confidence, feeling that AI was doing things better and faster than they could. Activities like this one help reframe that relationship, replacing passive anxiety with active, informed critical engagement.

Feedback became a skill, not a formality. Because feedback quality was the assessment criterion, students took it seriously. The result was feedback that was genuinely useful. Students on the receiving end of strong feedback made meaningful improvements to their Version 2. Those on the receiving end of weak feedback said so in their reflections, and understood exactly why it had not helped them. And in their final appendix, students themselves articulated how the feedback they received shaped their decisions, making the learning visible not just to the professor, but to the students themselves.

‍

Scaling from 30 to 800 students

The course has run three times, with each iteration refining the rubric and the structure of the activity. It is now being implemented in an 800-student course at EDHEC, a more than 25-fold increase, made possible because FeedbackFruits handles the infrastructure. The professor does not need to rebuild for a larger cohort. The platform scales with it.

That scalability has also sparked broader interest. A reusable template built on FeedbackFruits is now in development, designed so that any institution in the world can implement the same approach and achieve the same impact. Colleagues at other French universities have already asked how to replicate the model. And the conversation is going global: Claudia Carrone will be bringing the approach to EDULEARN26, one of the world's largest education and technology conferences, held in Palma de Mallorca from June 29th to July 1st, 2026, where she will animate a workshop titled From Listed to Lived: Bringing Soft Skills into Practice. With over 800 participants from more than 80 countries, it is a signal that this approach to AI literacy is attracting attention well beyond EDHEC's own walls.

"FeedbackFruits is giving me the place and the tools to implement this and make it scalable. It allows us to keep the evaluation structured and the quality high."
— Claudia Carrone, Learning Experience Designer & LDC Award Winner 2025, EDHEC

‍

Words from the students

"Something that I liked about this tool (Feedback Fruits) is that I was able to put screenshots about what I was prompting and what the outcome was coming. I think that was the best way to see it, to show them that I got something that wasn't the original plan."

"Reviewing others' work was really helpful: it helped us understand what others were doing and where we stood in comparison. What struck me most was realising that you're also learning from your peers: seeing that they did something you didn't, and thinking, 'I need to go and fix that.' That, I think, is one of the most important parts of the peer review process."

“We're so used to the linear classroom model: you learn something, you submit it, and the professor gives you a grade. This was completely different. The iterations, the fact that we could talk about each other's tools. You could go up to someone and say, 'hey, you wrote this about my work, can you elaborate?' That conversation between each other becomes part of the learning process.”

Key Takeaways

This is a new approach to integrating AI in education.

‍Most courses teach students to use AI. This one taught them to evaluate it. The difference between a user who trusts outputs and an evaluator who questions them is exactly the difference employers need from graduates entering AI-influenced workplaces.

Peer review changes what students are willing to see.

‍Students who tested peers' assistants found what was actually there, not what they expected to find. Thirty students across three criteria produced more than 360 individual qualitative responses per iteration. That volume of meaningful evaluative data is not easily achievable any other way.

The rubric is what makes peer review meaningful.  

Unstructured feedback produces unstructured learning. The three-criteria rubric gave students a professional evaluation framework that maps directly onto how AI tools are assessed in real organisations, and FeedbackFruits held that structure in place at scale.

Grading feedback quality changes student behaviour.

‍When students know their grade depends on the quality of their observations, the standard of those observations rises. This is a transferable design principle for any peer review activity where professional communication skills matter.

A 30-student pilot became an 800-student course in three iterations.

‍The infrastructure FeedbackFruits provides is what makes that trajectory possible without rebuilding from scratch each time.

A new standard for AI in Education

EDHEC did not set out to build a showcase activity. They set out to solve a real problem: students who could build AI tools but could not critically evaluate them. What emerged, through iteration, honest reflection, and the right infrastructure, is one of the most rigorous approaches to AI literacy we have seen in practice.

The students who went through this course left not just knowing how to prompt. They left knowing how to question. In a world where AI is moving faster than most institutions can track, that might be the most important thing a course can teach.

For institutions thinking carefully about how to introduce AI into teaching without compromising standards or overwhelming faculty, FeedbackFruits' AI Feedback and Literacy bundle is a practical starting point. It brings together Assignment Review, Automated Feedback, and AI Practice, the same infrastructure that made EDHEC's approach possible, and scalable.

‍