AI and authentic assessment

Embracing AI in assessment

The launch of ChatGPT came as a shock to educators. Many were confused about how to respond to this powerful new generative technology. In New York City, ChatGPT was banned from public schools in early 2023. The ban was lifted a few months later, with Chancellor David Banks announcing that New York City schools would instead “embrace its potential.”

The latter announcement seems to be the wiser move—rather than reject the technology, city administrators acknowledged its ubiquity and are engaging with it, providing educators with resources “to initiate discussion and lessons about AI in their classrooms.”

Other education experts agree. In “Authentic Assessment in the Era of AI,” Charles Knight of AdvanceHE points out:

“It is easy to get fixated on a deficit model, where students use these tools to commit academic malpractice, but there is a range of interesting and legitimate use cases that need to be considered.”

One of those use cases—which many critics of AI seem to miss—is assessment. AI can be extremely helpful as an assessment tool, even with authentic assessment.

Authentic assessment asks students to demonstrate mastery of knowledge and skills through inquiry and problem-solving. It goes deeper than multiple-choice tests or rote learning—it’s about real-world challenges that inspire curiosity, preparing students for the challenges of the workplace and inspiring a mindset of lifelong learning.

Read more: Authentic Assessment in the era of AI

Evaluating AI’s role in the grading process

AI grading tools, or automated grading systems, are increasingly available to instructors. They promise to streamline the grading process, allowing instructors to use their time to create more meaningful interactions with students. This sounds fantastic, of course. But as with any new technology, there are complications.

So what are the actual pros and cons of AI grading tools?

The pros:

• They save time, providing an easy, accurate way to grade quizzes, multiple-choice exams, and essays, freeing up instructors for other tasks.
• They’re consistent. AI tools use the same criteria for every assessment, reducing the potential for mistakes on the part of the instructor.
• They’re quick. Students are provided with immediate feedback.
• They can help with personalized learning. AI grading tools can analyze data from student submissions to demonstrate where students need more help.

The cons:

• They’re not foolproof. AI products can make factual errors and may not detect those of students.
• There are ethical issues. For example, the grading criteria may be unintentionally biased, and some are concerned about giving AI access to intellectual product.
• They’re not subtle. AI grading tools may not be useful for subjects that involve shades of meaning, such as literature or philosophy.
• Some grading tools are less flexible than others. For instance, students are often encouraged to use the same format or structure for their essays, resulting in intellectual conformity.

There is no denying that AI grading tools can be incredibly helpful and time-saving. Given the drawbacks, though, it seems best to consider them as an aid to the grading process.

For example, as we mentioned above, AI grading tools can help students address their grammar and citation issues, allowing the instructor more time to provide feedback on complex issues like argumentation.

AI grading tools will make life a lot easier for instructors, but they don’t replace the need for consistent mentorship, interaction, feedback, and guidance. (The International Journal for Educational Integrity has an interesting exploration of the issues associated with AI grading here.)

It’s interesting to consider the EU AI Act in light of this. The EU considers using AI in grading as a “high-risk” category perhaps due to this very reason: they’re concerned about education losing the human touch.

Read more: How to design AI policy

Nevertheless, with a balanced approach that combines the strengths of AI grading tools with human expertise and judgment, educators can reap the benefits of automation while preserving the integrity and effectiveness of a more holistic grading process.

Adapting assessment for AI

If AI has a place in authentic assessment, there are still reasonable concerns about how to adapt assessment practices to the age of AI. When it’s so easy to generate ideas or even complete research papers, how do we ensure students are still learning? The following suggestions take a multimodal approach, out of consideration for students of diverse backgrounds and learning preferences, and to encourage mutual learning.

• Presentations. ChatGPT can generate detailed slideshows replete with images, but Q and A sessions and peer feedback will reveal if a student has achieved mastery of a skill or topic.
• Teaching. Students can develop a research topic with the aid of AI and then teach a lesson on the topic to peers. As every educator knows, teaching involves not only mastery of a topic, but also the ability to communicate it—hallmarks of authentic assessment.
• Case study analyses. As mentioned, instructors can create a case study of a real-life situation and present it to students for further research and discussion of possible solutions. Feedback from instructors, peers, and AI will make assessment part of the process (see example below).
• Group projects. Students can work in groups, each with different tasks for a project. They might be required to document their meetings and their individual work, noting when and if AI tools were used, and providing feedback on each other’s tasks.
• Oral assessments. Oral assessments demonstrate if a student understands the material, and they work both for distance learning and in-person classes. Clear objectives and goals will help students focus on preparing for the assessment.

You can find more suggestions for authentic assessment in the age of AI in the FeedbackFruits article, “Practical activities to leverage AI for engagement and skills development.” See also useful suggestions from Dr. Siham Al Amoush, and Dr. Amal Farhat, in their article, “The Power of Authentic Assessment in the Age of AI.”

Elevating authentic assessment with FeedbackFruits AI

FeedbackFruits tools allow for learning journeys that use AI tools to support and enhance authentic assessment, including meaningful peer-to-peer interactions and one-to-one exchanges between instructors and students.

Here is an outline for a learning journey that incorporate FeedbackFruits AI to support students in developing real-life as well as academic skills. It’s a new take on the problem-solving approach, which stimulates meaningful multi-layer interactions.

You can download the full learning journey for your course here.

Step 1: Case study analysis

‍Instructors upload real-life case studies to the Interactive Document tool then annotate these with explanations, questions and discussion threads. To save time, instructors can use the Auto suggest feature in the tool (powered by AI)** to create context-based open questions. Students work in groups to study the use case by responding to the teacher’s prompts and discussing with their peers.

Step 2: In-class clarification

‍After students have studied the content, instructors can hold a clarification session, when they address the knowledge gaps and questions of students. This way, students have the opportunity to thoroughly understand the case study and know what they need to do next.

Step 3: Group discussion and first draft

Based on the previous research and discussions, groups can then discuss solutions to a case study problem, and then draft written reports.

Step 4: AI-generated feedback on the first draft

Students submit their first drafts to the Automated Feedback tool, which analyzes the report and provides formative feedback on students' technical writing aspects such as grammar, spelling, citation, etc.

Step 5: Peer feedback on the first draft

Students then submit the first draft to the Peer Review tool and provide feedback on other groups’ submissions based on a set of criteria. This step encourages critical thinking, accountability, and feedback skills. Instructors might also hold class discussions on appropriate methods of constructive criticism.

Step 6: Final draft and submission

‍Groups or individuals improve and finalize the writing based on the feedback and insights from teacher. They then submit the work and receive teacher feedback within the Assignment Review tool.

Step 6: Group presentation and teacher feedback

In addition to their written report, each group delivers a presentation on their solutions. Peers and instructors provide feedback here as well.

Step 7: Self-reflection and group evaluation

Finally, students have the opportunity to reflect on their own and others’ contributions to the group work based on a set of collaborative skills criteria in the Group Member Evaluation tool.