Anonymous Peer Review at the Vrije Universiteit

Course context:

This course, in the context of scientific research in the field of Pharmaceutical Sciences, aims to develop academic skills such as scientific writing, discussion, critical thinking and presentation skills. These skills are often referred to as academic soft skills. Furthermore, peer assessment of group reports is a mandatory pass/fail element of this course and participance is monitored by the instructor.

FeedbackFruits related learning goals / outcomes:

  1. Students are able to provide substantive and constructive feedback on the work of others based on predefined criteria.

Motivation to apply peer feedback:

In prior years, the quality of both feedback and end products was rather poor. By organising peer feedback, in which feedback can be directly connected to predetermined review criteria and placed in-line, the instructor assumed to enhance both the quality of feedback and the eventual end report. The previous logistics of organising the feedback process, in which groups receive feedback, was rather time consuming. The assumption was that organising peer feedback via FeedbackFruits would reduce the complexity, workload and improve the process overview.

Learning activities (Learner)

For this assignment, counting 20% towards the final grade, students give feedback on the draft literature report of one other group, consisting of three students. Students need to address all questions related to the assessment criteria (e.g. structure, contents, argumentation, etc.). In addition, students received further instructions and tips on how to give (and receive) feedback during lectures. Students rate the quality of the feedback they receive from three other students. Students also analyse whether the feedback addresses all relevant criteria. To finalise the activity students reflect on what they have learned. Learner activities based on the Bloom taxonomy are mainly at the level of:

  • Applying and Analysing: students practice critical thinking / critical reading, comprehension (judgment), presenting, discussing and writing, all important academic skills that receive a lot of attention in this course.
  • Evaluating: students also practice with collaborating, critical reflection on one’s own skills and the process of giving feedback.

Evaluation and assessment of learning outcomes:

With regard to outcome 2: ‘able to critically (self)reflect on skills’ and outcome 3: ‘Students are able to provide constructive feedback’ the instructor gives feedback and reviews the group report, presentation and completion of peer feedback. Additionally, as the quality of peer feedback is also graded by the receiving students, the teacher takes those ‘peer grades’ into account as well.

Instructor role in the peer feedback activity:

The instructor first explained to students that peer feedback was a mandatory part of the course, a pass or fail. Next it was emphasised why peer feedback is valuable for groups and how students can improve their learning by participating. It was explicitly announced that when groups score sufficient according to their reviewing peers, this does not imply or guarantee that the instructor would also agree with the sufficiency. Hence, student groups always have the final responsibility for their end product. After the submission deadline, the instructor gave instructions during class about how to give feedback with FeedbackFruits (20-25 min max). It was specified how to add a review or comment while keeping the sandwich feedback methodology in mind. For students who did not reflect at first (visible in the FbF overview), the teacher personally confronted / asked students to reflect in order to learn from it. As the quality of peer feedback is also rated by the receiving student groups, the instructor investigated how each student was valued in providing feedback. An average was taken from all the (receiving) students who graded one particular reviewer and subsequently discussed between reviewer and teacher.

Added value Technical support:

Working with FeedbackFruits to support peer feedback offered two main advantages:

First of all, FeedbackFruits provided a better overview of where all students are in the process (progress analytics). An example of this was that the instructor could take direct action when students did not meet the submission deadline, therefore in this particular case a late hand in was allowed. Secondly, using FeedbackFruits significantly optimised the logistics of the feedback process. According to the teachers, the process was less time consuming and also the process of organising feedback was less ‘fragmented’. Furthermore, the group support of importing existing LMS groups was a big benefit and was not possible with the available Turnitin tool. The possibility of linking feedback directly to the rubric assessment criteria was greatly appreciated. Also being able to have individuals of multiple groups allocated to group work of others was considered to create added value; as it prevented that members of relatively inactive groups were all allocated to the same deliverable. Finally, the ability of the Peer Review tool to allow students to reflect on the received feedback was advantageous.

Interesting outcomes:

There were several interesting outcomes:

Because comments were made mandatory per criterion, more complete feedback was given by students. Compared to previous years there were now no complaints anymore from students about the feedback process. The instructors found a strong positive correlation between the quality of final work after feedback and the quality of submitted drafts. There were mostly insufficient grades from groups that did not work well together, which provides an argument to enrich the set up by also including a Group Member Evaluation activity. After peers had rated the feedback of reviewers, random review reports were manually checked by the teachers. Teacher review scores matched the scores given by students.

The outcome: The quality of feedback from reviewers was highly valued by the receivers with a 7+ (out of 10) on average in class. This rating was only based on what students indicated and has not been adjusted by the instructor, and did not count towards the final grade. On average, students seem to fill in the provided rubrics more positively than the teachers would  

Previously, the feedback was given per group on another group. As a consequence, in the case of a weaker or less active group, the quality of the provided feedback may be rather poor. This issue was less apparent this year since groups received feedback from three different individuals. A selection of student comments on the use of Peer Review as a tool:

  • “I thought it was very useful and I learned a lot”, “Anonymity was greatly appreciated”
  • “Very complimentary feedback, since I got feedback from multiple individuals and not feedback from the same groups, reviewers were able to highlight different areas for improvement”

Don't miss these stories:

Copyright © 2020 FeedbackFruits Corp.