How Boston University enhanced peer feedback and engagement in a large online course

The problem: Facilitating effective peer feedback in a large student cohort

Facilitating a 15-week online course of 400 students is no simple feat, especially when the course design demands the incorporation of peer feedback, group work, and automated grading. So when setting up their online MBA program, Denise and Monty needed to find an effective tool that could put these goals into practice without sacrificing quality. The ‘ideal’ tool, according to the instructors, should 1) reduce the manual workload when grading for a large student cohort; 2) incorporate elements of peer feedback and engagement; and 3) allow students to work both individually and in teams to provide meaningful feedback. Denise and Monty finally found this ‘right’ tool, which was FeedbackFruits’ Peer Review.

“What was our solution? FeedbackFruits Peer Review tool, where students could work individually and in teams with auto-grading of the assignment in the LMS Grade book.” - Denise Kreiger

While Blackboard’s native tool for peer assessment failed to meet these requirements, Denise benefited from the fact that Peer Review - along with all FeedbackFruits tools - are fully integrated into Blackboard Ultra. This provided the ability to sync groups from the LMS to the FeedbackFruits activity, as well as enabling autograding of the assignment within the Blackboard Gradebook. The following section will cover just how Peer Review was implemented within the course.

The solution: Implementing Peer Review 

The course curriculum was structured with asynchronous lectures in the first half of the week, followed by live sessions in the second half. A Peer Review assignment was arranged after each live session, which was called ‘FeedbackFruits Executive Memo submission'. For this, each student had to write and submit an executive memo, then provide feedback on each other’s work based on a 3-criteria rubric. These submissions accounted for a total of 60% of the final grade. By the end of the course, students had to write a Formal Memo, which was marked by the instructors. This Formal Memo acted as a “summative assessment”, while the Executive Memo in the previous weeks were considered formative evaluations to help students practice and prepare for the final assignment. 

Besides providing the platform, Peer Review also facilitated the assignment design through a 5-step procedure.

An outline of the Peer Review activity is as follows: 

Step 1: Assignment instructions & Collaboration options

Step 2: Student submission and Settings

Step 3: Peer Review criteria & Settings

Step 4: Self-reflection (optional)

Step 5: Peer Review criteria & Settings

The first step is ‘Assignment instructions & Collaboration options’: instructors set the assignment guidelines, requirements, and expectations within the Peer Review environment. Using the tool, the instructor could also specify the hand-in and review option: individual or in group. 

The next step - ‘Student submission & Settings’ step is where the instructors “set the stage” or describe the technical aspects for the submission, (e.g. deadline, format, number of reviews, or timeline). This was also done via Peer Review. 

This is then followed by ‘Peer Review criteria & Settings’, in which students provided comments on peers’ executive memo based on 3 evaluation criteria:

1. The delivery of main ideas, situation, and complication: Are the main ideas, situation, and complication clearly presented and supported with evidence (with citations)? What worked well? Any suggestions for improvement?

2. The use of deductive reasoning: Is a deductive argument approach used? Were you convinced that the company could further create and/ or capture value by focusing on the two stakeholders group as a priority while minimizing potential discontent from the other stakeholder groups? What worked well? Any suggestions for improvement?

3. The conclusion: Does the conclusion effectively synthesize the main idea and findings? Any suggestions for improvement?

Peer Review allowed the instructors to present these criteria, allocate the peer reviewers, and most importantly, to facilitate meaningful peer feedback dialogue as students submitted reviews then replied to their received comments.

“I think it is interesting that not only do we have peer review going on here, but we have two students having an exchange from the Peer Review and the Feedback given.” - Denise Kreiger 

In the fourth step, students have the option to reflect on the peer feedback they received. 

Students’ submissions and reviews were then automatically graded in the final step - ‘Grade Weighting’. The instructors could specify the weight for specific assignment criteria using the Configurable Grading module of Peer Review. As shown figure 2, for each completed step, students will be assigned the corresponding grading weight. 

Once students have submitted the assignment and completed the peer feedback by the due dates, the “points” will be auto-published to the Blackboard Ultra Gradebook (i.e., full possible points for the assignment, or fewer points depending on meeting the criteria).

Both instructors acknowledged that the automation of grading within the Peer Review tool relieved the burden of manual workload,‍

“That [auto-grading] actually addressed the scalability issue for us around the grading"  - Monty Kaplan

‍

The outcome: Engagement and participation are optimised

The adoption proved to be a success, as shown by a critical rise in student engagement - 95% on-time submissions of the executive memo, and 97% peer feedback completion among students. All students benefited from Peer Review, according to Denise and Monty, as the tool supported large-size classrooms, helped students develop lifelong skills (such as critical thinking and collaborative learning), and enabled both formative and summative assessments.‍

“The FeedbackFruits Peer Review was highly scalable for large student cohorts and ideally suited for asynchronous online delivery.” - Denise Kreiger

You can watch Denise and Monty’s use case in their presentation at the inspirED 2021 Virtual Conference.