Join our Webinar on December 17th: From Regulation to Innovation: What the EU AI Act Means for EdTech
chevron_right

How Boston University enhanced peer feedback and engagement in a large online course

Nhi Nguyen
|
November 3, 2021
DOMAIN
Business & Management
Class Size
400
Instructor Workload
Learner Workload
ABOUT THE INSTITUTION

Boston University is one of the leading private research and teaching institutions in the world today, with two primary campuses in the heart of Boston and programs around the world.

ABOUT THE INSTRUCTOR(S)

Denise Kreiger, M.Ed. is a Digital Learning Designer in Instructional Production Services, Digital Learning & Innovation at Boston University where she designs and develops courses in the Questrom online MBA program. 

Monty Kaplan is a Platform Administrator in Educational Technology, Digital Learning & Innovation at Boston University where he administers the Questrom online MBA platform delivery, including Blackboard Ultra and FeedbackFruits.

ABOUT THE INSTITUTION

Boston University is one of the leading private research and teaching institutions in the world today, with two primary campuses in the heart of Boston and programs around the world.

ABOUT THE INSTRUCTOR(S)

Denise Kreiger, M.Ed. is a Digital Learning Designer in Instructional Production Services, Digital Learning & Innovation at Boston University where she designs and develops courses in the Questrom online MBA program. 

Monty Kaplan is a Platform Administrator in Educational Technology, Digital Learning & Innovation at Boston University where he administers the Questrom online MBA platform delivery, including Blackboard Ultra and FeedbackFruits.

Context

To facilitate effective peer feedback process for an online MBA program of 400 students, Denise Kreiger and Monty Kaplan of Boston University utilized pedagogical technology and smart course design.

Learning journey
Gamified peer review
A framework to integrate game design elements into peer review for enhanced engagement and critical thinking
access now
Learning journey
Self-assessment for learner engagement
A framework to integrate game design elements into peer review for enhanced engagement and critical thinking
access now
Learning journey
Developmental portfolio for lifelong learning
Promote skills transfer and career readiness through technology-enhanced student portfolios
access now

The problem: Facilitating effective peer feedback in a large student cohort

Facilitating a 15-week online course of 400 students is no simple feat, especially when the course design demands the incorporation of peer feedback, group work, and automated grading. So when setting up their online MBA program, Denise and Monty needed to find an effective tool that could put these goals into practice without sacrificing quality. The ‘ideal’ tool, according to the instructors, should 1) reduce the manual workload when grading for a large student cohort; 2) incorporate elements of peer feedback and engagement; and 3) allow students to work both individually and in teams to provide meaningful feedback. Denise and Monty finally found this ‘right’ tool, which was FeedbackFruits’ Peer Review.

“What was our solution? FeedbackFruits Peer Review tool, where students could work individually and in teams with auto-grading of the assignment in the LMS Grade book.” - Denise Kreiger

While Blackboard’s native tool for peer assessment failed to meet these requirements, Denise benefited from the fact that Peer Review - along with all FeedbackFruits tools - are fully integrated into Blackboard Ultra. This provided the ability to sync groups from the LMS to the FeedbackFruits activity, as well as enabling autograding of the assignment within the Blackboard Gradebook. The following section will cover just how Peer Review was implemented within the course.

The solution: Implementing Peer Review 

The course curriculum was structured with asynchronous lectures in the first half of the week, followed by live sessions in the second half. A Peer Review assignment was arranged after each live session, which was called ‘FeedbackFruits Executive Memo submission'. For this, each student had to write and submit an executive memo, then provide feedback on each other’s work based on a 3-criteria rubric. These submissions accounted for a total of 60% of the final grade. By the end of the course, students had to write a Formal Memo, which was marked by the instructors. This Formal Memo acted as a “summative assessment”, while the Executive Memo in the previous weeks were considered formative evaluations to help students practice and prepare for the final assignment. 

Besides providing the platform, Peer Review also facilitated the assignment design through a 5-step procedure.

An outline of the Peer Review activity is as follows: 

Step 1: Assignment instructions & Collaboration options

Step 2: Student submission and Settings

Step 3: Peer Review criteria & Settings

Step 4: Self-reflection (optional)

Step 5: Peer Review criteria & Settings

The first step is ‘Assignment instructions & Collaboration options’: instructors set the assignment guidelines, requirements, and expectations within the Peer Review environment. Using the tool, the instructor could also specify the hand-in and review option: individual or in group. 

The next step - ‘Student submission & Settings’ step is where the instructors “set the stage” or describe the technical aspects for the submission, (e.g. deadline, format, number of reviews, or timeline). This was also done via Peer Review. 

This is then followed by ‘Peer Review criteria & Settings’, in which students provided comments on peers’ executive memo based on 3 evaluation criteria:

1. The delivery of main ideas, situation, and complication: Are the main ideas, situation, and complication clearly presented and supported with evidence (with citations)? What worked well? Any suggestions for improvement?

2. The use of deductive reasoning: Is a deductive argument approach used? Were you convinced that the company could further create and/ or capture value by focusing on the two stakeholders group as a priority while minimizing potential discontent from the other stakeholder groups? What worked well? Any suggestions for improvement?

3. The conclusion: Does the conclusion effectively synthesize the main idea and findings? Any suggestions for improvement?

Peer Review allowed the instructors to present these criteria, allocate the peer reviewers, and most importantly, to facilitate meaningful peer feedback dialogue as students submitted reviews then replied to their received comments.

“I think it is interesting that not only do we have peer review going on here, but we have two students having an exchange from the Peer Review and the Feedback given.” - Denise Kreiger 

In the fourth step, students have the option to reflect on the peer feedback they received. 

Students’ submissions and reviews were then automatically graded in the final step - ‘Grade Weighting’. The instructors could specify the weight for specific assignment criteria using the Configurable Grading module of Peer Review. As shown figure 2, for each completed step, students will be assigned the corresponding grading weight. 

Once students have submitted the assignment and completed the peer feedback by the due dates, the “points” will be auto-published to the Blackboard Ultra Gradebook (i.e., full possible points for the assignment, or fewer points depending on meeting the criteria).

Both instructors acknowledged that the automation of grading within the Peer Review tool relieved the burden of manual workload,

“That [auto-grading] actually addressed the scalability issue for us around the grading"  - Monty Kaplan

Figure 1. Grading configuration in Peer Review

Assessment of learning outcomes

The outcome: Engagement and participation are optimised

The adoption proved to be a success, as shown by a critical rise in student engagement - 95% on-time submissions of the executive memo, and 97% peer feedback completion among students. All students benefited from Peer Review, according to Denise and Monty, as the tool supported large-size classrooms, helped students develop lifelong skills (such as critical thinking and collaborative learning), and enabled both formative and summative assessments.

“The FeedbackFruits Peer Review was highly scalable for large student cohorts and ideally suited for asynchronous online delivery.” - Denise Kreiger

You can watch Denise and Monty’s use case in their presentation at the inspirED 2021 Virtual Conference.

Notable outcomes

Possible variation

Share on social media

Recommended use cases

Learn how NHL Stenden made the transition from traditional portfolios to a streamlined, user-friendly approach that enables students to concentrate more fully on learning

Learn how instructors at ESSEC Business School utilized pedagogical tools and AI to transform formative assessment in large student cohorts

Discover how instructors at Reyjkavic University used gamification to enhance the peer review process

We gather the most valuable resources and learnings on pedagogy and email them to you every month.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Filling out this form means you agree to our Privacy Policy.