How to Create Rubrics That Encourage Active Learning (Feedback series)

In the previous article, we went into the nitty-gritty of rubrics, why we use them, the shift from teacher-centered to a student-centered approach, and the different types of rubrics and their ideal use cases. Now we are going to dive into methods for creating quality rubrics that can help meet these criteria.

The common misconceptions about rubrics

Before we really dive into the methods, it can be helpful to go over the common misconceptions surrounding rubrics, so we can avoid them [1]:

1. Confusing learning outcomes with tasks: Rubrics should not be assignment directions. Instead of being task-focused, they should focus on the proficiencies that we should expect to see students display after having finished the task.
2. Confusing rubrics with requirements of quantities: Instructors must be careful to avoid the rubric becoming a sort of checklist. In practice, it’s highlighting that students should display the ability to conduct thorough research, considering multiple criteria when selecting information sources, such as credibility, biases, scope, instead of requiring students to use five sources to score high.
3. Confusing rubrics with evaluative rating scales: Rubrics help structure observation, the rating scales should be descriptive instead of numerical so that it facilitates the bridge between what students display and the judgement of that.

How to approach creating rubrics

Approaching the creation of rubrics for courses or assignments can be intimidating and can vary depending on the type of rubric you choose to use. Regardless of the chosen rubric, most processes can be boiled down to similar elements [2]:

1. Identify which learning outcomes you want to focus on developing
2. Consider which tasks you will use to cover these
3. Select which criteria and descriptions of success you will use for these tasks

To help explore this process we’ll dive into some different approaches, this is by no means an exhaustive list but does provide a good starting point.

First, we’ll consider the adaptation of already existing rubrics, and then dive into how we can create rubrics from scratch, specifically using the bottom-up approach.

The Adopt and Adapt Approach

Time is precious and understanding the key components of rubrics, and what constitutes a good rubric in different scenarios can be helpful in recognizing good rubrics that can be adapted for your teaching. The approach sometimes called Adopt and Adapt, literally means adopt a rubric you found online, or that was shared by your colleagues, and adapt it to your teaching context.

If you find a rubric, say for a research paper, then there are a few steps you can take to best adapt it to your and your student’s needs.‍

          Step 1: Consider the assessment task and the context in which the rubric will be applied ‍

          Step 2: Evaluate whether the criteria can be kept as they are, for example, a criterion on “Synthesis and Evaluation” may still be relevant for         your learning outcomes on your research assignment too. ‍

          Step 3: Consider whether the description level performances of the criteria in the rubric you found suit your particular learning context.

In general, the process of adapting rubrics should be iterative over time. This means it is helpful and important to monitor how students are progressing over time, and how performance levels are changing. In this way, rubrics can be kept reliable and useful.

Ragaputhi and Lee suggest in their book chapter on rubrics, that a good starting place to test the adapt and adopt approach could be the VALUE rubrics provided by the American Association of Universities and Colleges.

The Bottom-up Approach 

Constructing a rubric from scratch can be a tough process. In their book on educational assessment Anthony Nitko and Susan Brookhart outline two valuable methods that can help scaffold the process of constructing rubrics to facilitate positive outcomes, these are called the top-down approach and bottom-up approach. In this case, we’ll focus on the bottom-up approach, but know that the top-down approach may be a good fit if instructors know what skills the course requires students to develop, and what levels of performance of these skills determine success. You can read more about this approach in the book linked above.

The bottom-up approach is an inductive process, which begins by sampling student work and using them to create a framework for assessment. This is best to use when still defining the descriptions of content and performance, or when you intend to involve students in defining their own criteria. The process is as follows:

• Step 1: Start by collecting up to a dozen of student work examples. These samples should all relate to the same overarching topic but should be different examples because the rubric should reflect the learning outcomes rather than the specific task.
• Step 2: Sort the work into three different levels: high, medium and low level.
• Step 3: Write descriptions of why each piece was sorted in the way it was. What made this paper this level, maybe the synthesis of information was thorough and creative.
• Step 4: Compare and contrast those descriptions of the work, if some descriptions come up often, e.g. “includes research to support claims well” then the ability to integrate information may emerge as a dimension.
• Step 5: For each of the criteria identified in the previous steps, try to write descriptions of performance along the dimensions (Beginning, Emerging…) for the appropriate amount of levels. There is no right number of levels but keep in mind that you should be able to clearly distinguish between each level, and they should be useful in guiding observations.

Criteria & Performance level descriptions in rubrics

In our previous article on rubrics, we highlighted that there are two main components to rubrics, the criteria and the description of success across levels for each criterion. No matter the type of rubric, they will at least include some form of these two components. So to ensure the success of the rubrics that we formulate, whether that be adapting or constructing from scratch, understanding what makes up good criteria and performance level descriptions is key.

Susan Brookhart provides us with a good list of recommendations for good criteria and performance level descriptions in her book on rubrics for formative education and assessment.

Let’s start with the criteria. They should be [1]:

           1. Appropriate: Each criterion should represent an aspect of the goal or objective that we intend for students to learn, for example, if it's teamwork, their contributions to the team's work may be a relevant criterion.  ‍

           2. Definable: It is important that each criterion consists of clear and agreed-upon meaning, that both teachers and students understand. Often times a major problem with rubrics can be students failing to understand what teachers really mean [4], so make sure that the criteria are simple and use terms that all students understand. ‍

           3. Observable: Criteria are used to guide judgement on learning, so to be useful and valid they should be something that teachers or anyone reviewing the work can observe. This could be in the form of behaviour in teamwork, for example, inclusivity, or it could be something visible in their work, such as argumentation. ‍

           4. Distinct: It is important that each criterion is unique, if we cannot distinguish between them they lose part of their value. It also becomes more difficult for students to assess and differentiate their learning progress if criteria overlap.‍

           5. Complete: Each criterion on its own may not represent the learning outcomes intended, but when taken all together the criteria should represent all the areas of learning and skills that we aim to foster in students.  ‍

           6. Support descriptions along continuums: It should be possible to describe the criteria over a range of levels, if this is not feasible then the criteria may not be ideal for use.

With the criteria established, the next part is to write the descriptions of performance. When starting off the descriptions, you can ask yourself “what does student performance on this criterion look like ranging from high to low?” [1] These levels should be on a continuum, and it’s important that the differences between these levels can be described precisely.

The descriptions of performance should be:

Descriptive: Performance is described in terms of what is observed in the work. Avoiding evaluative language is key. Instead of saying the student has excellent communication skills, try to break down what that actually means.‍

Clear: As highlighted above, mutual understanding of the criteria and descriptions for students and teachers is key to success.‍

Cover the whole range of performance: Performance is described from one extreme of the continuum to another for each criterion.‍

Distinguish among levels: It is important that the descriptions betweens levels are different enough so that there isn’t space for much ambiguity on both student and teacher's side when matching examples of work to performance descriptors.

‍Matching the target performance to the appropriate level: Make sure that the description of performance level is an appropriate level that one can expect from students, given the goal of the lesson, assessment or curriculum. Moreover, each description of level should be placed at the intended level on the rubric (developing, proficient, exemplary).

‍Feature parallel descriptions from level to level: Across the rubric for each criterion, there should be descriptions of performance, this should represent a continuum ranging from low to high.

Putting it all together

Creating rubrics is no easy task, from deciding which type of rubric to use, to deciding on the relevant criteria, and describing the levels of performance from high to low on each of the criteria.

From this article we hope to have made you aware of the most common misconceptions about rubrics, and how we can avoid those by scaffolding the rubric creation process, through adopting and adapting, or even by constructing the whole rubric from scratch, through the bottom-up approach.

With the continued use of effective rubrics, we can help students start to notice recurring patterns in their learning, including strengths and more troublesome areas. And through this reflection and monitoring of learning, we encourage students to become active participants in their learning.

More resources on assessment and feedback rubrics

• A guide to rubrics for educators: The article dives into understanding why we use rubrics, what they are, and how they can contribute to the student learning process.
• How to creative effective peer feedback criteria: Discover how to initiate high quality peer review by utilising these 7 essentials in designing peer feedback criteria.
• The Role of Feedback and Holistic Scoring in Building a Growth Mindset: Dr. Kim Chappell at Fort Hays State University shared best strategies to facilitate assessment and feedback that nurture a growth mindset.

References

[1] Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. ASCD.
[2] Ragupathi, K., & Lee, A. (2020). Beyond fairness and consistency in grading: The Role of Rubrics in Higher Education. In Diversity and inclusion in global higher education (pp. 73-95). Palgrave Macmillan, Singapore.
[3] Nitko, A. J., & Brookhart, S. M. (2011). Educational assessment of students (6th ed.). Boston: Pearson
[4] Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-517.