Do you feel frustrated when assignments are submitted that are lacking important criteria? Do you have several people grading and want to maintain consistency and fairness? Are you wishing to provide more feedback to students, to help them improve their skills? Answers to these questions can be targeted by building effective analytic rubrics. In a recent webinar, I shared how to build effective analytic rubrics, including what they are and why you should use them. Understanding the what, why, and how will help you to determine where to start when creating effective rubrics for your course. Let's begin!
It is important to provide the rubric to the students before they complete the activity or assessment. There aren’t any secrets to learning- we want them to know the goals that they are trying to achieve. Provide them with expectations for successful assignment submission. Help them hit the target!
I also suggest not simply attaching the rubric and never referring to it or explaining what a rubric is there for. If you plan to use rubrics, provide it, and point it out to students. Tell them where to find it, and why they should use it. I always tell my students, to review the rubric before they begin the assignment, and then again after they finish the assignment but before they submit it. This way they can self-assess their work, and decide if more effort is required (Cockett and Jackson, 2018). Or if they are ok with what they’ve accomplished. A rubric helps to clear confusion!
Rubrics are used for easier grading and provide reasoning for grades.
“Rubrics are viewed favorably by both staff and students as a method of enhancing consistency in assessment.” (Cockett and Jackson, 2018). If clearly set up, a rubric can make sure points are being applied or deducted accurately. Especially when you have several people grading in a large class, a rubric can support that grades are being applied objectively not subjectively. “It also makes grading feel more objective to students ('I see what I did wrong here'), rather than subjective ('The teacher doesn’t like me and that’s why I got this grade')" (Univ. of Michigan Center for Academic Innovation, 2020).
Rubrics provide feedback to students!
Be specific with the descriptions for each proficiency level, and what meets each level. Feedback is sadly often lacking in our current courses. And I get it, for large class sizes, it is tough to give specific feedback to each student. But rubrics can at least give some feedback, so the student knows how they can improve or what they did not meet.
When used in online courses especially, rubrics are an important component to clearly defining expectations, reasons for grading, and providing feedback. Rubrics help meet the online course quality standard from Quality Matters, #3.3 that states: “Specific and descriptive criteria are provided for the evaluation of learners’ work, and their connection to the course grading policy is clearly explained” (Quality Matters, 2020).
To build an analytic rubric, I’ve created a checklist you can follow. First, you’ll want to answer some questions, to gather information before building your rubric:
What is a rubric?
A rubric is “...a document that articulates the expectations for an assignment by listing the criteria or what counts, and describing levels of quality from excellent to poor" (Reddy and Andrade, 2010).
Typically comprised of rows and columns, where the rows are used to define the various criteria being used to assess an assignment. And the columns are used to define levels of performance for each criterion. Most have a range of points, to show the scoring strategy to meet the criteria.
Typically comprised of rows and columns, where the rows are used to define the various criteria being used to assess an assignment. And the columns are used to define levels of performance for each criterion. Most have a range of points, to show the scoring strategy to meet the criteria.
Types of Rubrics
There are many different types of rubrics. I will outline a few types here, but if you do a Google search for rubrics you will find all kinds. I will focus on the analytic rubric type, but first I want to cover the more basic rubric types so you can differentiate between them.Answer Key or Checklist
Often people consider an answer key or checklist as being a rubric. This is debatable. An answer key is simply the correct/incorrect answers, and perhaps some feedback attached to incorrect answers. I’ve seen this often as a document that is shared with TAs or graders and used to copy/paste for feedback, if given. The answer key is rarely given to students! A checklist is simply making sure the student included components for the assignment or discussion, for example: cited 2 sources, check. Or attached the completed document, check. But it is not really assessing if they LEARNED anything. Just that they followed directions, which is most likely not the learning objective!
Here’s an example of a holistic rubric that is evaluating the expectations for movie time with your best friend.
Holistic Rubric
A holistic rubric is a very simple rubric. It provides a point value and overall criteria to meet that point value. It works as a quick way to grade, but it often does not provide specific feedback to the student, which may lead to confusion. “Holistic rubrics emphasize the use of experts to judge performance assessment. They comprise a comprehensive assessment of the complex multi-faceted characteristics of the tasks undertaken” (Yune et al, 2018). Often a holistic rubric would be used to judge a science fair, or when generally assessing student understanding. These are not bad rubrics, they have their purpose for specific needs. However, they are missing key components that ware featured in analytic rubrics.Here’s an example of a holistic rubric that is evaluating the expectations for movie time with your best friend.
Movie Time with Your Best Friend: Holistic Rubric
Notice it provides a range of scores, as well as descriptions for the criteria needed to meet that score. Awesome! But often what happens with these types of rubrics is that a student may provide evidence of some of the criteria but it doesn’t fit neatly into the scores above.
For example, what happens when the popcorn is burned, but there are numerous selection options of candy and the recipient is very comfortable (lots of pillows and blankets!). What score would you give? Not a 1 because the other two areas are exceptional. Thus the grader must make a decision. Half points? What do they do? Will this be consistent or fair across students? Will the student be confused as to why they received their score?
For example, what happens when the popcorn is burned, but there are numerous selection options of candy and the recipient is very comfortable (lots of pillows and blankets!). What score would you give? Not a 1 because the other two areas are exceptional. Thus the grader must make a decision. Half points? What do they do? Will this be consistent or fair across students? Will the student be confused as to why they received their score?
Analytic Rubric
Let’s review an analytic rubric. Key features identified by Reddy and Andrade in their 2010 article, “A Review of Rubric Use in Higher Education” published in the journal, Assessment & Evaluation in Higher Education, describes the following components included on an analytic rubric:- Evaluation criteria: mapped to the learning outcomes or competencies that are to be measured.
- Quality definitions: qualitative descriptions of what is expected for a given grade or mark.
- Scoring system: grade ranges or degree classifications mapped to the quality description
Movie Time with Your Best Friend: Analytic Rubric
Notice more squares and explanations. The evaluation criteria are along the left side, the definitions are specifically described in the middle, and the scoring system is along the top. The key difference here, from the holistic rubric, is that the criteria are broken out into categories, typically tied to the objective or outcome for the assessment. In this example, the objectives could have been focused on the popcorn preparation, candy options, beverage availability, and the comfort level for the best friend. Then the objective would be further broken out into proficiency level. Having these criteria broken out allows for more feedback to be shared, and clarity for the student to understand what specifically they need to improve upon. (A shout out to the source for inspiration behind these examples, the blog Cult of Pedagogy.)
Why should I use a rubric?
Rubrics communicate expectations!It is important to provide the rubric to the students before they complete the activity or assessment. There aren’t any secrets to learning- we want them to know the goals that they are trying to achieve. Provide them with expectations for successful assignment submission. Help them hit the target!
I also suggest not simply attaching the rubric and never referring to it or explaining what a rubric is there for. If you plan to use rubrics, provide it, and point it out to students. Tell them where to find it, and why they should use it. I always tell my students, to review the rubric before they begin the assignment, and then again after they finish the assignment but before they submit it. This way they can self-assess their work, and decide if more effort is required (Cockett and Jackson, 2018). Or if they are ok with what they’ve accomplished. A rubric helps to clear confusion!
Rubrics are used for easier grading and provide reasoning for grades.
“Rubrics are viewed favorably by both staff and students as a method of enhancing consistency in assessment.” (Cockett and Jackson, 2018). If clearly set up, a rubric can make sure points are being applied or deducted accurately. Especially when you have several people grading in a large class, a rubric can support that grades are being applied objectively not subjectively. “It also makes grading feel more objective to students ('I see what I did wrong here'), rather than subjective ('The teacher doesn’t like me and that’s why I got this grade')" (Univ. of Michigan Center for Academic Innovation, 2020).
Rubrics provide feedback to students!
Be specific with the descriptions for each proficiency level, and what meets each level. Feedback is sadly often lacking in our current courses. And I get it, for large class sizes, it is tough to give specific feedback to each student. But rubrics can at least give some feedback, so the student knows how they can improve or what they did not meet.
When used in online courses especially, rubrics are an important component to clearly defining expectations, reasons for grading, and providing feedback. Rubrics help meet the online course quality standard from Quality Matters, #3.3 that states: “Specific and descriptive criteria are provided for the evaluation of learners’ work, and their connection to the course grading policy is clearly explained” (Quality Matters, 2020).
How can I build an effective analytic rubric?
Now that you know what a rubric includes, and why you should use them, let’s talk about how to create one! And specifically an analytical rubric! It is best to create a rubric when you’re creating the assessment. However, I do understand that it is possible you already have your assessments in a course and you want to create rubrics now to make grading easier, communicate expectations clearly, and give some good old feedback.To build an analytic rubric, I’ve created a checklist you can follow. First, you’ll want to answer some questions, to gather information before building your rubric:
- What are the objectives or learning outcomes for the assessment?
- Do the instructions within the assessment provide any details that will be evaluated in the rubric?
- How many points is this assessment worth?
- Create a draft of a rubric as a table using Word or Google docs.
- Define the criteria from the objectives or learning outcomes. (Evaluation criteria)
- Determine the levels of proficiency, and point values they represent. (Scoring system)
- Describe each criterion across the levels of proficiency. (Quality definitions)
- Share the rubric with a colleague, or locate past submissions and evaluate it with the rubric.
Rubrics in Canvas
Canvas provides the table to build out your rubric, with a place for criteria and ratings. I highly recommend using rubrics in Canvas, as they are quick for grading because it calculates the total points for you when you grade with a rubric in Speedgrader. But it can be confusing for how to set them up. A few nuances to be aware of:- Create rubrics first in a Google Doc or Word doc or spreadsheet, and then copy/paste into Canvas. This is just in case, so you have the rubric as a backup.
- If you plan to use the rubric for grading, you need to check the specific box in the options for it. Also consider freeform comments or not.
- For the 3 assessment types in Canvas (assignments, quizzes/exams, and discussions), rubrics are located in different places. For example, in an assignment the button for creating a rubric is near the bottom of the page, whereas for a quiz the button is within the three dots near the top of the page.
Here is a checklist you can save and follow, for when you are ready to add your rubrics to Canvas.
Summary
- Analytic rubrics include evaluation criteria tied to the learning outcomes/objectives for the assessment, as well as quality definitions that provide specific descriptions of criteria, and a scoring system for a proficiency range.
- It is important to provide rubrics to students in advance, so they see the expectations of the assessment, as well as so they can understand the score they received. Students can use rubrics to self-assess before submission.
- Develop as you are creating the assignment, quiz, or graded discussion. Do this because then you’re mindful of the outcomes you are expecting.
- Include specific descriptions for each point value for each criteria being assessed. This leads to fewer emails about expectations when it is clearly defined. And it’s providing feedback!
Are you interested in creating rubrics for your assessments? Email tic@asu.edu, and an instructional designer on our team will be happy to chat with you about how to make it happen!
Resources
- Video recording part 1 and part 2
- Presentation slides
- Webinar activity slides
- Checklist to Build an Effective Analytic Rubric
- Handout for How to Create Rubrics in Canvas
References
- Brookhart, S. M. (2016). Chapter 1. What Are Rubrics and Why Are They Important? In How to create and use rubrics for formative assessment and grading. Alexandria, VA: ASCD. Retrieved March 2, 2021, from http://www.ascd.org/publications/books/112001/chapters/What-Are-Rubrics-and-Why-Are-They-Important%C2%A2.aspx
- Brookhart, S. M. (2018). Appropriate criteria: Key to effective rubrics. Frontiers in Education, 3. doi:10.3389/feduc.2018.00022
- Cockett, A., & Jackson, C. (2018). The use of assessment rubrics to enhance feedback in higher education: An integrative literature review. Nurse Education Today, 69, 8-13. doi:10.1016/j.nedt.2018.06.022
- Gonzalez, J. (2017, October 03). How to turn rubric scores into grades. Retrieved March 03, 2021, from https://www.cultofpedagogy.com/calculate-points-with-rubrics/
- Gonzalez, J. (2017, October 03). Know your Terms: Holistic, analytic, and single-point rubrics. Retrieved March 03, 2021, from https://www.cultofpedagogy.com/holistic-analytic-single-point-rubrics/
- Hack, C. (2015). Analytical rubrics in higher education: A repository of empirical data. British Journal of Educational Technology, 46(5), 924-927. doi:10.1111/bjet.12304
- Jescovitch, L., et al (2019) "Deconstruction of Holistic Rubrics into Analytic Rubrics for Large-Scale Assessments of Students’ Reasoning of Complex Science Concepts," Practical Assessment, Research, and Evaluation: Vol. 24 , Article 7. DOI: https://doi.org/10.7275/9h7f-mp76 Available at: https://scholarworks.umass.edu/pare/vol24/iss1/7
- Quality Matters (2020). Specific Review Standards from the QM Higher Education Rubric, Sixth Edition. Retrieved March 02, 2021, from https://www.qualitymatters.org/sites/default/files/PDFs/StandardsfromtheQMHigherEducationRubric.pdf
- Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448. https://doi-org.ezproxy1.lib.asu.edu/10.1080/02602930902862859
- University of Michigan Center for Academic Innovation. (2020, July 08). Creating rubrics for effective assessment management. Retrieved March 02, 2021, from https://onlineteaching.umich.edu/creating_rubrics/
- Yune, S. J., Lee, S. Y., Im, S. J., Kam, B. S., & Baek, S. Y. (2018). Holistic rubric vs. analytic rubric for measuring clinical performance levels in medical students. BMC Medical Education, 18(1). doi:10.1186/s12909-018-1228-9
Post author:
Sarah Prosory is an Instructional Designer within the School of Life Sciences' Teaching Innovation Center at Arizona State University. She has worked in higher education for over ten years, supporting faculty in law, engineering, and biological sciences. Her experience includes assisting faculty with in-person, blended, and hybrid courses, as well as making the leap to fully online courses. She provides training to faculty and teaching assistants on how to use educational technologies, and shares evidence-based practices in course design to improve the student experience. She also teaches online for ASU's Mary Lou Fulton Teachers College.
Comments
Post a Comment