Given the challenges we are facing this semester in light of the pandemic, rethinking our exams is essential for the success of our students. Stress, anxiety, and navigating a new modality creates a perfect storm that may derail many of our students and their ability to do well in our courses. This is the time to rethink the design of our assessments to ensure our students are engaged and learning the skills and knowledge we want them to learn. Recently, Josh Caulkins and Sarah Prosory examined several options for adjustments to your exams, some of which may take a small amount of time to implement but may have a big impact overall.
There are many ways to assess learning outcomes, and really find out if your students are understanding the topic or have gained the necessary skills. Here are some ideas for ways to adjust your exams in this time of COVID and beyond. We describe a few ideas for exam adjustments on a scale based on approximately how much time it takes to prepare or run, so the effort on the instructor. Sometimes it may take you more time based on what needs to be done- perhaps you already have a process in place and are simply making an adjustment, or you have a new course and are looking to develop an assessment process. It all depends too on, the learning outcomes! Even the committed effort options are worth doing though, as you will see great results.
Unpack Multiple Choice Question Exams
We began with a poll: "What ways do you currently evaluate student learning in your course(s)?" Multiple choice was the leader with 89% of participants using them within their courses. A discussion followed describing the benefits and drawbacks of multiple choice questions (MCQ).
Concerns and Challenges
- High stakes exams may add additional stress to already stressed students
- Do we have an obligation to reduce students already traumatized by current pandemic, plus social, political, and/or personal issues?
- Example: The School of Life Sciences strongly advocates instructors to employ empathy and accommodate students as much as possible at this time.
- High stakes exams may add additional stress to already stressed students
- Do we have an obligation to reduce students already traumatized by current pandemic, plus social, political, and/or personal issues?
- Example: The School of Life Sciences strongly advocates instructors to employ empathy and accommodate students as much as possible at this time.
- Anti-cheating strategies are harder to employ effectively online
- Privacy concerns if we require use of Webcam and Lock-down browser
- Significantly more false positives than actual cheaters (suspicious behaviors may not be cheating)
- Difficult to enforce academic integrity rules
Backward Design & Principles of Learning
What are the skills and knowledge that we care the most about?
- Think deeply about what you value most
- Does your course currently allow students adequate time to practice and master the skills and knowledge you care about the most?
- What are the problems we want our students to be able to solve?
- These may serve as the basis of how we approach the design of our courses AND our assessments.
- Identify goals
- Determine tasks and criteria
- Plan learning activities
The focus for the webinar's topic is on the "how student learning will be assessed" or the middle part of the backward design process. It is also important to recognize here the need for knowledge of Bloom's Taxonomy, in order to create learning objectives that are measurable to the level of learning.
Ideas for Exam Adjustments
- Several shorter, weekly "exams" or quizzes
- Can be MCQ or short answer or even reflection questions
- Low effort: You already have exam questions.
- Check their understanding sooner than waiting until mid-way into semester.
- All of these mini exams replace the large summative, high-stakes, anxiety-inducing exams.
- Adjust the questions based on how well they are doing- have they mastered the skill or knowledge yet?
- Due by end of class meeting day, once per week.
- Consider Canvas quiz/exam features
- Shuffle answers- check box when editing! Super low effort! Watch for: “all of the above” - won’t work.
- One question at a time- only this, not the lock question after answering (not recommended). Gives the student focus. Still allows for jump list and tag questions to return to. Limits easy copy of whole exam. Less scrolling for students.
- Time limit- this is what should match your “class time” and clock starts ticking once students click Begin. Shows timer to countdown, but also can be hidden by student so as to lessen anxiety seeing it count down.
- Question groups (shuffle)- I believe the least used but low effort to do! Sounds like more work, but is not. Create 1 group, drag all questions into it, and set the parameters to select the amount of questions possible and how many points. Watch for: questions that are worth different amounts of points need to be in groups together. Questions that must go in a certain order cannot be put in the group.
- Open book or notes
- Could be MCQ or short answer/essay
- Questions should be more applied, higher Blooms level such as interpreting a graph and applying scientific principles
- Not for basic, easily searchable questions
- Moderate effort because questions may need to be revised.
- Essay exams/case studies
- Rubrics are essential for ease of grading
- Provide practical information, students can visualize scenario.
- Answer will allow them to apply their knowledge or skills
- Check out: National Center for Case Study Teaching in Science
- Presentations
- Team-based or individual
- Use for learning outcomes that ask students to describe, evaluate, apply
- Helpful for real-world skills
- Student colleagues evaluate, use Google Form to collect feedback. Try using the RISE Model from Emily Wray for students to follow when giving feedback, or even reflecting on their own work.
- Project-based or team-based
- "Project Based Learning is a teaching method in which students gain knowledge and skills by working for an extended period of time to investigate and respond to an authentic, engaging, and complex question, problem, or challenge." (PBLWorks).
- Clearly define what is expected
- Rubrics essential
- Scaffold skills to build up to final deliverable (very real-world)- have a list outlined of benchmarks and deliverables along the way
- Each team member has role, be clear with these
- Evaluate along the way: how is it going? What can be improved?
- Two-stage exams
- Also called “Group exams” or “Collaborative exams”
- 1st half of exam time: Individual (80-90% of total exam grade)
- 2nd half of exam time: Groups (10-20% of total exam grade)
- Benefits: Reduces stress, and turns assessment into a learning experience
- Challenge: Choreography takes time
- Pro-tip: Any student who earns a higher grade on individual exam section can opt to remove group grade from their overall exam grade.
- Groups should be assigned; students should not choose their own groups.
- Can be run online using Zoom rooms.
- Additional excellent resources at Univ. of British Columbia
- Specifications grading
- Students are graded pass/fail on assignments / tests or on “bundles” of assignments and tests.
- Instructors provide very clear, detailed specifications (specs) for what constitutes a passing (acceptable/ satisfactory, ~B-level) piece of work.
- Students are allowed at least one opportunity to revise an unacceptable piece of work, or start the course with a limited number of tokens that they can exchange to revise or drop unacceptable work or to submit work late.
- Bundles and modules that earn higher course grades require students to demonstrate mastery of more skills and content, more advanced/ complex skills and content, or both.
- Bundles and modules are tied to the learning outcomes of the course or the program. Students will not necessarily achieve all the possible outcomes, but their course grade will indicate which ones they have and have not achieved.
- For a brief overview of specs grading, read Nilson’s short piece “Yes, Virginia, there’s a better way to grade.”
Final Thoughts
- Acknowledge that our students may be experiencing trauma.
- What can we do to alleviate this?
- Foster trust and transparency through our teaching.
- What does it mean to trust our students?
- What might that look like?
- Design course activities and assessments to reduce stress and trauma.
- Which of the strategies we discussed today might you use in your classroom and with your students?
Call to Action
What alternative assessment type that we shared would you like to try in a future semester? How might you change your course in other ways to reduce student anxiety or challenges during this time of crisis? Need support with implementation? Email us at tic@asu.edu and schedule a consultation with an instructional designer.
Resources
- Video recording
- Presentation slides
- Activity slides
- Wiggins, Grant, and McTighe, Jay. (1998). Backward Design. In Understanding by Design (pp. 13-34). ASCD.
- Canvas Guides on Question Groups
- PBL Works page on Project-based Learning
- National Center for Case Study Teaching in Science
- Two-stage exam page at UBC
- Specifications grading article by Linda Nilson (Inside Higher Ed, 2016)
- Trauma-informed pedagogy article by Mays Imad (Inside Higher Ed, June 2020)
Post author
Sarah Prosory is an Instructional Designer within the School of Life Sciences at Arizona State University. She has worked in higher education for 10 years, supporting faculty in law, engineering, and biological sciences. Her experience includes assisting faculty with in-person, blended, and hybrid courses, as well as making the leap to fully online courses. She provides training to faculty and teaching assistants on how to use educational technologies, and shares best practices in course design to improve the student experience.
Comments
Post a Comment