Theme
Clinical Assessment and the OSCE
INSTITUTION
Flinders University, Centre for University Teaching
Flinders University, School of Medicine
It is increasingly clear that human judgement is indispensable in the assessment of students, especially in competency-based education. This requires a focus on assessor expertise, because assessing students requires similar expertise to clinical decision making. Managing judgement biases is important in developing such expertise, because such biases are incomplete (or incorrect) representations in the assessor’s mind of what has occurred during the assessment. But, biases are very hard if not impossible to train away so a viable approach is to focus assessor training on developing biases into full-blown person and performance scripts, because possessing such scripts are associated with expertise. This video provides a background on the role biases can play in assessment.
|
Judgement biases in assessment are not prejudices; instead they are incorrect representations in the assessor’s mind of what has occurred during the assessment. |
This pilot project focused on two biases: the Primacy Effect and Memory Errors of Commission. For each of these, we developed developed an assessor training package containing video presentations for the theoretical background, video vignettes of assessment situations to practice recognition in well-defined situations, real-life YouTube clips for ill-defined situations, activities to apply all this to the participants’ professional contexts and a compendium of practical strategies to prevent biases from unduly influencing the overall judgement. The resources were trialled in two workshops with 56 particpants and can be found at www.flinders.edu.au/better-judgement.
The two biases addressed in this project were the Primacy Effect and Memory Errors of Commission. The Primacy Effect occurs when the assessor excessively weights what happens at the beginning of the assessment compared to the rest of the assessment. Memory Errors of Commission are where the assessor remembers something as having happened, when in fact it did not. These are more fully explained in these short videos.
Primacy Effect
Memory Errors of Commission
When asked what they already knew about biases before the workshop, most participants indicated that they had no or negligible prior knowledge. They valued the opportunity to exchange experiences and interpretations of biases and to exchange strategies. Participants also gained a ‘language’ from the workshops. This was even clearer among experienced examiners who stated that they were previously aware of factors influencing their judgement in an unwanted way, but they had now acquired a language to describe and manage these factors. Typical quotes for this were:
“Had a feeling that as an assessor I do engage in this type of bias … now have a term on which to anchor this bias and have opportunity to self-reflect.”
“Aware of existence but not as a formal name (had taken steps to eliminate them) … [by end of workshop realised there are] more examples of [errors of commission] than thought. Aware of ones that affect me regularly.”
Participants were able to recognise biases in action in the video vignettes. For example, the following video shows Memory Errors of Commission and contains captions to explain what may be happening.
The key to educating assessors in dealing with biases is to not try to train them away, but to provide them with a ‘language’ about biases, thus enabling the assessors to discuss the biases within their communities of practice.
The issue of how to manage biases can be addressed by medical education institutions by incorporating this training package in their faculty development programs.
Biases need to be matched with the appropriate counteracting strategy:
-
memory errors → documentation strategies
-
generalisation errors → rubrics strategies
-
interpretation errors → cognitive strategies
- emotional factors → organisational/institutional strategies.
|
Biases are natural and cannot be stopped from occurring, but we can train assessors to recognise, name and manage them. |
Support for this activity has been provided by the Australian Government Office for Learning and Teaching. The views expressed in this poster do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.
Post your comments
Send Email