Theme: 9JJ Admission to Medicine and Postgraduate Training Programmes
  • Currently 5.00/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: 5.0/5 (2 votes cast)

The gap between first impression and multiple mini-interview performance ratings: A comparison between different rater groups
Authors: Mirjana Knorr
Johanna Hissbach
Anja Bath
Wolfgang Hampe
Susanne Sehner
Institutions: University Medical Center Hamburg

The Multiple Mini-Interview (MMI) is currently the most popular method to measure non-academic attributes in candidates for medical school.

Although MMI scores rely on behavioral observations by multiple raters the problem of rater bias should still be considered.

Possibilities to reduce rater bias include:

  1. the use of standardized, anchored rating scales by trained raters
  2. and a balanced assignment of raters to stations according to gender, profession or other characteristics.


Since these measures require a lot of preparatory effort our goal was to display their effects by comparing two different rating types:

  • first impression ratings
  • and MMI performance ratings

formed by raters of different

  • profession
  • and gender.
Summary of Work


  • 2012 MMI for admission to medical school at the Hamburg University Medical Center
  • 192 candidates
  • 1 day, 5 sets, 4 simultaneous circuits
  • 8 different stations with 2-3 raters per station

Raters were either physicians, medical students, psychologists, psychology students or of other professional backgrounds (e.g. sociologists or dentists).

Data analysis

In addition to descriptive statistics we analyzed the effects of

  • rating type (first impression, MMI performance)
  • rater characteristics (gender, profession)
  • and candidate characteristics (gender, age)

on performance ratings in a linear mixed model.

Summary of Results
  1. Overall mean ratings significantly dropped from first impression to MMI performance ratings across all groups
    • First impression: M = 3.64 (95% CI: 3.62-3.67, SD = 0.85)
    • MMI performance: M = 3.48 (95% CI: 3.44-3.50, SD = 0.98)
  2. The change in ratings was influenced by rater (profession, gender) but not by candidate characteristics. Most notably the three-way interaction between rater’s gender, rater’s profession and rating type was significant.
  3. The mean difference varied between rater groups with male psychologists showing the largest gap of 0.3 points between ratings.

  1. Most rater groups adjust their initial first impression rating after using the standardized, anchored rating scheme.
  2. Rater groups vary in their
    • levels of severity for both ratings
    • and in their adjustment from one to the other rating.

These findings implicate differences in rating behavior. Therefore, we advise a balanced assignment of raters of different profession and gender.

Take-home Messages

Observed differences between rating types and rater groups support costly or time-consuming measures like the development and use of standardized, anchored rating scales by trained raters and a balanced assignment of raters according to rater characteristics.

Summary of Work
Summary of Results
Take-home Messages
Send ePoster Link