Theme
Assessment: Written and Feedback
INSTITUTION
Royal College of Surgeons in Ireland (Dublin & Bahrain campuses)
It has previously been suggested that the use of illustrations in assessment has variable effects on individual items (1). This study examines the effect of illustrated text, as opposed to text alone, in MCQ vignettes to discern if any overall bias between the two formats is detectable in an undergraduate histology examination.
We reviewed 6 Histology MCQ examinations from our Medical Junior Cycle (1st medical year). All items were in single best answer format and analysed using classical test theory analysis (2).
Items were then divided into two groups, depending on whether their vignettes used text alone (TA) or illustrated text (IT).
- Vorstenbosch et al, 2013
- Englehardt, 2009
- Schuwirth LW & van der Vleuten, 2004
- Levie & Lentz, 1982
- Carney & Levin, 2002
See more detail for full citations
We analysed 195 single best answer MCQs; 95 used text alone and 100 used illustrated text. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions.
-
There was no difference in item difficulty between the two groups
- (0.76 vs. 0.80; p = 0.862, Mann-Whitney-U).
-
The item discrimination (point biserial correlation), also showed no difference between groups
- (0.305 vs. 0.304; p = 0.948; Independent t-test).
With text-alone MCQs, the decision as to whether to use context-free or context-rich questions depends on the specific item context and information required (3). While the use of appropriate illustrations in learning has been shown to facilitate understanding and increase information retention, particularly for delayed recall (4, 5), information on the effect of illustrations in examinations is more limited but suggests that their influence is dependent on context (1).
- We found no overall bias or effect on either item difficulty or discrimination resulting from the addition of illustrations to Single Best Answer MCQs.
- We suggest that illustrations may useful additions to textual vignettes; whether they test basic recall or more complex reasoning will depend on exactly how they are employed.
- Both illustrated text and text alone MCQs are valid formats for undergraduate histology examinations and may test a range of cognitive levels depending on context.