Abstract Title
Is a picture worth a thousand words?

Authors

JC Holland
R O'Sullivan
R Arnett

Theme

Assessment: Written and Feedback

INSTITUTION

Royal College of Surgeons in Ireland (Dublin & Bahrain campuses)

Background

It has previously been suggested that the use of illustrations in assessment has variable effects on individual items (1).  This study examines the effect of illustrated text, as opposed to text alone, in MCQ vignettes to discern if any overall bias between the two formats is detectable in an undergraduate histology examination.

Summary of Work

We reviewed 6 Histology MCQ examinations from our Medical Junior Cycle (1st medical year).  All items were in single best answer format and analysed using classical test theory analysis (2).  

Items were then divided into two groups, depending on whether their vignettes used text alone (TA) or illustrated text (IT).

References
  1. Vorstenbosch et al, 2013
  2. Englehardt, 2009
  3. Schuwirth LW & van der Vleuten, 2004
  4. Levie & Lentz, 1982
  5. Carney & Levin, 2002

See more detail for full citations

Summary of Results

We analysed 195 single best answer MCQs; 95 used text alone and 100 used illustrated text.  The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions.

  • There was no difference in item difficulty between the two groups
    • (0.76 vs. 0.80; p = 0.862, Mann-Whitney-U).
  • The item discrimination (point biserial correlation), also showed no difference between groups
    • (0.305 vs. 0.304; p = 0.948; Independent t-test).
Conclusion

With text-alone MCQs, the decision as to whether to use context-free or context-rich questions depends on the specific item context and information required (3). While the use of appropriate illustrations in learning has been shown to facilitate understanding and increase information retention, particularly for delayed recall (4, 5), information on the effect of illustrations in examinations is more limited but suggests that their influence is dependent on context (1).

  • We found no overall bias or effect on either item difficulty or discrimination resulting from the addition of illustrations to Single Best Answer MCQs. 
 
  • We suggest that illustrations may useful additions to textual vignettes; whether they test basic recall or more complex reasoning will depend on exactly how they are employed.
Take-home Messages
  • Both illustrated text and text alone MCQs are valid formats for undergraduate histology examinations and may test a range of cognitive levels depending on context.
Background
Summary of Work

MCQs were divided into two groups according to whether their vignettes used text alone (TA) or illustrated text (IT).  Both TA and IT MCQs were used to test a range of cognitive levels and an example of both a context-free and context-rich MCQ from each group may be seen in the image below.

Examples of MCQs

References

1. Vorstenbosch MATM, Klaassen TPFM, Kooloos JGM, Bolhuis SM, Laan RFJM. Do images influence assessment in anatomy? Exploring the effect of images on item difficulty and item discrimination. Anatomical Sciences Education. 2013;6(1):29-41.

2. Engelhardt PV. An Introduction to Classical Test Theory as Applied to Conceptual Multiple-choice Tests. Getting Started in PER, edited by C Henderson and KA Harper (American Association of Physics Teachers, College Park, MD, 2009), Reviews in PER. 2009;2.

3. Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 2004 Sep;38(9):974-9.

4. Levie WH, Lentz R. Effects of text illustrations: A review of research. ECTJ. 1982; 30(4):195-232.

5. Carney R, Levin J. Pictorial Illustrations Still Improve Students' Learning from Text. Educational Psychology Review. 2002;14(1):5-26.

Summary of Results

Q-Q plots were performed of all data to identify distribution.

Item difficulty did not significantly differ between the TA and IT groups.

Item discrimination, as measured by point biserial correlation, also did not significantly differ between the TA and IT groups.

Conclusion
Take-home Messages
Send ePoster Link