Raters' opinions of the first national pilot OSCE in Taiwan

Authors

Keh-Min Liu
Chung-Sheng Lai
Jer-Chia Tsai
Chun-Hsiung Huang
Min Liu

Theme

OSCE and Standard Setting

Category

OSCE

INSTITUTION

Kaohsiung Medical University College of Medicine and Kaohsiung Medical University Hospital, TAIWAN

Background

Starting from 2013, Taiwanese medical graduates must pass the National Objective Structured Clinical Examination (OSCE) before taking the National Medical Licensure Examination (NMLE) Step II.

 

Prerequisites for taking the NMLE Step II include: (1) graduate from a medical school, (2) pass the NMLE Step I, and (3) pass the National OSCE.


Conclusion

The National pilot OSCE was successfully administered. However, the cases needed to be refined and SPs needed more training. Changes made in the 2012 pilot OSCE:

 

n Each medical school provided 25 cases to create a test bank of 300 cases. The OSCE Committee spent substantial time reviewing and refining the case materials, and created training videos.

n Case writers, raters, SPs trainers and SPs must have obtained required training and experience.

n To avoid breaches of test security, case materials weren’t given to raters, SPs trainers and SPs, until the examination day. However, this decreased their time to become familiar with the case materials. To give them more time, the examination schedule was changed from three circuits per day, four days over two consecutive weekends, to two circuits per day, six days over two consecutive weekends.

n Training videos were included in the case materials to assist SPs portray their roles.

Take-home Messages

Extensive preparation is needed to implement a national OSCE.

Summary of Work

In 2011, a 12-station pilot OSCE was simultaneously administered in eleven medical schools. The National OSCE Committee developed the examination format and content, held training workshops, oversaw the examination process, and set passing standards.


Each school recruited its standardized patients (SPs) and internal raters. Student participation wasn’t mandatory. The borderline regression method was adopted for standard setting. To pass the examination, a student must achieve the minimum total score and pass at least nine stations.

Summary of Results

The pilot OSCE was successfully completed in four days over two consecutive weekends. 111 final year medical students took the examination in Kaohsiung Medical University (KMU) and all passed. 


Internal raters in KMU were asked to fill out feedback questionnaires about their opinions on the examination and SPs performance. Raters agreed that the examination venue was well set-up, examination ran smoothly, the test content and difficulty were appropriate, SPs followed instructions and played the patient role during the examination. Raters pointed out the problems they observed, and gave suggestions on case instructions, checklist items and rating standards.

Acknowledgement

We would like to thank all the faculty, staff and medical students in Kaohsiung Medical University and Kaohsiung Medical University Hospital who participated in the pilot OSCE.

The pilot OSCE administered in Kaohsiung Medical University was supported by Department of Health, Ministry of Education, Ministry of Examination, Taiwan Association of Medical Education, Kaohsiung Medical University and Kaohsiung Medical University Hospital.

References
  • Liu M, Huang YS, Liu KM. Assessing core clinical competencies required of medical graduates in Taiwan. Kaohsiung J Med Sci. 2006 Oct;22(10):475-83.
  • Huang YS, Liu M, Huang CH, Liu KM. Implementation of an OSCE at Kaohsiung Medical University. Kaohsiung J Med Sci. 2007 Apr;23(4):161-9.
  • Liu M, Liu KM. Setting pass scores for clinical skills assessment. Kaohsiung J Med Sci. 2008 Dec;24(12):656-63.
Background
Conclusion
Take-home Messages
Summary of Work

The 12-station pilot OSCE was composed of 8 stations of clinical encounters with standardized patients and 4 stations of procedural skills. 

Summary of Results

Acknowledgement
References
Send ePoster Link