Internship OSCE, average grades and written exams scores at UNAM: Is there a correlation?

Authors

Trejo-Mejía Andrés
Martínez-González Adrian
Sánchez-Mendiola Melchor

Theme

OSCE and Standard Setting

Category

OSCE

INSTITUTION

Universidad Nacional Autónoma de México

Background

Internship is a critical period during which fifth-year medical students develop clinical competence because they have clinical activities with patients and they select the hospital to do it with an average of 4 years previous since the highest to the lowest. During this year they pass 2 months in each discipline area of Internship and at the end of this period they have a written exam with clinical cases and multiple choice questions. UNAM Faculty of Medicine currently uses OSCE exams for formative and end-of-career summative purposes. Several studies show low correlation between OSCE and written exams. (van der Vleuten, 1990, Prislin 1998, Wilkinson, 2004)

Conclusion

The correlation of post-internship OSCE scores with average grades and pre-internship OSCE show students' consistency. The correlation between written test and OSCE scores in the end-of-career summative exam provide supportive evidence for validity. There was no unique variable with high predictive value for the final OSCE (postest). 

Summary of Work

The objective of the study was to correlate OSCE scores before Internship (5th year of medical school), with average grades of previous 4 years, written exams during Internship, written summative exam, and the end-of-career OSCE summative exam. We applied an 18 stations OSCE to 278 medical students before the start of internship to assess clinical competence(pre-test), and an equivalent OSCE at the end of the year (postest). We used check lists at the stations and global ratings for communication skills. The clinical cases in the stations were focused on general medicine (history taking, physical exam, diagnostic and management, X-ray interpretation, laboratory studies interpretation and Analysis of published literature). The clinical cases of stations were classified into the six disciplines: Surgery, Gyn & Obstetrics, Family Medicine, Internal Medicine, Pediatrics and Emergencies. We got the average of 4 years previous and the written exams scores of the same 278 students to do the correlation studies. 

Take-home Messages

Generalized use of formative OSCE before the Internship could allow for provision of feedback to students, faculty and curriculum developers.

Summary of Results

 

Pre-test OSCE global mean score was 55.4 ± 6.6 and the post-test score was 63.2 ± 5.7 (p<0.001). Cronbach’s alpha was 0.62 for the pre-test and 0.64 for the post-test. Pearson's correlation coefficients were:

1) Moderate correlations   among OSCE postest and written summative exam (r=0.44), with OSCE pretest (r=0.44) with average grades of the previous 4 years (r=0.39), all were significant (p<.0001).

2) Low correlation between OSCE (postest) with written exams during Internship ( r= 0.21) and with each discipline.

Acknowledgement
References

Wilkinson TJ, Frampton CM. Comprehensive undergraduate medical assessments improve prediction
of clinical performance. Med Educ.  2004; 38: 1111-1116.
Prislin MD,  Fitzpatrick CF, Lie D, Giglio M, Radecki S, Lewis E. Use of an Objective Structured Clinical
Examination in Evaluating Student Performance, Fam Med. 1998; 30(5): 338-44.
Van der Vleuten CS, Swanson DB.  Assessment of Clinical Skills With Standardized Patients: State of the Art. Teach
Learn Medicine.   1990, 2(2): 58-76
 
Background

Grafico Internship exam

Conclusion
Summary of Work
Take-home Messages
Summary of Results
Table 1 Correlations OSCE postest with written exams (WE) during Internship
  OSCE postest:

Surgery (WE)

 OSCE postest:

Gynecobste-

trics (WE)

  OSCE postest:

Pediatrics (WE)

 

OSCE postest:

Internal Medicine (WE)
 

OSCE postest:

Family Medicine (WE)

  OSCE postest:

Emergen-cies (WE)

  r=0.17

p=0.003

 r=0.11 p=0.05

 r=0.11

p= 0.05

 r=0.23

p=0.0001

 r=0.08

p= 0.14

 r=0.21

p= 0.0004

 

 

Table 2 Correlations written summative exam with OSCE postest areas
 Written summative exam: Surgery OSCE  

Written summative exam:

Gynecobs-tetrics OSCE
 Written summative exam: Pediatrics OSCE  

Written summative exam:

Internal Medicine OSCE
 

Written summative exam:

Family Medicine OSCE
 Written summative exam: Emergencies OSCE
 

r=0.22

p= 0.0002

 

r=0.25

p= <.0001

 

r=0.13

p= <.0001

 

r=0.32

p= <.0001

 

r=0.32

p= <.0001

 

r=0.25

p= <.0001

 

 

 

Table 3 Correlations OSCE pretest with written exams (WE) during Internship
 

OSCE pretest:

Surgery (WE)

 

OSCE pretest:

Gynecobs-tetrics (WE)

 

OSCE pretest:

Pediatrics (WE)

 

OSCE pretest:

Internal Medicine (WE)

 

OSCE pretest:

Family Medicine (WE)

 

OSCE pretest:

Emergencies (WE)

 

r= 0.15

p= 0.01

 

r= 0.15

p= 0.01

 

r= 0.18

p= 0.002

 

r= 0.31

p=<0.0001 

 

r= 0.11

p=  0.06

 

r= 0.22

p= 0.0001

 

 

Table 4 Correlations OSCE postest by areas with written exams (WE) during Internship by areas
 

OSCE Surgery:

Surgery (WE)

 

OSCE G&Obst:

Gynecobs-tetrics (WE)

 

OSCE Pediatrics:

Pediatrics

 (WE)

 

OSCE internal Medicine:

Internal Medicine(WE)

 

OSCE Family Medicine:

Family Medicine(WE)

 

OSCE Surgery:

Surgery (WE)

r=0.10
p= 0.07
r=0.10
p= 0.07
r= -0.08
p= 0.15
 
r=0.12
p= 0.03
r=0.12
p= 0.03
r=0.21
p=0.0004

 

Table 5. Correlation and determination coefficient between OSCE Postest with OSCE pretest, written exams and average grades previous 4 years.
OSCE  postest:
Written sumative exam
OSCE postest:
OSCE pretest
 
 
OSCE postest:
Average grades previous 4 years
 
OSCE postest:
Written exams during internship
 
r=0.44
 
r2=0.20
p<0.001
r=0.44
 
r2=0.19
p<0.001
 
r=0.39
 
r2=0.15
p< 0.001
 
r=0.21
 
r2=0.04
p= 0.0003
 

  

Table 6. Multiple regression analyses to assess summative predictive value of written exams and average grades previous 4 years.
OSCE postest:
 
 
OSCE pretest +
Written summative exam +
Average grades previous 4 years +
Written exams during internship
 
R2=0.30
p<0.001
 
Acknowledgement
References
Send ePoster Link