• Currently nan/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: nan/5 (0 votes cast)

Authors Institution
Jesper Hessius Medical Student
Niclas Lewisson MD
Jakob Johansson MD, PhD

Contact Details

Jesper Hessius: Hessius@me.com
Faculty of Medicine Uppsala University
Department of Surgical Sciences Uppsala University
Department of Surgical Sciences Uppsala University
Theme
eLearning
Using a Mobile Application as a Supplemental Tool for Student Evaluations - Adding iEvaluation to e-Evaluation
Background

In 2012, we developed a mobile application (app) for the iOS platform enabling students to access our clinical teaching assessment instrument using their smartphone. It is not yet known if application-based evaluation is a useful addition to web-based evaluation. 

Summary of Work

The assessment instrument, a web-based questionnaire of 10 questions (Likert scale 1–6) and text box for comments, was ported to a mobile app. The app was launched for the autumn semester of 2012 for third- to fifth-year medical students at Uppsala University (Sweden). Evaluations submitted through the app were compared to those submitted through the website.


 

Application screenshots

References

Lewisson N, Hellgren L, Johansson J. Quality Improvement in Clinical Teaching through Student Evaluations of Rotations and Feedback to Departments. In press in Medical Teacher

Summary of Results

Approximately 49 % of the students in the study owned an iPhone at the end of the semester. Out of 1734 submitted evaluations 21.7 % were submitted using the app (n=376) versus 78.3 % submitted through the website (n=1358). Female students were more prone to evaluate using the app (female: 25 % versus male: 17 %) (p<0.001).

Fig 1Fig 2



No difference in mean ratings±SD was observed between app evaluations (4.30±1.12) and web evaluations (4.36±1.08) (p=0.38). The distribution of ratings on the scale (1-6) was similar in both groups. The prevalence of free text comments did not differ between app evaluations (49 %) and web evaluations (53 %) (p=0.16). 

 


Hint: Click me to switch graph!
Conclusion

Evaluations submitted through a mobile application did not differ in mean ratings or prevalence of free text comments compared to web evaluations.

Take-home Messages

Mobile app-based evaluation appears to be a useful addendum to web-based instruments.

Designed & Managed by Innovative Technology®
Background

For a more in depth background of the clinical assessment instrument for which the app was developed, see this excerpt from the upcoming Medical Teacher article "Quality Improvement in Clinical Teaching through Student Evaluations of Rotations and Feedback to Departments" by N. Lewisson, L. Hellgren, J. Jakobsson.


The number of students admitted to medical schools in Sweden has increased considerably over the past few years. This implies a specific challenge for the concept of clinical teaching since an increased number of students in a clinical rotation has been demonstrated to have a negative effect on the perceived quality of education (Dolmans et al. 2002). Moreover, continuous reductions in healthcare and decreasing numbers of hospital beds might further challenge the quality of clinical teaching (Lindgren et al. 2011). Thus, awareness regarding the quality of clinical teaching is needed and there is already some data indicating that there is need for improvement. In a large nationwide evaluation from the Swedish National Agency for Higher Education (SNAHE 2007) it was concluded that clinical education often is delivered without reference to expected learning outcomes, that clinical skills are not systematically assessed and that the students too often are spectators rather than active participators.


At Stanford University, a Faculty Development Program (SFDP) for clinical teaching was developed (Skeff et al. 1992). The programme identifies a framework of seven key categories of clinical teaching which can be used to encourage teacher reflection and improve the effectiveness of teaching behaviour (Litzelman et al. 1998). This programme has been successfully transported to a Swedish setting (Johansson et al. 2009) and has been used at the Uppsala Medical School, Sweden, for many years. In 2009, the programme committee of Uppsala Medical School concluded that a standardised assessment of clinical teaching was needed and should, therefore, be implemented. Building on a 26 item questionnaire based on the framework of clinical teaching from Stanford (Litzelman et al. 1998), we developed an evaluation instrument adapted to the needs of Uppsala Medical School. A questionnaire in which students would rate their clinical rotations was generated consisting of 10 questions reflecting the seven categories of clinical teaching from Stanford (Kihlberg et al. 2011). The purpose of the questionnaire was twofold. First, it would enable evaluation of the entire faculty during a clinical rotation rather than focus on the individual teacher. This is in alignment with the recommendations from a recent review proposing recognition of collaborative work instead of individual awards (Huggett et al. 2012). Second, it would be possible to continuously measure the quality of clinical teaching in order to provide specific feedback to the faculty as a group and constitute the basis for quality improvement when necessary.


References

  • Dolmans DH, Wolfhagen HA, Essed GG, Scherpbier AJ, Van Der Vleuten CP. Students' perceptions of relationships between some educational variables in the out-patient setting. Med Educ 2002; 36(8): 735–741
  • Huggett KN, Greenberg RB, Rao D, Richards B, Chauvin SW, Fulton TB, Kalishman S, Littlefield J, Perkowski L, Robins L, Simpson D. The design and utility of institutional teaching awards: a literature review. Med Teach 2012; 34(11): 907–919
  • Johansson J, Skeff K, Stratos G. Clinical teaching improvement: the transportability of the Stanford Faculty Development Program. Med Teach 2009; 31(8): .e377–82
  • Kihlberg P, Perzon M, Gedeborg R, Blomqvist P, Johansson J. Uniform evaluation of clinical teaching - an instrument for specific feedback and cross between departments. Högre utbildning 2011; 1(2): 139–150
  • Lindgren S, Brännström T, Hanse E, Ledin T, Nilsson G, Sandler S, Tidefelt U, Donnér J. Medical education in Sweden. Med Teach 2011; 33(10): 798–803.
  • Litzelman DK, Stratos GA, Marriott DJ, Skeff KM. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med 1998; 73(6): 688–695
  • Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: a comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof 1992;15(3): 350–366
  • SNAHE 2007 Evaluation of undergraduate programmes in medicine and health care at Swedish universities and university colleges. Swedish National Agency for Higher Education. Retrieved November, 4, 2012.
    http://www.hsv.se/aboutus/publications/reports/reports/2007/
    evaluationofundergraduateprogrammesinmedicineandhealthcareat
    swedishuniversitiesanduniversitycolleges.5.3c1b9686119162e52038000604.htm
Summary of Work
References
Summary of Results
Conclusion
Take-home Messages
Send ePoster Link