Theme: Assessment: Written and Feedback
  • Currently nan/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: nan/5 (0 votes cast)

Logo
Student's feedback on their performance based on summative and formative results - extended by a structured selection using combined performance criteria
Authors: Alexander Schiffel
Johann Arias
Sonja Finsterer
Institutions: RWTH Aachen University (Germany), Medical Faculty
 
Background

Since 2012, the Aachen medical faculty combines students’ results in both summative (individual course test) and formative tests (Progress Test Medicine, Berlin) to give a better view on the students’ performances. This new visibility of performances points to students’ problems that previously were hard to detect. To identify students which have problems in certain areas, an extensive selection of students and thus focussing on special advisory is helpful.

Summary of Work

Each mentor is in charge of approx. 250 students. To be able to handle this large number, lists of students who are in need of assistance are automatically generated to support mentors. Based on these lists, students are invited for individual advisory talks. Students’ strengths can be identified not only on summative data but also based on longitudinal formative data.

Before

After

Summary of Results

The application of the students’ selection criteria has led to a more specific feedback and to the need of knowledge in very specific aspects of learning, like learning methods, long-time acquisition of knowledge, and exam nerves. As a result, the feedback sessions can be more specific. The new process shows that the workflow needs to be optimized and well documented since additional feedback sessions for the same students are often necessary.

As an example, figures for third year students right before their first state exams are presented.

Analysis of number of attempts

Difference of summative and formative ranks

Analysis of combined performance

Conclusion

Mentors are able to identify problems early and to analyze problems on a well-founded basis.

A more specific selection of students and the active invitation by the deanery of study affairs has a different outcome than self-motivated mentoring. It is also more time-consuming and needs new structures in certain aspects.

However, currently the measurement of benefit for students and the faculty is difficult.

Take-home Messages

Application of different evaluation criteria enriches the process of students’ feedback, making it more effective yet more time-consuming.

Background

In a single-student view, individual summative results can be compared to those of the formative tests which are associated to the same domain. To allow for a better evaluation, all of these results are shown along with the results of the comparison group.

Summary of Work
Summary of Results

Methods which can be applied in a mentoring talk according to the presented selection criteria include:

Analysis of number of attempts

  • Analyse approach to learning and exam preparation
  • Cover different methods of assessment
  • Suggest learning groups
  • Refer to experts in the specific medical field, if appropriate

Difference of summative and formative ranks

  • Encourage sustainability of knowledge
  • Redefine approach to learning and goals
  • Exam nervest & Training for different methods of assessment
  • Simulate different exam situations (OSPE, OSCE, etc.) in learning groups

Analysis of combined performance

  • Individual counselling to identidy causes of decline in performance
  • Apply aforementioned methods, if appropriate
Conclusion

Individual strengths & flaws can be analysed both in a cross-sectional and in a longitudinal view using the data provided by the Progress Test.

Cross-sectional

Longitudinal

Take-home Messages
Send ePoster Link