Theme: Assessment: Written and Feedback
  • Currently 3.00/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: 3.0/5 (1 vote cast)

Logo
Self-monitoring as a strategy to improve performance: a diagnosis based on tests
Authors: Institutions: Federal University of Minas Gerais
 
Background

 

Self-monitoring mistakes, suggesting an inaccurate transition between an automatic mode of practice and reflexive reasoning, are a valuable stimulus to induce self-directed learning among medical students (Eva and Regehr 2007; Eva and Regehr 2011).

By applying tests at the end of the course, it was demonstrated that- at least in experimental conditions - senior medical students are capable of using efficient mechanisms for self-monitoring. When answering questions that "bordered the limits of their competence" students changed their strategy of clinical reasoning - from pattern recognition to reflexive thinking - and improved their performance (Agrawal, Norman and Eva 2012).

However, this may not be true at earlier phases of the course. Comparing performance to solve clinical problems with or without analytic techniques, Mamede et al. (2010) showed that deliberation-without-attention really improved novices’ decisions in simple problems, whereas consciously thinking about complex problems did not help. In this last case, the use of purely analytic techniques could have exposed junior students to "a virtual torrent of clinical features, making it difficult to reconcile the observed pattern with a single diagnostic entity" (Eva 2004). Indeed, an undesirable high cognitive load, while attempting to elucidate clinical cases using reflexive thinking, was one of the hypotheses proposed to explain the inferior performance of medical students, when compared to colleagues who were working with pattern recognition (Mamede et al. 2012).

In order to shed some light on this area, we investigated the accuracy of self-monitoring at the intermediate phase of the course. The study was developed in a "real-life"situation, and used a quite simple method of assessment.

Summary of Work

 

During the final test of a discipline of internal medicine, medical students of the 4th year were freely asked to select among the total 30 multiple choice questions, the ten questions they considered to be the easiest and the ten most difficult. The students were informed that questions that they selected as "easy" would be worth 20% more (1.2) and those selected as "difficult" would be worth 20% less (0.8). The value of each of the 10 remaining questions would be 1.0, totaling 30 to the test. The time to complete the test was recorded.

Summary of Results

From all students in this class (n = 141), one refused to take part in the study and one did not follow the directions to fill the answer sheet. Among the remaining 139 (98.6%, from which 52.5% were men), the mean score of the questions selected as ”easy” (9.1 out of 10) was more than twice the score of the questions selected as “difficult” (4.1 out of 10).

Correct answers among those selected as the easiest, average and most difficult

 

Easiest

Average

Most difficult

Range

5-10

2-10

0-9

Mean

9.1

6.3

4.1

Standard deviation

1.0

1.8

1.5

 

In the group of students with scores below median (20/21 out of 30), almost 2/3 completed the test before the median time (94/95 minutes). On the other hand, almost 2/3 of students whose scores were higher than the median spent a longer time to complete the test (chi-square=7.43, p=0.006).

The time to complete the test and the scores were similar for men and women, but women scored higher in the questions they selected as "more difficult" (F=4.44, M=3.81, t-test=2.5, p=0.016).

Conclusion

Students in an intermediate phase of the medical course, when faced with questions with moderate levels of difficulty, were capable of identifying the limits of their knowledge.

The process seems to be more accurate among those who took longer to complete the test, insinuating an advantage of more extensive usage of reflexive thinking.

Take-home Messages

Self-monitoring processes seem to be accurate at the intermediate phases of the medical course. This ability should be taken in account when designing strategies to induce self-regulated learning.

Reflexive thinking seems to improve performance among these students; its processing may involve gender related specifities.

References

Agrawal S, Norman GR, Eva KW. Influences on medical students’ self-regulated learning after test completion. Med Educ 2012;46:326–35.

Eva KW. What every teacher needs to know about clinical reasoning. Med Educ 2004;39:98–106.

Eva KW, Regehr G. Knowing when to look it up: a new conception of self-assessment ability. Acad Med 2007;82(10 Suppl.), S81–S84.

Eva KW, Regehr G. Exploring the divergence between self-assessment and self-monitoring. Adv Health Sci Educ 2011; 16:311–29.

Mamede S, Schmidt HG, Rikers RMJP, Custers EJFM, Splinter TAW, van Saase JLCM. Conscious thought beats deliberation without attention in diagnostic decision making: at least when you are an expert. Psychol Res 2010;74:586–92.

Mamede S, van Gog T, Moura AS, de Faria RMD, Peixoto JM, Rikers RMJP, Schmidt HG.  Reflection as a strategy to foster medical students acquisition of diagnostic competence. Med Educ 2012;46:464-472.

Background
Summary of Work
Summary of Results
Conclusion
Take-home Messages
References
Send ePoster Link