Abstract Title
Simulation-Based Directed Self-Regulated Learning vs. Instructor Regulated Learning of Advanced Cardiac Life Support (ACLS) Skills - A Randomized Trial

Authors

Luke Devine
Jeroen Donkers
Rodrigo Cavalcanti
Vsevolod Perelman
Ryan Brydges
S. Barry Issenberg

Theme

Postgraduate Education

INSTITUTION

University of Toronto
Maastricht University
Gordon Center for Research in Medical Education, University of Miami

Background
  • Simulation-based mastery learning of Advanced Cardiac Life Support (ACLS) skills is effective
  • Traditional ACLS courses incorporate instructor-regulated learning (IRL)
  • Directed self-regulated learning (DSRL) may prove to be an effective and less expensive way to teach ACLS skills
  • DSRL involves an educator structuring self-regulated learning to ensure appropriate scaffolding exists to optimize learning

Research Questions

  • How does the ACLS skill performance of PGY1 Internal Medicine residents who participate in a DSRL simulation-based mastery learning course compare to residents who participate in an instructor regulated learning (IRL) mastery learning course?
  • What is the cost effectiveness of a DSRL simulation-based mastery learning course?
Summary of Work

Procedure

  • 40 residents randomly assigned in blocks of four into DSRL or IRL group
  • Residents in each group received standard orientation (video - more details), completed a pre-test, received the assessment instruments and minimum passing standard
  • Following the intervention, a post-test and post-course survey were completed

Intervention

  • Modeled after mastery learning curriculum of Wayne et al. (2006b) and focused on pulseless ACLS events
  • Simulation-based training sessions in groups of 4

       DSRL Group

  • Each group member rotated in role of ACLS team leader for practice scenarios
  • After each practice scenario, groups conducted a structured debriefing utilizing a provided debriefing tool
  • During the debriefing, residents could review ACLS algorithms, review tips and teaching points developed in advance for each scenario, access any relevant online materials and review the assessment instrument

 

      IRL Group

  

  • Each group received instruction from experienced ACLS instructors
  • Scenario selection for testing and practice identical to DSRL group
  • Debriefing led by the instructor with access to same debriefing materials as DSRL group

  

Instruments

  • Pre-test and post-test were scored using assessment instruments developed by Wayne et al. (2009), that had been modified based on updated ACLS algorithms
  • Pre-tests and post-tests were video-recorded and later scored by a single rater
  • 25% of scenarios scored by a second rater

Analysis

  • Post-test checklist scores were primary outcome
  • Within group differences determined using paired samples t-tests
  • Differences between groups determined using ANCOVA, with pre-test scores as covariate
  • Intra-class correlation coefficients (ICC) were determined
References
Summary of Results
  • ICC = 0.88 - strong inter-rater reliability
  • Significant improvement was seen in each group between pre-test and post-test (Table 1)
  • Effect size (r) was 0.91 for IRL and 0.95 for DSRL
  • No difference between post-test scores in the IRL and DSRL group (controlling for pre-test scores)   F(1,37) = 0.007, p = .935

 

Table 1 - Pre-test and post-test performance. IRL= instructor regulated learning, DSRL - directed self-regulated learning

 


IRL= (n=20) DSRL (n=20) P value
Mean Pre-test score 71.46 (1.96) 71.23 (1.31) .92
Mean Post-test score 92.97 (1.20) 93.11 (1.20) .94


  • Cost of IRL was higher than DSRL (Table 2)  
  • Costs of simulation lab rental, equipment, etc. was identical in each group
  • Instructor costs were higher in the IRL group, while preparation costs were higher in the DSRL group

Table 2 - Costs associated with each intervention ($Cdn)  IRL = instructor regulated learning, DSRL = directed self-regulated learning

  IRL DSRL
Sim Lab/Simulators $1200 $1200
Instructors $2000 $0
Technician $400 $800
Preparation time $0 $1200
Total $3600 $3200
Conclusion
ACLS Skills
  • DSRL and IRL significantly improve resident performance in responding to pulseless ACLS events, with an average improvement of 22% and with a large effect size (r = 0.9)
  • No difference in ACLS skill performance achieved in the DSRL group compared to the IRL group

Costs

  • DSRL was more cost effective than IRL for a group of 20 residents
  • Although DSRL had higher costs associated with planning and development, savings in instructor costs easily compensated for this
  • The cost for 80 residents to complete a DSRL session would be $10400
  • The cost for 80 residents to complete an IRL  session would be $14400
Take-home Messages
  • Directed Self-Regulated Learning (DSRL) is as effective as Instructor Regulated Learning (IRL) for learning ACLS skills in simulation-based mastery learning 
  • DSRL is more cost-effective than IRL
Acknowledgement

This work was completed as part of an MHPE thesis at the University of Maastricht.

This work was supported by a Royal College of Physicians and Surgeons of Canada Fellowship for Studies in Medical Education and through a fellowship provided by the Herbert Ho Ping Kong Centre for Excellence in Education and Practice (CEEP) at the University Health Network, Toronto.

Technical support was provided by  Finch Taylor and SimSinai at Mount Sinai Hospital, Toronto. 

Background
  • Advanced Cardiac Life Support (ACLS) skills are a required competency for Internal Medicine trainees in Canada
  • Clinical exposure to ACLS events among trainees is decreasing (Mickelson et al., 2011) 
  • Existing ACLS training courses may lead to suboptimal skill acquisition and retention (Semeraro et al., 2006)
  • Half of Internal Medicine residents surveyed felt they have inadequate training opportunities to develop ACLS skills (Hayes et al., 2007)
  • Many Internal medicine residents are not able to demonstrate ACLS skills at the completion of their training (Wayne et al., 2006a)

Mastery Learning

  • Mastery learning has been shown to significantly improve residents' ACLS skills (Wayne et al., 2006b)
  • Mastery learning incorporates principles of deliberate practice to ensure all learners surpass the objectively set level of mastery performance (McGaghie et al., 2008)

Simulation

  • Frequently used for ACLS skills training and mastery learning allowing for training of rare and critical events in a safe environment
  • Defined as "a technique, not a technology, to replace or amplify real experiences with guided experiences, often immersive in nature, that evoke or replicate substantial aspects of the real world in a fully interactive fashion" (Gaba, 2004, p. 126)
  • Cost is often a barrier to implementing simulation-based training 
  • Cost is infrequently reported in simulation-based medical education research (Zendejas et al., 2013)

Directed Self Regulated Learning (DSRL)

  • In DSRL, an educator designs and structures the learning process to direct trainees in their learning (Brydges et al., 2010)
  • DSRL allows for scaffolding to support learning, while maintaining independence to enhance motivation and metacognitive activity
  • DSRL effective in teaching procedural skills (Brydges et al., 2012, Jowett et al., 2007)
  • DSRL can be applied to groups of learners with a shared identity and goals

Research Questions

  • How does the ACLS skill performance of PGY1 Internal Medicine residents who participate in a DSRL simulation-based mastery learning course compare to residents who participate in an instructor regulated learning (IRL) mastery learning course?
  • What is the cost effectiveness of a DSRL simulation-based mastery learning course?
  • It was hypothesized that DSRL would be as effective as IRL and be more cost effective
Summary of Work

Setting

  • University of Toronto - SimSinai and Bloomberg School of Nursing Simulation Centre 
  • Utilized full body mannequin (SimMan®) that could be preprogrammed to respond to real-time interventions 

Participants

  • PGY1 Internal Medicine residents at the University of Toronto, Canada (n=40)
  • University of Toronto REB approval obtained

 

Video - Introduction to simulation session 

References

Brydges, R., Dubrowski, A., & Regehr, G. (2010). A new concept of unsupervised learning: directed self-guided learning in the health professions. Academic Medicine, 85(10 Suppl), S49-55.

Brydges, R., Nair, P., Ma, I., Shanks, D., & Hatala, R. (2012). Directed self?regulated learning versus instructor?regulated learning in simulation training. Medical Education, 46(7), 648-656.

Gaba, D. M. (2004). The future vision of simulation in health care. Quality & safety in health care, 13 Suppl 1, i2-10.

Jensen, A. R., Wright, A. S., Levy, A. E., McIntyre, L. K., Foy, H. M., Pellegrini, C. A., . . . Anastakis, D. J. (2009). Acquiring basic surgical skills: Is a faculty mentor really needed? The American Journal of Surgery, 197(1), 82-88.

Jowett, N., LeBlanc, V., Xeroulis, G., MacRae, H., & Dubrowski, A. (2007). Surgical skill acquisition with self-directed practice using computer-based video training. The American Journal of Surgery, 193(2), 237-242.

McGaghie, W. C. (2008). Research opportunities in simulation?based medical education using deliberate practice. Academic emergency medicine, 15(11), 995-1001.

Mickelsen, S., McNeil, R., Parikh, P., & Persoff, J. (2011). Reduced resident “code blue” experience in the era of quality improvement: New challenges in physician training. Academic Medicine, 86(6), 726-730.

Semeraro, F., Signore, L., & Cerchiari, E. L. (2006). Retention of CPR performance in anaesthetists. Resuscitation, 68(1), 101-108.

Wayne, D. B., Butter, J., Didwania, A., Siddall, V., & McGaghie, W. C. (2009). Advanced Cardiac Life Support checklists for simulation-based education Retrieved June 6, 2012, from MedEdPORTAL www.mededportal.org/publication/1773

Wayne, D. B., Butter, J., Siddall, V. J., Fudala, M. J., Wade, L. D., Feinglass, J., & McGaghie, W. C. (2006a). Graduating internal medicine residents' self-assessment and performance of advanced cardiac life support skills. Medical Teacher, 28(4), 365-369.

Wayne, D. B., Butter, J., Siddall, V. J., Fudala, M. J., Wade, L. D., Feinglass, J., & McGaghie, W. C. (2006b). Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice.Journal of General Internal Medicine, 21, 251-256.

Zendejas, B., Wang, A. T., Brydges, R., Hamstra, S. J., & Cook, D. A. (2013). Cost: The missing outcome in simulation-based medical education research: A systematic review. Surgery, 153(2), 160-176.

 

 

 

Summary of Results

Table 3 - Baseline demographic data, mean (standard error).  IRL = instructor regulated learning, DSRL = directed self-regulated learning

  IRL (n=20) DSRL (n=20) P value
Age 27.4 (0.73) 27.0 (0.54) .65
Male 12 13 .74
Cdn. medical school graduate 19 14 .09
Intl. medical school graduate 1 6 .09
Months since completed ACLS training 8 (0.97) 8.26 (0.70) .83
Number of times participating in pulseless ACLS event 4.70 (1.06) 6.10 (0.97) .33

 

  • On a post-course survey (1=strongly disagree, 5=strongly agree), residents in the DSRL group were significantly more likely to want to train in the IRL group (mean = 3.5), than residents in the IRL group were to have wanted to train in the DSRL group (mean = 2.2, = .001) (Table 4)
  • Both groups felt that participation in their assigned group was an effective way to learn ACLS, although those in the IRL group agreed with this more strongly (Table 4) 

Table 4 - Summary of post-course survey.  Mean score (standard error) on a 5-point Likert scale (1=strongly disagree, 5=strongly agree).  IRL = instructor regulated learning, DSRL = directed self-regulated learning

Question IRL (n=15) DSRL (n=16) P value
I feel that the standard ACLS training is sufficient training to lead a cardiac arrest team 2.80 (0.24) 2.31 (0.25) .094
I am more comfortable leading a resuscitation after this session 4.20 (0.14) 4.25 (0.11) .785
I felt that participating in an IRL group is an effective way to learn ACLS skills 4.40 (0.16)    
I felt that participating in a DSRL group is an effective way to learn.016 ACLS skills   3.75 (0.19) .016
I would have preferred to learn in a self-directed learning group 2.20 (0.29)    
I would have preferred to learn in a group led by an instructor   3.50 (0.18) .001
       


Conclusion
Resident Perception
  • Majority of residents did not feel that traditional training was sufficient preparation to lead a cardiac arrest team
  • Despite equivalent efficacy in improving resident ACLS skills performance, residents in both groups were more likely to express a desire to learn in an instructor led group
  • Trainee preference for instructor regulated learning has been noted previously (Jensen et al., 2009)
  • Residents in this study may be more familiar and comfortable with the traditional model of didactic and instructor led learning because of their prior educational experiences

Limitations

  • Impact of DSRL and IRL on skill retention or transfer to clinical practice was not examined
  • Only 2 hours of deliberate practice provided, however, scope of material was narrow
  • Different scenarios used for pre-test and post-test prevents direct comparison, but addresses potential for test-retest bias and was adjusted for using pre-test score as a covariate in analyses
  • Checklist used in the study and minimum passing scores were not validated specifically in this setting

Future Work

  • Assessment of retention of skills and transfer to clinical setting
  • Determine how to optimize DSRL and what mechanisms make it effective
  • Determine why residents prefer IRL over DSRL despite its effectiveness

Implications

  • ACLS skills are taught to many health care providers
  • DSRL represents a potentially cost-effective way to teach these skills
  • Given apparent learner preference for IRL, educators who wish to engage learners in DSRL may need to acknowledge these preferences and outline the potential advantages of DSRL
Take-home Messages
Acknowledgement
Send ePoster Link