Is CME effective in improving Physicians` knowledge? Second Order Systematic Review


Mohammad Al-Onazi


Teaching and learning: the teachers toolkits and context for learning


Postgraduate education and continuing medical education (CME)




Continuous medical education (CME) consists of educational activities in the medical field that is designed by faculty who are experts in certain clinical areas. These educational activities might be live programs, online courses, workshops, small group discussion, academic detailing, written materials, or any activity that serve to maintain, or increase the knowledge, skills, and professional competence of health professionals



In summary, pre-educational intervention need assessment and an interactive learning activity is required in designing CME to ensure a positive effect on physician`s knowledge. For sustaining such effect a follow up educational intervention should be planned 6 months to one year later

Take-home Messages

Recommendations& Implications for future policy and research:


An initial step in planning, and designing an activity for CME or CPD is to conduct a systematic process to determine and address the learning needs of the targeted audiences. This review shows clearly that interactive learning activities such as small group discussion, audit and feed back, and academic detailing are effective in improving physicians` knowledge as compared to didactic, and lecture-based activities. Due to the knowledge decay property of human being a follow-up intervention is advisable to sustain the knowledge gained. Most of the educational studies suggested 6 months to one year post-intervention as the appropriate timing for re-intervention. This review clearly found that the included primary studies in the systematic reviews are of poor quality. This finding calls for future researches of good quality to be able to build a firm inferences from such good quality researches

Summary of Work


It is a 2nd Order Systematic Review of the literatures, conducted in an office setting. The study subjects are the Systematic Reviews of a well-designed and properly executed reviews which provide the best evidence on the efficacy of health care interventions.

Inclusion criteria:

The studies will be included if they (were):

  • Systematic reviews
  •  Written in English
  • Published after 2004.
  • Used CME interventions of a didactic or interactive nature.
  • Objectively determining the effect of CME on physician's knowledge.
  • More than 50% of the participants are practicing physicians, or residents in training.


 All included studies passed the following stages:

 Three screening phases.

Assessment of quality of the systematic review.

Collecting general study characteristics.

Collecting CME activity characteristics.

Compiling studies data for comparison.

We utilize the measurement tool developed by Shea and Colleagues in 2007 to assess the quality of reviews and reported ("A MeaSurment Tool to Assess Reviews"


Summary of Results


The search of the electronic databases yielded 184 systematic reviews, while hand Searches yielded 14 systematic reviews. As a total we located 198 systematic reviews. After initial screening of the titles, 127 potentially relevant papers were considered for the second screening of the abstract. This process yielded 54 systematic reviews, which underwent a third screening of the full article to determine wither they satisfied the inclusion criteria. Nine of these studies are found to be relevant. The average quality of the of the included reviews according the AMSTAR were of moderate type. The resulting synthesis can therefore be described as sufficiently robust to sustain the subsequent interpretation and analysis. Participants were primary care physicians (four studies), clinicians ( two studies), resident in training (one study), and obstetrics and gynecologist (one study)


To My wife Dr. Wedad Al-Onazi, for her assisstance in retrieving researches, and to the secretarial staff in the department of Medical Education in College of Medicine in King Saud Ibn Abdulaziz for Health Sciences (COM, KSAU-HS), Mrs Rose and Annabelle, for their valuable administrative support.


Dr. Mohammad Al-Onazi

Dr. Rana Tamim

Prof. Mohi Eldin Majzoub

Take-home Messages
Summary of Work



Yes. No. Can’t Answer. Not Applicable

1-Was a `priori` design provided?

The research question and inclusion criteria should be established before the conduct of the review

Yes. No. Can’t Answer. Not Applicable

2-Was there duplicate study selection and data extraction?

There should be at least two independent data extractors, and a consensus procedure for disagreements should be in place

Yes. No. Can’t Answer. Not Applicable

3-Was a comprehensive literature search performed?

At least two electronic sources should be searched. The report must include years and databases used (eg Central, EMBASE and MEDLINE). Key words and/or MESH terms should be stated and where feasible the search strategy should be provided …

Yes. No. Can’t Answer. Not Applicable

4-Was the status of publication (i.e. grey literature) used as an inclusion criterion?

The authors should state that they searched for reports regardless of their publication type

Yes. No. Can’t Answer. Not Applicable

5-Was a list of studies (included and excluded) provided?

A list of included and excluded studies should be provided

Yes. No. Can’t Answer. Not Applicable

6-Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes.

Yes. No. Can’t Answer. Not Applicable

7-Was the scientific quality of the included studies assessed and documented?

A priori’ methods of assessment should be provided (eg for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo-controlled studies, or allocation concealment as inclusion criteria)

Yes. No. Can’t Answer. Not Applicable

8-Was the scientific quality of the included studies used appropriately in formulating conclusions?

The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the review, and explicitly stated in formulating recommendations.

Yes. No. Can’t Answer. Not Applicable

9-Were the methods used to combine the findings of studies appropriate?

For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity

Yes. No. Can’t Answer. Not Applicable

10-Was the likelihood of publication bias assessed?

An assessment of publication bias should include a combination of graphical aids (eg funnel

plot, other available tests) and/or statistical tests

Yes. No. Can’t Answer. Not Applicable

11-Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both the systematic review

and the included studies.

  Table: Study quality rating:


Quality Rating

Underlying Methodology


8 – 11


6 - 7


 < 6

Summary of Results



The result of this second order systematic review was in agreement with the previous reports (Maliheh et al, Bernard S. Bloom, and  Woosung et al) which indicate that educational interventions most likely to result in improvement of physician knowledge involve multiple interactive educational efforts that is clinically relevant such as, interactive small group sessions, workshops, and written materials or tool kits combined with audit or feedback, and strong communication channels between instructors and learner


Result table:


Evaluation in individual studies

Outcome Measure

Teaching Strategies

Target Audiences

Review Quality



Included Studies Design

Number of Studies Included, (Reported  quality)



Three studies shows some form of evaluation and follow-up.

Pre-/post test

Follow-up, mail survey, audit. and



Improving Knowledge of practicing physicians

Improving practitioner behavior and practice towards older patients

Small –group learning




Didactic lectures

Academic detailing

Opinion leaders

Multiple methods

Primary Care Physicians and community internists


Moderate (6/11)

Interviewing  22 directors of geriatrics CME Practicing Physician Education (PPE) programs that emphasized active-mode learning

RCT*, Systematic reviews, Meta-analysis.

(Details are not specified)


(Not specified)



David C. Thomas17

Self-administered questionnaires in all studies except three

Interviews with GP in two interventions

Audits in three interventions



In 14 interventions measuring physicians` knowledge and attitude, an increase in the knowledge and attitude towards palliative care issues or cancer pain management. One RCT did not find differences in knowledge or attitude between the intervention and control group.

Role model educators, didactic courses, small group discussions, case studies, clinical rounds, home visits, outreach programs, opinion leaders, mini-fellowship, educational materials, guidelines, small group learning, and media events

Primary care physicians





1 Systematic review

11 Interventions as before and after trials

3Educational courses

2 Evaluation comparing to guidelines






(Consort Statement)


Marta P. Alvarez18


Effective intervention in improving the clinicians knowledge and their   prescription of recommended antibiotics for acute outpatients infections

 The median effect of interventions combining clinician education with audit and feedback was smaller than that for clinician education alone.

Academic detailing, educational outreach, workshops, printed materials, educational programs, feedback, mailed information, and symposiums,






13 RCT

12 Controlled before-after trials

1 Interrupted time series




(EPOC Recommendation)^


Michael A. Steinman19

Questionnaires, knowledge tests, observations, surveys, pre-/post-intervention questionnaires

Changes in physicians knowledge, attitude, clinical practice, and patient outcome.

Training courses, written materials, programs, internet courses, seminars, outreach visits, multifaceted intervention, academic detailing, and in practice educational sessions

General Practitioners (GP)





21 RCT

6 Clustered RCT

5 Quasi-experimental


(not specified)


Jing Tian20








Effectiveness of CME intervention on physician's knowledge.

Academic detailing, educational outreach, workshops, printed materials, educational programs, feedback, symposiums. and multifaceted programs.

Practicing physicians




Comparative studies with adequate control group

(RCT, Non RCT**, Observational studies)




(Jadad Score)


Georges Bordage21

Self evaluation, structured and standardized surveys, direct observation of teaching behavior, videotaped teaching sessions, objective structured teaching sessions, pre-and post-test, and resident questionnaires.

Effectiveness of resident-as-teacher interventional programs on the attitude, knowledge, skills, and behavior of the resident.

Lectures, small group discussion, debriefing, role plays, simulation, retreat, feedback, workshop, and seminars

Resident in training





10 Non-RCT

10 Single group study

2 descriptive case series




(Cochrane Handbook for Systematic Reviews of Intervention (Version 5.0.1))


Andrew G. Hill22

Comparative results, feedback, questionnaires, self-assessment,

Learning achievement for knowledge, skills, attitude, and behavior

Educational meetings, interactive sessions, didactic sessions, e-learning






7 Non RCT

13 Before and after comparison studies




(Not specified)





Amer Raza23


1-Health care outcomes

2-health professional behavior.

3- PCPs` knowledge and learning

Multifaceted interventions, interactive seminars, outreach visits, small group workshops, and educational group meetings.

Primary care providers (PCP)





5 Cluster RCT

1 Controlled before and after study




(EPOC Quality Criteria) ^


M. Perry24

Pre- and post-training test, audit, and questionnaires

Kirkpatrick`s four-level model

Computer-based training programs, audit with feedback, lectures, case studies, clinical supervision, simulated clinical scenarios,

Obstetrics and Gynecologist





2 Non RCT

6 Non-comparative

6 Comparative time-series




(Not specified)



Pehrson C25

Send ePoster Link