Metode Penelitian

Systematic Review/Meta-analysis
Siti Rizny F. Saldi
Respati W. Ranakusuma
Indah S. Widyahening
Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
RSUPN Dr. Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia
Acknowledgements
• Paul Glasziou, Elaine Beller (IEBH, Bond
University Australia)
• Steve McDonald, Sally Green, Miranda
Cumpston (Cochrane Australasia)
• Cochrane Training
[Link]
• Cochrane Handbook of Systematic Reviews of
Interventions
[Link]
Let’s Refresh!
Type of Article and Best study design
Type of Question Type of Study/Methodology
Therapy Information needed about Systematic review/Meta-
treatments (effectiveness, cost, analysis of RCTs
etc.) Double-Blind RCT
Diagnosis Information needed about a Systematic review/Meta-
diagnostic test (sensitivity, analysis of diagnostic studies
accuracy, etc.) Cross-sectional study with
random or consecutive sample
Prognosis Information needed about the Systematic review/Meta-
course of the disease over time, analysis of prognostic studies
expected complications, etc. Cohort/survival study
Aetiology/ Information needed about Systematic review/Meta-
Harm causes of disease or analysis of RCTs/obsv. studies
contributing factors of disease RCTs, cohort study, case-
control Study
Level of evidence

SR-MA

RCTs

Cohort studies

Case-controlled studies

Case series/reports

Background information/
expert opinion
So how should we go about summarizing medical
information?
How do we summarize medical
information?
Traditional approach
• Expert opinion
• Narrative review articles
• Consensus statements
(group expert opinion)

New approach
(Systematic reviews)
 Explicit quantitative
synthesis of ALL the
evidence
Narrative/traditional reviews
(unsystematic)
What makes a review
“Systematic”?
NARRATIVE
SYSTEMATIC
(OVERVIEW)
QUESTION Broad Focused
SOURCES/ Usually unspecified; Comprehensive sources;
SEARCH possibly biased explicit search strategy
SELECTION Unspecified; possibly Criterion-based;
biased uniformly applied
APPRAISAL Variable Rigorous
SYNTHESIS Often qualitative only Either
INFERENCE/ Seldom evidence-based Usually evidence-based
CONCLUSION

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best
evidence for clinical decisions. Ann Intern Med. 1997;126(5):376-80.
Systematic review
• “The application of strategies that limit bias
in the assembly, critical appraisal, and
synthesis of all relevant studies on a specific
topic.”
Oxford Centre of Evidence Based Medicine (OCEBM) Levels Table

• Ensures that all available evidence is taken

into account and minimises ‘cherry-picking’
• Not performing SRs can be dangerous and/or
unethical!
Streptokinase for Myocardial Infarction
intravenous streptokinase vs. a placebo or no therapy in patients who had been hospitalised for acute myocardial infarction.

Mulrow CD. Rationale for systematic reviews. BMJ. 1994;309(6954):597-9.

What is a Systematic Review?
• A systematic review is an overview of primary studies of a clearly
formulated question which contains systematic and explicit
statement of objectives, materials and methods (identification,
CRITICAL
selction, appraisal relevant studies), and systematically
APPRAISAL combine
included studies to arrive at conclusion; and has been conducted
according to explicit and reproducible methodology.

• Two types:
– Qualitative (non-statistical)

– Quantitative (statistical) or meta-analysis

Integrative Literature
Overview (review article)
Unsystematic
No sufficient appraisals
Prone to severe bias

Systematic review
Systematic
No formal statistical methods

Meta-analysis
Systematic
Formal statistical methods
Integrative Literature
Overview (review article)
Unsystematic
No sufficient appraisals
Prone to severe bias

Meta-analysis
Systematic review Systematic
Systematic Formal statistical methods
No formal statistical methods
What is meta-analysis?
• Statistical methods (meta-analysis) may or may not be used to analyze
and summarize the results of the included studies.
• Meta-analysis = quantitative approach for systematically combining
results of previous research to arrive at conclusions about the body of
research
• Advantages:
– To quantitatively summarize estimate from previous studies (i.e.
resolve controversies)
– Helps guide for further research
– Increase statistical power
– Improves generalizability
Steps in Systematic Review/Meta-Analysis
•Formulate question
A. Identification •Develop search strategy
•Searching relevant studies

•Select eligible studies

B. Selection
•Choose variables (comparison & outcomes)

•Assess quality
C. Abstraction •Extract data and calculate individual summary
measures

•Quantitative pooling
D. Analysis •Sensitivity analysis
•Investigate heterogeneity & biases

E. Write-up •Clear presentation

A. Identification
Be methodical: plan first → protocol

▪ Formulate your review questions

▪ Develop search strategy
▪ Searching relevant studies
▪ A comprehensive and reproducible literature
search is the foundation of a systematic
review
How to search?
▪ Sources:
▪ Find published relevant articles:
▪ Major bibliographic databases: PubMed/MEDLINE,
EMBASE, Cochrane.
▪ Textbooks, printed journals, reference lists
▪ Unpublished materials:
▪ Theses, dissertations, trial registries, contact with
experts/personal communication (peer group), etc.
▪ Search strategies:
▪ Database: MeSH terms and text words
▪ Others: hand-searching → a manual page-by-page
examination of the entire contents, to identify all eligible
reports
▪ No limitation on years and languages
Literature search challenges
• Database Bias - “No single database is likely to
contain all published studies on a given subject.”
• English-language bias - occurs when reviewers
exclude papers published in languages other
than English
• Citation bias - occurs when studies with
significant or positive results are referenced in
other publications, compared with studies with
inconclusive or negative findings
• Publication Bias - selective publication of articles
that show positive treatment of effects and
statistical significance.
Is finding all published studies enough?
• Negative studies less likely to be published than ‘Positive’
ones
• Editors tend to accept studies with positive than negative
results
• Positive results tend to be submitted to international
journals, whilst negative results are submitted to local
journals
• Many negative results are product of studies with small
sample size
• Inference by manufacturers
Identifying publication bias

Symmetrical funnel plot

0

1
Standard Error

3
0.1 0.33 0.6 1 3 10
Source: Matthias Egger & Jonathan Sterne Effect
Funnel Plots
Symmetrical plot in the absence of
bias (smaller studies with no
significant effect shown by open
circles)

Asymmetrical plot in the presence

of reporting bias

Asymmetrical plot in the presence

Other reasons for offunnel plotsome
bias because asymmetry:
smaller
studies (open circles) are of lower
True Heterogeneity
methodological quality and
Chance
therefore produce exaggerated
intervention effect estimates.
To minimize publication bias:
(some) solutions
• All trials registered at inception in meta-registry
of trial registries
– [Link]
– [Link]

• Search in other sources:

– Conference proceedings
– Technical reports (research and government
agencies)
– Dissertations, theses
– Contact with primary researchers
B. Selection
Be methodical: plan first → protocol
Aim: to include all relevant studies, based on:
• Study design: randomization (for clinical trial), cohort, etc.
• Patients’ characteristics: diagnosis (including severity), gender, age,
group, race, etc.
• Similarity of exposure or treatment (e.g. drug class, dosage)
• Similarity of outcomes (case definitions)
• Setting (emergency department, outpatient, inpatient)
• Year of publication or conduct (especially if technology or typical dosing
changes)
• Minimum sample size
• Etc.
Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and
exemplars for reporting systematic reviews. BMJ 2021;372:n160
C. Abstraction

• Compile relevant data

– May need information from author(s)
• Summarize for analysis
– Analyze for risk of biases
Abstraction - example
Bias assessment - scales and
checklists
• many scales available
• not supported by empirical evidence
• different scales, different conclusions
• may include criteria not related to bias
• numerical weighting not justified
• difficult for readers to interpret the score
Intervention
RCT

Intervention
Non-RCT

Diagnosis
Overall Bias
D. Analysis
• Combine data to arrive at a summary, 3
measures
– Effect Size (OR/RR/MD), with CI of each study
– Variance with 95% Confidence Interval
– Test of heterogeneity
• Two Graphs
– Forest Plot
– Funnel Plot
• Examine why the studies are heterogeneous (if
they are) → test of heterogeneity
• Use Statistical Packages, several choices
Study level Review level
↓ ↓

Study A Outcome data Effect measure

Study B Outcome data Effect measure

Effect measure

Study C Outcome data Effect measure

Study D Outcome data Effect measure

Source: Jo McKenzie & Miranda Cumpston

What is Forest Plot?
Understanding evidence: The big picture

Systematic reviews are the best way to

understand the effects of treatments because they
consider all the relevant, reliable evidence.

These slides will teach you how to interpret the

main graph used to present the results of a
systematic review of randomized trials.
Forest Plot Favours Treatment A Favours Comparator B
Main Graph

Trial 1
Trial 2
Trial 3
Trial 4
Many similar trials

Trial 5

Trial 6

Trial 7

Trial 1+2+3
+4+5+6+7

Relative Risk (RR)

Steroids given before preterm delivery
reduce the number of babies who die.
Favours steroid Favours placebo

percent who died with steroid

RR = 11.2%
percent who died with placebo "No difference"
RR=1
10%
5% die
10% die with
with steroid
steroid
10%
5% die
10% die with
with placebo
placebo

RR
RR =
= ??
??

0.01 0.1 0.5 1 2 10

Relative Risk
Relative Risk of Death
(RR)
(RR) of death
Favours steroid Favours placebo
Trial Deaths
Deaths
Babies Steroid Placebo

#1 1070 6.8
36 % 11.2
deaths 60 deaths
%
538 babies
532 babies

6.8%
percent who died with steroid
RR = = 0.61
11.2%
11.2%
percent who died with placebo

0.01 0.1 0.5 1 2 10

Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
Favours Point
steroid Favours placebo
estimate
Trial Deaths
Babies Steroid Placebo "best guess at truth"
#1 1070 6.8% 11.2%

95% Confidence interval

"truth probably in here"

0.01 0.1 0.5 1 2 10

Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
Favours steroid Favours placebo
Trial Deaths
Babies Steroid Placebo "Cumulative Meta-analysis"
#1 1070 6.8% 11.2%

#2 130 1.4% 8.2%

#3 126 3.0% 11.9%

#4 127 9.0% 10.0%

#5 146 1.4% 6.7%

#6 144 2.5% 15.9%

#7 743 9.7% 10.0%

Total 2486

0.01 0.1 0.5 1 2 10

Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
0.1 0.5 1 2 10
Forest plot

The label tells you what the comparison and outcome of

interest are
Forest plot

Each study has an ID

Forest plot

The data for each trial are here, divided into experimental and
control groups
Forest plot
• This is the % weight given to each study in the pooled analysis
• For the mean difference: SDs are used together with the sample
sizes to compute the weight given to each study.
Forest plot

Results from each trial are also given numerically

Forest plot

The label above the graph tells you which statistic has been
used
Forest plot
• Mid point of the box shows point effect estimate of each study
• The size of the block for each study is proportional to the % weight.
• The horizontal line represents the confidence interval.
Forest plot
• Take care to read the labels
• Things to the left do not always mean the treatment is
better than the control
Forest plot
• The pooled analysis is given a diamond shape
• The widest bit is located on the point estimate, and the horizontal width in the
confidence interval.
• Does the confidence interval cross the line of no effect?

Overall effect
estimate
Forest plot

Results of the pooled analysis are also given numerically

Forest plot
• Test for heterogeneity
• The result of a Z test with a p value are also given for the overall
effect
1. Which is the largest trial?
2. Which is the smallest trial?
3. How many trials favour treatment?
4. How many trials are statistically significant?

1
2
3
4
5
Study results (1)
• For binary/nominal outcomes:
– Example: death, relapse, cure, may be
continuous originally (e.g. pain relief)
– Presented as OR for individual study with
corresponding 95% CI
– Represented by square, the area = relative
weighted value
– Summarized as combined OR with CI as diamond
– Vertical line → OR = 1 (no effect)
Meta-analysis: Example of graph presentation of RCTs
with nominal outcome

Study I 1992
Study II 1994
Study III 1995
Study IV 1995
Study V 1996
Study VI 1997
Study VII 1999
Study VIII 2000
Combined
0.1 10
OR = 1
Favor drug Favor placebo
Study results (2)
• For continuous/numeric outcomes:
– Example: volume (e.g. blood loss), quality of life,
physical measurement (e.g. lung function), duration
(e.g. stay in hospital)
– Presented as mean difference for individual study
with corresponding 95% CI
– Represented by square, the area = relative weighted
value
– Summarized as combined mean difference with CI as
diamond
– Vertical line → mean diff = 0 (no effect)
Meta-analysis: Example of graph presentation of RCTs
with numerical outcome (xe-xc)

Study I 1992
Study II 1994
Study III 1995
Study IV 1995
Study V 1996
Study VI 1997
Study VII 1999
Study VIII 2000
Combined
-0.3 +0.3
Favor drug Xe- Xc= 0 Favor placebo
Looking for causes of heterogeneity
What is heterogeneity
Variation or Differences
between the true intervention effects underlying
the different studies

Differences across studies in meta-analysis.

Studies should have similar participants,
interventions, comparisons, and outcomes

Clinical Methodological Statistical

Clinical diversity/heterogeneity
• Participants
– e.g. condition, age, gender, location, study
eligibility criteria
• Interventions
– intensity/dose, duration, delivery, additional
components, experience of practitioners, control
(placebo, none, standard care)
• Outcomes
– follow-up duration, ways of measuring,
definition of an event, cut-off points
Methodological
diversity/heterogeneity
 Design
 e.g. randomised vs non-randomised

 Conduct
 e.g. risk of bias (allocation concealment,
blinding, etc.), approach to analysis
Statistical heterogeneity
 There will always be some random variation
between the results of different studies
 Heterogeneity is variation between the true
intervention effects underlying the different
studies
 caused by clinical and methodological diversity
 alternative to homogeneity (identical true effects
underlying every study)
 study results will be more different from each other
than if random variation is the only reason for the
differences between the estimated intervention
effects
How to deal with heterogeneity
• Ignore it and use fixed effects model (not
recommended)
• Test for it and do not pool results if studies
are significantly heterogeneous
• Incorporate it:
– assume that heterogeneity is due to random
differences among studies whose sources can
not be identified
– use random effects model
Identifying heterogeneity
• Visual inspection (eyeball test) of forest plots
• Chi-squared (χ2) test (Q test)
• I2 statistic to quantify heterogeneity
Visual inspection of forest plots
The chi-squared (χ2) test (1)
• The Chi-squared (χ2) is a statistical test for
heterogeneity which assesses whether
observed differences in results are
compatible with chance alone
• The consistency of studies’ results → if CI for
the results of individual studies have poor
overlap, could indicates the presence of
statistical heterogeneity
The chi-squared (χ2) test (2)
 Low power in small size studies or few in
number
 while a statistically significant result may indicate a
problem with heterogeneity, a non-significant result
must not be taken as evidence of no heterogeneity
(p-value 0.10 → to determine statistical significance)

 High power in many studies in a meta-analysis

 may detect clinically unimportant differences with
many studies
The chi-squared (χ2) test (3)
The I2 statistic (1)
• I2 statistic describes the percentage of variability
due to heterogeneity rather than chance (0%-
100%)
– Low values indicate no, or little heterogeneity
• 0%-40%: might not be important
– High values indicate a lot of heterogeneity
• 30%-60%: may represent moderate heterogeneity
• 50%-90%: may represent substantial heterogeneity
• 75%-100%: considerable heterogeneity
• Be cautious in interpreting: can be misleading
since the importance of inconsistency depends
on several factors
The I2 statistic (2)
Analysis
• Many techniques
• Focus on variability
– Fixed effects model
• Only intra-study, ignore inter-study variability
• Narrower CI
– Random effects model
• Considers also inter-study variability
• Broader CI
• Unless the studies are very heterogeneous →
fixed and random model usually similar
• Pre-specify your planned approach in your
protocol
Fixed-effect model

Random (sampling)
error • assumes all studies are
measuring the same treatment
effect
• estimates that one effect
• if not for random (sampling)
error, all results would be
identical

Study
result
Common
Source: Julian Higgins true effect
Random-effects model

Random
error • assumes the treatment effect
varies between studies
• estimates the mean of the
distribution of effects
• weighted for both
within-study and between-study
variation (tau2, 2)

Study-
specific
effect

Mean of true
Source: Julian Higgins effects
What’s the difference?
Random-effects meta-analyses are:
• almost identical to fixed-effect when there is
no heterogeneity
• similar to fixed-effect but with wider
confidence intervals when there is
heterogeneity of the sort assumed by RE
model
• different from fixed-effect meta-analyses
when results are related to study size
No heterogeneity
Fixed Random

Adapted from Ohlsson A, Aher SM. Early erythropoietin for preventing red blood cell transfusion in preterm
and/or low birth weight infants. Cochrane Database of Systematic Reviews 2006, Issue 3.
Some heterogeneity
Fixed Random

Adapted from Adams CE, Awad G, Rathbone J, Thornley B. Chlorpromazine versus placebo for schizophrenia.
Cochrane Database of Systematic Reviews 2007, Issue 2.
Small study effects
Fixed Random

Adapted from Li J, Zhang Q, Zhang M, Egger M. Intravenous magnesium for acute myocardial infarction.
Cochrane Database of Systematic Reviews 2007, Issue 2.
Investigating heterogeneity
• Does the intervention effect vary with
different populations or intervention
characteristics (such as dose or duration) →
Interaction and effect modification
• Methods to identify this:
– Subgroup analysis
– Meta-regression
Subgroup analysis
• Splitting all the participant data into subgroups to
make comparisons between them
• Aims:
– To investigate heterogeneous results
– To answer specific questions about particular patient
groups, types of intervention or types of study.
• Example: Subsets of participants (males and
females), study settings (different geographical
locations).
Participant subgroups

Based on Stead LF, Perera R, Bullen C, Mant D, Lancaster T. Nicotine replacement therapy for smoking cessation. Cochrane Database
of Systematic Reviews 2008, Issue 1. Art. No.: CD000146. DOI: 10.1002/14651858.CD000146.pub3.
Based on Linde K, Berner MM, Kriston L. St John's wort for major depression. Cochrane Database of
Systematic Reviews 2008, Issue 4. Art. No.: CD000448. DOI: 10.1002/14651858.CD000448.pub3.
Intervention subgroups
Sensitivity analysis
• A sensitivity analysis is a repeat of the primary
analysis/meta-analysis in which alternative
decisions or ranges of values are substituted for
decisions that were arbitrary or unclear.
• Potential domains for sensitivity analysis:
– Searching for studies
– Eligibility criteria (P – I – C – O – S)
– Data for analysis (missing SDs, change score vs post-
intervention values, cut-point value )
– Analysis methods (fixed-effect vs. random-effect
methods, OR vs. RR, using several scales)
Critical appraisals of
Systematic Review/Meta-analysis
How to read and appraise a Systematic Review:
The qFAITH tools
Validity:
• Question
• Find
• Appraise
• Include
• Total up
• Heterogeneity
Paul Glasziou Importance
Centre for Research in Evidence Based Practice
Bond University, Gold Coast, Australia
Applicability
Other tools:
Q-

F-
A-

I-

T-

H-
Question

Introduction Section in SR/MA

 Paragraphs containing:
• State the reasons why the review is needed
• Based on problems in clinical setting (high volume, high risk, high cost)
• Any controversies in the literatures? → In treatment, choice of diagnostic
modalities, determination of prognosis
• Ended with objective of SR based on clinical question (PICO)
I

O’s
Final paragraph of the introduction

C
Study
Design

Problem /
Inclusion and Comparison
Population
exclusion
criteria Intervention
Outcome -
Time
Find
✓  ✓
✓
✓ ✓
✓ ✓
✓
✓ ✓
✓ ✓

clinical trial registry (?)

Appraise

Include
How to appraise/assess the individual studies

• The validity of a systematic review ultimately

depends on the scientific method of the retrieved
studies and the reporting of data
• In systematic review to assess treatment effect,
RCTs are considered to be more rigorous than
observational studies
• Quality assessment should be performed by at
least two assessors
Total up

• Meta-analysis = calculated “best guess” of the true effect size

• The statistical combination of the results gives a pooled,
weighted average of the primary results
• It weights the effect size (result) of each study in relation to
sample size of the study
• Optional part of SR
Forest plot
Dichotomous Effect Estimate &
P–I–C–O-T versus Confidence
Continuous Intervals

Pool Effect
and
Interpretation
Heterogeneity
Methods
How well was the research done? (Internal validity) Yes No Unclear

✓
Question – Does the systematic review address a focused
question (PICO)?

……and use it to direct the search and select articles

for inclusion? ✓
Have all
Find – Did the search find all the relevant evidence?  important
domains of
risk of bias
Appraise – Have the studies been critically appraised?
✓ been
addressed?

Include – Did they only include high quality studies?  Explaining

heterogeneity
and excluding

✓
Total up – Have the results been totaled up with low quality
appropriate summary tables and plots? studies only for
MPCE outcome,


Heterogeneity – ……and heterogeneity between studies how about other
assessed and explained? outcomes?
What were the results?
Importance: what were the results?

RR = 0.82 (95% CI, 0.49 to 1.36)

RR = The ratio of the risk of the outcome (MPCE) in the experimental group relative to
the risk of the same outcome in the control group
RR = 0.82 (95% CI, 0.49 to 1.36)
RR = The ratio of the risk of the outcome (MPCE) in the experimental group relative to
the risk of the same outcome in the control group

• Supposed, PEER = 20% ( = CER in your setting)

– in your practice, out of 100 patients having non-cardiac surgery and
received no treatment/control treatment, 20 patients had a major
perioperative cardiovascular event
– PEER (the patient expected event rate) refers to the rate of events we
would expect in a patient who received no treatment or conventional
treatment.
• Experimental event rate (EER) = RR x PEER = 0.82 x 20% = 16.4%
• Absolute risk reduction (ARR) = PEER – EER = 20% – 16.4% = 3.6 %
ARR:
-blocker treatment reduced the risk of major perioperative cardiovascular events in patients having
non-cardiac surgery, of 3.6% from those who received placebo or standard treatment
• Number needed to treat (NNT) = 1/ARR = 1/0.036 = 27.8  28

NNT: We need to treat 28 patients having non-cardiac surgery with perioperative -blocker
treatment for one additional person to avoid major perioperative cardiovascular events
Importance: what were the results?

3rd paragraph

RR = 2.27 (95% CI, 1.53 to 3.36)

RR = The ratio of the risk of the outcome (BNT) in the experimental group relative to
the risk of the same outcome in the control group
RR = 2.27 (95% CI, 1.53 to 3.36)
RR = The ratio of the risk of the outcome (BNT) in the experimental group relative to
the risk of the same outcome in the control group
• Supposed, PEER = 4% ( = CER in your setting)
– in your practice, out of 100 patients having non-cardiac surgery and received no
treatment/control treatment, 4 patients had a bradycardia needing treatment.
– PEER (the patient expected event rate) refers to the rate of events we would
expect in a patient who received no treatment or conventional treatment.
• Experimental event rate (EER) = RR x PEER = 2.27 x 4% = 9.08%
• Absolute risk reduction (ARR) = PEER – EER = 4% – 9.08% = – 5.08%
Absolute risk increase (ARI)
ARI:
-blocker treatment increased the risk of bradycardia needing treatment in patients having non-
cardiac surgery, of 5.08% from those who received placebo or standard treatment

• Number needed to harm (NNH) = 1/ARI = 1/0,051 = 19,6  20

NNH: We would have to treat 20 patients having non-cardiac surgery with perioperative  -
blocker for one additional bradycardia needing treatment
Importance: what were the results?

Could it have been

due to chance?
95% CI for NNT/NNH?
Example
Applicability:
Will they help me look after my patients?
• Refer to questions related to ‘applicability’ in each critical
appraisal worksheets (etiology/diagnosis/therapy/
prognosis)
• Diagnosis:
– Can I do the test in my setting?
– Do results apply to the mix of patients I see?
– Will the result change my management?
– Costs to patient/health service?
• Therapy:
– Is my patient so different to those in the study that the results
cannot apply?
– Is the treatment feasible in my setting?
– Will the potential benefits of treatment outweigh the potential
harms of treatment for my patient?
Applicability

Will the potential benefits of treatment

outweigh the potential harms of treatment for
my patient?

NNT: We need to treat 28

patients having non-
BENEFIT cardiac surgery with
perioperative -blocker
treatment to have one
HARM additional person who
NNH: We would have to avoid major perioperative
treat 20 patients having cardiovascular events
non-cardiac surgery with
perioperative  -blocker, to
cause 1 extra bradycardia
needing treatment
Conclusions
EBM and Systematic Review
EBM Systematic Review
• Steps • Steps
– Answerable Question – Answerable Question
– Search – Search ++++
– Appraise – Appraise x 2
– Synthesize
– Apply – Apply

• Time: 90 seconds • Time: 6 months

• < 20 articles • < 2,000 articles
• This patient survives! • This patient is dead

Find a systematic review!!

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
RSUPN Dr. Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia
Thank you
respatiranakusuma@[Link]