Systematic Review and Meta-

Analysis
Moderator : Dr.Nigussie Yohanes (Assistant
professor)

By Esrael Ayele C-11

Contents

• Introduction and Definitions

• Methods
• Summary
• References
Introduction and Definitions

• systematic review and meta-analysis are both research

methods that use a set of criteria to answer a research
question about a specific topic

• A systematic review involves a detailed and comprehensive

pre-defined criteria and search strategy with the goal of
reducing bias by identifying, appraising, and synthesizing all
relevant studies on a particular topic/ research question
Introduction and Definitions

• A meta-analysis is a quantitative, formal, epidemiological

study design used to systematically assess the results of
previous research to synthesize the data into a single
quantitative estimate or summary effect size and derive
conclusions

• Not every systematic review can include a meta-analysis, as

the studies may have different designs or measure different
outcomes but every meta-analysis require systematic review
Value of SRMAs
• A systematic review answers a research question by
summarizing almost all available empirical data's

• A meta-analysis uses statistics to combine the results of

multiple studies to produce more precise estimates of the
effects of interventions than individual studies
.
• There fore the biggest strength of well conduced SRMAs are
the potential to provide the best evidence for researchers,
clinicians, and policymakers.
Potential drawbacks
• The biggest limitations of well conduced SRMAs are; they are
dependent on the data available and on the quality of those
studies from which the data is obtained
• There are also many other potential pitfalls
– Lack of funding
– Need of training
– Difficulty to synthesize data from variety of study design
– Requires at least two individuals
– Time consuming nearly 6 to 12 months
The key characteristics of a systematic review

• A clearly stated set of objectives with predefined eligibility

criteria for studies;
• An explicit and reproducible methodology;
• A systematic search that attempts to identify all studies that
meet the eligibility criteria;
• An assessment of the validity of the findings of the included
studies
• A systematic presentation and synthesis of the attributes and
findings from the studies used.
• Provide more reliable findings from which conclusions can be
drawn and decisions made.
Key characteristics of meta-analysis

• Provides Quantitative evidence

• Uses statistical methods to combine results of various
independent but similar studies

• Gives more precise estimate of the effect of treatment or risk

factor for disease, or other outcomes, than any individual study
contributing to the pooled analysis
• Examination of heterogeneity is the most important task

• Examining heterogeneity of a group of studies, and

generalizability of responses can lead to more effective
treatments or modifications of management.
Methods
1. Clarify your question
• Develop a review title it is often best to keep titles as short
and descriptive as possible, by using the following formula:
Intervention for population with condition.
• Formulate your question and hypotheses
• Clearly state the objective of your review using the Cochrane
acronym PICOS which stands for population/participant,
intervention/exposure, comparison/context/conditions,
outcomes, study design
2. create a protocol.
• Is useful to ensure that one decides on all key components prior to starting the
review which includes:
• The review question
• Searches (MEDLINE, EMBASE, cocharine library)
• What types of studies to include and exclude (e.g., RCTs only, RCTs and quasi-
experimental designs, qualitative research)
• Domains and conditions to be included, for example, population age range,
conditions, outcomes, and type(s) of interventions and control groups.
• It is also critical to operationally define the minimum number of participants in each
group,
• Methods to be used in data quality assurance, data extraction and screening,
• Plan for data synthesis, analysis and dissemination
• Other additional information
– Date to start and complete
– Review team members
– Etc
3. Conduct your search
• Identify resources to be searched and search terminology
• Construct and run your search
• Generally, it is important to come up with a comprehensive
list of key terms related to each component of PICOS to be
able to identify all relevant trials in the area. .
• Searches generally include several relevant electronic
databases such as MEDLINE, EMBASE, cocharine library), but
it can also include checking article reference lists, hand-
searching key journals, and personal communication with
experts or key researchers in the field.
• Save references using a citation manager (REFWORKS)
4. Select studies

• Once a comprehensive list of abstracts has been retrieved and

reviewed; first remove duplicates
• Studies appearing to meet inclusion criteria would then be
obtained and reviewed in full.
• Make final decision on the studies to be included
• This process of review is generally done by at least two
reviewers to establish inter-rater reliability.
• It is recommended that authors keep a log of all reviewed
studies, with reasons for inclusion or exclusion, and it may be
necessary to contact study authors to obtain missing
information needed for data pooling (e.g., means, standard
deviations).
• Document the screening protocol using PRISMA flow chart
available on internet
5. Extract data
• Extract all the data will allow you to answer the question (e.g.,
authors, publication year, number of participants, age range,
study design, outcomes, included/excluded).
• Data extraction by at least two reviewers is important again
for establishing inter-rater reliability and avoiding data entry
errors.
• Use a simple data extraction form or table to organize the
information extracted from each reviewed study
• Extraction forms
– Create your own (excel sheet, MS access)
– Distiller SR
– EPPI-Reviewer
– SRDR(systematic review data repository)
6. Appraise included studies

• Assess weakness and strengths of each study

– Risk of bias
– Appropriateness of study design
– Quality of reporting
– Choice of outcome measures
– Statistical issues
– Generalizability
• Choose appraisal tool suitable for the review from online
– The GRADE approach (grade of recommendations, assessment, development
and evaluation) currently recommended
– CEBM( center for evidence based medicine)
– CASP (critical appraisal skills program)
– Strengthening the Reporting of Observational Studies in Epidemiology
(STROBE) checklist
– Consolidated Standards of Reporting Trials (CONSORT),
– Etc …
7. Synthesis

• All systematic reviews should contain summary of findings

tables and texts to provide initial descriptive summary and
explanation for the characteristics and findings of the
included studies
• Synthesis involves the combination and summary of
individual studies included in the review
• It can be done quantitatively using formal statistical
techniques such as meta-analyses or
• If statistical pooling of results is not appropriate you can use
narrative synthesis approach
• Both Statistical and narrative approaches can be used
together for quantitative data's
Statistical approach

• There are various statistical programs available to calculate

effect sizes for meta-analyses, such as
– Review Manager (RevMan)
– Comprehensive Meta analysis software
– MetaLight
• Meta-analyses can be used to
– To establish statistical significance with studies that have conflicting
results
– To develop a more correct and precise estimate of effect magnitude by
increasing statistical power
– To examine potential reasons for variability or heterogeneity in study
results
– To examine subgroups with individual numbers that are not
statistically significant
Statistical

• The final pooled effect estimate is the key outcome

• The most common measures of effect used for dichotomous
data are the risk ratio (also called relative risk) and the odds
ratio
• The dominant method used for continuous data are
standardized mean difference (SMD) estimation
• Effect sizes are stated along with a 95 % confidence interval
(CI) range, and presented in both
– Quantitative format and
– Graphical representation (e.g., forest plots).
Statistical

• Forest plots visually depict each trial as a horizontal diamond

shape with the middle representing the effect size and the
end points representing both ends of the CI.
• These diamonds are presented on a graph with a center line
representing the zero mark.
• Often the left side of the graph (< zero) represents the side
favoring treatment, while the right side (> zero) represents
the side favoring the control condition
Statistical

• At the bottom of the graph is a summary effect size or

diamond representing all of the individual studies pooled
together.
• Ideally, we would like to see this entire diamond (effect size
and both anchors of the CI) falling below zero, indicating that
the intervention is favored over the control
• In addition, most programs also calculate a heterogeneity
value to indicate whether the individual studies are similar
enough.
• In this case, it is preferable to have non-significant findings for
heterogeneity.
Statistical
• Typically used heterogeneity metrics include the Cochran's Q
statistic, a chi-squared (χ2) test of heterogeneity with k-1
degrees of freedom and the inconsistency index I2, which
describes the percentage of total variation across studies

• Some ways to investigate reasons for heterogeneity are

subgroup analysis and meta-regression

• It is still possible to pool studies when significant

heterogeneity exists, although these results should be
interpreted with caution or reasons for the heterogeneity
should be explored.
Narrative synthesis approach
• Project for economic and social research counsel (ESRC)
provide guidance on the conduct of narrative synthesis for SR
• The general frame work includes
– Theory of how the intervention work
– Summary of findings for the studies
– Relation ship b/n studies
– Overall assessment of the strength of the study
8. Write up the report
• To write up your report you can use
– PRISMA checklist
– EQUATOR network toolkit

• As with all papers, the last step in the writing process involves
summarize the findings, and providing recommendations:
– For clinical work (e.g., which interventions are efficacious,
for whom, and under what conditions) and
– For research (e.g., what areas/ topics /interventions
require further research)
Disseminate findings

• Although reviews conducted through the Cochrane

Collaboration get published in the online Cochrane Database
of Systematic Reviews, they are often quite lengthy and
detailed.

• Thus, it is also possible and encouraged to publish

abbreviated versions of the review in other relevant academic
journals, as long as they are clearly indicated

• Plain language summaries for families and patients are also

commonly provided, and there is an expectation that reviews
should be regularly updated to ensure they are always up-to-
date and relevant.
Summary
• Systematic reviews and meta-
analyses are considered at the top
of the hierarchy of evidence
• They can provide invaluable
information to healthcare
providers, researchers, and
policymakers given that they are
conducted properly
• The conduct of systematic reviews
and meta-analyses requires
multiple steps that need to be
transparent, well documented and
reproducible
References
• Gopalakrishnan, S. and Ganeshkumar, P., 2013 Systematic
reviews and meta-analysis: understanding the best evidence
in primary healthcare. Journal of family medicine and primary
care
• Centers for Disease Control and Prevention, 2019. Systematic
Reviews. Available from:
https://www.cdc.gov/library/researchguides/sytemsaticrevie
ws.html
• Egger, M., Smith, G.D. and Phillips, A.N., 1997. Meta-analysis:
principles and procedures. BMJ, 315(7121), pp.1533-1537
• Uman, L.S., 2011. Systematic reviews and meta-analyses.
Journal of the Canadian Academy of Child and Adolescent
Psychiatry, 20(1), p.57.
Thank U