Study selection and critical appraisal
Study selection is determined by the inclusion/exclusion criteria you have specified in your project. At a minimum, two reviewers will independently screen search results.
There are two levels of screening: title and abstract screening and full-text article review.
First level: screening of title and abstracts of records to see if they potentially meet the criteria. Those meeting the criteria will move forward to the second level.
Second level: full-text article review.
Track the number of excluded records and the reasons for exclusion, review the PRISMA checklist.
Once you have selected your studies, you can begin the data collection/data extraction process. Data extraction helps authors to summarize and appraise studies.
What are the essential elements/characteristics you will need to know about your included studies?
There are data extraction tools and forms to help in this process.
Below is an example of a data extraction form used for RCT studies in a systematic review.
This form was created by the Cochrane Collaboration.
Critical appraisal is the careful analysis of studies to determine their relative value. The Health and Medical Division of the National Academy of Sciences includes Standard 3.6: “Critically appraise each study:
3.6.1 Systematically assess the risk of bias, using predefined criteria
3.6.2 Assess the relevance of the study’s populations, interventions, and outcome measures
3.6.3 Assess the fidelity of the implementation of interventions”
See also: What is Critical Appraisal?
Critical appraisal tools and checklists
AMSTAR Checklist (A Measurement Tool to Assess Systematic Reviews)
A valid, reliable and useable instrument that helps users differentiate between systematic reviews, focusing on their methodological quality and expert consensus.
CATMaker and EBM Calculators
Centre for Evidence-Based Medicine (Oxford, UK). CATmaker is a computer-assisted critical appraisal tool, now free. It provides guided appraisal and calculations for therapy, diagnosis, prognosis, etc.
GRADE Working Group
The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations.
The ROBINS-I tool (Risk Of Bias In Non-randomized Studies - of Interventions)
by the Cochrane Methods Bias Group
SIGN Critical Appraisal Notes and Checklists
Includes checklists for systematic reviews, RCTs, and other types of studies.
E-books on critical appraisal and research design
Available online from the UBB
(Link to swissbib list)
Print books on critical appraisal and research design
Available for borrowing from the UB
(Link to swissbib list)