Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions

Christopher W Halladay; Thomas A Trikalinos; Ian T Schmid; Christopher H Schmid; Issa J Dahabreh

doi:10.1016/j.jclinepi.2014.12.017

Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions

J Clin Epidemiol. 2015 Sep;68(9):1076-84. doi: 10.1016/j.jclinepi.2014.12.017. Epub 2015 Feb 7.

Authors

Christopher W Halladay¹, Thomas A Trikalinos², Ian T Schmid³, Christopher H Schmid¹, Issa J Dahabreh⁴

Affiliations

¹ Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Biostatistics, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
² Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Health Services, Policy, and Practice, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
³ Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA.
⁴ Center for Evidence Based Medicine, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA; Department of Health Services, Policy, and Practice, School of Public Health, Brown University, Box G-S121-8, Providence, RI 02912, USA. Electronic address: issa_dahabreh@brown.edu.

PMID: 26279401
DOI: 10.1016/j.jclinepi.2014.12.017

Abstract

Objectives: Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions.

Study design and setting: We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results.

Results: A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set.

Conclusion: For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.

Keywords: Databases; EMBASE; Health technology assessment; Literature search; Meta-analysis; PubMed; Rapid review; Systematic review.

Publication types

Review
Systematic Review

MeSH terms

Bibliometrics
Databases, Bibliographic*
Humans
Meta-Analysis as Topic
PubMed
Research Design
Review Literature as Topic*