Background: In the development of search strategies for systematic reviews, "conceptual approaches" are generally recommended to identify appropriate search terms for those parts of the strategies for which no validated search filters exist. However, "objective approaches" based on search terms identified by text analysis are increasingly being applied.
Objectives: To prospectively compare an objective with a conceptual approach for the development of search strategies.
Methods: Two different MEDLINE search strategies were developed in parallel for five systematic reviews covering a range of topics and study designs. The Institute for Quality and Efficiency in Health Care (IQWiG) applied an objective approach, and external experts applied a conceptual approach for the same research questions. For each systematic review, the citations retrieved were combined and the overall pool of citations screened to determine sensitivity and precision.
Results: The objective approach yielded a weighted mean sensitivity and precision of 97% and 5%. The corresponding values for the conceptual approach were 75% and 4%.
Conclusion: Our findings indicate that the objective approach applied by IQWiG for search strategy development yields higher sensitivity than and similar precision to a conceptual approach. The main advantage of the objective approach is that it produces consistent results across searches.
Keywords: Data mining; Information storage and retrieval; Medline; Prospective studies; Reproducibility of results; Sensitivity and specificity.
Copyright © 2016 The Author(s). Published by Elsevier Inc. All rights reserved.