Background: Different approaches can be adopted for the development of search strategies of systematic reviews. The objective approach draws on already established text analysis methods for developing search filters. Our aim was to determine whether the objective approach for the development of search strategies was noninferior to the conceptual approach commonly used in Cochrane reviews (CRs).
Methods: We conducted a search for CRs published in the Cochrane Library. The studies included in the CRs were searched for in MEDLINE and represented the total set. We then tested whether references previously removed could be identified via the objective approach. We also reconstructed the original search strategies from the CRs to determine why references could not be identified by the objective approach. As we performed the validation of the search strategies without study filters, we used only sensitivity as a quality measure and did not calculate precision.
Results: The objective approach yielded a mean sensitivity of 96% based on 13 searches. The noninferiority test showed that this approach was noninferior to the conceptual approach used in the CRs (P < 0.002). An additional descriptive analysis showed that the original MEDLINE strategies could identify only 86% of all references; however, this lower sensitivity was largely due to one CR.
Conclusion: To the best of our knowledge, our findings indicate for the first time that the objective approach for the development of search strategies is noninferior to the conceptual approach.
Keywords: Data mining; Information storage and retrieval; Medline; Reproducibility of results; Retrospective studies; Sensitivity and specificity.
Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.