Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies

Anil P S Ori; Ake T Lu; Steve Horvath; Roel A Ophoff

doi:10.1186/s13059-022-02793-w

Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies

Genome Biol. 2022 Oct 24;23(1):225. doi: 10.1186/s13059-022-02793-w.

Authors

Anil P S Ori¹, Ake T Lu², Steve Horvath^{2

3}, Roel A Ophoff^{4

5

6}

Affiliations

¹ Center for Neurobehavioral Genetics, Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles, 695 Charles E. Young Drive South, Los Angeles, CA, 90095-176, USA. anilori.contact@gmail.com.
² Department of Human Genetics, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA.
³ Department of Biostatistics, Fielding School of Public Health, University of California Los Angeles, Los Angeles, CA, USA.
⁴ Center for Neurobehavioral Genetics, Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles, 695 Charles E. Young Drive South, Los Angeles, CA, 90095-176, USA. ophoff@ucla.edu.
⁵ Department of Human Genetics, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA. ophoff@ucla.edu.
⁶ Department of Psychiatry, Erasmus University Medical Center, Rotterdam, The Netherlands. ophoff@ucla.edu.

Abstract

Background: DNA methylation (DNAm)-based predictors hold great promise to serve as clinical tools for health interventions and disease management. While these algorithms often have high prediction accuracy, the consistency of their performance remains to be determined. We therefore conduct a systematic evaluation across 101 different DNAm data preprocessing and normalization strategies and assess how each analytical strategy affects the consistency of 41 DNAm-based predictors.

Results: Our analyses are conducted in a large EPIC DNAm array dataset from the Jackson Heart Study (N = 2053) that included 146 pairs of technical replicate samples. By estimating the average absolute agreement between replicate pairs, we show that 32 out of 41 predictors (78%) demonstrate excellent consistency when appropriate data processing and normalization steps are implemented. Across all pairs of predictors, we find a moderate correlation in performance across analytical strategies (mean rho = 0.40, SD = 0.27), highlighting significant heterogeneity in performance across algorithms. Successful or unsuccessful removal of technical variation furthermore significantly impacts downstream phenotypic association analysis, such as all-cause mortality risk associations.

Conclusions: We show that DNAm-based algorithms are sensitive to technical variation. The right choice of data processing strategy is important to achieve reproducible estimates and improve prediction accuracy in downstream phenotypic association analyses. For each of the 41 DNAm predictors, we report its degree of consistency and provide the best performing analytical strategy as a guideline for the research community. As DNAm-based predictors become more and more widely used, our work helps improve their performance and standardize their implementation.

Keywords: Biomarkers; Consistency; DNA methylation; DNAm predictors; Infinium MethylationEPIC array; Jackson Heart Study; Replicability.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

DNA Methylation*

Grants and funding

U01 AG060908/AG/NIA NIH HHS/United States