Validating model output in the absence of ground truth data: A COVID-19 case study using the Simulator of Infectious Disease Dynamics in North Carolina (SIDD-NC) model

Abstract

As the COVID-19 pandemic has progressed, various models have been developed to forecast changes in the outbreak and assess intervention strategies. In this study we validate the Simulator of Infectious Disease Dynamics in North Carolina (SIDD-NC) model against an ensemble of proxy-ground truth infections datasets. We assess the performance of SIDD-NC using Spearman Rank Correlation, RMSE, and percent RMSE at a state and county level. We conduct the analysis for the period of March 2020 through November 2020 as well as in shorter time increments to assess both the recreation of the pandemic curve as well as day-to-day transmission of SARS-CoV-2 within the population. We find that SIDD-NC performs well against the datasets in the ensemble, generating an estimate of infections that is robust both spatially and temporally.

Keywords: Agent-based modeling; COVID-19; Micro-simulation modeling; Pandemic; SARS-CoV-2.