Interrater agreement in visual scoring of neonatal seizures based on majority voting on a web-based system: The Neoguard EEG database

Abstract

Objective: To assess interrater agreement based on majority voting in visual scoring of neonatal seizures.

Methods: An online platform was designed based on a multicentre seizure EEG-database. Consensus decision based on 'majority voting' and interrater agreement was estimated using Fleiss' Kappa. The influences of different factors on agreement were determined.

Results: 1919 Events extracted from 280h EEG of 71 neonates were reviewed by 4 raters. Majority voting was applied to assign a seizure/non-seizure classification. 44% of events were classified with high, 36% with moderate, and 20% with poor agreement, resulting in a Kappa value of 0.39. 68% of events were labelled as seizures, and in 46%, all raters were convinced about electrographic seizures. The most common seizure duration was <30s. Raters agreed best for seizures lasting 60-120s. There was a significant difference in electrographic characteristics of seizures versus dubious events, with seizures having longer duration, higher power and amplitude.

Conclusions: There is a wide variability in identifying rhythmic ictal and non-ictal EEG events, and only the most robust ictal patterns are consistently agreed upon. Database composition and electrographic characteristics are important factors that influence interrater agreement.

Significance: The use of well-described databases and input of different experts will improve neonatal EEG interpretation and help to develop uniform seizure definitions, useful for evidence-based studies of seizure recognition and management.

Keywords: Consensus agreement; Electrographic seizures; Interrater agreement; Neoguard; Neonatal multichannel EEG; Seizure detection algorithms.