An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

Stijn van Lierop; Daniel Ramos; Marjan Sjerps; Rolf Ypma

doi:10.1016/j.fsisyn.2024.100466

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

Forensic Sci Int Synerg. 2024 Apr 17:8:100466. doi: 10.1016/j.fsisyn.2024.100466. eCollection 2024.

Authors

Stijn van Lierop¹, Daniel Ramos², Marjan Sjerps^{1

3}, Rolf Ypma¹

Affiliations

¹ Netherlands Forensic Institute, Laan van Ypenburg 6, The Hague, 2497GB, the Netherlands.
² AUDIAS Lab, Universidad Autonoma de Madrid, Escuela Politécnica Superior, Calle Francisco Tomàs y Valiente 11, 28049, Madrid, Spain.
³ University of Amsterdam, KdVI, PO Box 94248, Amsterdam, 1090 GE, the Netherlands.

Abstract

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (C_llr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. C_llr = 0 indicates perfection while C_llr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" C_llr is unclear. Aiming to provide handles on when a C_llr is "good", we studied 136 publications on (semi-)automated LR systems. Results show C_llr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting C_llr remains stable. Noticeably, C_llr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

Keywords: Automated likelihood ratio systems; Benchmarking review; Cllr; Forensic datasets; Forensic evaluation; Log-likelihood ratio cost; Performance of LR system.

Publication types

Review