Developing leading indicators from OHS management audit data: Determining the measurement properties of audit data from the field

Abstract

Introduction: OHS management audits are one means of obtaining data that may serve as leading indicators. The measurement properties of such data are therefore important. This study used data from Workwell audit program in Ontario, a Canadian province. The audit instrument consisted of 122 items related to 17 OHS management elements. The study sought answers regarding (a) the ability of audit-based scores to predict workers' compensation claims outcomes, (b) structural characteristics of the data in relation to the organization of the audit instrument, and (c) internal consistency of items within audit elements.

Method: The sample consisted of audit and claims data from 1240 unique firms that had completed one or two OHS management audits during 2007-2010. Predictors derived from the audit results were used in multivariable negative binomial regression modeling of workers' compensation claims outcomes. Confirmatory factor analyses were used to examine the instrument's structural characteristics. Kuder-Richardson coefficients of internal consistency were calculated for each audit element.

Results: The ability of audit scores to predict subsequent claims data could not be established. Factor analysis supported the audit instrument's element-based structure. KR-20 values were high (≥0.83).

Conclusions: The Workwell audit data display structural validity and high internal consistency, but not, to date, construct validity, since the audit scores are generally not predictive of subsequent firm claim experience. Audit scores should not be treated as leading indicators of workplace OHS performance without supporting empirical data.

Practical applications: Analyses of the measurement properties of audit data can inform decisionmakers about the operation of an audit program, possible future directions in audit instrument development, and the appropriate use of audit data. In particular, decision-makers should be cautious in their use of audit scores as leading indicators, in the absence of supporting empirical data.

Keywords: Audit; Leading indicator; Management system; Performance measurement.