Objective: To evaluate the reliability of a new tool, the LIV-MAAS, in assessing consultation competence in UK general practice.
Design: These were pilot studies, with small numbers of participants. Videoed general practitioner (GP) consultations were analysed by trained lay and professional raters, using the LIV-MAAS. The inter-rater reliabilities were assessed. Four videos were assessed by five raters in a pilot study. After this, 71 consultations from eight doctors were assessed by sets of three raters.
Main measures: Inter-rater reliabilities and inter-consultation reliabilities.
Results: For the pilot study, the estimated inter-rater reliability ranged from 0.69 (one rater) to 0.91 (five raters). For the main study, the estimated inter-rater reliability for the LIV-MAAS checklist using two raters was 0.71, and using three raters it was 0.78. Mean differences in reliability within each series of nine consultations were 0.20 (three raters) and 0.42 (two raters).
Conclusions: As a measure of 'consultation competence', administered by trained raters (medical or lay) to real GP consultations, the LIV-MAAS instrument shows adequate reliability and stability but would benefit from considerable shortening. Further development of the LIV-MAAS and testing with larger samples are required.