Date and Time
April 29, 2009
1:30 PM - 3:00 PM

Availability
Open to the public
No RSVP required

Speaker
Paul J. Sharek - Stanford University

Research Objective:
To assess the suitability of the IHI GTT for use as a measure of patient harm at individual hospital and national levels and to inform the methodology required for a national harm measurement system.

Study Design:
A retrospective review of 10 patient charts per quarter from 10 randomly selected acute care hospitals over a 6 year period. Charts were reviewed by teams internal and external to the hospital. Further, 10% of the charts were reviewed by a ‘gold standard' team of experienced IHI GTT reviewers. Each team applied the GTT methodology, and harms identified were compared within and between team (using Kappa statistics). A secondary analysis explored harm over time (using a random effects poison regression model). 

Population Studied:
The study population comprised patients admitted to 10 randomly chosen acute care hospitals in North Carolina between 2002 and 2007. 

Principal Findings:
Internal, external and ‘gold standard' teams found harm rates of 22.9 (95% CI 21.4, 24.9) per 100 patients (537 harms, 2344 charts), 17.2 (95% CI 15.6, 19.0) per 100 patients (403 harms, 2337 charts) and 36.6 (95% CI 28.8, 46.0) per 100 patients (74 harms, 202 charts). Internal teams displayed an almost perfect level of intra-rater agreement, where their associated Kappa statistics were higher than external teams for indicating any harm (internal Kappa=0.87 vs external Kappa=0.57), number of harms (0.93 vs 0.72) and severity of harm (0.87 vs 0.59). Internal teams also displayed more (inter-rater) agreement with the ‘gold standard' reviewers than external teams for indicating any harm (0.49 vs 0.32), number of harms (0.37 vs 0.30) and severity of harm (0.53 vs 0.26).

No significant change in harm over time was detected from internal reviews, 0.99 relative reduction per year, (95% CI 0.94, 1.04 p=0.613) and external reviews 0.97 per year (95% CI, 0.92, 1.02 p=0.268). There was a significant reduction in preventable harm found by external reviews 0.91 per year (95% CI 0.84, 0.99 p=0.023), but not internal 0.99 per year (95% CI 0.93, 1.05 p=0.770).

Conclusions:
The relatively high levels of agreement within and between the review teams in this study of a random sample of hospitals in one US State suggests the IHI GTT may be a suitable measure of harm both for individual hospitals and nationally.

Although the study was not powered to detect subtle changes in harm rates over time, there was no suggestion of an improvement trend over this over this 6 year period. However, there has been a contemporaneous change in case mix and severity of illness in US hospitals, and the results are unadjusted for these factors. The results may indicate that current strategies for improving specific aspects of patient safety are not yet resulting in measureable overall improvement.

Implications for Policy, Delivery or Practice:
In order to more clearly understand and assess the impact of policies, recommendations and associated initiatives aimed at improving patient safety, a nationally applicable measure of patient harm is required. The time may now be right for establishing such a measurement system and performing robust studies to assess changes in harm rates over time.