JAMA article considers Hospital Performance Measures and mortality rates

In the December 13, 2006 issue of JAMA, Drs. Rachel Werner and Eric Bradlow analyzed data from 3657 acute care hospitals looking at the relationship between certain performance measures and mortality rates. They used 10 measures which are available from CMS and the Joint Commission 's web site. For example, in regard to pneumonia the measures are: 1) timing of initial antibiotic administration, 2) pneumococcal vaccination and 3)whether oxygenation was measured in the first 24 hours.

The authors concluded that:

"hospital performance measures predict small differences in hospital risk-adjusted mortality rates"
and said further:

"Based on these results, the ability of performance measures to detect clinically meaningful differences in quality across hospitals is questionable"

Dr. Susan Horn in her related editorial makes several good points:

1.If these performance measures are not strongly associated with outcomes, why should we bother with them either as basis for P4P of for consumers to use as tools for judging hospitals?
2.Since many( but not all) of these measures were chosen,at least,in part because of the results of RCTs, why are these interventions not associated with better outcomes in practice?

Her answer to the second question is that improving outcomes in actual real world practice is much more complex and multidimensional than using a few, discrete interventions that seemed efficacious and safe in RCTs that typically focus on a single condition in a fairly homogeneous population. In part, it is the question of efficacy versus effectiveness. Measuring quality of care is much more complex and slippery than the ten measures analyzed in their study.

In an earlier article Dr. Werner pointed out some possible unintended consequences of hospital "report card" including treating the chart and excluding sicker patients.

One message should be: we have no business using simple and simplistic measures as a basis for pay for performance or for claiming to be able to distinguish between different hospitals quality of care. Not only may they not deliver on what they promise, they may be harmful.