Can Dr. Cassel and her collaborators at the National Quality Forum repeal Goodhart's Law?

In the December 4, 2014 issue of the NEJM, Dr. Chistine K.Cassel et al expound on performance measures.She freely admits there have been some problems but also apparently some successes.It is just a matter of learning from the mistakes and  leveraging the successes. We just have to work harder and be smarter. "Getting More Performance from Performance Measurements" Cassel, CK et al , NEJM 371,23 2014). When people talk about "leveraging" you can be sure the tripe level is dangerous high.

She offers her explanation of  why the notorious "four hour pneumonia rule" did not work out well. That particular clinical situation was "not the right place to intervene". .."There was too much clinical variability for the measure to help physicians on exactly the right course of action." True enough but that was not the reason.

The underlying insight is found in Goodhart's Law which states that when a measure becomes a target it loses its value as a measure.A more basic insight is that people respond to incentives. When folks are graded, or rewarded or punished based on some measure they will find ways to achieve that target and if better care results fine but there is not reason to think apriori  that it will. Teachers will teach to the tests and students will study for the test regardless of how well or how badly the test reflects the students' mastery of the subject.


Goodhart wrote about his "law " in 1975 and in 1976 Donald Campbell wrote in regard to education and testing :

"achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.

It is tempting to substitute medical care for  the teaching process and we get "But when quality measures become the goal of the medical care process , they both loose their value as indicators of medical care and distort the process in undesirable ways."

I submit that devising better measures will not change the situation.

Dr. Cassel et al would disagree with my view and she closes her commentary with  the usual boiler plate words about "stakeholders"  and better quality down the road. "All stakeholder groups are now invested in getting more  performance out of measurements ,which should ultimately drive the care improvements that patients need and deserve."

 The economist,Arnold Kling, give his take on performance measures or P4P  here.

I have commented on this general subject before and here  the issue of a conflict between P4P and traditional medical ethics is talked about.

The folks at the National Quality Forum can no more negate Goodhart's law than they can make it now longer the case that people respond to incentives.The stakeholders of which Cassel speaks , of course, included the NQF itself as it is in the business of writing quality standards .