Proton pump inhibitors, hip fracture and small relative risks

A JAMA article has generated considerable media attention citing a case control study that linked PPIs (proton pump inhibitors) to an increased risk of hip fracture.

The overall adjusted odds ratio was 1.44 with a confidence interval of 1.3-1.59. There was a dose response relationship demonstrated with duration of therapy.

Observational studies, such as this case-control study, are generally accepted to be so-called "hypothesis generating" studies rather than studies that more suggest a causative relationship (o.k. I'll stipulate causation is a deep, complicated philosophical issue and I use the word here just to indicate that case-control studies are just the beginning of efforts to demonstrate likely causal relationships).

A dramatic example of how observational trials can be misleading can be found in the juxtaposition of two studies in the New England Journal of Medicine dealing with relationship between hormone replacement therapy and coronary artery disease. One study demonstrated a two fold increase in coronary disease risk while the companion article showed a coronary artery risk reduction of 50% in those who used HRT.

There has been a number of conflicting case control studies regarding the relationship between statin use and colon cancer risk which again points our the problems involved when you take the results of one such study too seriously.

Let's quickly apply Bradford Hill's criteria to the PPI-hip fracture issue. The temporal relationship is there. There is a dose response relationship is a reasonable biological plausibility- low stomach acid conditions can interfere with absorption of some forms of calcium.
However, the association (1.44. OR) is not very large and there is not a strong consistency of results in other observational studies. (The JAMA article quote another study with similar results and one that did not demonstrate the association). At the end of the mini-analysis we still don't know if the reported relationship is valid or really know if we should somehow change our practice in regard to PPIs.

How big should the relative risk (RR) or odds ratio (OR) be for us to be concerned with the results? This is the very question asked by Dr. David Sackett in his book " Evidence-Based Medicine. How to practice and Teach EBM", Churchill Livingtone. Second edition,pg 162).For an answer to this the pioneers in EBM turn to argument by authority as opposed to an evidence based reason.

In regard to a case control study he said

We might not want to label an odds ratio from a case-control study as impressive unless it is greater than 4 for minor adverse events and set this value progressively lower [for more serious events].

Further, he says in regard to a cohort study, where there is less potential bias , that a relative risk of greater than 3 might be convincing for a more serious adverse event.I have posted before on the issue of RR in the 1-2 range.

John Ioannidis stirred up the medical community a bit in his article "Why Most Published Research Findings are False" and in regard to small relative risk determinations he is quoted as saying:

The smaller the effect sizes in a scientific field, the less likely the research findings are to be true...more likely true in scientific fields with ...relative risks [ in the 3- 20 range] ...than in scientific fields where the postulated effects are small [in the 1.1-1.5 range]

Could this study, and so many more with RRs less than 2 hit that hit the headline news, not really be worth much concern at all since the reported increased risk is so small ?

Dr. Marcia Angell, NEJM editor, has been quoted as saying that generally they only accept papers if the relative risk is 3 or more, particularly if it is biologically implausible or if it's a brand new finding. In the PPI case, there is some biological plausibility and occurs in the context of at least one study with findings pointing in the same direction.

In some legal venues, a relative risk of 2 or more is required to meet the legal standard of "more likely than not".

The PPI study authors suggest we should recommend that calcium supplements be taken with meals and I would add recommend adequate amounts of vitamin D ( thought to be about 800-1000 units per day now) and to perhaps have a looser trigger finger on the indications for measurement of bone density in those patients who are on long term PPI therapy and/or higher dose. However, I do not think patients whose GERD symptoms have been controlled by PPIs-and sometimes this change has been dramatic-should throw away their pills.