Recognition of adverse effects-no quick and dirty way

The recent flurry of scientific comment, speculation, 6 o'clock news type hype, accusations about everyone concerned-except perhaps patients- and thoughtful commentary about the possibility that Avandia increases the risk of ischemic events makes me sit back and think about how did we get to where we are.

The randomized clinical trial is unrivaled in its ability-when properly done-to defy the confounders and the biases and point to a treatment that is efficacious and provides some information about adverse effects and can compare fairly the treated and untreated groups. It deserves its place as king of the hill in the realm of evidence and evidence based medicine because of its ability to eliminate selection bias and ideally chase away the known and unknown confounders that seem to lurk everywhere.

However, in regard to adverse effects (AE) detection RCTS are imperfect and not the most powerful tool in the shed. RCTs are typically short lived and even the larger ones often are too small to have adequate power to detect low frequency AEs. Further,RCTs are designed to demonstrate efficacy-or in some cases the much weaker standard of proof- non-inferiority. The patients in the trial often are less diverse and complicated that a typical subset of patients from real clinical life that will likely be taking the medication.This commentary by Ioannidis,Goodman and Mulrow in the Annals of Internal Medicine speaks more authoritatively that I can in regard to this point emphasizing the limitations of randomized trials in this regard. Further, they make suggestion to make postmarketing surveillance more "routine and efficient".

"Routine and efficient" certainly seems lacking as lately alarm about AEs associated with two blockbuster drugs was sounded by cardiologists and statisticians at Cleveland Clinic rather than the drug manufacturers or the FDA. Regardless of how correct or misleading their analysis may ultimately turn out to be (if there is ever an "ultimately"), I doubt we can always rely or a Dr. Nissen or whoever to oversee the vast drug related adverse effect data set.

It falls on the shoulders of case reports and observational data and meta-analyses to fill in the gaps regarding those less common AEs and those that only develop after the patient has been on the medication for a longer period of time.Sometimes these AEs burst on the headline news with alarming findings as with drug eluting stents,Vioxx, Ketek, and now,it seems, Avandia. They certainly can be alarming but they should not be surprising.

With more patients, and more time, more AEs will become apparent but the AEs have to be plucked out of a data which is plagued by potential confounders, the management of which has generated various statistical techniques which ,in my opinion,exceeds the level of statistical expertise possessed by most physicians and medical journal editors. I have written before about the difficulty in choosing between competing techniques proposed to tame the observational data.

So the RCT may just be the beginning of the data accumulation. The RCT is also the beginning of the onslaught of information,dis-information, useful educational material, misleading spin and well thought out marketing efforts of the drug manufacturer,educational efforts by patient organizations, guidelines from various groups and delicious "educational company"sponsored dinners punctuated by "thought leader" physicians showing slides typically prepared by the company.

Hopefully, postmarketing trials will come more efficient and routine and timely (Nissen's Avandia meta-analysis appeared about 8 years after FDA approval), but this will require a greater or more forceful role for the FDA-an agency which currently enjoys a reputation only slightly less tarnished than big Pharma. Something positive may come out of congressional hearings into various activities of the FDA and then perhaps remedial legislation but is not the current FDA the product of earlier congressional efforts to mitigate what was considered at the time by some a dangerously slow medication approval process?

Meanwhile, until a new day appears, we need to not be overly reassured by "negative findings" regarding side effects derived from RCTs. Ioannidis and co-authors included this quote in the above referenced commentary:

"It is almost always inappropriate to make statements about "no difference"in adverse event rates between groups on the basis of nonsignificant P values. Rates of adverse events that are derived from single,modest sized trials that are not statistically different typically do not exclude with certainty the possibility of major, clinically important differences in harms between groups."