Medical journalist, James Le Fanu, quoted in a 2002 BMJ article made the above -presumably tongue-in-cheek-comment in the context of increasingly common mass media medical news articles claiming increasing or decreasing risk of some disease only to find contradictory data soon after.Data dredging refers to statistically analyzing large numbers of variables and number of outcomes, a method well recognized to produce false positives particularly using the p<0.05 level.
Data dredging is not new nor are the attendant problems.A letter to the editor quoted James Lee's "grand equation of truth". All observations are subject to error. What we observe is equal to the truth plus or minus the errors associated with chance,bias and confounding. With increase in the number of possible associations as in data dredging,the play of change looms larger. Our techniques for "controlling" bias and confounding in case control studies ( a major source of data dredging exercises) leaves a lot to be desired.(Of course you can dredge around in RCTs as well looking a various subsets and this also can be the source of epistomologic mischief). None of this is new . What seems new to me are three things: 1)the reporting of scientific studies in the mass media without the appropriate context and in a simplified and simplistic form,2)increasing numbers of dredge reports.(My cynical explanation for part of this increase is the large number of academic MDs in various departments who need to write papers.Having said that I recognize there are many excellent clinician-researchers who turn our worthwhile research without resorting to the quick and dirty case-control multi-comparison expediency) and 3) the ease with which dredging can now be done with modern computer power and software usuable by folks who is the past would have been at the mercy of the few wizards of the mainframes.
Closing down the epi departments may be a bit drastic. Physicians can always fall back on the basics, i.e. look the latest breakthough through the lenses of context and biological plausibility and warn your patients when asked about the latest finding accordingly.
No comments:
Post a Comment