Another VItamin E lesson-statistics experts do not agree on how to analyze a meta-analysis

There were 11 letters to the editor published in the July 19,2005 issue of the Annals of Internal Medicine critical of the methodology used in the meta-analysis by Miller et al that concluded high dose vitamin E supplements may increase all-cause mortality.

I do not believe a non-statistician could weigh the various arguments and then the counterarguments offered in reply to the critiques.

Here are some to give a sense of this (the letters in the Annals require a subscription)

Dr. Antonio Possolo says in part " instead of the authors' spline (if you Google 'spline' you will harvest a long list of also incomprehensible articles) I employed a nonparametric, locally quadratic, weighted regression to model the relationship between relative risk and the logarithm of dose" He found no statistical significance.

 Dr. Kent J DeZee et. al criticized Miller's use of the " hierarchical logistic regression rather than traditional meta-analytic approaches, reanalyzed some of Miller's data and found a non significant result. Miller answers point-for-point the criticisms but I challenge a non-statistician to plow through it all with anything close to an understanding.

Here's the thing. We have a meta-analysis ( 19 trials with 135,000 subjects) that found a slight
increase in all cause mortality ( the risk ratio for " high dose" Vitamin E was 1.03 !) In the other corner we have a crowd of epidemiologists and statisticians who put forth arguments that the methodology is inappropriate and the conclusions reached are not justified. Clearly the matter is in dispute because experts cannot agree on how we should analyze the data.

There is very little evidence that the high hopes some had for Vitamin E have been fulfilled. But when we are asked, "Doctor, should I stop my Vitamin E, I have read it may kill me ?", I think we can say the benefits we believed might be forthcoming do not seem to be likely but I do not believe we have good reason for you to worry about having hurt yourself.The researchers cannot agree if there was or was not a very slight increase in risk.

The first Vitamin E lesson is don't get carried away with preventive measures based on observational and case control studies. The second lesson is we have a lot to learn about how to do the analysis part of meta-analysis and at this point the experts don't seem to agree on how to do it except in the simplest, most straight-forward situations.