A tip of the blogging hat to DB's Medical Rants for this reference.
This article by Ian Shrier and others from McGill investigated the subjectivity of meta-analyses and concluded the following:
The interpretation of the results of systematic reviews with meta-analyses includes a subjective component that can lead to discordant conclusions that are independent of the methodology used to obtain or analyse the data. And things get even more mushy when the statistical experts differ as to what methodology should be used.
In their discussion this paragraph says it well :
Our results suggest that a systematic review with a meta-analysis must be viewed with the perspective that it represents one study conducted by specific investigators with a specific methodology. At each step of the methodology (defining the general criteria, search strategy, inclusion/exclusion criteria, data abstraction, and analysis), subjective decisions are required that could affect the validity of the study; the relative importance of each will likely depend on the topic of inquiry and the data acquired. Our study demonstrates that disagreements in the conclusions of systematic reviews with meta-analyses can also be due to subjective interpretations of the results and not only of the methodology. The inclusion-exclusion criteria often are determinative of the outcome.Meta-analyses can be thought of as observational studies in which the subjects are trials.
Of course meta-analyses involve subjective judgment calls and various type of personal bias that the investigators bring to the table. How could it possibly be otherwise?
This gives me still another opportunity to reference the classic editorial in the Annals of Internal Medicine by Steve Goodman of Johns Hopkins which I discussed at some length here. To sum it I can do no better than to quote Goodman:
Judgment determines what evidence is admissible and how strongly to weigh different forms of admissible evidence. When there is consensus on these judgments and the data are
strong, an illusion is created that the evidence is speaking for itself and that the methods are objective. But this episode[ the mammogram controversy mentioned above] should raise awareness that judgment cannot be excised from the process of evidence synthesis and that the variation of this judgment among experts generates uncertainty just as real as the probabilistic uncertainty of statistical calculations.
I never tire of repeating my rant that meta-analyses should never have been placed at the top of the evidence based medicine evidence hierarchy.And for that matter biological plausibility should never have relegated to lower rings of the ladder.
Sunday, August 24, 2008
Tuesday, August 19, 2008
Merck replies to the criticism of the Advantage trial
Dr. Jonathan M. Edelman, Director of Scientific affairs for Merck, has offered this reply to the charges leveled in the Annals of Internal Medicine article which had characterized the Advantage trial as a "seeding trial".
Dr. Edelman denies that it was, in fact, a seeding trial and that it was designed to answer a significant, scientific question describing the trial as one in which Vioxx was compared with a widely used drug Naproxen in a "real life" setting and involving patients who were allowed to take aspirin as well and - as such- answered questions that were relevant and pertinent to practicing physicians. He also denies that participants were not properly informed about the purpose of the trial.
Edelman also says that the trial "was designed, conducted, analyzed and interpreted by the scientific department of Merck." I had implied otherwise in my comments in an earlier entry on this subject perhaps inappropriately embellishing this following statement from the Annals article, "Merck's marketing division, ...handled the scientific and marketing data including collection, analysis and dissemination." However, which group or groups within the company designed the trial does not settle the question of if this a seeding trial or not.
Dr. Carlat has also taken up this topic and interviewed one of the authors of the Annals article.See here for the interview. The WSJ.Com Health Blog also writes about this issue here. The Med Page Today blog tackles this topic also.
Details of how much and what was disclosed to the patients and the participating physicians is not made clear by either the Annals article or the Merck reply.
Dr. Edelman denies that it was, in fact, a seeding trial and that it was designed to answer a significant, scientific question describing the trial as one in which Vioxx was compared with a widely used drug Naproxen in a "real life" setting and involving patients who were allowed to take aspirin as well and - as such- answered questions that were relevant and pertinent to practicing physicians. He also denies that participants were not properly informed about the purpose of the trial.
Edelman also says that the trial "was designed, conducted, analyzed and interpreted by the scientific department of Merck." I had implied otherwise in my comments in an earlier entry on this subject perhaps inappropriately embellishing this following statement from the Annals article, "Merck's marketing division, ...handled the scientific and marketing data including collection, analysis and dissemination." However, which group or groups within the company designed the trial does not settle the question of if this a seeding trial or not.
Dr. Carlat has also taken up this topic and interviewed one of the authors of the Annals article.See here for the interview. The WSJ.Com Health Blog also writes about this issue here. The Med Page Today blog tackles this topic also.
Details of how much and what was disclosed to the patients and the participating physicians is not made clear by either the Annals article or the Merck reply.
Monday, August 18, 2008
Annals Internal Medicine article dissects the "seeding trial" aka "Marketing trial"
A number of e-mails and other internal Merck documents became fair game for analysis and critique when they became available as part of the discovery process of a civil suit involving the cardiovascular safety of Vioxx and this analysis is published in the August 19,2008 issue of the Annals of Internal Medicine . (The Advantage Seeding Trial: A review of Internal Documents, Hill KP et al, Annals Int Med. 2008;149;251-258).
The documents discussed in this article are concerned with a trial called ADVANTAGE which was said to assess the differences between Vioxx and Naproxen regarding side effects and effectiveness in the treatment of osteoarthritis.The trial involved 600 investigators and 2785 patients and was ultimately published in the Annals of Internal Medicine giving the results of a 3 month followup.
The current Annals article(needs subscription) takes the reader into the real life example of the world of a clinical trial that is designed and executed by the marketing ( not the research) division of a pharmaceutical company. This type of trial has been referred to as a "marketing trial" or a "seeding trial". The term " seeding trial" is actually used by involved employees although one company e-mail advised against the use of the term even in in-house writings.Although it has been suspected and alleged that drug companies sponsor these marketing trials this article is said to be the first documentation of at least some of what goes on in the company as the trial is planned and carried out.
Perhaps new to some physicians the seeding trial is nothing new at all in the world of marketing as explained in great detail in this reference which puts the idea in the context of opinion leaders, the Hawthorne effect and the tipping point. (for some reason I could only pull up the cached version.)
If the idea worked for Post-Its-as the above reference explains-why not for prescription drugs?
Psaty and others have decried the use of these low quality ( from a scientific or statistical point of view) trials and I wrote about in 2006 with some fairly unkind words about the physician opinion leaders who participate and make these marketing trials possible.
Doctors Harold Sox and Drummond Rennie wrote the editorial in the August Annals entitled "Seeding trials: Just say "no" " They wrote that the key to a successful seeding trial is deception. I would like to believe that many or most of the physician investigators were deceived as the alternative is that they knew it was wink-wink-nudge-nudge pretense of a trial and the term co-conspirator would apply.
Sox and Rennie said the Annals published the trial because no one told them it was a seed trial.One has to wonder why the editors had to be told. A 1994 NEJM special article made clear the characteristics of such trials and the Advantage trial was published in the Annals in 2003.David Kessler and co-authors in their NEJM article said the following about these marketing trials.
Features that distinguish such trials from scientifically rigorous studies include the use of a design that does not support the stated research goals, the recruitment of investigators not because they are experts or leading researchers but because they are frequent prescribers of competing products in the same therapeutic class, disproportionately high payments given to "investigators" for their work (although the only work may be to write prescriptions for the drug), sponsorship of the studies by the company's sales and marketing division rather than its research department, minimal requirements for data, and the collection of data that are of little or no value to the company. Typically, these trials involve introducing a new drug in a crowded therapeutic class. The success of such a new product may depend on undoing physicians' comfortable habits of prescribing a competing, more established product.
Hopefully with the spotlight of the current Annals article and editorial, editors,review boards and potential physician investigators will be less likely to be deceived and will "say no"and protect the real victims of these trials namely the patients who often in good faith thought they were furthering science and helping other people with the disease under study while in fact they may put themselves at risk for side effects for no reason other than increasing the sales of the medication.
The documents discussed in this article are concerned with a trial called ADVANTAGE which was said to assess the differences between Vioxx and Naproxen regarding side effects and effectiveness in the treatment of osteoarthritis.The trial involved 600 investigators and 2785 patients and was ultimately published in the Annals of Internal Medicine giving the results of a 3 month followup.
The current Annals article(needs subscription) takes the reader into the real life example of the world of a clinical trial that is designed and executed by the marketing ( not the research) division of a pharmaceutical company. This type of trial has been referred to as a "marketing trial" or a "seeding trial". The term " seeding trial" is actually used by involved employees although one company e-mail advised against the use of the term even in in-house writings.Although it has been suspected and alleged that drug companies sponsor these marketing trials this article is said to be the first documentation of at least some of what goes on in the company as the trial is planned and carried out.
Perhaps new to some physicians the seeding trial is nothing new at all in the world of marketing as explained in great detail in this reference which puts the idea in the context of opinion leaders, the Hawthorne effect and the tipping point. (for some reason I could only pull up the cached version.)
If the idea worked for Post-Its-as the above reference explains-why not for prescription drugs?
Psaty and others have decried the use of these low quality ( from a scientific or statistical point of view) trials and I wrote about in 2006 with some fairly unkind words about the physician opinion leaders who participate and make these marketing trials possible.
Doctors Harold Sox and Drummond Rennie wrote the editorial in the August Annals entitled "Seeding trials: Just say "no" " They wrote that the key to a successful seeding trial is deception. I would like to believe that many or most of the physician investigators were deceived as the alternative is that they knew it was wink-wink-nudge-nudge pretense of a trial and the term co-conspirator would apply.
Sox and Rennie said the Annals published the trial because no one told them it was a seed trial.One has to wonder why the editors had to be told. A 1994 NEJM special article made clear the characteristics of such trials and the Advantage trial was published in the Annals in 2003.David Kessler and co-authors in their NEJM article said the following about these marketing trials.
Features that distinguish such trials from scientifically rigorous studies include the use of a design that does not support the stated research goals, the recruitment of investigators not because they are experts or leading researchers but because they are frequent prescribers of competing products in the same therapeutic class, disproportionately high payments given to "investigators" for their work (although the only work may be to write prescriptions for the drug), sponsorship of the studies by the company's sales and marketing division rather than its research department, minimal requirements for data, and the collection of data that are of little or no value to the company. Typically, these trials involve introducing a new drug in a crowded therapeutic class. The success of such a new product may depend on undoing physicians' comfortable habits of prescribing a competing, more established product.
Hopefully with the spotlight of the current Annals article and editorial, editors,review boards and potential physician investigators will be less likely to be deceived and will "say no"and protect the real victims of these trials namely the patients who often in good faith thought they were furthering science and helping other people with the disease under study while in fact they may put themselves at risk for side effects for no reason other than increasing the sales of the medication.
Friday, August 15, 2008
Physicians who are confused and mislead by drug companies can rest easy-maybe, the government is coming to help
Enticed by pizza and promotional pens and mugs, enchanted by attractive, tricky drug reps and hoodwinked by cleverly crafted medical articles that have been funded by Big Pharma the woefully undereducated and incredibly gullible and naive medical doctors may look forward to having help from the government . At least that is being proposed, see here for news about that.
The plan is for the government to hire unbiased docs who will go forth and detail the office docs but this time the detailing will be done by folks who know the truth. It has become very difficult for someone with only a college degree, four years of med school and from 3-8 years of further education to sort the truth from the hype in regard to medication use. Fortunately, once the government hires someone with similar educational background to disseminate the unbiased truth they apparently- through the magic of government funding- become able to pluck the nuggets of truth from the dung of pharma manufactured propaganda and the razzle-dazzle of the traveling hired guns.
There are some cynics who believe that government decisions themselves have an unfortunate but predictable tendency to be biased, confused and self interested but the high-school civics class notion of the governmental agency or agent who is free of self interest and serves only the public good has considerable currency in spite of overwhelming evidence to the contrary.
As suggested by "The Last Psychiatrist" blog -does it not worry anyone that Congress believes or claims that physicians are so clueless that the government has to send out "academic detailers" to instruct the docs as to proper patient care?
I also wonder how have they been able to practice medicine before without that help?
Were not the "academic detailers" hired by the medical education companies hired by the drug companies thought to be part of the problem? Well, with a different source of funding things will change.
The plan is for the government to hire unbiased docs who will go forth and detail the office docs but this time the detailing will be done by folks who know the truth. It has become very difficult for someone with only a college degree, four years of med school and from 3-8 years of further education to sort the truth from the hype in regard to medication use. Fortunately, once the government hires someone with similar educational background to disseminate the unbiased truth they apparently- through the magic of government funding- become able to pluck the nuggets of truth from the dung of pharma manufactured propaganda and the razzle-dazzle of the traveling hired guns.
There are some cynics who believe that government decisions themselves have an unfortunate but predictable tendency to be biased, confused and self interested but the high-school civics class notion of the governmental agency or agent who is free of self interest and serves only the public good has considerable currency in spite of overwhelming evidence to the contrary.
As suggested by "The Last Psychiatrist" blog -does it not worry anyone that Congress believes or claims that physicians are so clueless that the government has to send out "academic detailers" to instruct the docs as to proper patient care?
I also wonder how have they been able to practice medicine before without that help?
Were not the "academic detailers" hired by the medical education companies hired by the drug companies thought to be part of the problem? Well, with a different source of funding things will change.
Tuesday, August 05, 2008
Physicians learning to be shift workers?
In a recent Medscape edition (July 1,2008) we find a disturbing essay by a practicing academic pediatrician who expresses concern about some of the unexpected consequences of the restrictions placed on medical house staff work hours. I find it disturbing in part because what the author describes could not be further from the notion that the physician (house staff) should not leave until his patient is taken care of.
Dr. Jane R. Gilsdorf, from the University of Michigan Medical Center, pulls no punches with this sentence:
The current system is creating a legion of shift-worker physicians who leave when the clock strikes a certain hour rather than when the job has been completed.
In discussing this with a former partner, who taught internal medicine residents for over 30 years and retired one year after the work rule changes were implemented, he had this cynical comment to make:
"Maybe it is good that the residents learn that approach early on because it seems that most docs are becoming shift-workers. Consider how things often work. You have a primary care doctor who has defined office hours and when you call his number after hours you either get a "triage nurse" or a recording that basically says call back tomorrow if it is not an emergency or go to the ER if it is an emergency. At the ER you see an ER doc who is working a shift and if you are admitted you are likely seen by a hospitalist who also is a shift worker. The notion of a personal physician whose role transcends duty hours is dead in the water."
Dr. Gilsdorf opines that more funds are needed to help correct the problems brought about by the ACGME work restriction rules. Basically, someone needs to be paid to do much of what the house staff used to do and now cannot. Dr. R. Centor discusses some of the problems brought about by the ACGME work rules and suggests with good planning,teaching and hard work many of them can be at least to a significant degree mitigated and I believe he is probably right.
However,how the broader "shift-workerization" of physicians can be reversed is another matter.Major shifts in the economic forces regarding physician and hospital compensation have brought that about and that big ship will be hard to turn around.
Dr. Jane R. Gilsdorf, from the University of Michigan Medical Center, pulls no punches with this sentence:
The current system is creating a legion of shift-worker physicians who leave when the clock strikes a certain hour rather than when the job has been completed.
In discussing this with a former partner, who taught internal medicine residents for over 30 years and retired one year after the work rule changes were implemented, he had this cynical comment to make:
"Maybe it is good that the residents learn that approach early on because it seems that most docs are becoming shift-workers. Consider how things often work. You have a primary care doctor who has defined office hours and when you call his number after hours you either get a "triage nurse" or a recording that basically says call back tomorrow if it is not an emergency or go to the ER if it is an emergency. At the ER you see an ER doc who is working a shift and if you are admitted you are likely seen by a hospitalist who also is a shift worker. The notion of a personal physician whose role transcends duty hours is dead in the water."
Dr. Gilsdorf opines that more funds are needed to help correct the problems brought about by the ACGME work restriction rules. Basically, someone needs to be paid to do much of what the house staff used to do and now cannot. Dr. R. Centor discusses some of the problems brought about by the ACGME work rules and suggests with good planning,teaching and hard work many of them can be at least to a significant degree mitigated and I believe he is probably right.
However,how the broader "shift-workerization" of physicians can be reversed is another matter.Major shifts in the economic forces regarding physician and hospital compensation have brought that about and that big ship will be hard to turn around.
Subscribe to:
Posts (Atom)