Medical journalist, James Le Fanu, quoted in a 2002 BMJ article made the above -presumably tongue-in-cheek-comment in the context of increasingly common mass media medical news articles claiming increasing or decreasing risk of some disease only to find contradictory data soon after.Data dredging refers to statistically analyzing large numbers of variables and number of outcomes, a method well recognized to produce false positives particularly using the p<0.05 level.
Data dredging is not new nor are the attendant problems.A letter to the editor quoted James Lee's "grand equation of truth". All observations are subject to error. What we observe is equal to the truth plus or minus the errors associated with chance,bias and confounding. With increase in the number of possible associations as in data dredging,the play of change looms larger. Our techniques for "controlling" bias and confounding in case control studies ( a major source of data dredging exercises) leaves a lot to be desired.(Of course you can dredge around in RCTs as well looking a various subsets and this also can be the source of epistomologic mischief). None of this is new . What seems new to me are three things: 1)the reporting of scientific studies in the mass media without the appropriate context and in a simplified and simplistic form,2)increasing numbers of dredge reports.(My cynical explanation for part of this increase is the large number of academic MDs in various departments who need to write papers.Having said that I recognize there are many excellent clinician-researchers who turn our worthwhile research without resorting to the quick and dirty case-control multi-comparison expediency) and 3) the ease with which dredging can now be done with modern computer power and software usuable by folks who is the past would have been at the mercy of the few wizards of the mainframes.
Closing down the epi departments may be a bit drastic. Physicians can always fall back on the basics, i.e. look the latest breakthough through the lenses of context and biological plausibility and warn your patients when asked about the latest finding accordingly.
Featured Post
Is the new professionalism and ACP's new ethics really just about following guidelines?
The Charter ( Medical Professionalism in the New Millennium.A Physician's Charter) did not deal with just the important relationship of ...
Monday, October 31, 2005
Thursday, October 27, 2005
Are Canadian physicians really going back?
With the introduction of the Canadian single-payer health care system, large numbers of physicians moved south in search of a different practice setting. A news item this week in the AMA News ( American Medical News,October 23/31/2005,)-now requiring a subscription-hints that now the pattern is being reversed. It is probably too early to conclude that but there are some data suggesting that Canadian docs are going back home after finishing training in the U.S. The AMA article is not all that convincing and is lacking in much hard data. Here are the facts they present: For the first time in the last 30 years more docs are leaving the U.S.for Canada than are coming in (202 out, 162 in),there is a growing shortage of physicians in Canada and the government is said to be attempting to upgrade facilities and in some provinces offer higher reimbursements and some other recruitment incentives. I suspect there is no great exodus in the making but the "M and M's" of medical practice discontent ( e.g. malpractice premiums and managed care) are making more than a few docs to leave practice and at least the malpractice premiums seem lower up north and perhaps the paperwork less onerous. DB's Medical Rants recently posted a typical sad situation of a not very old OB leaving practice because of the first "M" even though she had no history of law suits. Her story is not unique.
Tuesday, October 25, 2005
'Why I did not become an internist"
Through the incredible-almost most endless- amount of material that reverberates through the internet I came across a now 10 year old letter to the Annals of Internal Medicine. It is short and expresses much of what I have thought over the last 10 years. Twenty plus years ago, the author did not become an internist to A) be a family doctor, B) to lead the way in informatics C) to master systems and lead quality programs. He learned to take care of sick patients, sometimes very sick patients and quarterback the details of complex, multisystem clinical situations and to try and unravel difficult clinical problems.The author, Dr. Michael Pauszek, receiving his four years of post medical school education in the late 70s, was trained, as was I , to take care of sick patients and have the breath and depth of medical knowledge to diagnose difficult cases. General internists are still needed to do such things but the economic and structural landscape of medicine has changed even more since his 1995 letter that now more than then internists may well not be able to make a living doing what they do best and more and more are relegated to being more like family doctors and would-be office gynecologists and amateur dermatologists in the office while fighting underfunded turf battles with hospitalists and intensivists (words that were not coined when he and I trained) in the hospital (the long ago domain of the internist.) We read of more straw grasping for internists. For example, recently a IM newspaper touted the virtues of an internists developing a niche in sports medicine to supplement income.I recently injured my knee while running. The first thought I had was not "Let me try a find a good internist who dabbles in sports medicine". Another was for internists to consider becoming expert in ob cases with complicated medical problems providing - in my opinion-a unique opportunity to combine physician stress, the same relatively low internist compensation rates plus probably malpractice premiums closer to those of the high risk ob warriors than the procedureless general internist.Of the two suggestions at least the second involved what internists do best, taking care of complicated sick patients, not pretending to be something else.
Monday, October 24, 2005
Yet another problem with guidelines
Guidelines are great; guidelines are terrible. Internists are of two minds-at least two- regarding guidelines. We resist the harness of being told what to do (although the authors of the guides always include a declaimer that they are not telling what to do and clinically judgment and patient preferences must be given weight) but welcome the relief of finding an easy answer to a given clinical issue. We resist even more vigorously the threat and the reality of being graded-and with P4P being paid- according to this or that guideline. Should I get a pap test this year, doctor? Well the ACOG or the ACS says..... Kevin MD calls our blogish attention to a recent article casting a cloud over the process by which at least some guidelines are prepared. The issue here is the putative influence of the drug manufacturers on the process and the ties the guideline writers have to Big Parma. HCRENEWAl and others have done a great job in directing our attention to various ways in which seemingly reputable medical journals are and have been greatly influenced in ways that may put the volume of sales of certain medications about the priority of scientific inquiry and good faith reporting of data. (If we believe the comments of some journal editors,they were as duped as we were). Well, should anyone be surprised if the same entities have considerable influence on the particulars of various panel's recommendations regarding the indications for various drug and procedures and if the default position is to use more medication(s) for more indications? If we accept RCTs as the top of the line evidence and we have good reason to be at least skeptical about the influence of Big Pharma on those trials it follows that guidelines based to a large degree on those studies will be smeared with the same mud.
Monday, October 17, 2005
Internal Medicine academics discuss why fewer student choose IM as career
In the October 2005 issue of the American Journal of Medicine. Dr. Karen Hauer and a group of other academic internists tackle the issue of declining medical student interest in the field of IM particularly general internal medicine. The online version of the journal which is the official publication of the Association of Professors of Medicine offers some full text journals for free but not this one. It is entitled (Educational responses to declining student interest in internal medicine careers) and the reference is Hauer,K et al , 118, no. 10,1164, Oct 2005).
They do a good job pointing out numerous factors influencing students. A few of them are: perceived vastness of the IM knowledge base perceived to be difficult or impossible to master,heavy administrative burden,work hassle,income ,less control over lifestyle, unhappy IM residents' influence on students as they rotate on service, economic pressures that promote short patient encounters and leave no time for reflection and pathophysiologic considerations traditionally said to be core values of IM. The panel realizes that, while changes in the medicine clerkship may at least make some directional change in students views of IM that most of the factors are out of the control of clerkship directors. The usual suspects of unmanageable administrative burdens (Let us call it correctly insurance plan hassles), low compensation and time demands and tanking prestige are the real culprits. The medical schools cannot change them. The authors said that " practicing clinicians and their national organizations must address flaws in the current practice environment" I have not given up completely on meaningful change taking place but it sounds like Will Roger's "boil the oceans" [ to eliminate German U-boats]. When I was a medical student or house officer, we did not discuss "lifestyle" specialties and-believe it or not-income differentials were not a pivotal factor in the decisions many of us made. The 21st century medical student is much more savvy regarding what happens in the life post training. I believe the major blow to general internists was and is managed care. I seriously doubt that that juggernaut will be stopped or meaningfully altered by practicing doctors or their largely ineffective and sometimes misdirected national organizations. But I can still hope.
They do a good job pointing out numerous factors influencing students. A few of them are: perceived vastness of the IM knowledge base perceived to be difficult or impossible to master,heavy administrative burden,work hassle,income ,less control over lifestyle, unhappy IM residents' influence on students as they rotate on service, economic pressures that promote short patient encounters and leave no time for reflection and pathophysiologic considerations traditionally said to be core values of IM. The panel realizes that, while changes in the medicine clerkship may at least make some directional change in students views of IM that most of the factors are out of the control of clerkship directors. The usual suspects of unmanageable administrative burdens (Let us call it correctly insurance plan hassles), low compensation and time demands and tanking prestige are the real culprits. The medical schools cannot change them. The authors said that " practicing clinicians and their national organizations must address flaws in the current practice environment" I have not given up completely on meaningful change taking place but it sounds like Will Roger's "boil the oceans" [ to eliminate German U-boats]. When I was a medical student or house officer, we did not discuss "lifestyle" specialties and-believe it or not-income differentials were not a pivotal factor in the decisions many of us made. The 21st century medical student is much more savvy regarding what happens in the life post training. I believe the major blow to general internists was and is managed care. I seriously doubt that that juggernaut will be stopped or meaningfully altered by practicing doctors or their largely ineffective and sometimes misdirected national organizations. But I can still hope.
Sunday, October 16, 2005
Pertussis is back giving adults persistent cough,but help is on the way
Pertussis-whooping cough-was something we saw at Big Charity hospital on our pediatrics rotation.After that I rarely even thought of it. In the last several years docs are suggesting it as a cause for their own or their patient's lingering coughs. Adults usually don't get the whooping part . There are now two new vaccines that contain with Tetanus and diphtheria antigens an acellular pertussis antigen which in one form or another has been put into practice in Europe and Canada and efforts are underway to use DTap in place of Td in the U.S. The current issue of NEJM features an article documenting the efficacy of this acellular pertussis vaccine in adolescents and adults. I believe that the reason pertussis was eliminated for the "tetanus" shot for adults was the reaction rate with the old cellular pertussis. Apparently this is not a problem with the new formulations.
Saturday, October 15, 2005
AMA offer programs on lesions we should learn from Nazi doctors.
Beginning in 2004,Tte AMA's Institute for Ethics and the U.S. Holocaust Memorial Museum in Washington, D.C. have offered a lecture series that examines Nazi Medicine and offers perspective on what can be learned from this dark (darkest?) chapter in medical history. A recent AMA newspaper editorial reports that the program has been presented to 12 med schools so far. All of the schools should sign up for this. I did not learn of the incredible behavior of many German doctors until long after I graduated from medical school and I heard nothing about it in college or high school. Here is the link for the AMA web site.CME credit may be obtained. The book " The Nazi Doctors" by Robert Jay Lifton give details of the behavior of individuals who were physicians in name only that is so chilling that I have no words to express the magnitude of the horror. The terrible things that have been done in the name of the "greater good " and the sacrifice of the individual to the collective and when individual responsibility is removed know no bounds.
Friday, October 14, 2005
A blood test to diagnose mesothelioma? Osteopontin
Blood tests for cancer(s) have been widely sought after. It could be so easy. Order a test, voila a diagnosis. So far, only one blood test is widely use to screen for cancer, the PSA and it is not without detractors and critics and the issue of screening is contentious.Generally blood tests for cancer have looked pretty good early on only to disappoint as more data is accumulated. But hope springs eternal.
There is recent data that at least suggests a blood test may be useful in the diagnosis of mesothelioma. Ostopontin, a glycoproten that mediates cell matrix interactions and is overexpressed in mesothelioma, may be of value to distinguish patients with asbestos exposure who do and who do not have mesothelioma. The October 13, issue of NEJM features an article by H. I .Pass et al who studied 76 patients with pleural mesothelioma as well as 69 patients with nonmalignant asbestos disease and 45 subjects with no history of asbestos exposure.Tumor tissue was examined for osteopontin and serum levels were measured. Levels were not increased from asbestos exposure alone and patients with pleural plaques and no pulmonary fibrosis did not demonstrate an increased blood level but mesothelioma patients did. 36/38 cases of mesothelioma studied demonstrated osteopontin staining of tumor cells.
Osteopontin, however, cannot be cannot be called MSA ( mesothelioma specific antibody) because colon cancer cases have also shown overexpression of osteopontin correlated with progression. The colon cancer study linked above did not do serum levels.
The typical mesothelioma case has a median survival of at most one year. There are cases of five year survivals in patients with early disease whose tumor were resected. Certaintly a means of detecting early disease would be welcome. The latency of mesothelioma is very longer, sometimes 30 or 40 years. OSHA has a mandated asbestos surveillance program but that program ends when the workers leave employment. In my experience consulting with a number of petrochemical companies over the past 25 years, most of the mesothelioma cases in refinery and chemical plant workers have occurred in retirees and most companies do not offer an exam program for retirees. The population to monitor and screen for early mesothelioma is the retiree group. On the other hand, pleural plaques in active employee exposed to asbestos in the years before the current OSHA standards were in place are not uncommon. The following issue will likely arise-should employees with pleural plaques be screened with serum osteopontin tests? Pleural plaques are not premalignant per se, that is a plaque is not transformed into a tumor. However, an asbestos exposed person whose exposure is signaled by a plaque is at risk of a mesothelioma. It may be that serial changes in the osteopontin level may be used in a similar manner to the PSA. The so-called PSA velocity may signal prostate cancer and perhaps serial increases in the osteopontin value may indicate early mesothelioma. Perhaps the utility of the test-if further data provides more validation-would be in periodic measurements in retirees with pleural plaques.
There is recent data that at least suggests a blood test may be useful in the diagnosis of mesothelioma. Ostopontin, a glycoproten that mediates cell matrix interactions and is overexpressed in mesothelioma, may be of value to distinguish patients with asbestos exposure who do and who do not have mesothelioma. The October 13, issue of NEJM features an article by H. I .Pass et al who studied 76 patients with pleural mesothelioma as well as 69 patients with nonmalignant asbestos disease and 45 subjects with no history of asbestos exposure.Tumor tissue was examined for osteopontin and serum levels were measured. Levels were not increased from asbestos exposure alone and patients with pleural plaques and no pulmonary fibrosis did not demonstrate an increased blood level but mesothelioma patients did. 36/38 cases of mesothelioma studied demonstrated osteopontin staining of tumor cells.
Osteopontin, however, cannot be cannot be called MSA ( mesothelioma specific antibody) because colon cancer cases have also shown overexpression of osteopontin correlated with progression. The colon cancer study linked above did not do serum levels.
The typical mesothelioma case has a median survival of at most one year. There are cases of five year survivals in patients with early disease whose tumor were resected. Certaintly a means of detecting early disease would be welcome. The latency of mesothelioma is very longer, sometimes 30 or 40 years. OSHA has a mandated asbestos surveillance program but that program ends when the workers leave employment. In my experience consulting with a number of petrochemical companies over the past 25 years, most of the mesothelioma cases in refinery and chemical plant workers have occurred in retirees and most companies do not offer an exam program for retirees. The population to monitor and screen for early mesothelioma is the retiree group. On the other hand, pleural plaques in active employee exposed to asbestos in the years before the current OSHA standards were in place are not uncommon. The following issue will likely arise-should employees with pleural plaques be screened with serum osteopontin tests? Pleural plaques are not premalignant per se, that is a plaque is not transformed into a tumor. However, an asbestos exposed person whose exposure is signaled by a plaque is at risk of a mesothelioma. It may be that serial changes in the osteopontin level may be used in a similar manner to the PSA. The so-called PSA velocity may signal prostate cancer and perhaps serial increases in the osteopontin value may indicate early mesothelioma. Perhaps the utility of the test-if further data provides more validation-would be in periodic measurements in retirees with pleural plaques.
Thursday, October 13, 2005
Interview with physician who was in Charity Hospital during Katrina aftermath
Dr. Ruth Berggren,who is on the staff at Charity and faculty at Tulane, give an audio sketch of her days in Big charity In New Orleans as everything fell to pieces after Katrina hit.An audio presentation is available-I think only to subscribers-at the NEJM site. It is worth listening to. Her first hand account points to the pivotal role of gun fire and snipers in aborting the effort to get patients - and everyone else- out in a timely manner. I have written before about the heroics of the medical personal at Charity at this time. She gives her voice to what took place. There was no power, little food and water, a diarrhea outbreak among patients with no working plumbing, heat, darkness, needing to practice medicine in the dark bereft of laboratory assistance and- incredibly- fear of men with guns. The hospital personnel and patients were abandoned on a hellish island. City and state governmental officials-either powerless or ignorant of their plight- were not to their immediate aid. Were there no plans in place to evacuate the critical patients in the event of a catastrophe? Finally on 9/3/05 staff and patients were evacuated. For decades the poor and uninsured in Louisiana relied on Charity Hospital for medical care. We now learn that Charity has been declared past the point of salvage and recovery and can only be torn down. The state of Louisiana talks of plans to build a new hospital. It now looks like many more patients than the ones in hospital on the day the levees broke may be abandoned at least for now.
Monday, October 10, 2005
"Cultural Competency" is not a competency to be checked off like a thoracentesis
Articles on cultural competency seem to be found in almost every journal lately. Certain aspects of this movement have not felt right to me. Linda M. Hunt, an Associate Prof at Michigan State , does an excellent job of explicating some of these aspects. Her article is worth reading;here are some of the highlights:
"Cultural competency (CC-my abbreviation) remains a vaguely defined goal with no explicit criteria established for its accomplishment or assessment. This lack may in part be due to the elusive nature of its central construct:culture". Nevetheless, there are increasing requirements for physicians to receive training in area for which no or few valid criteria for assessment exist.Various residency program include it as do medical schools. The influential IOM recommended training CC for medical students and in New Jersey has passed a law mandating physicians receive training.
She says further, " It is not possible to predict the beliefs and behaviors of individuals based on their race, ethnicity, or national origin."...Paradoxically..what originated in a desire to promote respect for individual differences may instead promote stereotyping and essentializaing...It may reinforce the belief that culture can be diagnosed and treated."
Much of what she said is encapsulated in a comments made by a letter writers to DB's Medical Rants. Letters no 3 and 4 seem on target . Physicians treat individuals not members of some arbitrary sociological construct. Treat patients with respect. Don't worry that a given patient might have some attitude or belief based on their alleged membership in some cultural classification subgroup.
She continues ;"..it [the cultural competency movement] has too often represented culture as a decontextualized set of traits providing a template for the perceptions and behaviors of of group members"..."specific ethnic cultures are represented as a codified body of characteristics that can be identified and then either modified or manipulated to facilitate clinical goals."
Does she offer an alternative? ( my alternative would be do drop the programs, but I don't see that happening for a while). She recommends two things. First be respectful of the unique perspective each patient brings . Respect is certaintly a core value. The second is a bit harder to put into real life clinical experience. Hunt recommends what she calls "cultural humility" by which she means to explore through conversations with the patient similarities and differences between his own and each patient's goals, capacities and priorities. That is quite an assignment even with limitless time but not likely to be achieved in a 10-15 clinical encounter let alone a 7 minute one.
Allegedly, somehow the clinical competency movement is supposed to further the goal of reducing or elliminating the gap between the medical care haves and the medical care havenots.
How programs with poorly defined goals, a flawed conceptual foundation and an intellectual regression to treating folks as members of groups will achieve that remains unclear. Even the much more restricted goal of improving doctor patient communication is not likely to be greatly improved by putting old wine (racial streotypes) into a new bottle. A recent Johns Hopkins study is said to have demonstrated that CC training improved some aspect(s) of patient satisfaction and physician understanding but - not surprisingly-no data are found demonstrating improved health care.
Hunt's closing paragraph says in part " ... being appropriately cognizant of and more responsive to cultural issues shoudl not be thought of a reaching a 'competency' " but she says an ongoing process in which there is a "respectful recognition" of the unique set of charateristics and attitudes and concerns of individual patients.
"Cultural competency (CC-my abbreviation) remains a vaguely defined goal with no explicit criteria established for its accomplishment or assessment. This lack may in part be due to the elusive nature of its central construct:culture". Nevetheless, there are increasing requirements for physicians to receive training in area for which no or few valid criteria for assessment exist.Various residency program include it as do medical schools. The influential IOM recommended training CC for medical students and in New Jersey has passed a law mandating physicians receive training.
She says further, " It is not possible to predict the beliefs and behaviors of individuals based on their race, ethnicity, or national origin."...Paradoxically..what originated in a desire to promote respect for individual differences may instead promote stereotyping and essentializaing...It may reinforce the belief that culture can be diagnosed and treated."
Much of what she said is encapsulated in a comments made by a letter writers to DB's Medical Rants. Letters no 3 and 4 seem on target . Physicians treat individuals not members of some arbitrary sociological construct. Treat patients with respect. Don't worry that a given patient might have some attitude or belief based on their alleged membership in some cultural classification subgroup.
She continues ;"..it [the cultural competency movement] has too often represented culture as a decontextualized set of traits providing a template for the perceptions and behaviors of of group members"..."specific ethnic cultures are represented as a codified body of characteristics that can be identified and then either modified or manipulated to facilitate clinical goals."
Does she offer an alternative? ( my alternative would be do drop the programs, but I don't see that happening for a while). She recommends two things. First be respectful of the unique perspective each patient brings . Respect is certaintly a core value. The second is a bit harder to put into real life clinical experience. Hunt recommends what she calls "cultural humility" by which she means to explore through conversations with the patient similarities and differences between his own and each patient's goals, capacities and priorities. That is quite an assignment even with limitless time but not likely to be achieved in a 10-15 clinical encounter let alone a 7 minute one.
Allegedly, somehow the clinical competency movement is supposed to further the goal of reducing or elliminating the gap between the medical care haves and the medical care havenots.
How programs with poorly defined goals, a flawed conceptual foundation and an intellectual regression to treating folks as members of groups will achieve that remains unclear. Even the much more restricted goal of improving doctor patient communication is not likely to be greatly improved by putting old wine (racial streotypes) into a new bottle. A recent Johns Hopkins study is said to have demonstrated that CC training improved some aspect(s) of patient satisfaction and physician understanding but - not surprisingly-no data are found demonstrating improved health care.
Hunt's closing paragraph says in part " ... being appropriately cognizant of and more responsive to cultural issues shoudl not be thought of a reaching a 'competency' " but she says an ongoing process in which there is a "respectful recognition" of the unique set of charateristics and attitudes and concerns of individual patients.
Tuesday, October 04, 2005
Grand Rounds is up again
First-year Tulane medical student, Niels Olson, offers a very well done Grand Rounds, found here. Hurricane Katrina has given new meaning to the nickname "Green Wave". Tulane Medical School restarted its year in Houston hosted by Baylor Med just last week. It will take more than two hurricanes to keep those folks from going to med school. (Rita delayed the opening of Tulane West a few days before)
Monday, October 03, 2005
Provocative PLOS article claims "Most published research findings are False"
John P.A Ioannidis, writing in the August 2005 issue of PLOS, entitled his essay "Why most published research findings are false". If that claim is true, how solid are the foundations of the recently venerablized Evidence Based Medicine"(EBM)? Obviously, not solid at all. He states " It can be proven that most claimed research findings are false" After one rather dense paragraph, he says " ... a research finding is more likely true than false if (1-beta)R >0.05, where 1-beta is the power of the study and R is the ratio of true relationship to no relationships and 0.05 is the typically used alpha( type I error) value. It seems to me that if we knew the ratio of true to no relationships we would have to have had some aprior knowledge of what was true, which we do not. A letter to the editor by Jonathan Wren of the University of Oklahoma goes down this same counter-argument road when he says that yes, the probability that a research findings is true depends in part on the prior probability of it being true but we do not know that probability, we merely makes guesses or estimates about it.
Ioannidis's thesis cannot be verified ( or falsified) unless we had some independent method of determining if study results were true. An individual study can be falsified-if robust contradictory evidence is adduced- or it can become stronger when attempts at falsification fail but the generalization that most studies are false seems to not be the type of statement readily falsified on empirical grounds, since you cannot ever really prove a given study true. You can sometimes prove one is false but to prove most is a Herculean task particularly given the current increasing pace of new studies being published. Wren evokes a type of Russell Paradox argument saying that if most studies are wrong, what about the studies that Ioanndis marshaled to support his thesis and what does that say about Ioannidis's published study.
My eyes glaze over a more than a little bit when I dive into Ioannidis's theoretical argument but his so called " corollaries " seem to ring true and have been widely discussed in the medical literature. These include various factors that tend to make studies less reliable. For example,
smaller studies are less likely to be true as are studies that target small effects ( eg. RR of 1.05) .
Further, the greater the number and the lesser the selection of tested relationship in a scientific field, the less likely the research findings are to be true.
Ioannidis's paper will not derail medical or scientific research-after all science marched on even after David Hume's critique of induction-but I do welcome efforts that might medical students develop a healthy skepticism about research findings. However, even in that light the letter to the editor by Stephen Paulker also worth reading. For a researcher the issue may well be is the finding more likely than not or even a higher standard of evidence but for the clinical doctor the question is- given the circumstances of the patient- what is the best thing to do. Even if we are not sure of a given therapy's efficacy and even if the probability of it being efficacious is less than .5, offering that treatment may be the best thing to do. I think he is saying that even though there may be many good reasons to be skeptical using the best (realizing it is not perfect or guaranteed) evidence we have is really the best we can do. Often-maybe most of the time- we have to go with the evidence we have- not the evidence we want to have.
Ioannidis's thesis cannot be verified ( or falsified) unless we had some independent method of determining if study results were true. An individual study can be falsified-if robust contradictory evidence is adduced- or it can become stronger when attempts at falsification fail but the generalization that most studies are false seems to not be the type of statement readily falsified on empirical grounds, since you cannot ever really prove a given study true. You can sometimes prove one is false but to prove most is a Herculean task particularly given the current increasing pace of new studies being published. Wren evokes a type of Russell Paradox argument saying that if most studies are wrong, what about the studies that Ioanndis marshaled to support his thesis and what does that say about Ioannidis's published study.
My eyes glaze over a more than a little bit when I dive into Ioannidis's theoretical argument but his so called " corollaries " seem to ring true and have been widely discussed in the medical literature. These include various factors that tend to make studies less reliable. For example,
smaller studies are less likely to be true as are studies that target small effects ( eg. RR of 1.05) .
Further, the greater the number and the lesser the selection of tested relationship in a scientific field, the less likely the research findings are to be true.
Ioannidis's paper will not derail medical or scientific research-after all science marched on even after David Hume's critique of induction-but I do welcome efforts that might medical students develop a healthy skepticism about research findings. However, even in that light the letter to the editor by Stephen Paulker also worth reading. For a researcher the issue may well be is the finding more likely than not or even a higher standard of evidence but for the clinical doctor the question is- given the circumstances of the patient- what is the best thing to do. Even if we are not sure of a given therapy's efficacy and even if the probability of it being efficacious is less than .5, offering that treatment may be the best thing to do. I think he is saying that even though there may be many good reasons to be skeptical using the best (realizing it is not perfect or guaranteed) evidence we have is really the best we can do. Often-maybe most of the time- we have to go with the evidence we have- not the evidence we want to have.
Subscribe to:
Posts (Atom)