« Putting your green juice pulp to work | Main | Thank You from Dhrumil & Company »

Why most published studies are either exaggerated or flat out wrong

In 2005 epidemiologist John Ionnidis published a paper which concluded that most published scientific research later turns out to be false.

This weekend NPR's On The Media caught up with Dr Ionnidis to ask him just exactly why this is the case:

Bad Study Habits

Most studies published in scientific journals, it turns out, are either exaggerated or wrong. How come? According to epidemiologist John Ionnidis, editors of science journals are no different than everyone else in media. Sensationalism sells.

Full interview here

Here's an excerpt from Brooke Gladstone interview with Dr John Ionnidis that really drives the point home:

DR. JOHN IOANNIDIS: There's several studies in the literature that suggest that when you have studies that have been equally well designed and equally well conducted, the ones with positive results will be printed far more frequently or they will be published far more quickly.

For example, there's an empirical study that looked at studies on antidepressants. About half of them had found positive results in favor of the drugs, and another half of them or so found negative results — no effectiveness, the drug didn't seem to make a difference compared to placebo. However, the published literature on these drugs is comprised entirely of positive studies.

BROOKE GLADSTONE: So the obvious question is, why? Is it just that scientific journals are subject to the same commercial pressures that all other journals are and they just have to move paper?

DR. JOHN IOANNIDIS: I think that this is part of the truth. There’s a lot of competition, and I think that any journal is interested to publish research that does seem to matter and does seem to make a difference, and something that other journals have not published.

BROOKE GLADSTONE: What’s the overall impact of all of these exaggerated scientific stories in the journals?

DR. JOHN IOANNIDIS: The impact could be very substantial. For example, if we're talking about research that deals with whether treatments are effective or not, if the literature is distorted, that means that we believe that some treatments are worth it and they are effective, while, in fact, they're not, or that they are more worth it and more effective than they actually are.

It could lead the whole field of researchers down the wrong path. One could have what we call “herding,” where someone publishes a paper in a major journal and then everybody has to try to work on that field and try to find similar results so as to be trendy, so as to attract funding, so as to seem that one is working in research that is attractive and it was published in that major journal, so why not me as well?

When I first embarked on my raw journey back in 2001, the second most popular question I would hear was, "Are there any published scientific studies to back this up?" I use to feel like an idiot when I would reply, "Not that I know of, but..."

A few years would pass before I would learn just how loaded and weighted this question was. It would be a few years later that I would learn about the influence that corporations have on what gets funded and what gets published. It would be a few years later that I would learn about the lack of attention given to preventive and holistic tools, (not out of conspiracy, they just aren't profitable). And it would be a few years later before wise MDs, like Alejandro Junger and Gabriel Cousens, would tell me that the best study is self study and that nothing speaks louder than true results.

The scientific method and research are great things. In fact we'll need more research as the raw food community grows and continues to get more sophisticated.

I have a feeling that with the cost of research coming down, the awareness of holistic living going up, we'll see more truly authentic studies conducted and published in decentralized methods. The work that Gabriel Cousens MD is doing to promote the cure for diabetes is a clear example of that. I know we will continue to see other examples emerge.

Comments