Summary
In this episode of The Drive podcast, host Peter Etia interviews John Ionides, a physician scientist and Stanford University professor who studies scientific research itself, primarily in clinical medicine but also somewhat in the social sciences. Ionides is one of the world's foremost experts on the credibility of medical research and is the co-director of the Meta-Research Innovation Center at Stanford. The episode covers Ionides' journey from Greece to the United States, his background in mathematics and medicine, and his work on the credibility of medical research.
The discussion covers topics such as nutritional epidemiology and the use of mathematical models as tools to create frameworks in science. Etia references Ionides' famous paper, "Why Most Published Clinical Research Is Untrue," which explains through a mathematical model why most published research in the biomedical field is incorrect. The paper was published in PLOS Medicine, an open-source journal that was a transformative move at the time.
The context discusses a mathematical model that attempts to explain and predict the validity of research in evidence-based medicine. The model takes into account factors such as the prior chance of finding something in a particular field, the power of the study, and the presence of biases. The model shows that in most circumstances, if a study produces a statistically significant result with a p-value of 0.05, the chances of it being a false positive are higher than 50%. The context also explains the difference between statistical significance and clinical significance and suggests that a more stringent threshold for statistical significance should be used.
The speaker discusses the issue of misinterpretation of research results due to a lack of rules and guidelines in scientific research. They argue that many scientists are not well-trained in statistics and therefore misuse and misinterpret statistics. The speaker suggests that rules and guidelines for certain types of research could be useful in minimizing harm and error and maximizing clinically significant findings.
The context discusses the need for more data and standardization in research practices, particularly in fields like nutrition. The principles of large coalitions of researchers sharing data and standardizing analysis are being used in some fields but not to the same extent in nutrition. The context also mentions Austin Bradford Hill's 10 criteria for epidemiology and how they apply to tobacco as an example of a clear signal of causality. However, in nutritional epidemiology, strength and consistency are often lacking, making it difficult to determine causality.
The speaker discusses the issue of beliefs and how they can influence research and interpretation of findings. They emphasize the importance of transparency and imposing safeguards to minimize the chances of fooling oneself in the research process. They mention the case of Brian Wansick at Cornell, who urged students to cut corners and torture data to generate nice-looking results. The speaker acknowledges that fraud is uncommon in science, but questionable research practices are widespread.
The speaker discusses the importance of public funding for high-risk research and the need for clear communication between scientists and the public. They argue that the traditional narrative that every research grant delivers important results is false, and that most grants will likely fail to find anything significant. However, out of a thousand grants, there may be five that make a significant contribution.
The context also discusses the problematic design and recruitment of a medical trial, the Mediterranean diet trial, which was retracted and republished with additional analysis. The trial lacked credibility due to its partly randomized and partly observational nature. The article also discusses the events of 2020 and the studies done in Santa Clara and LA County that showed the coronavirus was much more widely spread than previously thought. The study was validated with several prevalent studies done around the world. The virus is rapidly and widely spreading, and there is a steep risk gradient with some people having minimal risk and others having a high risk of a bad outcome. The study received criticism and attention due to the toxic political environment.
In this context, a scientist discusses the backlash he received on social media regarding his research paper on COVID-19. He explains that while some scientists offered helpful comments, others who were not epidemiologists became "twitter or facebook epidemiologists" and had vocal opinions on how things should be done. The scientist believes that most of the animosity was related to the toxic political environment and that science should be completely dissociated from politics. Despite the challenges, the scientist remains optimistic about the future of science and his desire to continue learning and correcting his knowledge.
