Skip to main content

The Human Operating Manual

By Jay Downes9 min readPublished 19 November 2026

Why Most Published Research Findings Are False

Quick answers5 questions this page answers

What is this paper about?

John Ioannidis’s 2005 argument that, for most kinds of study and most fields, a claimed research finding is more likely to be false than true.

What decides whether a research finding is true?

How likely the idea was to be true before the study, the study’s power to detect a real effect, the significance threshold, and how much bias went in.

The model →

What makes a finding less likely to be true?

Smaller studies, smaller effects, more relationships tested with less preselection, more flexible designs and analysis, greater financial and other interests, and a hotter field with more teams competing.

The six corollaries →

How often are findings from different kinds of study true?

In his model, about 85% for a well-powered randomised trial or a meta-analysis of good trials, about 20% for a well-powered exploratory epidemiological study, and about 12% for an underpowered one.

How often is a finding true? →

Has the paper been challenged?

Yes. Goodman and Greenland argued the maths doesn’t prove the title, and Jager and Leek estimated that about 14% of significant findings in top medical journals were false. Replication projects have since found many published results don’t repeat.

Reading this in 2026 →

Author: John P. A. Ioannidis

Source: PLoS Medicine, 2(8), e124 (2005). doi:10.1371/journal.pmed.0020124. Open access under a Creative Commons Attribution licence.

Topics: the scientific method, statistics, p-values, bias, reading research, replication, conflicts of interest

All information is attributed to the author. Except in the case where we may have misunderstood a concept and summarized incorrectly. These notes are only for reference and we always suggest reading from the original source.

Contents

1: The Claim

2: The Model

3: Bias

4: Many Teams, One Question

5: The Six Corollaries

6: How Often Is a Finding True?

7: An Example: Hunting for Schizophrenia Genes

8: Findings as a Measure of Bias

9: How to Improve the Situation

10: Reading This in 2026 (HOM note)


1: The Claim

Published findings are often overturned by later evidence, in clinical trials, epidemiology, and molecular research. Ioannidis argues that for most kinds of study and most fields, a claimed finding is more likely to be false than true.

  • He defines a “research finding” as any relationship that reaches statistical significance, such as an effective treatment, a risk factor, a predictor, or an association.
  • He blames the habit of treating a single study as conclusive because it crossed p < 0.05. P-values don’t summarise research well, but medical papers are widely read as if they do.
  • He says “negative” research is useful too, and that the label is misleading. The paper looks only at relationships researchers claim exist.

2: The Model

Building on earlier work by other methodologists, Ioannidis says the chance a finding is true depends on:

  • Pre-study odds (R): the ratio of true relationships to non-relationships among everything tested in a field. Some fields test likely ideas, and others look for a few true relationships among thousands or millions of possibilities.
  • Power: the study’s chance of detecting a real effect if there is one.
  • Significance threshold: the false-positive rate the field accepts, usually 0.05.

The chance a claimed finding is true is its positive predictive value (PPV), the same measure used for diagnostic tests. Without bias, a finding is more likely true than false only when power multiplied by R is greater than 0.05.

Our worked example, not the paper’s: if one in twenty ideas tested in a field is true, a study with perfect power gives a finding about a 50% chance of being true. With 50% power, the chance drops to about a third.

3: Bias

  • Ioannidis defines bias as design, data, analysis, and presentation factors that produce findings when they shouldn’t. In his model, u is the share of analyses that wouldn’t have been findings but are reported as findings.
  • Bias is separate from chance. A well-run study can still produce a false finding by chance. Selective or distorted reporting is a typical form of bias.
  • The more bias, the lower the chance a finding is true.
  • Reverse bias loses true relationships, through large measurement errors, inefficient use of data, or conflicts of interest that bury results. He thinks it is less common, but says there is no good large-scale evidence on how often it happens.

4: Many Teams, One Question

Several teams, often dozens, usually work on the same or similar questions, and attention goes to whichever one claims a finding. The more independent studies on a question, the lower the chance that any one positive result, looked at on its own, is true.

5: The Six Corollaries

A finding is less likely to be true:

  1. The smaller the studies: small samples mean low power. Findings from large cardiology trials with thousands of participants are more likely true than those from molecular predictor studies with samples a hundred times smaller.
  2. The smaller the effects: smoking raises the risk of cancer or heart disease three- to twenty-fold, while genetic risk factors for complex diseases raise it by 10–50%. If most real genetic or nutritional causes of complex disease raised risk by less than 5%, he says those fields would be close to hopeless.
  3. The more relationships tested, and the less they were chosen in advance: confirmatory work, such as large phase III trials and meta-analyses of them, does better than exploratory experiments. Research that tests huge numbers of things at once, such as gene microarrays, should have extremely low PPV.
  4. The more flexible the designs, definitions, outcomes, and analysis: flexibility lets researchers turn “negative” results into “positive” ones. Agreed standards, clear outcomes such as death (instead of many different rating scales), and standard analyses all help. Selective outcome reporting is common even in randomised trials, so stopping selective publication wouldn’t fix the problem on its own.
  5. The greater the financial and other interests and prejudices: conflicts of interest are common in biomedical research and poorly reported. Prejudice can also come from commitment to a theory or to one’s own findings, and some studies are done mainly for promotion or tenure. Prestigious researchers can use peer review to block findings that contradict their own. He says expert opinion has been shown to be extremely unreliable.
  6. The hotter the field: with many teams competing, each one rushes out its most impressive positive results, and negative results get published mainly to refute someone else. He calls the swing between extreme claims and extreme refutations the Proteus phenomenon, and says it is very common in molecular genetics.

The six factors affect each other. Prejudice can be strong in a hot field, but a hot field or strong interests can also bring larger studies and better standards.

6: How Often Is a Finding True?

Ioannidis ran his formulas for common situations. In his model, it is hard for a finding to reach a PPV above 50%.

Type of studyChance it’s true
Adequately powered randomised trial with little bias and 1:1 pre-study oddsPower 0.80 · pre-study odds 1:1 · bias 0.100.85
Confirmatory meta-analysis of good-quality randomised trialsPower 0.95 · pre-study odds 2:1 · bias 0.300.85
Meta-analysis of small, inconclusive studiesPower 0.80 · pre-study odds 1:3 · bias 0.400.41
Underpowered but well-performed early-phase (I/II) trialPower 0.20 · pre-study odds 1:5 · bias 0.200.23
Underpowered, poorly performed early-phase (I/II) trialPower 0.20 · pre-study odds 1:5 · bias 0.800.17
Adequately powered exploratory epidemiological studyPower 0.80 · pre-study odds 1:10 · bias 0.300.20
Underpowered exploratory epidemiological studyPower 0.20 · pre-study odds 1:10 · bias 0.300.12
Discovery research with massive testingPower 0.20 · pre-study odds 1:1,000 · bias 0.800.0010
As above, but with less bias (more standardised)Power 0.20 · pre-study odds 1:1,000 · bias 0.200.0015
Adapted from Ioannidis (2005), Table 4, under a Creative Commons Attribution licence. All figures assume a significance threshold of 0.05 for a single study.
  • Pooling small, inconclusive studies in a meta-analysis probably gives a false finding if the pre-study odds are 1 in 4 or worse.
  • Even well-powered epidemiological studies may be right only about one time in five when 1 in 11 of the ideas tested is true.

7: An Example: Hunting for Schizophrenia Genes

A team tests 100,000 gene variants for links to schizophrenia. Based on its heritability, perhaps ten are truly involved, each with a modest effect, so the chance any one variant is real is about 1 in 10,000. The study has 60% power.

  • A result that just crosses p = 0.05 raises the probability about twelve-fold, to about 12 in 10,000.
  • With some bias (from choices about which patients or controls to include, unplanned subgroup analyses, changed definitions, or selective reporting), it falls to about 4.4 in 10,000. He notes that commercial “data mining” software was advertised on its ability to produce significant results.
  • With no bias, if ten teams run similar studies and one finds a significant link, the chance it is true is about 1.5 in 10,000, barely higher than before any of the research was done.

8: Findings as a Measure of Bias

Ioannidis says much modern biomedical research is done where the chance of a true finding is very low, before and after the study. In a “null field”, where there is nothing real to find, reported effects should scatter around zero by chance, so any consistent pattern of effects measures the field’s bias.

  • His example: if no nutrient affects the risk of a particular tumour, but the literature has linked 60 nutrients to it, each with a 20–40% change in risk between the highest and lowest intakes, those effect sizes measure only the bias in that literature.
  • Among null fields, the ones claiming the biggest effects (often with claims of medical importance) would have the worst biases.
  • Large, highly significant effects are usually seen as exciting. He argues that in most fields of modern research they are more likely to be signs of large bias, and a reason to look for what went wrong.
  • Researchers resist the idea that their field might be null, but new evidence or technology can eventually dismantle one.

9: How to Improve the Situation

The truth can never be known with certainty, but Ioannidis suggests ways to raise the chance that findings are true:

  • Better-powered evidence: large studies and low-bias meta-analyses. Aim them at questions that are already fairly likely to be true, so a positive result is close to definitive, and at major concepts instead of narrow questions, so a negative result can rule out a whole line of research. Running large studies for narrow reasons, such as marketing a drug, is largely wasted. Very large studies can also make trivial effects statistically significant.
  • All the evidence: many teams work on most questions, so it is misleading to focus on one team’s significant result.
  • Better standards and less prejudice: this may need a change in scientific culture, which is hard to achieve.
  • Registering studies in advance: this is done for randomised trials. It is harder for exploratory research, but registering data collections, networking researchers, and writing and following a protocol could be used more widely.
  • Thinking about pre-study odds: researchers should estimate how likely it is they are testing something true before running an experiment. Established findings should be retested in large, low-bias studies to see how often they hold up. He expects several established “classics” to fail.
  • Caution about significance: a significant result in one study gives only part of the picture, because you can’t see how much testing was done elsewhere. Rough assumptions about the odds are unavoidable, and still useful for putting claims in context.

10: Reading This in 2026 (HOM note)

This section is ours, not the paper’s. The paper is one of the most cited on research reliability, and it has been debated since it came out.

  • Critique of the maths: Goodman and Greenland (2007) agreed there are more false claims than many would suspect, but argued the model doesn’t prove that most published claims are false. It counts only whether a result crossed p = 0.05, not how small the p-value was, and its assumptions about bias limit how much evidence any study can provide. Ioannidis replied that their methods and his were practically the same, and that even without bias the chance of a finding being true probably stays below 50% for most studies that aren’t randomised or large.
  • Estimates from published data: Jager and Leek (2014) used the p-values reported in top medical journals and estimated that about 14% of significant findings were false. In their reply to critics, they said 14% was probably optimistic but the true figure was unlikely to be above 50% for well-planned studies. Ioannidis called their approach flawed.
  • Replication projects: when the Open Science Collaboration (2015) repeated 100 psychology studies, about 36% of the replications produced significant results, compared with 97% of the originals. Camerer and colleagues (2018) replicated 13 of 21 social science experiments published in Nature and Science (62%), and the successful replications tended to find smaller effects.
  • How we use it on the HOM: whether “most” findings are false depends on the field and on how “false” is defined. We use the paper as a checklist. How plausible was the idea? How big was the study and the effect? How many things were tested, and how flexible was the analysis? Who had a stake, and how crowded was the field?

References

  • Camerer, C. F., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644. doi:10.1038/s41562-018-0399-z
  • Goodman, S., & Greenland, S. (2007). Why most published research findings are false: Problems in the analysis. PLoS Medicine, 4(4), e168. doi:10.1371/journal.pmed.0040168
  • Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. doi:10.1371/journal.pmed.0020124
  • Ioannidis, J. P. A. (2007). Why most published research findings are false: Author’s reply to Goodman and Greenland. PLoS Medicine, 4(6), e215. doi:10.1371/journal.pmed.0040215
  • Jager, L. R., & Leek, J. T. (2014). An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics, 15(1), 1–12. doi:10.1093/biostatistics/kxt007
  • Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. doi:10.1126/science.aac4716

More to explore