How the file drawer hides truth
Publication bias occurs when research with positive or significant results is more likely to be published than research with null or negative results. Imagine 100 studies are conducted, each with 30 percent statistical power to detect a real effect. By chance, 30 yield positive results while 70 are negative.
The diagram illustrates the outcome: journals publish the 30 positive studies, while researchers file the 65 negative studies away. The 5 remaining studies (false positives at alpha = 0.05) are also published. A meta-analysis of the published 35 studies appears to show a strong, consistent effect, when in reality the true effect is weak or absent.
Distortion of evidence and meta-analysis reliability
This filtering effect inflates the apparent strength of findings and narrows confidence intervals. Published effect sizes become systematically larger than true population effects. A meta-analyst combining only published studies will overestimate both the mean effect and its precision.
Solutions include preregistration of studies (public commitment before data collection), open science practices (sharing null and negative results), and funnel plots (visuals that reveal asymmetry caused by publication bias). Recognizing publication bias is essential for interpreting any literature review or meta-analysis.