Fewer large clinical trials are reporting positive findings since the introduction of measures designed to improve transparency

Kaplan and Irwin analysed all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. Trials were included if direct costs >$500,000/year, participants were adult humans, and the primary outcome was cardiovascular risk, disease or death. The 55 trials meeting these criteria were coded for whether they were published prior to or after the year 2000, whether they registered in clinicaltrials.gov prior to publication, used active or placebo comparator, and whether or not the trial had industry co-sponsorship. We tabulated whether the study reported a positive, negative, or null result on the primary outcome variable and for total mortality.

The analysis of 55 trials meeting the inclusion criteria showed that 17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ²=12.2,df= 1, p=0.0005).

The authors also looked into various aspects of transparent reporting: improvement in time in including diagrams reporting patient flow throughout the study, registration of studies on ClinicalTrials.gov and a prospective declaration of the primary outcome (see the last 3 paragraphs of the Results).

The authors note various important initiatives improving transparency of RCTs – including major journals requiring trial registration and following the CONSORT recommendation when reporting trial methods and findings. They state: “All of their (NHLBI) large trials published after 2000 were preregistered and transparently reported. Although we cannot say that stricter reporting requirements caused the trend toward more null reports from NHLBI trials, we do find the association worthy of more investigation.”

And implications for us: “A growing collection of trials suggests that promising treatments do not match their potential when systematically tested and transparently reported. Publication of these trials may lead to the protection of patients from treatments that use resources while not enhancing patient outcomes. For example, a recent economic analysis of the Women’s Health Initiative clinical trial suggested that the publication of the study may have resulted in 126,000 fewer breast cancer deaths, and 76,000 deaths from heart disease between 2003 and 2012. The economic analysis estimated that there was about $140 returned for each dollar invested in the study. Transparent and impartial reporting of clinical trial results will ultimately identify the treatments most likely to maximize benefit and reduce harm.”

Reference:

Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS One 2015; http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132382