Enhancing the QUAlity and Transparency Of health Research
PLOS Medicine Editorial Director, Virginia Barbour, reflects on the publication of the CONSORT and PRISMA guidelines and reminds us of the importance of checklists to medical publishing. This blog was originally published as one of eight in PLOS Medicine’s 10th Anniversary blog series on the most interesting and influential articles of the last ten years. You can read the original blog here
Gilbert and Sullivan’s Lord High Executioner has, sadly, given lists a bad name. Rather than tools of revenge, lists in healthcare, however, have the power to do much good. Atul Gawande’s book on lists has explained why they should be core to medical practice. I’d argue that in medical publishing too they are critical. When Robert Boyle, one of the founders of the UK’s Royal Society, wrote the Spring of the Air he was probably the first to write in such a way that allowed other men (it was only men then of course) to repeat and test his findings. In this way, he was, in turn, one of the first to legitimize research by making it reproducible.
More than 300 years later we have fully reproducible literature with everything fully reported, right? Wrong. There is a current crisis of confidence in research, with increasing and appropriate concern that many results, especially the most dramatic, often cannot be trusted. Contributing fundamentally (but not exclusively, obviously) to this problem is that whole swathes of the medical and scientific literature are not described in sufficient detail that anyone else can even test. In medicine this crisis literally is life-threatening; patients given treatments as a result of inadequately described studies may at best be treated sub-optimally, at worst harmed or killed.
However, in one important corner of the research endeavour, a group of individuals have, for many years now, been making a determined effort to change this poor reporting and PLOS Medicine is proud to have played its part in it. The CONSORT reporting guidelines for clinical trials and the PRISMA guidelines for Systematic Reviews and Meta-Analyses are, I’d argue two of the most important papers the journal has ever published. The premise behind both of these documents, and the checklists in them and the many other guidelines that we and other PLOS journals have published is very simple: tell us what you did so others can test it.
Is there a downside to this reporting? Yes, perhaps as a weary researcher, having another guideline to abide by and a checklist to fill can seem like one too many hurdles to jump over. And what if guidelines were used in such a way as to judge significance—would they be used to discriminate against them? Are such worries legitimate? I’d say not. And furthermore, in several ways, beyond the simple message of clarity of reporting they illustrate values that are core to PLOS.
Reporting guidelines above all allow us to judge what a paper reports, and what it does not (hint, the clue is in the name). Was this trial randomized or not; how was the search strategy done; these are the most basic of questions. The secondary questions of whether these results are ones that will change your medical practice are not for guidelines to answer. My favourite analogy of a guideline is that using one is akin to turning on a light in a room. It tells you what the room looks like; it doesn’t (unless you live in some Hogwartian dream word) clean the room for you. More prosaically, reporting guidelines are a tool, not a panacea; much like Open Access.
The two guidelines here are themselves revisions on earlier guidelines and this also illustrates an important point i.e. that such guidelines must evolve, that every iteration with more accumulated input to it will improve it. Arguably, publishing these papers in an Open Access journal such as PLOS Medicine will accelerate this process by enabling visibility, dissemination, reuse and thus ultimately improvement—again something very close to PLOS’ heart.
And the final way in which these guidelines indirectly reflect PLOS’ values are in the way that they are the product of many people’s time, intellect and energy; none of them would have come about unless, as one of the leaders of the EQUATOR initiative once said, when drawing these guidelines up everyone was able to ”leave their egos at the door”.
So, dear researcher, when you feel wearied by a checklist, remember the philosophy, energy and lack of egos that went into making them. If Robert Boyle and colleagues could understand the importance of good reporting 300 years ago, surely it’s time now to make it a cornerstone of the scientific and medical literature.
You can read the full guidelines here:
How can reporting quality interfere with reproducibility issues and overall trust in science results? With that question in mind, we participated in the Reproducibility, Replicability and Trust in Science conference organised by the Wellcome Genome Campus from 9 to 11...