Imagine you’re a scientist. You published a paper in a reputable journal a couple of years ago, and that paper has made an impact in your field of study. It has shaped your own research and has inspired others to pursue similar lines of investigation. Thanks to that paper, you’re enjoying a certain status among your peers. Now you find out that one of the works you cited in that paper has just been retracted. Whether it was an honest mistake, total fabrication, or something in between, that work has now been flagged and no longer stands up to scientific scrutiny.
What do you do?
Jerome Samson is a data science, media, and research communication consultant. He’s principal at 3.14 Research Management Group, a research and communications agency, and was previously managing editor of scientific publications at Nielsen.
Modern science is a group effort. It’s slow, it’s complicated, it often involves multiple disciplines, and it’s almost always expensive — all excellent reasons for researchers to team up to share the burden and check each other’s equations. A 2015 paper in Physical Review Letters had a record-breaking 5,154 co-authors. That’s a bit extreme, of course, but there’s no question that across many fields today, single-authorship is shrinking and multi-authorship is growing.
And science is built on top of science. Standing on the shoulders of giants, as Google Scholar makes clear on its home page. Researchers read up on prior work before starting their own lines of inquiry, often replicating past experiments to lay a foundation for their own research. They cite references scrupulously, and peer-review is the norm before anything gets published.
Scientists form a giant altruistic community. There are turf considerations and funding to compete for, as in any other walk of life, but by and large, scientists are doing what they do because they’re motivated by the greater good, and collaboration is in their DNA.
So when a paper is retracted, potentially jeopardizing everything that came after it, the scientific community goes into self-correction mode and pulls all the papers that, at one point or another, directly or indirectly, have made a reference to that work. Right? Until further notice? Not exactly.
Direct citations for a single paper, before and after retraction (for the legend, see the end of this story).
For one thing, the word that a paper has been retracted rarely makes the news. Unless, of course, that paper is this piece of malfeasance and your name is Andrew Wakefield. Most of the time though, the original publisher simply posts a retraction notice, slaps a “retracted” watermark on the online version of the paper, and notifies the online databases.
For example, the PubMed listing for the stem cell paper used in the illustration above contains a large “Retracted article” label at the top of it. Cancer Research, the journal that first published the paper in 2005, lists this at the top: “This article has a correction, but has also been retracted.” This is all nice and well, but for every retracted paper, the original unmarked copy still lives on in print (where you might have read it in the first place). And if you have cited that paper in your own work, you don’t receive an alert that one of your citations has just imploded.
Which means that you might be totally in the dark.
Can your own work survive without that one citation? In the vast majority of cases, it will. But why keep the citation in when it’s likely to cast a shadow on your work? In those few situations where the retraction shakes the very foundations upon which your house is built, you’re going to want to address the situation head on , and sooner rather than later.
It’s no secret that science is under attack. The outstanding team at Retraction Watch catalogs around 600 new retractions every year. That’s not necessarily a bad thing — after all, a high number of retractions is a healthy sign that science is actively correcting itself. And when you compare that figure to the volume of scientific papers out there (2.5 million new papers every year, published by more than 35,000 journals worldwide), it’s a very small percentage. But a good half of all retractions are due to misconduct. Political interests are quick to pounce and use those cases to tar an entire field of study. In today’s out-of-control news cycle, allowing a single retraction to have a ripple effect on the credibility of future research is simply unacceptable.
So, what do we do about it?
Darpa has recognized the problem and recently issued a request for information with the following grim assessment:
“Publications today lack mechanisms for rapidly incorporating dynamic changes in the literature, such as retractions, replications, or new findings, which presumably should alter a user’s confidence in a specific study’s conclusions.”
The agency is valiantly inviting researchers to help it devise some automated system to assign confidence levels to research results. Beyond p-values, impact factors, h-indices and altmetrics, Darpa is looking for some combination of natural language processing, sentiment analysis, network analytics, and machine learning to quickly assess a research claim’s reliability, reproducibility, and robustness.
This is commendable and definitely worth exploring, but it’s hard to imagine that the solution won’t involve some seriously low-tech grunt work: namely, alerting all authors who have cited the retracted work, directly or indirectly, and asking them to formally evaluate the impact of that retraction on their own work. We can debate how formal that process ought to be (should all affected papers be shelved until they’ve been cleared?), but there’s little doubt that it needs to happen.
This is no small matter. As the animation at the top of the article showed, one stem cell paper published in 2005 and retracted in 2010 has been cited 667 times — so far. Nearly half of those citations occurred after the retraction was made official. Here we are, in 2017, seven years after its retraction, and authors continue to refer to it as if nothing happened (including half a dozen times in the past couple of months alone). Nobody knows the extent to which the mistakes in that paper have affected any of those papers downstream.
And the animation at the top only accounts for direct citations. Each one of those 667 papers was, in turn, cited by other papers, and so on and so forth. See how the ripple effect from that one retraction ends up affecting more than 33,000 papers:
This illustration shows direct (first-degree) and indirect (second-degree only) citations for a single paper, before and after retraction. The ripple effect can reach tens of thousands of papers.
Science is a powerful thing. And part of what makes it powerful is how interconnected everything is. But that interdependence also makes it vulnerable. It doesn’t have to be that way. By staying on top of retractions and jumping at the opportunity to reassess their work when a source has been tainted, scientists will deny science-deniers the energy they feed off, and go a long way to improve the standing of their own research.
Chart legend: For the illustrations in this article, each blob represents a separate paper, and the size of the blob is a function of that paper’s total number of citations. A blob’s distance to the center represents the years since publication (2005 to 2017), and the colors correspond to different journals. In the second illustration, the lines between blobs further represent citation relationships between papers.
WIRED Opinion publishes pieces written by outside contributors and represents a wide range of viewpoints. Read more opinions here.