This past week an editorial by Drs. Hershel Raff and Dennis Brown titled the “Civil, sensible, and constructive peer review in APS journals” was published in the American Journal of Physiology: Regulatory, Integrative and Comparative Physiology. Dr. Brown is the editor-in-chief of Physiological Reviews and Dr. Raff is the chair of the publications committee for the American Physiological Society.
Drs. Raff and Brown address the “recent’ interest in open publishing and post-publication peer review, reiterate their perceived value of pre-publication peer review, and state their ongoing vision for the publication process. Now, I have to disclose that I have found value in pre-publication peer review. I haven’t really had any real run-ins with Reviewer #3. I’ve certainly had some reviewers that I thought were perhaps a smidge over-zealous in their requests, but I’ve never been asked for anything unreasonable and I have found the editors to be helpful. I think that my manuscripts have come out of pre-publication peer review improved. That said, I am also a child of the internet era and I wonder if perhaps my own beloved academic society isn’t being closed-minded about the potential of the internet, instead entrenching themselves in the soft, warm familiarity of what they know how to do best. Allow me to prattle on, since it’s my blog and I’ll prattle if I want to…
Raff and Brown begin by acknowledging the call by some for post-publication peer review. They write:
…some have even advocated eliminating prepublication peer review altogether (1, 7). The underlying premise of this initiative is that anything in the correct format is publishable, and that postpublication scrutiny using blogs and comments, a vox populi of sorts, will ultimately expose the truth. This was called “scholarly skywriting” almost two decades ago (5) and more recently “trial by twitter” (11). While proponents of this school of thought have increased in number and influence, the Publications leadership of the American Physiological Society (APS) continues to believe that prepublication peer review is worth the effort and cost and is critical to maintaining the scientific integrity of our publications.
I find it interesting that Raff and Brown begin by drawing such a dichotomy. The alternative to pre-publication peer review is the lawless, wild west-style world of open access, post-publication peer review. Post-publication peer review happens necessarily at the elimination of pre-publication review and the opposite of peer review is 4chan. I find this to be disingenuous. For example, PLoS allows for post-publication commenting, but it has not eliminated pre-publication review. They close the paragraph with the intimation that not having pre-publication review would damage the scientific integrity of the journal. I find that fascinating. Perhaps elimination of pre-publication review would lead to a barrage of LOLCats submissions and INTERWEBZ WARZ!! I, for one, would enjoy seeing PhysioProf’s recipes paired next to offerings from the Journal of Applied Physiology’s own Wine Wizard, but I digress…
Raff and Brown go on to acknowledge that not everyone is content with the pre-publication review process, with some finding it uncivil and overly demanding. But, it’s interesting how they then make the case against adulterating pre-publication peer review, They write:
To reduce the types of inappropriate behavior mentioned above, it has been proposed that revealing reviewer identities to authors might be helpful (3). It has also been suggested that removing author names from submitted manuscripts (and for that matter, grant applications) may improve the process. The APS Publications Committee has discussed and rejected both of these tactics and disagrees with the idea that they would improve peer review. This decision has been confirmed consistently by several thorough scientific analyses that failed to find a beneficial effect of blinding reviewers to authorship or of asking reviewers to sign their reviews (4, 8, 14, 15). It is virtually impossible to disguise authorship of well-known scientists within an area of focus (8), and many highly qualified and sought-after reviewers would not agree to participate in the process if they could not maintain anonymity (15).
Their decisions are steeped in data. The dismissal of post-publication peer review?
Most importantly, they imply in titling their article “civil, sensible and constructive” that open peer review would be neither of those things. I know of no data to suggest that post-publication peer review is necessarily any less “civil” than pre-publication peer review. I find the reluctance to consider web-based scientific discourse to be inherently unscientific and usually rooted in individual bias and discomfort. I see no need to consider this as a black or white issue. Post-publication review and discussion can happen without eliminating pre-publication review. It’s been happening for centuries in our lab meetings, pubs and bars, and scientific meetings. The internet is an extension of these places.
Importantly, whether the journals come on board or not, people are using the internet as a forum to discuss science. Whether the editors of our favorite scientific journals want to acknowledge it or now. I’m not advocating that we go to the extreme of eliminating pre-publication review, but the journals need to decide whether they want a hand in the internet discourse. Do they want these conversations happening on their websites, in their sandboxes, or are they happy seeing them (or, probably not seeing them, if I know many of my colleagues) happen in the distances of the internet? Does the internet enhance the discussion in any way, or is it really best to keep driving the ship the way they have for the last century? I hope that my colleagues will consider this question in the future, as they have others – thoughtfully and evidence-based.