“Civil, Sensible, and Constructive” in Peer Review…But Why Can’t We Have That Too?

This past week an editorial by Drs. Hershel Raff and Dennis Brown titled the “Civil, sensible, and constructive peer review in APS journals” was published in the American Journal of Physiology: Regulatory, Integrative and Comparative Physiology. Dr. Brown is the editor-in-chief of Physiological Reviews and Dr. Raff is the chair of the publications committee for the American Physiological Society.

Drs. Raff and Brown address the “recent’ interest in open publishing and post-publication peer review, reiterate their perceived value of pre-publication peer review, and state their ongoing vision for the publication process.  Now, I have to disclose that I have found value in pre-publication peer review. I haven’t really had any real run-ins with Reviewer #3. I’ve certainly had some reviewers that I thought were perhaps a smidge over-zealous in their requests, but I’ve never been asked for anything unreasonable and I have found the editors to be helpful. I think that my manuscripts have come out of pre-publication peer review improved.  That said, I am also a child of the internet era and I wonder if perhaps my own beloved academic society isn’t being closed-minded about the potential of the internet, instead entrenching themselves in the soft, warm familiarity of what they know how to do best.  Allow me to prattle on, since it’s my blog and I’ll prattle if I want to…

Raff and Brown begin by acknowledging the call by some for post-publication peer review.  They write:

…some have even advocated eliminating prepublication peer review altogether (1, 7). The underlying premise of this initiative is that anything in the correct format is publishable, and that postpublication scrutiny using blogs and comments, a vox populi of sorts, will ultimately expose the truth. This was called “scholarly skywriting” almost two decades ago (5) and more recently “trial by twitter” (11). While proponents of this school of thought have increased in number and influence, the Publications leadership of the American Physiological Society (APS) continues to believe that prepublication peer review is worth the effort and cost and is critical to maintaining the scientific integrity of our publications.

I find it interesting that Raff and Brown begin by drawing such a dichotomy. The alternative to pre-publication peer review is the lawless, wild west-style world of open access, post-publication peer review. Post-publication peer review happens necessarily at the elimination of pre-publication review and the opposite of peer review is 4chan. I find this to be disingenuous. For example, PLoS allows for post-publication commenting, but it has not eliminated pre-publication review. They close the paragraph with the intimation that not having pre-publication review would damage the scientific integrity of the journal. I find that fascinating.  Perhaps elimination of pre-publication review would lead to a barrage of LOLCats submissions and INTERWEBZ WARZ!! I, for one, would enjoy seeing PhysioProf’s recipes paired next to offerings from the Journal of Applied Physiology’s own Wine Wizard, but I digress…

Raff and Brown go on to acknowledge that not everyone is content with the pre-publication review process, with some finding it uncivil and overly demanding. But, it’s interesting how they then make the case against adulterating pre-publication peer review, They write:

To reduce the types of inappropriate behavior mentioned above, it has been proposed that revealing reviewer identities to authors might be helpful (3). It has also been suggested that removing author names from submitted manuscripts (and for that matter, grant applications) may improve the process. The APS Publications Committee has discussed and rejected both of these tactics and disagrees with the idea that they would improve peer review. This decision has been confirmed consistently by several thorough scientific analyses that failed to find a beneficial effect of blinding reviewers to authorship or of asking reviewers to sign their reviews (4, 8, 14, 15). It is virtually impossible to disguise authorship of well-known scientists within an area of focus (8), and many highly qualified and sought-after reviewers would not agree to participate in the process if they could not maintain anonymity (15).

Their decisions are steeped in data. The  dismissal of post-publication peer review?

No Data

Most importantly, they imply in titling their article “civil, sensible and constructive” that open peer review would be neither of those things. I know of no data to suggest that post-publication peer review is necessarily any less “civil” than pre-publication peer review. I find the reluctance to consider web-based scientific discourse to be inherently unscientific and usually rooted in individual bias and discomfort. I see no need to consider this as a black or white issue. Post-publication review and discussion can happen without eliminating pre-publication review. It’s been happening for centuries in our lab meetings, pubs and bars, and scientific meetings. The internet is an extension of these places.

Importantly, whether the journals come on board or not, people are using the internet as a forum to discuss science. Whether the editors of our favorite scientific journals want to acknowledge it or now. I’m not advocating that we go to the extreme of eliminating pre-publication review, but the journals need to decide whether they want a hand in the internet discourse. Do they want these conversations happening on their websites, in their sandboxes, or are they happy seeing them (or, probably not seeing them, if I know many of my colleagues) happen in the distances of the internet?  Does the internet enhance the discussion in any way, or is it really best to keep driving the ship the way they have for the last century? I hope that my colleagues will consider this question in the future, as they have others – thoughtfully and evidence-based.

7 responses to ““Civil, Sensible, and Constructive” in Peer Review…But Why Can’t We Have That Too?

  1. Comradde PhysioProffe

    I’ve never been asked for anything unreasonable[.]

    This suggests that you are not reaching far enough in what journals you are submitting to and/or the breadth of the conclusions you are drawing from your results.

  2. Isis the Scientist

    No. It doesn’t.

  3. If not a single reviewer of any of your work has ever asked for something that you consider unreasonable, that is a sign that you could be shooting for higher-impact more-selective journals (which require more for what they consider a “compelling story”) and/or drawing bolder broader conclusions from your experimental results (against which reviewers will push back to at least some extent on the sufficiency of the presented results to justify conclusions of such breadth).

    Since this is all a relatively stochastic process, dependent on the happenstance of what reviewers one ends up with, the fact that you are *never* getting what you consider to be unreasonable pushback from *any* reviewers, it is strong evidence that you are being more conservative than necessary with the venues you shoot for and/or the conclusions you draw.

    Unless, of course, your goal is not to gain as much prominence for your work as possible, but rather to as easily as possible get your work published. This is certainly a possible goal, and not one that I would deign to judge.

  4. Post-pub peer review seems a waste of time, and wouldn’t work. Who would be motivated enough to read something in enough critical detail to volunteer a review if it’s already published? That said, I have been on the receiving end of what I think are snarky and inappropriate reviews (comments about how long and how many weekends the authors must have worked, etc). I’d rather that there was some way to evaluate the quality of reviewers (kinda like a credit score) and have this reflected on their records for all to see.

  5. Isis the Scientist

    There’s a big difference between unreasonable and inappropriate, nor did I say that I am not getting push back. That is as inappropriate conclusion.

  6. Eeke: “Who would be motivated enough to read something in enough critical detail to volunteer a review if it’s already published?”

    You mean like this? http://neurodojo.blogspot.com/2013/01/what-we-know-and-dont-know-about.html

    Happens on blogs all the time.

  7. ZF – yes, of course, and I appreciate these. We also see perspectives printed right in the journal that some articles are published in. However, reviews are critical evaluations by at least two people (I’ve been a single reviewer on a manuscript, or, occasionally, one of four. It’s hard to say what sort of impact this will have (without pre-publication review), except that a lot more crappy articles will make it to print.

What do you have to say about that?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s