Why I love the peer review system! (Or rather, how it used to be…)

By Joern Fischer

Okay, so rejections aren’t nice. It’s easy to take them personally. And so on. After you’ve published a few dozen papers, you know what it’s like: sometimes you get lucky, sometimes not. It’s all a bit random, but overall, good stuff wins.

I LOVE PEER REVIEW!

But hey, let me qualify this: I LOVE WHAT PEER REVIEW USED TO BE!

I feel more and more like the peer review system is broken. I publish mainly in reasonably reputable, international ecology journals — and I complete about one review every two weeks for one of these journals. In other words, I am neither new to this ‘game’ nor overly naive. The journals I might typically submit my papers to are (in no particular order, neither from good experience to bad, nor from high to low impact): Ecography, Ecological Applications, Oikos, Frontiers, Landscape Ecology, Journal of Applied Ecology, Conservation Letters … and so on. Those of you working in conservation biology will know the general set of journals I’m talking about.

I am disappointed, outraged, and frustrated by how random the peer review process has become. In the last few months, I have had some very constructive experiences, with superbly constructive and professional reviews. I’ve also had a range of rejections without review, with those papers later doing well in other journals. And I’ve got good papers for review, as well as absolute junk that should have never made it to review. The point is: all this by leading journals. It’s completely random.

Here is what I think the main problem is: many so-called leading journals now reject papers without peer review. I guess it is to keep the workload for reviewers under control, and I guess some kind of solution is needed for this. But essentially, I feel like functional peer review is all but dead. It’s all about which gate keepers (= editors) are on a similar wavelength, or if you’re more cynical, it’s all about who you know and which network you are part of. Once a paper goes for peer review it’s no worse than it used to be: but editors are vastly more powerful than in the past. Hence, a lot of good science is rejected out of hand — the process of reject-without-review is incredibly subjective!

So here’s an appeal to all journals actually interested in promoting the best science, as opposed to promoting their impact factor and playing games as to what’s their current ‘style’, or publishing only things that include global datasets, or who-knows-what-else they’re into: send relevant papers for review unless they are evidently ‘bad’. The model of editors as gate keepers is subjective nonsense that makes a farce of the peer review system.

In my experience, some journals are excellent and really well run — but the gap between those that are ‘professional’ and those that are all about who you know, to my mind, seems to be widening. Current impact factors don’t reflect this kind of difference.

So again … I LOVE THE PEER REVIEW SYSTEM! — or rather, what it used to be. Many of us used to whinge about peer review even before the onset of mega-subjective editor-gate-keepers. We should know better now: compared to now, the system was great when virtually nothing was rejected without review!

Advertisements

4 thoughts on “Why I love the peer review system! (Or rather, how it used to be…)

  1. Indeed… With huge respect to the editors, I would say that peer review system is better than leaving all the control on the hands of one single person. For example, if s/he is very busy, may be unpatient and reject fast some really good papers. I just speculate that manuscripts coming from ‘no name’ authors and ‘no name’ countries may be less favoured by increasingly busy and unpatient editors.

  2. Hear Hear!

    Are you aware of this attempt to challenge the system, at least the Elsevier approach to the system. Last I checked it’s gaining momentum.
    http://thecostofknowledge.com/
    http://www.nytimes.com/2012/02/14/science/researchers-boycott-elsevier-journal-publisher.html

    On a lighter note (and in relation to what Tibor just wrote), I’ve been reading Kahneman’s “Thinking fast and slow”. In there he discusses the propensity for people to shift into “default” decision making once their glucose levels drop. He cites the example of parole boards for which the vast majority of applications are rejected. When the data was examined more closely researchers found a pattern linking time since lunch/glucose levels with rejection of applications. In other words, if the default position is rejection, then the likelihood that the brain will resort to default positions increases with hunger/decision tiredness.

    Summary here (brilliant article, well worth reading): http://www.nytimes.com/2011/08/21/magazine/do-you-suffer-from-decision-fatigue.html?pagewanted=all

    I have little doubt that such random aspects of human decision making, combined with ridiculously high rejection rates, helps to explain at least some of the seemingly arbitrary decisions we keep experiencing.

  3. Hi Joern,

    Agree with almost everything in your post. I’ve also been getting a niggling feeling that Editorial powers are increasing. From own experience and from what I’ve been hearing from colleagues. On two occasions in the past year I’ve submitted positive reviews of papers aimed for publication in two well-known ecological journals. On both occasions, all other reviewers resonated my positive reviews. On both occasions the editor, for quite vague reasons, decided to override the reviewers and reject the papers.

  4. phew! I just thought you were referring to our latest article. But- there’s no decision yet. Let’s just hope the editor will read through it carefully 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s