Life as a reject (without review)

By Joern Fischer

This blog has a bit of a history of commenting on “academic life”, rather than just on the content of sustainability research (e.g. here). In line with that history, and following a series of rejected papers, I thought I’d share my latest thoughts on rejections with the rest of the world.

Before I do this, I might just highlight some of the less amusing recent experiences I’ve had with the peer review process – all of these with so-called reputable international journals:

  • One paper was rejected without review because it did not fit the scope of a journal – but that same journal had published a paper with virtually the exact same scope in the previous year (same threatened species as a focus, arguably less interesting study area, no more or better data).
  • One paper I co-authored was attacked in a response article. We had never been asked to look at the response article before it was published. Once it was published, we asked to write a response. We were then told our response could not be published, because, in fact, our original paper had not actually been within the scope of the journal. It was rejected without review.
  • One paper was rejected without review for not being global enough. The same journal later that year published a paper that addressed only a single village within the same study area we had focused on (we had worked in 30 villages, mapped out implications to an area encompassing over 300 villages, and of course also embedded all this in a global discourse).
  • One paper was rejected by a conservation journal without review because it did not contain specific recommendations for a protected area or similar. The same journal regularly publishes papers that also do not do this.
  • Two papers were rejected without review for addressing a similar topic to another paper recently accepted. One then encouraged us to submit a response letter to the original paper instead. The content was not considered in these decisions (i.e. whether ours was better or different than the other paper previously accepted), just the topic area being the same.

Most colleagues I talk to still say that peer review is the best we have. I guess so. But can we call the editor-as-gate-keeper model peer review? The above experiences show that the judgment of single handling editors (or editors in chief) is far from consistent. It upholds the standard of individual subjectivity, more than the standard of good science.

It appears that with too much pressure on journal pages, subjectivity is increasing. Journals typically say they “reject things that are unlikely to make it through our very rigorous peer review process”. Well, those papers are probably amongst those rejected without review. But then, I would argue, there are many other papers that might well make it through peer review, but never get the chance. It used to be journals like Nature and Science that were subjective and took what was “cool”, as opposed to focusing exclusively on what was good. Now, that phenomenon is probably down journals with an impact factor of about 3, with great amounts of inconsistency within editorial boards as to what qualifies as “cool enough” to let through in the first place.

There are two ways of being an editor: the editor-as-gate-keeper model versus the editor-as-wise-arbitrator model. In the latter, the editor seeks advice by trusted experts, and weighs their arguments. In the former case (now much more common), the editor seeks reasons to reject, and primarily trusts her (or more commonly, his) instincts to know what is good science.

The vast amount of science I see is “reasonable”, and this is why PLoS One (for example) accepts two thirds of papers. Most papers submitted to good journals are alright. But far more get rejected, increasingly because editors seem to believe in one kind of “right science” (= theirs).

Yes, pressure on pages is real, and so I don’t have a simple solution for this either. But one thing is for sure: the current system is broken, wastes people’s time, sucks motivation out of graduate students, and is highly subjective. Perhaps Peerage of Science is one of the more promising options for the future?

Advertisements

9 thoughts on “Life as a reject (without review)

  1. A blog post like this is long overdue! Rather than just ‘in house’ rants that barely echo past the lab and office doors. Thank you!
    Yes, the system is broken. But we need alternatives that not only publishers but also the writers will agree on.
    While were at it, let’s overturn the precious ‘impact factor’, too.

  2. Hi Joern – I completely agree with your comments but would add that it’s possible to challenge editorial decisions, though sometimes it works and sometimes it doesn’t. I would offer the following two examples:

    – an editor at a prominent journal recently rejected an experimental manipulation study of a plant-pollinator network as it was unreplicated. When I pointed out that the previous month the same journal had published an unreplicated observational study of a plant-pollinator network his response was that “observational studies are different to experimental studies”. I argued that this was nonsense but he would not shift his position. This was partly responsible for Brian McGill’s recent post on the Dynamic Ecology blog: http://dynamicecology.wordpress.com/2014/04/14/does-good-science-require-replication/

    – an editor at another international journal turned down a manuscript because a reviewer, in about 6 lines, said that the manuscript was “uninteresting” and no one would read it. When I challenged this the editor sent it out for further review and it was subsequently accepted and published in 2011. Since then it’s been listed as “Highly Cited” by Web of Science (137 citations according to WoS; 223 on Google Scholar). So much for “uninteresting”….

    So editors sometimes make poor decisions, but sometimes are willing to concede that they are wrong when a sound argument is made.

    Jeff

    • I had an interesting exchange with an EIC of a journal in which they suggested our study was too small scale for their journal, in terms of the size of the study area. I pointed out that our study area was larger than the study area of a paper currently ‘in press’ in the journal, a paper that was in fact co-authored by the EIC. It worked 😉

      • That’s a nice story Dale! I pointed out to the editor I was dealing with that the Large Hadron Collider was also an unreplicated experiment but he wasn’t convinced 🙂

  3. Thanks Joern – We have recently experienced similar paper rejection.

    We submitted one paper to the appropriate journal in May, 2014. Our paper is trying to show the impacts of land-use change on sacred and non-sacred forests in Africa. It characterizes the general land-use change surrounding these forests within 15 years. Then, analyse past fragmentation effect on selected sacred and non-sacred forest patches by comparing changes in shape, edge density and forest size between these two forest types. Our core objective is to see whether the sacred forests are suffered lower rate of area loss because of their sacred status as compared to the non-sacred forests and to indicate the relative importance of sacred forests in biodiversity conservation.

    We followed the format required by the journal during the submission. After two weeks later, the Editor-in-Chief sent us back without reviewing the paper stating that the paper would be better suited to a different journal that focuses on the geographic region or the specific research topic of our study.

    Then, we again submitted our paper to other journal we think that is appropriate for our bits. The second journal Editor wanted to transfer the paper without reviewing to one of the journals within his network and responded to our submission to accept the transfer services or decline. We have accepted it because it saves the time we spend formatting the paper for the new journal. What infuriating us is not the rejection of the paper as it is becoming a normal but reformatting of the paper to submit to other journals as each journal has its own format. It also sometimes requires rewriting part of the paper to reduce the words so that it fits, as you said, to the maximum page as well as word limit.

    So, if the paper keeps bouncing back and forth between Editors and Authors, it is really discouraging. I think Peerage Science, as you said, may be an option.

    Desalegn

  4. Great post Joern!

    The pressure on journal page space is an illusion, because many journals are now online, all should be, and even print-based journals have found ways around their page limits (Science’s “enhanced abstract + online article” format is a good example).

    I think the more binding constraint for good journals is reviewers. Good reviewers are hard to come by, and need to be protected from less-relevant, less-interesting, or poorly-written manuscripts. Some journals explicitly tell editors that if more than x referees refuse to review the paper (based often solely on the author list and the title), it should be returned to the authors as insufficiently interesting.

  5. Pingback: Paper recommendation: Rejecting Editorial Rejections Revisited: Are Editors of Ecological Journals Good Oracles? | Ideas for Sustainability

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s