By Joern Fischer
Readers familiar with this blog will know that I struggle with occasional frustration about modern academia — its competitiveness, its speed, its often polarised debates (see this previous entry and below that one, those citing it). Talking with two of my colleagues, Jan Hanspach and Euan Ritchie, we felt it was time to make these points somewhere prominent — where people would actually read them.
To that end, we hope we have now made a step in the right direction. We just published a letter in Trends in Ecology & Evolution, entitled “Academia’s obsession with quantity”. In it, we argue that more does not equal better; though our academic accounting systems try to make us believe that those are basically the same things. We argue for more time for reflection and personal relationships, and more balanced lifestyles for academics. Utopia? We think not.
You can download the full paper here: 2012_quantity_obsession_TREE. Comments, as always, welcome, and I’d especially encourage you to share this one with colleagues — perhaps it will foster a bit of debate in people’s institutions (which I believe would be much needed).
Ideas for Sustainability 
when # of refereed publications is the primary factor in determining tenure track there is a quantity problem…
Very good point – the confusion of quality with the quantity.
I guess that is asked individually, most if not all researchers perceive their own papers as qualitatively excellent. And if they have 50 papers per year, that probably will mean 50 excellent papers per year. More than the just 10 excellent papers of the other researchers therefore the first researcher is better-in his (and maybe others?) perception. The researcher may have serious reason to believe that its ‘mental babies’ (i.e. the papers) are of good quality. First they go through a peer review system. This creates or contributes to the illusion of good quality – if the paper pass 3-4 referees it should be good. Second, maybe the very fact that the research (which produce the papers) is funded may be an extra reason of the perceived good quality. In a world when money is less and less, if one is able to attract them it should be ‘good’, and the science produced also should be good and valuable. Third (and maybe it may be more?) the researcher (and its team) may perceive that he (they) have some very good things to say to the world.
Probably the research paper quality is also a perception problem (i.e. who perceives what and how). And also a rationality problem (i.e. in Aumann sense: a person behaviour [i.e. the publication of many papers – note TH] is rational when it is on his benefit given his best information.
I dont know what to suggest. It seems it is a selection which stabilizes this huge number of papers…i really dont know if kicking out the amount (i.e. the amount of papers and the amount of impact factors accumulated as first or corresponding authors blabla…) from the paper landscape (paperscape:))) is possible. Although I would like to be…so I agree with you. We are just too well grounded in the quantity (in each respect of life not only science…see the new buildings, modern art, music, literature – they become cheaper and more commercial…i.e. responding to the requirement of the ‘amount’)
Great post! As a young research student, the focus on quantity in academia and the cut-throat behavior that it nurtures, creates a less than appealing image. In addition, the constant need to maximize publications favors research projects with ‘concrete’ publication outputs, while risky projects that may have great potential, are ignored.
perhaps a change in the nature of publishing??? I think as we move toward a more Open Access atmosphere data can be reviewed while it is being collected (by anyone) instead of the public having to wait for numbers and methods to be doctored first. This will do many things for academia including: increase validity of results, increase breadth of results due to increased collaborative opportunities, and increase the project nature of research (a good thing).
Personally, my problem is primarily with the fact that quantity now seems to be the ONLY thing that “the system” can reliably detect (and therefore value). I do think it is useful and sensible to reward productive academics, somehow — but if the system does only this, blindly, without accounting for other important factors (that can’t be captured as simple performance criteria?) … well, then we run the risk of unintended side-effect. An open access culture may be part of what might help, indeed — but I do feel that different/other (fewer??) top-down incentives also would be necessary.
I am really happy to have found this post, and read your attached paper! As a PhD student I have been feeling this way for a while – it’s great to read that other more experienced people in the field notice it too. I just posted a blog on this idea last week (http://manuelinor.wordpress.com/2012/06/17/scientist-or-writer/) and have elaborated on the theme in previous posts too – I think this obsession with ‘quantity’ discourages some young researchers having to compete with ‘established’ prolific publishers in their field, and it also increases the communication gap between science and the non-scientific public. ‘Quantity’ doesn’t just refer to number of publications – it can also become a stumbling block with data analyses, jargon usage, title length etc. If these aspects of a paper become too convoluted, they will distance a reader, rather than enlighten them – but convolution and complexity seem to be a pre-requisite to being published today.
In ecology especially, simple ‘observational’ style science has become something we’re encouraged to steer clear of – yet, to me, this is the foundation of Ecology! To put it simplistically, how can we truly claim to understand ecological interactions without ‘observing’? In my humble opinion, modelling and meta-analysis can only predict, they can’t be certain – I appreciate there is a place for modelling in policy-making and some research, but I don’t believe it should become the normality.
Thanks again!
Hi Manu — thanks for your comment. I’m glad the paper resonated with you. I think the problem is that we are scaring some of the very best young people out of academia, by it being increasingly non-sensical. Don’t get me wrong: I expect several papers from all of my PhD students, so it’s not like I don’t believe in productivity. But as a goal in its own right? — That’s where it’s gone wrong. We should produce because otherwise we’re wasting public money. If you receive a government scholarship, you should produce, for example. BUT — there’s more to good science than output. Output does not equal outcome (ha, that’s just about Australian government speak!).
So, I think the brightest, best, and most devoted to changing the world through their thinking should be academics. If those leave though, because they think academia is a heap of competitive bullshit — then academia has failed. I think we’re very close to that being the case in an increasing number of cases.
Again, thanks for your thoughts!
Joern
PS: if you’re writing yourself, you may find this helpful: http://writingajournalarticle.wordpress.com/
I agree with you completely. I believe (quality) research should be shared – indeed, what is the point of doing research if you don’t share it with the wider community, scientific or otherwise? And I also see the embryos of Change in many people’s minds – as a whole, I think we can create a more supportive industry, rather than a rank-centric rat race. Of all professions, Science should be focussed more on education and enlightenment, rather than competitiveness!
Pingback: Addendum… « Ecology is not a dirty word
The article blames “quantity,” but “quality” can be just as harsh a mistress: Impact Factor, glamour mags, number of citations, etc.
Also, the paper asks, “Is it possible to obtain and communicate deep insights via ‘twitteresque’ research sound bites?” Twitter is not about sound bites, in my experience: it’s about conversation. And conversation does lead to insights.
Thanks Zen — I agree there is a place for short messages (like through twitter), and I agree that not all that looks like quality actually is. Moreover, how do we judge quality? I don’t have the perfect solution, either, obviously. But it seems obvious to me that we need a balance of communication styles — including the hard and fast, short papers, but also there must be a place (that is valued!) for other types of work. Increasingly, it feels to me we are only valuing the “twitteresque” — or if you like, we no longer value the long letters between friends, only their text messages. That in science, seems very unbalanced to me. You are of course welcome to disagree — thank you for engaging in the discussion!
Oh there is a very simple “measure” of quality of a paper: It is the Answer to the question “Dear reader: Was this helpful or inspiring to you in your research?”. Scientific papers should neither be produced in order to meet the body-er…paper-count desired by the institutions responsible for controlling nor to show off. Both is done to get more or at least a continuation of funding. During my research I have read lots of disappointing papers that sounded very promising in the abstract but did not convey any substantial information. At best I could get some references to more interesting papers, second best I judge what the authors were busy with until the point when the felt it was time to justify their existence by writing a paper. Only to produce a follow-on paper a half year later that looked almost the same as the first one. For most of the papers, reading them was a waste of my time – time better spent on observing, programming or whatever researches do when not writing papers.
A side note: I really liked to read older papers (pre. 2000) – they were of substantial length and degree of detail. But with research positions for young researchers being granted for 3 years or even less how is one supposed to do some thorough research work on something new and get a good paper about one’s findings published in a higher ranking journal (and have a life). I have chosen life.
Pingback: Financial dependency, time and system constrains as innovation filters? | Ideas for Sustainability
Pingback: Is it possible to “train” scientific creativity? | QuantitativeConservationBiology
Pingback: Measuring academic activity — and heads up for ESA Portland! | Ideas for Sustainability
Pingback: Caught in the adaptive cycle – reflecting about Europe | Ideas for Sustainability
There is no need to confuse quality with quantity. If one has P papers, that is a measure of quantity. Assume that these P papers receive C citations. Then if impact is defined as i = C/P, i is a measure or indicator of quality. From these, it is possible to think of P, C = iP and X = iC as zeroth, first and second order indicators of performance. I personally favour using X.
This is very interesting. Can you expand on the rationale of these metrics? From your website, it looks like something you have thought about a fair bit, and it would be interesting to hear more of your views. Thank you!
Pingback: Eyes wide open « Ecology is not a dirty word
Pingback: Self-confidence of women in science and a camel « Nothing in biology makes sense!
Your article shocked me. It shocked me in a good way. When I saw that senior researcher talk about the serious and painful problem I felt a germ of hope! Your reply to to Loyola et al. and Halme et al. “An academia beyond quantity” I found even more exited and useful, because you proposed a realistic roadmap to improve the situation. I totally agree that everyone who care about the future of science must do some work by himself too, instead of only blaiming the system.
May be one of the method to judge the quality of research is to make authors writing a short note “so what?” in a language understandable for a broad audience and publish it in a local newspaper (nowadawys newspapers publish all kind of silly things). This “so what?” note should explain the idea and the importance of particular study in a popular way, understandable for kids. This aproach can be useful for several reasons:
1. it connects science and public
2. it helps to show whether the studies are good and useful (simple language disclose the real meaning)
3.it broaden the audience that can judge research (if particulr research get responce from audience, it can be a ground for further funding. With a time public became more and more educated and envolved in a scientific discussion)
4. scientifically based news will substitute the rubbish that is very often published now in the newspapers
5. scientists will get experience to talk in understandable language
Thank you one more time for a courage of bringing the problem out of the silence!
Pingback: A science revolution… within our lifetime? | Ideas for Sustainability
Pingback: Thoughts about academia`s obsession with quantity | Ideas for Sustainability
Pingback: Life as a reject (without review) | Ideas for Sustainability
Pingback: Academia’s obsession with quantity revisited | Ideas for Sustainability
Pingback: Ten things I wish I could tell myself in advance of starting graduate school |
Pingback: Managing time and expectations | Ideas for Sustainability