Last year, you will not be astonished to learn, a great many peer-reviewed research papers on Covid-19 were published. Upwards of 75,000, to be exact, with another 10,000 or so available on pre-print servers, i.e. awaiting peer review (New Scientist, 16 December). Great, you might think. The brainiacs of the world rolling up their sleeves and getting down to business on a somewhat pressing matter. The world of science should be delighted.
Er, not exactly. Papers are being shunted out at the rate of 4,000 a week, and who can possibly read all that? Scientists say they’re not reading but drowning, and they want new smart tools to wade through it all (bit.ly/3bNJIGD).
But still, it’s been an unusual year, so you’d expect a tsunami of research papers, wouldn’t you? Actually, this is no different from any other year. Scientists have for a long time been decrying their ‘attention decay’ in the midst of an ever-increasing flood of academic studies (Independent, 11 March 2015). And their libraries need a lot of magazine subscriptions too. No one knows how many scientific journals there are, but several estimates point to around 30,000, with close to two million articles published each year (bit.ly/2OrH4h4).
Alright, but presumably all this stuff must get read eventually?
Again no. A 2007 study estimated that half of all academic papers are read by only three people, their author, and two journal editors (Smithsonian Magazine, 25 March 2014 ).
Yes, but all the same, it’s all worthwhile research, isn’t it?
You’d presume so. But a recent discussion with a friendly in-house editor in the field of epidemiology sheds a somewhat different light on things. It seems that much of what gets sent to high-impact journals is, basically, pretty low grade. An editor’s job is to decide which papers are worth sending out for peer review, but this editor says that 85-90% of the papers she receives effectively ‘go in the bin’, with only around 10-15% going out to peer review. Some journals save money by not bothering with an in-house sifting process, and send everything straight out to review, but this inevitably creates a bottleneck. And reviewers, it needs to be borne in mind, don’t get paid, their motivation being assumed to be keeping abreast of new developments in the field. If they have to provide detailed recommendations on papers that should have been ‘binned’ by the journal’s editor, that’s less overall incentive for the reviewer and more useful research time wasted.
That might just be one overworked and jaded view though?
Not according to psychologist Stuart Ritchie: ‘We think of science as being this objective thing that […] produces all these scientific papers, which are almost sacred things. But a lot of people don’t see how the sausage is made. […] In a lot of cases, the science is useless, not worth the paper it is written on’ (New Scientist, 19 August 2020). In his view, the review system isn’t up to much either. Even if reviewers try to check, they usually can’t get access to the raw data, so they can’t really verify what they’re reading. And on top of that, they can often guess where a paper has come from, despite the supposed anonymity, so bias can creep in.
But even so, it can’t be right that people are writing useless papers, surely? What incentive is there to do that? Are they simply incompetent? Here the in-house editor becomes especially illuminating. To paraphrase the conversation: ‘You have to understand how the system works. The way academics are judged is in terms of publications. These days, most staff in university departments don’t have tenure, they have to write funding bids to cover their wages. If they are managing research projects, they have to write even more extravagant funding bids to cover their workers’ wages as well as their own. Funding agencies get their money ultimately from the government, and they are also under the cosh to justify what they have funded. How do they do that? By promising the government ‘deliverables’. What are these deliverables? Generally, academic papers. Government bean-counters aren’t best placed to tell good papers from bad, so it becomes a matter of ‘never mind the quality, feel the width’. The upshot is that researchers pay the rent by writing vast numbers of papers, many of which will never be read. The whole thing is a funding merry-go-round, driven not by the needs of science so much as the needs of people to chase their next pay cheque.’
To make matters worse, there’s ‘a growing trend in doctoral education […] to dispense with the traditional PhD dissertation and replace it with the requirement for doctoral students to publish several articles based on their research in academic journals, in effect moving responsibility for evaluating doctoral research from university committees to journal editors and reviewers’ (bit.ly/2OwkAv6). So much for the appliance of science.
Well ok, but papers are at least reliable because they’re full of citations of other work, right? Right, but 90 percent of papers are never cited at all (Smithsonian, above), so citations tend to come disproportionately from a very few papers. Then these citations in turn spawn further citations, in an ever lengthening and uncritical chain, until the original paper can end up all but forgotten.
To give one example, low-fat diets have been a nutritional shibboleth for decades, yet when the lo-carb craze kicked off a few years ago and sent seismic shocks through the weight loss industry, some researchers actually looked again at the published research on dietary fat. It turned out that all the established ‘knowledge’ about fats in diets, and the official advice given by the US and UK for decades, stemmed from just 6 studies in the 1970s. Even those studies were heavily qualified by the researchers at the time, with some saying dietary advice based on them should never have been issued (bit.ly/2NilfzI). What everyone assumed was rock-solid science in reality stood on very shaky foundations.
Wasted effort is bad enough at the best of times, but when humans are faced with a crisis on the scale of the Covid pandemic, capitalism’s cash-fixated approach to science reveals itself as hopelessly inept. It’s a no-brainer to say that researchers should not write bad papers in the first place. But for that to happen, the pressure for funding would have to come off, so that they could stop chasing their own tails. Could capitalism do this? Unlikely. It hates unquantifiable returns. As they say about cynics, it knows the price of everything, and the value of nothing. But socialism, by making everything free, would do it at a stroke, scrap the entire byzantine funding hierarchy, from government down, and scrap the capitalist money system that engenders the whole ludicrous business. Then researchers would be able to focus on the real work of expanding our knowledge, instead of churning out sops to fulfil the next funding bid.
Paddy Shannon
No comments:
Post a Comment