From Peer Review to the Wisdom of Crowds? Open Access & Peer Review

Open Access has potentially dramatic consequences not only for the dissemination of research results, but for how they are produced and published. It is clear that for many in the OA vanguard, there is more at stake than shifting from a subscription model to either Gold or Green OA.  Developments in the wider field of scientific publishing arguably call into question the traditional model of peer review completely.

One model is PLOS (Public Library Of Science) ONE, now the world’s largest OA journal. PLOS ONE, which is online-only and run entirely on a Gold OA basis, asks reviewers to assess articles only on the validity of their methodology, not on their potential importance and impact. ‘The job of the PLOS ONE reviewer is not to decide whether the study represents a significant advance to the field, or whether additional experiments need to be performed to increase the impact, or whether it is suitable for a broad interest journal. The reviewer must simply ascertain whether the study has been performed correctly, and whether the data support the conclusions.’  As a result, PLOS ONE has a much higher acceptance rate than many scientific journals – around 70%. There is a valuable lesson here about the economics of Open Access: science OA journals work best when they are scaled up. Nor does PLOS ONE copy edit the articles it publishes, one effective way to keep its APCs at a competitive level. Fans of the PLOS ONE model – of which there are many – argue that readers make their own judgments as to the quality of the research, So, for example, in the course of time an article can be judged by the number of times it is cited.

Other journals have begun to experiment with ‘crowdsourcing’ peer review.  A few, such as BMC Medicine, publish reviewers’ names and reports as an account of the editorial process. So Open Access, for some at least, goes hand in hand with a move towards Open Evaluation.  Perhaps the most radical model of all is that of unrefereed repositories such as arXiv, which allow authors to make their manuscripts available at any point.

Where does this leave humanities journals? Dan Cohen, the influential director of the Roy Rosenzweig Center for History & New Media at George Mason University has already asked: ‘Imagine if a history journal published all intelligent, well-written, properly cited, on-topic articles and left assessment to the audience?’ Rumour has it that plans are already afoot in the UK for a PLOS-style Public Library of the Humanities.

One view, trenchantly expressed by Glen Newey on the London Review of Books’ blog, is that journals are now all-but-obsolete, and only hang on as a tool of disciplinary power and a means of ranking individuals and departments. Why not opt out of all this, stick our work on the Internet, and let people actually read it?

It is hard not to have some sympathy with Newey’s discontent with the bean-counting tendencies of British higher education. But his proposal of letting ‘everything slosh around in the paddling-pool of the net’ raises more questions than it answers. Firstly, it dispenses not only with peer review, but with editors’ curatorial function. Are we prepared to dispense with the book reviews, historiographical reviews, and discussion fora commissioned and cajoled by journal editors? Without the familiar landscape of journals, how will we find new work in our fields? Will our in-boxes collapse under the deluge of self-promoting emails drawing attention to our colleagues’ latest publications? If we let social media take the strain, will only those early career scholars whose supervisor is an energetic and influential Tweeter have their work read? It’s worth noting that PLOS also has ‘flagship journals’ such as PLOS Biology with a far higher rejection rate that PLOS ONE

Secondly, Newey – and many OA advocates – see peer review as largely serving an elitist and gate-keeping function. But journals, in the humanities at least, do much more than select fully-formed articles from a pool. In my experience as a writer and editor, peer reviewers and journal editors can help an author coax an argument out of a not-fully-cooked paper, point out crucial literatures that have been missed, and help the author explain the significance of his/her work. For early career authors in particular, the peer review and editing process should play a mentoring role. Only a very lucky few will have any sort of instruction in the uncertain craft of writing a journal article: the rest of us have to learn it on the job. And that’s before we get to the invaluable copy-editing provided by both editors and professional copy-editors. A hands-on editorial process improves the vast majority of articles considerably.

This is not to say that the peer review process could not be improved. Environmental Microbiology’s ever-amusing yearly round up of referees’ quotes shows the good, the bad, and the ugly of peer review in all its glory.  We should take very seriously voices of discontent, and proposals for improvement – for example that reviewers should sign their reviews. But I am not convinced that moving to a universal system of no or light-touch peer review would do authors – or readers – any favours.

This may be one of the areas where OA models developed in the sciences do not translate easily. Martin Coward has made some perceptive points about how the nature of humanities and social science research affects scholars’ attitudes towards copyright. Similar issues may arise in post-publication peer review. Does the nature of humanities scholarship, particularly single authorship, mean that more midwives are needed before an argument sees the light of day? Do historians have the time and inclination to carry out voluntary peer review in a timely-manner without the nudging emails of an editorial assistant? Can a PLOS ONE-style journal work in a very different academic culture, where citation is less important, and scholars read even journal articles in their field a year or more after publication?

There is only one way to answer these questions. By all means let’s try new models of academic publishing. The History Working Papers Project is an exciting experiment in open peer review. A PLOS One–style journal for the humanities and social sciences already exists: Sage Open. It seems highly likely that more will emerge in the near future. But it would be premature to declare the traditional scholarly journal dead just yet.