UK enquiry into peer review Part I: effectiveness of standard peer review
Posted Jun 01 2011 1:26pm
The UK House of Commons Select Committee on Science and Technology launched an enquiry on peer review earlier this year, and invited Faculty of 1000 ( F1000 ) to take part. The real rationale for holding this enquiry is still somewhat of a mystery to many of us who were invited to present but it appears to have stemmed from the original enquiry into Open Access about 7 years ago. Ninety-five pieces of written evidence have been submitted to-date and a few of those submitters, including F1000, were then invited to provide oral evidence before the committee.
The standard opening question on these oral sessions seemed to be:
‘Peer review is perceived to be “critical to effective scholarly communication”. If it disappeared tomorrow, what would be the consequences?’
to which most answered that another version of peer review would just be developed instead to replace it. Everyone agreed that peer review serves an important purpose and while it does indeed miss many things (it’s not great at picking up fraudulent behaviour for one), we would be worse off without it.
On the issue of dealing with fraudulent behaviour, Dr Elizabeth Wager, Chair of COPE (Committee on Publication Ethics) and Board Member of the UK Research Integrity Office suggested that:
‘You should say, “Don’t go to a university that hasn’t had at least one person fired for misconduct, because it means they are not looking for it properly”.’
This was later put by the committee to Prof Teresa Rees, former Pro Vice Chancellor (Research) at Cardiff University, and Prof Ian Walmsley, Pro Vice Chancellor at Oxford University who both said they were not aware of anyone ever being fired at their universities for fraudulent behaviour. They clarified that they have their own processes which deal with similar issues in other ways and that their internal processes are reasonably robust, and more robust now than they have been. This of course led to the accusation from the Committee Chair, MP Andrew Miller:
‘Hang on a minute. You have not come across cases of fraud. How do you know that the processes in place to deal with them are robust?’
There was also agreement that the concept of cascading peer review was a good one. When practiced within publishers between journals (e.g. at the likes of Nature Publishing Group, Public Library of Science (PLoS), BioMed Central (BMC)), it works well in saving the journal, authors and reviewers time. However, when practiced between publishers (e.g. in the Neuroscience Peer Review Consortium ), the problem of competition comes in and disappointment was voiced with the uptake of the Consortium experiment. To highlight the point further, Nature Publishing Group Editor-in-Chief Dr Philip Campbell stated in his written evidence that Nature Neuroscience wouldn’t have participated in the Consortium if their main competitor Neuron (published by Cell Press at Elsevier) had joined, demonstrating the unfortunate realities of capitalism when it comes to trying to improve peer review.
The overwhelming feeling seemed to be that although standard peer review practices were not ideal and did not pick up all the problems with articles, the publishing community, institutions, funders and other key stakeholders were working on different ways to try and improve it and direct intervention from the UK Government wasn’t required to help solve the problems. Tomorrow, I’ll be posting a second blog covering the discussion that took place in our oral session around the concept of splitting peer review into the two separate constituent parts of scientific analysis versus impact assessment, and touch on some of the related issues around data sharing.
[Please note that the quotes taken from the latest evidence transcripts are still uncorrected documents and the final form of their publication have not yet been approved by the Committee]