• WebmedCentral submission to UK Parliament

Peer review

Written evidence submitted by Kamal Mahawar (PR 92)

I am a British surgeon who has closely experienced and also written on various aspect of biomedical publishing. I am also the founder of web portal WebmedCentral (www.webmedcentral.com), which is now owned by Webmed Limited, UK. I am a shareholder and director in the company. I welcome this enquiry into peer review by Science and Technology select committee of House of Commons and appreciate a chance to participate in it. The views expressed here are my own and do not necessarily reflect those of Webmed Limited UK, but I do touch upon practices adopted by WebmedCentral to illustrate my point. WebmedCentral is the first publishing platform globally to have adopted a unique approach of publishing biomedical scientific literature without any prior peer review.

For clarity of discussion, I have divided this submission into various sections, each covering an aspect of peer review that the committee wishes to investigate. It is also worth emphasising that this response focuses on peer review in scientific publishing and not on the role of peer review in grant applications. Lastly, given an opportunity, I will be pleased to present this response as oral evidence to the committee.

1. THE STRENGTHS AND WEAKNESS OF PEER REVIEW AS A QUALITY CONTROL MECHANISM FOR SCIENTISTS, PUBLISHERS AND THE PUBLIC

Quality control in scientific matters can be tricky business. It is at best a presumption that science needs a quality control mechanism. The second presumption is that review by editors and a few peers is the best way to carry this out. I have attempted to cover several important aspects of biomedical publishing which are impacted by pre publication peer review.

1.1 QUALITY ASSURANCE: It is believed that pre publication peer review provides quality assurance in biomedical literature and generates a basic level of trust and standards in science. There is a perception that without the gate keeping function of pre publication peer review, we will all be inundated with unscientific rubbish.

I believe that we are inundated with low quality science as such; it often comes in the garb of peer reviewed literature. Over the last few decades, whilst the peer review has got more and more entrenched, the number of articles published has actually increased dramatically. This mainly stems from the huge pressure on academics to publish, not just to carry out research. Prestige of institutions and those of individuals are measured by the numbers of articles published. Professional careers are built and destroyed on these numbers. In this environment, whether or not an article is ever cited does not matter. I believe that we can correct this discrepancy by making the act of publication a non issue. Removing the barrier of pre publication peer review will hence, on the contrary, improve the quality of science by moving the emphasis away from the number of publications to the quality of research

1.2 MANUSCRIPT IMPROVEMENT: Proponents of Pre publication peer review argue that it improves the quality and presentation of manuscripts. There is little doubt that peer review improves quality of manuscripts, but there is no evidence that it has to be carried out before publication and that a similar system after publication will not lead to improvement in quality of manuscripts. If manuscripts are published without prior peer review there would be little academic gains to be had by simply publishing the manuscript unless authors actually defend criticisms levelled against it, provide all the supporting data and submit revised versions if necessary. Serious authors concerned about their reputation will take these responsibilities seriously whereas casually constructed manuscripts will not survive the test of time. In current system, where publication is seen as the ultimate end point of any research, there is no incentive for reviewers and authors to engage in any meaningful post publication peer review.

1.3 PEER REVIEW IN BIOMEDICAL SCIENCES: There are those who believe that biomedical sciences needs pre publication peer review more than other disciplines of science because of its importance, subjective nature and difficulty in verifying claims. Surely these arguments also then apply to any potential reviewer trying to pass an arbitrary judgment on a manuscript. Others have said that without peer review doctors would potentially make erroneous decisions on the basis of incorrect information. This is a very naive argument as peer reviewed articles do not always give correct information either. False assurance that all peer reviewed information is correct may be more detrimental. All published literature including peer reviewed medical literature should at best be considered "debatable" and must not form the basis of any medical decision making until repeatedly verified and incorporated into "standard" practice.

1.4 PEER REVIEW AS A GATE KEEPER: It is important to differentiate pre publication peer review which is used as a gate keeper of science to post publication peer review which can be useful to generate debate around an article. Pre publication peer review is not an effective gatekeeper. Occasionally it fails to keep the bad guys out and far too often it does not let the good deserving people in. As a gate keeper, it is an arbitrary, biased, non transparent, time consuming, inefficient, non-standardized, costly, and unscientific process with no evidence base.

1.5 RESEARCH VALIDATION: The best validation for any research is its reproducibility. Peer review does not and cannot reproduce research or even ensure that it is accurate. Reviewers do neither have access to raw data nor the time to examine it. Research replication and validation will potentially become easier if scientists share their methodologies and data openly, something which internet has easily made possible today. If such a practice becomes the norm, it will have the additional advantage of discouraging fraudulent scientists.

1.6 COST OF BIOMEDICAL PUBLISHING: Even though reviewers themselves provide services for free, pre publication peer review is a costly exercise and somebody has to pay for it. A lot of the current cost of publishing comes from pre publication editorial and peer review. Moreover a large number of publishers are businesses intended to generate profits for shareholders. Broadly speaking the burden of this cost falls on the shoulders of two major bodies- libraries and funding agencies. These costs can be greatly reduced if pre publication barriers to publishing were removed and businesses focussed on profiting, not profiteering.

1.7 LOST RESEARCH: A proportion of research carried out is never published and hence lost because of peer review’s function as a gate keeper. Apart from researchers themselves, the other people who lose out in the process are funding agencies. It is incomprehensible that a funding agency will ever want a research, which it funds, not to come to public domain, whether or not peers approve of it.

1.8 PUBLIC PERCEPTIONS: It is a popular public misconception that peer reviewed science represents the truth. Scientists know this simply not to be the case and several examples can be cited to illustrate this point. Publishing articles without any prior peer review will remove this misconception and all articles will then be viewed by the scientific community as well as the public with a healthy dose of scepticism, which is what science needs to develop itself.

1.9 COLLECTIVE VS INDIVIDUAL WISDOM: Peer review places higher value on collective wisdom of editors and reviewers than those of individual scientists. It risks suppressing individual creativity and breakthroughs in research. It does not permit voicing opinion opposed to the mainstream and far more commonly forces authors to dilute their opinion and take a more community friendly stand in order to publish their work. I would like to make a broader point here that excellence does not need policing; it actually flourishes best in an open environment where views can be expressed freely without any fear.

1.10 SCIENTIFIC DEBATE: Because of undue emphasis on the act of publication alone, there is no incentive for authors or reviewers to engage in post publication peer review. If emphasis was removed from the act of publication to the merit of the manuscript, scientists will defend their work with blood and sweat. Furthermore, if generating debate is the aim of peer review, it would certainly be best done post publication in a transparent manner when people declare themselves and their interests. Pre publication peer review behind closed doors involving hidden reviewers is probably not the best way to have such a debate.

1.11 THE ROLE OF EDITORS: There is a further problem of editorial peer review in scientific literature which has nothing to do with science as such. Its main objective is to maintain journal’s prestige and financial viability. A large number of manuscripts are turned down at this stage and it is not even peer review by experts. There is no standardisation of such practices in the publishing world and practices vary from journal to journal.

1.12 THE ROLE OF REVIEWERS: Peer Review is a time consuming exercise and busy researchers who are asked to review manuscripts may not have the time to examine minute details. They are asked to decide, with limited time and no obvious rewards, the fate of a manuscript which somebody has spent weeks, months or years preparing. There is no academic or financial incentive for them to engage in the process to make this exercise more productive. As mentioned above, I believe, reviews should be carried out post publication and reviewers should be adequately compensated for it. Publishing industry needs to explore different ways of rewarding reviewers for this act of altruism.

1.13 SCIENTIFIC INTEGRITY: There is another wider point here. Pre publication peer review as a gate keeper of science relies on the integrity and knowledge of the reviewers. It however starts with a position of mistrust towards researchers. Surely scientific systems should be geared towards researchers and give them complete freedom to express them instead of being sceptic towards them and police them. Who is likely to know more about a research topic? A researcher who has spent months or years working on it or a reviewer who has no more than a few hours at the very most to decide its fate.

2. MEASURES TO STRENGTHEN PEER REVIEW

I believe that measures to strengthen peer review should be focussed at post publication peer review. Pre publication peer review as a gate keeper of science, for the reasons mentioned above, has no place in modern scientific literature. The purpose of peer review should not be to judge whether a manuscript deserves to be published, but to help improve its presentation and debate the science in it.

The purpose of peer review is only to generate a constructive debate around every piece of research and suggest ways to improve its scope, methodologies, and presentation. Post publication peer review can easily fulfil these functions. Since authors are in best position to lead and generate such debates, WebmedCentral follows a system of author driven post publication peer review when authors suggest people that should be invited to review the article. We encourage authors to invite as many scientists (reviewers) as they can and we then further reinforce this process through hundreds of Scholarly Reviewers and in near future WebmedCentral Faculties.

3. THE VALUE AND USE OF PEER REVIEWED SCIENCE ON ADVANCING AND TESTING SCIENTIFIC KNOWLEDGE

Science that has gone through peer review or peer approval is not necessarily the best science. Pre publication peer review forces scientists to tone down their controversial opinion and at its worst prevents growth of new ideas. It forces scientists to think as a group and lose their individuality. At the same time, peer review does not test any scientific knowledge at all. Reviewers do neither have the time nor are they expected to replicate results. All they do is ensure that manuscript is well written, can stand as itself and is not self contradictory. All these goals can easily be achieved through post publication peer review by engagement of scientific community.

4. THE VALUE AND USE OF PEER REVIEWED SCIENCE IN INFORMING PUBLIC DEBATE

It is science which informs and shapes public debate, irrespective of the gate keeping function of peer review. It would be wrong to believe that science would not exist without peer review, as it certainly existed even before peer review began and was in good health despite the recent undue emphasis on pre publication peer review. A vigorous post publication peer review of scientific material will potentially enable a wider and freer public debate.

5. THE EXTENT TO WHICH PEER REVIEW VARIES BETWEEN SCIENTIFIC DISCIPLINES AND BETWEEN COUNTRIES ACROSS THE WORLD

One of the problems of pre publication peer review is lack of standardisation and its tendency to pass arbitrary judgment on manuscripts. Practices vary considerably from country to country and journal to journal. Lack of transparency and uniformity with regards to editorial reviews and policies adopted by different journals further compounds the problems. What is rejected by one journal is often found suitable for publication by a second, third or fourth journal after many months of delay. Editors have the prestige of the journal and the interests of the publishers to look after.

Different scientific disciplines apply different rigour to peer review processes. As the British Medical Association’s submission to this debate exemplifies, biomedical researchers believe that their discipline needs more rigorous peer publication peer review. As I have discussed above, this does not have to be the case.

6. THE PROCESSES BY WHICH REVIEWERS WITH THE REQUISITE SKILLS AND KNOWLEDGE ARE IDENTIFIED, IN PARTICULAR AS THE VOLUME OF MULTI-DISCIPLINARY RESEARCH INCREASES

Pre publication peer review depends mainly on the editors selecting reviewers and in some cases authors suggesting reviewers. Both approaches have problems. Editor selected reviewers may be subconsciously or consciously promoting journal’s agenda and author selected reviewers may pass "biased" judgement on manuscript in the name of peer review. WebmedCentral’s method of post publication peer review relies on multiple tiers of reviewers. Our first tier of reviewers is suggested by authors themselves. We believe that authors know best who else is doing similar research and are hence in best position to choose people to generate a debate around the article. Our authors can invite as many reviewers as they like. Our second tier of reviewers comprises of scholarly reviewers who have registered with us and faculty members that we will soon invite. Thirdly our scientific team intermittently intervenes, finds out researchers who have published on similar topics in the past and then invites them to review the article.

7. THE IMPACT OF IT AND GREATER USE OF ONLINE RESOURCES ON THE PEER REVIEW PROCESS

Internet technologies have enabled publication of unlimited amounts of data making it possible to remove some of the traditional barriers to academic publishing. Unlimited publishing space has made it possible for researchers to share even the minutest details of their research with the community. Generating a post publication debate around a published article will be much easier with the full use of technology.

8. POSSIBLE ALTERNATIVES TO PEER REVIEW

There is widespread belief that pre publication peer review, ineffective and harmful as it may be, is a necessary evil. I believe time has now come for scientific community to discard pre publication peer review and concentrate on reinforcing post publication peer review. New article level metrics need to be developed to identify merit of each article and hence those of the authors. Scientific career progression should move away from number of articles published in journals with high impact factor to the true impact of one’s research. Such measures will not be easy to develop and this poses unique challenges for the scientific community in 21st century.

COMPETING INTERESTS: I am the founder of web portal www.webmedcentral.com which publishes biomedical research without prior peer review. We then encourage a post publication debate around each published work and request authors to engage in it. The portal is now owned by Webmed Limited, UK of which I am a share holder and director.

Kamal Mahawar

28 March 2011

http://www.publications.parliament.uk/pa/cm201011/cmselect/cmsctech/writev/856/m92.htm