My opinion
 

By Mr. Kamal K Mahawar
Corresponding Author Mr. Kamal K Mahawar
Department of General Surgery, Sunderland Royal Hospital, Kayll Road - United Kingdom SR4 7TP
Submitting Author Mr. Kamal K Mahawar
MISCELLANEOUS

Biomedical Publishing, Peer Review

Mahawar KK. Role of Peer Review in Biomedical Publishing. WebmedCentral MISCELLANEOUS 2011;2(4):WMC001863
doi: 10.9754/journal.wmc.2011.001863
No
Submitted on: 08 Apr 2011 11:22:45 PM GMT
Published on: 09 Apr 2011 06:51:31 AM GMT

My opinion


Following is a copy of my evidence submitted to the enquiry on Peer Review launched by Science and Technology committee of House of Commons of UK Parliament.
I am a British surgeon who has closely experienced and also written on various aspect of biomedical publishing. I am also the founder of web portal WebmedCentral (www.webmedcentral.com), which is now owned by Webmed Limited, UK. I am a shareholder and director in the company. I welcome this enquiry into peer review by Science and Technology select committee of House of Commons and appreciate a chance to participate in it. The views expressed here are my own and do not necessarily reflect those of Webmed Limited UK, but I do touch upon practices adopted by WebmedCentral to illustrate my point. WebmedCentral is the first publishing platform globally to have adopted a unique approach of publishing biomedical scientific literature without any prior peer review.
For clarity of discussion, I have divided this submission into various sections, each covering an aspect of peer review that the committee wishes to investigate. It is also worth emphasising that this response focuses on peer review in scientific publishing and not on the role of peer review in grant applications. Lastly, given an opportunity, I will be pleased to present this response as oral evidence to the committee.
1. THE STRENGTHS AND WEAKNESS OF PEER REVIEW AS A QUALITY CONTROL MECHANISM FOR SCIENTISTS, PUBLISHERS AND THE PUBLIC
Quality control in scientific matters can be tricky business. It is at best a presumption that science needs a quality control mechanism. The second presumption is that review by editors and a few peers is the best way to carry this out. I have attempted to cover several important aspects of biomedical publishing which are impacted by pre publication peer review.
1.1 QUALITY ASSURANCE: It is believed that pre publication peer review provides quality assurance in biomedical literature and generates a basic level of trust and standards in science. There is a perception that without the gate keeping function of pre publication peer review, we will all be inundated with unscientific rubbish.
I believe that we are inundated with low quality science as such; it often comes in the garb of peer reviewed literature. Over the last few decades, whilst the peer review has got more and more entrenched, the number of articles published has actually increased dramatically. This mainly stems from the huge pressure on academics to publish, not just to carry out research. Prestige of institutions and those of individuals are measured by the numbers of articles published. Professional careers are built and destroyed on these numbers. In this environment, whether or not an article is ever cited does not matter. I believe that we can correct this discrepancy by making the act of publication a non issue. Removing the barrier of pre publication peer review will hence, on the contrary, improve the quality of science by moving the emphasis away from the number of publications to the quality of research.
1.2 MANUSCRIPT IMPROVEMENT: Proponents of Pre publication peer review argue that it improves the quality and presentation of manuscripts. There is little doubt that peer review improves quality of manuscripts, but there is no evidence that it has to be carried out before publication and that a similar system after publication will not lead to improvement in quality of manuscripts. If manuscripts are published without prior peer review there would be little academic gains to be had by simply publishing the manuscript unless authors actually defend criticisms levelled against it, provide all the supporting data and submit revised versions if necessary. Serious authors concerned about their reputation will take these responsibilities seriously whereas casually constructed manuscripts will not survive the test of time. In current system, where publication is seen as the ultimate end point of any research, there is no incentive for reviewers and authors to engage in any meaningful post publication peer review.
1.3 PEER REVIEW IN BIOMEDICAL SCIENCES: There are those who believe that biomedical sciences needs pre publication peer review more than other disciplines of science because of its importance, subjective nature and difficulty in verifying claims. Surely these arguments also then apply to any potential reviewer trying to pass an arbitrary judgment on a manuscript. Others have said that without peer review doctors would potentially make erroneous decisions on the basis of incorrect information. This is a very naive argument as peer reviewed articles do not always give correct information either. False assurance that all peer reviewed information is correct may be more detrimental. All published literature including peer reviewed medical literature should at best be considered "debatable" and must not form the basis of any medical decision making until repeatedly verified and incorporated into "standard" practice.
1.4 PEER REVIEW AS A GATE KEEPER: It is important to differentiate pre publication peer review which is used as a gate keeper of science to post publication peer review which can be useful to generate debate around an article. Pre publication peer review is not an effective gatekeeper. Occasionally it fails to keep the bad guys out and far too often it does not let the good deserving people in. As a gate keeper, it is an arbitrary, biased, non transparent, time consuming, inefficient, non-standardized, costly, and unscientific process with no evidence base.
1.5 RESEARCH VALIDATION: The best validation for any research is its reproducibility. Peer review does not and cannot reproduce research or even ensure that it is accurate. Reviewers do neither have access to raw data nor the time to examine it. Research replication and validation will potentially become easier if scientists share their methodologies and data openly, something which internet has easily made possible today. If such a practice becomes the norm, it will have the additional advantage of discouraging fraudulent scientists.
1.6 COST OF BIOMEDICAL PUBLISHING: Even though reviewers themselves provide services for free, pre publication peer review is a costly exercise and somebody has to pay for it. A lot of the current cost of publishing comes from pre publication editorial and peer review. Moreover a large number of publishers are businesses intended to generate profits for shareholders. Broadly speaking the burden of this cost falls on the shoulders of two major bodies- libraries and funding agencies. These costs can be greatly reduced if pre publication barriers to publishing were removed and businesses focussed on profiting, not profiteering.
1.7 LOST RESEARCH: A proportion of research carried out is never published and hence lost because
of peer review’s function as a gate keeper. Apart from researchers themselves, the other people who lose out in the process are funding agencies. It is incomprehensible that a funding agency will ever want a research, which it funds, not to come to public domain, whether or not peers approve of it.
1.8 PUBLIC PERCEPTIONS: It is a popular public misconception that peer reviewed science represents the truth. Scientists know this simply not to be the case and several examples can be cited to illustrate this point. Publishing articles without any prior peer review will remove this misconception and all articles will then be viewed by the scientific community as well as the public with a healthy dose of scepticism, which is what science needs to develop itself.
1.9 COLLECTIVE VS INDIVIDUAL WISDOM: Peer review places higher value on collective wisdom of editors and reviewers than those of individual scientists. It risks suppressing individual creativity and breakthroughs in research. It does not permit voicing opinion opposed to the mainstream and far more commonly forces authors to dilute their opinion and take a more community friendly stand in order to publish their work. I would like to make a broader point here that excellence does not need policing; it actually flourishes best in an open environment where views can be expressed freely without any fear.
1.10 SCIENTIFIC DEBATE: Because of undue emphasis on the act of publication alone, there is no incentive for authors or reviewers to engage in post publication peer review. If emphasis was removed from the act of publication to the merit of the manuscript, scientists will defend their work with blood and sweat. Furthermore, if generating debate is the aim of peer review, it would certainly be best done post publication in a transparent manner when people declare themselves and their interests. Pre publication peer review behind closed doors involving hidden reviewers is probably not the best way to have such a debate.
1.11 THE ROLE OF EDITORS: There is a further problem of editorial peer review in scientific literature which has nothing to do with science as such. Its main objective is to maintain journal’s prestige and financial viability. A large number of manuscripts are turned down at this stage and it is not even peer review by experts. There is no standardisation of such practices in the publishing world and practices vary from journal to journal.
1.12 THE ROLE OF REVIEWERS: Peer Review is a time consuming exercise and busy researchers who are asked to review manuscripts may not have the time to examine minute details. They are asked to decide, with limited time and no obvious rewards, the fate of a manuscript which somebody has spent weeks, months or years preparing. There is no academic or financial incentive for them to engage in the process to make this exercise more productive. As mentioned above, I believe, reviews should be carried out post publication and reviewers should be adequately compensated for it. Publishing industry needs to explore different ways of rewarding reviewers for this act of altruism.
1.13 SCIENTIFIC INTEGRITY: There is another wider point here. Pre publication peer review as a gate keeper of science relies on the integrity and knowledge of the reviewers. It however starts with a position of mistrust towards researchers. Surely scientific systems should be geared towards researchers and give them complete freedom to express them instead of being sceptic towards them and police them. Who is likely to know more about a research topic? A researcher who has spent months or years working on it or a reviewer who has no more than a few hours at the very most to decide its fate.
2. MEASURES TO STRENGTHEN PEER REVIEW
I believe that measures to strengthen peer review should be focussed at post publication peer review. Pre publication peer review as a gate keeper of science, for the reasons mentioned above, has no place in modern scientific literature. The purpose of peer review should not be to judge whether a manuscript deserves to be published, but to help improve its presentation and debate the science in it.
The purpose of peer review is only to generate a constructive debate around every piece of research and suggest ways to improve its scope, methodologies, and presentation. Post publication peer review can easily fulfil these functions. Since authors are in best position to lead and generate such debates, WebmedCentral follows a system of author driven post publication peer review when authors suggest people that should be invited to review the article. We encourage authors to invite as many scientists (reviewers) as they can and we then further reinforce this process through hundreds of Scholarly Reviewers and in near future WebmedCentral Faculties.
3. THE VALUE AND USE OF PEER REVIEWED SCIENCE ON ADVANCING AND TESTING SCIENTIFIC KNOWLEDGE
Science that has gone through peer review or peer approval is not necessarily the best science. Pre publication peer review forces scientists to tone down their controversial opinion and at its worst prevents growth of new ideas. It forces scientists to think as a group and lose their individuality. At the same time, peer review does not test any scientific knowledge at all. Reviewers do neither have the time nor are they expected to replicate results. All they do is ensure that manuscript is well written, can stand as itself and is not self contradictory. All these goals can easily be achieved through post publication peer review by engagement of scientific community.
4. THE VALUE AND USE OF PEER REVIEWED SCIENCE IN INFORMING PUBLIC DEBATE
It is science which informs and shapes public debate, irrespective of the gate keeping function of peer review. It would be wrong to believe that science would not exist without peer review, as it certainly existed even before peer review began and was in good health despite the recent undue emphasis on pre publication peer review. A vigorous post publication peer review of scientific material will potentially enable a wider and freer public debate.
5. THE EXTENT TO WHICH PEER REVIEW VARIES BETWEEN SCIENTIFIC DISCIPLINES AND BETWEEN COUNTRIES ACROSS THE WORLD
One of the problems of pre publication peer review is lack of standardisation and its tendency to pass arbitrary judgment on manuscripts. Practices vary considerably from country to country and journal to journal. Lack of transparency and uniformity with regards to editorial reviews and policies adopted by different journals further compounds the problems. What is rejected by one journal is often found suitable for publication by a second, third or fourth journal after many months of delay. Editors have the prestige of the journal and the interests of the publishers to look after.
Different scientific disciplines apply different rigour to peer review processes. As the British Medical Association’s submission to this debate exemplifies, biomedical researchers believe that their discipline needs more rigorous peer publication peer review. As I have discussed above, this does not have to be the case.
6. THE PROCESSES BY WHICH REVIEWERS WITH THE REQUISITE SKILLS AND KNOWLEDGE ARE IDENTIFIED, IN PARTICULAR AS THE VOLUME OF MULTI-DISCIPLINARY RESEARCH INCREASES
Pre publication peer review depends mainly on the editors selecting reviewers and in some cases authors suggesting reviewers. Both approaches have problems. Editor selected reviewers may be subconsciously or consciously promoting journal’s agenda and author selected reviewers may pass "biased" judgement on manuscript in the name of peer review. WebmedCentral’s method of post publication peer review relies on multiple tiers of reviewers. Our first tier of reviewers is suggested by authors themselves. We believe that authors know best who else is doing similar research and are hence in best position to choose people to generate a debate around the article. Our authors can invite as many reviewers as they like. Our second tier of reviewers comprises of scholarly reviewers who have registered with us and faculty members that we will soon invite. Thirdly our scientific team intermittently intervenes, finds out researchers who have published on similar topics in the past and then invites them to review the article.
7. THE IMPACT OF IT AND GREATER USE OF ONLINE RESOURCES ON THE PEER REVIEW PROCESS
Internet technologies have enabled publication of unlimited amounts of data making it possible to remove some of the traditional barriers to academic publishing. Unlimited publishing space has made it possible for researchers to share even the minutest details of their research with the community. Generating a post publication debate around a published article will be much easier with the full use of technology.
8. POSSIBLE ALTERNATIVES TO PEER REVIEW
There is widespread belief that pre publication peer review, ineffective and harmful as it may be, is a necessary evil. I believe time has now come for scientific community to discard pre publication peer review and concentrate on reinforcing post publication peer review. New article level metrics need to be developed to identify merit of each article and hence those of the authors. Scientific career progression should move away from number of articles published in journals with high impact factor to the true impact of one’s research. Such measures will not be easy to develop and this poses unique challenges for the scientific community in 21st century.
COMPETING INTERESTS: I am the founder of web portal www.webmedcentral.com which publishes biomedical research without prior peer review. We then encourage a post publication debate around each published work and request authors to engage in it. The portal is now owned by Webmed Limited, UK of which I am a share holder and director.
Kamal Mahawar

Source(s) of Funding


none

Competing Interests


As in the article text

Disclaimer


This article has been downloaded from WebmedCentral. With our unique author driven post publication peer review, contents posted on this web portal do not undergo any prepublication peer or editorial review. It is completely the responsibility of the authors to ensure not only scientific and ethical standards of the manuscript but also its grammatical accuracy. Authors must ensure that they obtain all the necessary permissions before submitting any information that requires obtaining a consent or approval from a third party. Authors should also ensure not to submit any information which they do not have the copyright of or of which they have transferred the copyrights to a third party.
Contents on WebmedCentral are purely for biomedical researchers and scientists. They are not meant to cater to the needs of an individual patient. The web portal or any content(s) therein is neither designed to support, nor replace, the relationship that exists between a patient/site visitor and his/her physician. Your use of the WebmedCentral site and its contents is entirely at your own risk. We do not take any responsibility for any harm that you may suffer or inflict on a third person by following the contents of this website.

Reviews
3 reviews posted so far

The manuscript needs revisions
Posted by Dr. Abhijit Bal on 27 May 2011 09:01:02 AM GMT

Excellent overview of a significant problem
Posted by Dr. Devon D Brewer on 11 Apr 2011 05:15:33 AM GMT

Comments
7 comments posted so far

Stratified Levels of Publication Posted by Dr. Abu Dzarr on 14 Apr 2011 06:23:11 AM GMT

Response to Reviews and Comments Posted by Mr. Kamal K Mahawar on 27 May 2011 10:21:31 PM GMT

Response to Mr Mahawar Posted by Dr. Abhijit Bal on 28 May 2011 10:35:12 AM GMT

post-publication peer review Posted by Dr. PRATIK PATEL on 10 Feb 2012 09:41:05 AM GMT

Please use this functionality to flag objectionable, inappropriate, inaccurate, and offensive content to WebmedCentral Team and the authors.

 

Author Comments
1 comment posted so far

Response to Dr. Carabas Posted by Mr. Kamal K Mahawar on 05 Mar 2012 03:46:25 PM GMT

 

What is article Popularity?

Article popularity is calculated by considering the scores: age of the article
Popularity = (P - 1) / (T + 2)^1.5
Where
P : points is the sum of individual scores, which includes article Views, Downloads, Reviews, Comments and their weightage

Scores   Weightage
Views Points X 1
Download Points X 2
Comment Points X 5
Review Points X 10
Points= sum(Views Points + Download Points + Comment Points + Review Points)
T : time since submission in hours.
P is subtracted by 1 to negate submitter's vote.
Age factor is (time since submission in hours plus two) to the power of 1.5.factor.

How Article Quality Works?

For each article Authors/Readers, Reviewers and WMC Editors can review/rate the articles. These ratings are used to determine Feedback Scores.

In most cases, article receive ratings in the range of 0 to 10. We calculate average of all the ratings and consider it as article quality.

Quality=Average(Authors/Readers Ratings + Reviewers Ratings + WMC Editor Ratings)