|Year : 2015 | Volume
| Issue : 8 | Page : 465-470
Scholarly peer reviewing': The art, its joys and woes
Madhuri S Kurdi
Department of Anaesthesiology, Karnataka Institute of Medical Sciences, Hubli, Karnataka, India
|Date of Web Publication||17-Aug-2015|
Madhuri S Kurdi
Department of Anaesthesiology, Karnataka Institute of Medical Sciences, Hubli, Karnataka
Source of Support: None, Conflict of Interest: None
Research and publications are an important part of academics. Nowadays, there is an increasing trend amongst professionals including anaesthesiologists to submit scientific articles to journals for publication. Most journals are peer-reviewed which means that the articles they publish go through the peer review process. Peer review is carried out for assessing the inadequacies of research and manuscript preparation so that the best papers are published in a journal. Although peer review is a key part of the process for the publishing of medical research, there are some limitations in the system. Keeping this in mind, all aspects of peer reviewing were searched from books and journals for full text from PubMed and Google search. The information so gathered is presented in this article which focuses on the general aspects of the peer review process.
Keywords: Editorial, journal article, peer review, role
|How to cite this article:|
Kurdi MS. Scholarly peer reviewing': The art, its joys and woes. Indian J Anaesth 2015;59:465-70
| Introduction|| |
Peer review is the heart of the scientific publication process and represents the critical phase based on which papers are published, academics promoted and Nobel prizes are won.  Peer is a person who is equal in ability, standing, rank or value.  Peer reviewers are experts who have knowledge, experience and have interest in the manuscript topic.  Scientific peer review is defined as the evaluation of research findings for competence, significance and originality by qualified experts.  Peer review, also known as 'refereeing', is the critical assessment of manuscripts submitted to journals by experts who are not a part of the editorial staff.  Peer reviews represent some of the most valuable and interesting reflections on other peoples' work. 
Of late, the trend to submit manuscripts to journals for publication is increasing rapidly amongst medical professionals, including anaesthesiologists. Most scholarly journals are peer-reviewed, which means that they publish articles which go through the peer review process. People have varied notions about the peer review process,  and the process has been increasingly misunderstood.  Authors may not clearly realise how and why their article was accepted or rejected after peer review. A sound knowledge of the process of peer reviewing would be beneficial to both the reviewers and authors and also would help to improve the peer review system. A literature search was performed on the topic of peer review with a focus on the meaning, types, nature, benefits and limitations of the peer review process. The information was derived from journal articles and the internet via Google Scholar using the words/phrases 'peer review', 'types of peer review', 'benefits and drawbacks of peer review' and 'the role of peer reviewers'. The literature search was performed between the years 2000 up to date. Few older articles gave insight into some basic aspects of peer reviewing including the history.
| History of Scholarly Peer Reviewing|| |
Earlier, editors of scientific journals often made publication decisions without seeking outside input.  The process of soliciting peers to evaluate scholarly work prior to publication was initiated by Henry Oldenburg, editor of the first scientific journal, 'Philosophical Transactions'.  The first peer-reviewed publication might have been the 'Medical Essays and Observations' published by the Royal Society of Edinburgh in 1731.  The preface to the first volume of this publication stated: 'Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author'.  Peer review in the systemised and institutionalised form as we know today has developed largely since the second world war partly as a response to the large increase in scientific research during this period.  Currently, peer review is accepted as a key part of the process for publishing of medical research. 
Goals of peer review
Peer review is the means through which journal editors can confer authenticity and authority upon scientific and scholarly papers.  Peers act as sentinels on the road of scientific discovery and publication.  Peer reviewers are mainly expected to provide constructive comments and suggestions (called as the 'gift from anonymous') to improve the quality and value of the manuscript.  They can also detect errors and fraud in a publication. 
| The Peer Review Process|| |
Manuscript review is an intellectual process with objective and subjective elements.  On receipt, editors first determine the overall quality and general suitability of the manuscript. The specifications and guidelines related to the technical aspects such as the word count, the sequence of manuscript subtitles and subheadings, language, correctness of citation entries in the article and the corresponding references, overall study design and methodology are assessed in technical review. If judged to be good, the manuscript is sent to peer reviewers for assessment.  The editors need to ensure that appropriate reviewers are selected.  An electronic mail invitation is sent to the reviewer with a date within which the completed review is requested (e.g., 'within 30 days').  After receiving the invitation to review, the reviewers first confirm that the scientific content of the manuscript is within their area of expertise. They may check for conflicts of interest, academic or financial, if obvious. They may accept the invitation or decline.  They go through the manuscript and submit their remarks. The journal editor thoughtfully considers the remarks of the reviewers and makes a decision about the acceptability or otherwise of the manuscript for publication. , When a paper is resubmitted after 'major revisions', it is sent to one or more of the original reviewers to get their opinion regarding the revision. This is the process of re reviewing.  The manuscript may be sent for additional revisions ('re-revision') as felt necessary any number of times, till the referee and/or the editor is satisfied. Traditionally, comments of two referees is the mandatory norm (discussed below) but the editor may decide on more based on his assessment of the manuscript and the quality of remarks received from the two. At any stage, opinion of additional reviewers may also be solicited by the editor if he/she is still not sure about the academic quality.  The reviewers' opinions are normally respected by the editor.  The reviewers merely give a recommendation. Ultimately, it is the editor who takes the decision and determines what gets published. ,
| Selection and Grading of Peer Reviewers|| |
Skill in scientific peer reviewing may be ill-defined.  The reviewer selection processes of most journals, and thus, the qualifications of their reviewers, are ill-defined. , The editor or his team members select the reviewers. A minimum of two professionals are selected on the basis of qualities like familiarity with the topic, diversity, skill with the review process, sensitivity, honesty and punctuality. , Many journals keep an electronic database of reviewers with their names and area of expertise.  Ideally, editors should monitor the performance of peer reviewers, maintain and update their database and cease to use reviewers who consistently produce poor quality, discourteous or late reviews.  Sometimes, a diverse group of reviewers is purposefully sought to gather opinions from various angles.  A study showed that the most popular reason given by reviewers for doing the work of reviewing was to play their part as members of the academic community, to enhance their reputation, to achieve fame, to increase their chance of being offered a role in the journal's editorial team, reviewer receptions at conferences etc.  Having too little experience in review work and not being good in passing critical comments are some of the reasons which people cite when not agreeing to become peer reviewers. 
Types of scholarly peer review
There can be peer reviews either pre-publication or post-publication. Pre-publication reviews include varieties like single-blind review, double-blind review and open review. 
In a single-blind review, the identity of the reviewer is not revealed to the author but the identity of the author is known to the reviewer.  Single-blind review shelters reviewers from their review consequences and helps them to detect various conflicts of interest concerning the authors.  In a double-blind review, both the reviewer and the author remain anonymous. The article file itself is blinded at submission; that is, no identity of author or institution is allowed in the file, and the same file is sent for reviews; or such details are removed before sending the file to referees. The reviewer/reviewers' comments that the author receives are compiled by the editor through the system where the reviewer identity is not available for the author. 
In an open review, the reviewer and author are known to each other.  The benefits of open peer review include transparency, accountability and giving credit to reviewers.  However, young junior reviewers may be intimidated into writing inappropriately favourable reviews of their powerful senior colleagues.  Furthermore, reviewers may become less critical, scientific standards may decline and professional relationships may suffer because of the creation of inappropriate dialogues between author and reviewer in this system.  Some studies have revealed that reviewer reports operating under an open peer review system were of higher quality than those under a closed system. , A study found that open reviews were more courteous and took longer to complete than unsigned reviews.  Open review though currently adopted by some reputable journals is still described as an experimental system. 
It is a variant of open review, in which all readers are able to review, comment on the paper and rate it on a numerical scale following publication. This can occur with or without the conventional pre-publication review.  It is an experiment designed to increase the speed of the review process.  In a study, it was found that it served as a useful supplement to formal peer review. Encouragement of instant reactions and discouragement of thoughtful review are some disadvantages of post-publication review.  Newer models of peer review have also been described  [Table 1].
Reviewers generally ask questions based on checklists (guidelines) sometimes provided by editors ,,,, [Table 2].
Effect of peer review comments on the author
Editors usually share all revisions with the author. Sometimes, certain reviews that are biased, not respectful or contain comments that focus on the author rather than the manuscript may be discredited by the editor and not sent to the author.  Bad comments can affect the author's confidence or make the author feel humiliated and get confused. Good comments may affirm the author's worth and give specific support directed at improving his/her work. Feedback from peers leads to a thoughtful reception of criticism, a search for confirmation by going to other sources, questioning and self-corrections by the authors. 
| Drawbacks of the Peer Review Process|| |
The peer review system though very important, has some imperfections and drawbacks.  Many professionals, particularly novice scholars do not possess an adequate knowledge of how to effectively critique research.  An experiment on peer review with a fictitious manuscript found that peer reviewers failed to detect some manuscript errors.  The most widely recognised failure of peer review is its inability to ensure the identification of high-quality work.
In 1796, the journal editor of 'Philosophical Transactions' rejected Edward Jenner's report of the first vaccination against smallpox.  A study found that peer reviewers often fail to detect important deficiencies in the reporting of the methods and results of randomised trials.  The responses received in a survey of a sample of scientists who were authors of highly cited articles indicated that a majority of them had faced the problems of manuscript rejection, scepticism, ignorance and incomprehension by the peer reviewers. 
The reviewer may adopt a stringent approach in an attempt to serve as the journal's gatekeeper, and this can lead to harshness in the tone and content of reviewers' comments. 
Different reviewers may offer conflicting reviews because they may expect, notice and value different qualities in an academic submission leading to problems in the editorial decision.  A common source of conflicting advice is the length of the manuscript. 
There could be nationality, language and speciality related bias.  There may be a strong bias against 'negative studies'.  Peer reviewers can suffer from intellectual suppression due to: (1) The Matthew effect (the rich get richer and the poor get poorer): The manuscripts of famous researchers have greater chances of getting published whereas less popular authors' works may get rejected. (2) Heider's assimilation-contrast theory: We experience concordant affective reactions to the ideas of persons who belong to our in-groups and discordant reactions to those who do not.  Single-blind review according to some authors encourages an unconscious bias towards prominent authors or prestigious institutions.  Gender bias is a possibility when reviewers know the identity of the author. Many studies provide evidence that double-blind review is more fair to authors from less prestigious institutions and to women authors.  Double-blind review decreases the enthusiasm of reviewers, and reduces their timeliness. It places extra burdens on the editorial team, reviewers and authors. 
It is difficult to identify and motivate high quality peer reviewers because they are increasingly busy and often find it difficult to free up time to do reviews. , Many reviewers devote considerable amounts of time and energy, frequently reviewing for multiple journals without incentive. ,, Many capable intellectuals avoid review work and if they agree to do it, they give it their last priority. Many reviewers do not finish the review process before the set deadline, thus causing a delay in the publication process.  Some reviewers take their task too seriously. They become a menace by decimating an article, or becoming quasi-authors themselves. 
Journals vary in their peer review standards. Acceptance of an article by a peer reviewed journal does not tell much about the quality or originality of the article.  Nowadays, there are many predatory journals that charge publication fees but deliberately omit the peer review process. This amounts to editorial misconduct. 
Lack of facilities like non-availability of access to full text articles on PubMed or other sources to the majority of reviewers and to some editors is one of the major limitations of the peer review process. 
| Measures to Improve the Peer Review Process|| |
- Blinding can reduce bias in the review process and encourage reviewers to give their honest appraisal.  A survey showed that 56% open respondents preferred double-blind review followed by 25% for single-blind, 13% for open and 5% for post-publication review. 
- Specialisation and formal training of young reviewers showed improvement in peer review in a randomised trial.  Reviewers of research reports should be well versed in the scientific method and statistics.  Training, ongoing appraisal and revalidation if provided to individuals who peer review randomised controlled trials can help them to improve. 
- Reviewers can be recognised and rewarded by using reviewer centric approaches like Reviewer Index, Reviewer Index Directory and Global Reviewer Index Directory. This can help produce high-quality reviewers.  Incentives like free subscription to the journal, acknowledgement in the journal and offering of discounts in author publication charges by the journal can encourage scientists for reviewing. 
- A committed peer reviewer system and use of beneficial technology by peer reviewers can overcome aberrant attitude in the authors and prevent scientific fraud. 
- Adopting the Broad Daylight Publication Model may lead to better reviews. This model includes openness at three levels-disclosing submissions and reviews, making reviewers accountable for their actions, reviewer rating by readers and opening up the editorial hierarchy for reviewers with good ratings and reputation. 
- A key to the success of the peer review process is the journal editor who must be rigorous in selecting and deselecting reviewers, be vigilant about the subjective elements of the process and ensure that it is fair. ,
| Conclusion|| |
Peer review is the best way to ensure quality control of submitted scientific material.This can be achieved by adopting different types of scholarly peer review. As with any other system, the peer review system has some imperfections. Several strategies are being tried to improve the system. Honest, timely, competent and fair work by peer reviewers combined with competent and sincere editorial supervision can ensure quality assurance of the peer review process and evolve the system into a process that produces a good scientific output.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Smith R. Peer review: A flawed process at the heart of science and journals. J R Soc Med 2006;99:178-82.
Triggle CR, Triggle DJ. What is the future of peer review? Why is there fraud in science? Is plagiarism out of control? Why do scientists do bad things? Is it all a case of: "All that is necessary for the triumph of evil is that good men do nothing"? Vasc Health Risk Manag 2007;3:39-53.
Kotur PF. How to write a scientific article for a medical journal? Indian J Anaesth 2002;46:21-5.
Benos DJ, Bashari E, Chaves JM, Gaggar A, Kapoor N, LaFrance M, et al.
The ups and downs of peer review. Adv Physiol Educ 2007;31:145-52.
International Committee of Medical Journal Editors. Uniform Requirement for Manuscripts Submitted to Biomedical Journals; 2008. Available from: http://www.icmje.org
. [Last accessed on 2015 Feb 17].
Wicherts JM, Kievit RA, Bakker M, Borsboom D. Letting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science. Front Comput Neurosci 2012;6:20.
Van der Wall EE. Peer review under review: Room for improvement? Neth Heart J 2009;17:187.
Daniel K. Einstein versus the physical review. Phys Today 2005;58:43-8.
Berkenkotter C. The power and perils of peer review. Rhetor Rev 1995;13:245-8.
Callaham ML, Tercier J. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med 2007;4:e40.
Rojewski JW, Domenico DM. The art and politics of peer review. J Career Tech Educ 2004;20:41-54.
Annesley TM. Seven reasons not to be a peer reviewer - And why these reasons are wrong. Clin Chem 2012;58:677-9.
Coelho RJ, LaForge J. Manuscript characteristics affecting reviewers′ decisions for rehabilitation counseling-related journals. J Rehabil 2000;66:4-8.
Rees M. Code of Conduct and Best Practice Guidelines for Journal Editors, 2011. Committee on Publication Ethics (COPE). Available from: http://www.publicationethics.org/
. [Last accessed on 2015 Feb 16].
Lovejoy TI, Revenson TA, France CR. Reviewing manuscripts for peer-review journals: A primer for novice and seasoned reviewers. Ann Behav Med 2011;42:1-13.
Schramm HL Jr, Miranda LE. Responding to peer review and editor′s comments. In: Jennings CA, Lauer TA, Vondracek B, editors. Scientific Communication for Natural Resource Professionals. Bethesda, Maryland: American Fisheries Society; 2012. p. 135-42. Available from: http://www.web.fisheries.org/proofs/sci
. [Last accessed on 2014 Dec 09].
Callaham M. The evaluation and training of peer reviewers. In: Godlee F, Jefferson T, editors. Peer Review in Health Sciences. 2 nd
ed. London: BMJ Books; 2003. p. 164-82.
Steinman WC, Lebeau DL, Michaels RK. A Survey of Journal Editors Regarding the Review Process for Original Clinical Research. [Abstract]; Third International Peer Review Congress; 1997, 18 September; Prague, the Czech Republic; 1997.
Patel J. Why training and specialization is needed for peer review: A case study of peer review for randomized controlled trials. BMC Med 2014;12:128.
Moylan EC, Harold S, O′Neill C, Kowalczuk MK. Open, single-blind, double-blind: Which peer review process do you prefer? BMC Pharmacol Toxicol 2014;15:55.
Hymans KC. Letter. Lancet 1996;34:132-3.
Kowalczuk M, Dudbridge F, Nanda S, Haaiman SL, Moylan EC. A comparison of the quality of reviewer reports from author-suggested reviewers and editor-suggested reviewers in journals operating on open or closed peer review models. F1000 Posters 2013;4:1252. Available from: http://f1000research.com/posters/1094564
[Last accessed on 2015 Jul 29].
Walsh E, Rooney M, Appleby L, Wilkinson G. Open peer review: A randomised controlled trial. Br J Psychiatry 2000;176:47-51.
Zellmer WA. Editorial. What editors expect of reviewers. Am J Health Syst Pharm 1977;34:819.
Winck JC, Fonseca JA, Azevedo LF, Wedzicha JA. To publish or perish: How to review a manuscript. Rev Port Pneumol 2011;17:96-103.
Yang M, Badger R, Yu Z. A comparative study of peer and teacher feedback in a Chinese EFL writing class. J Second Lang Writ 2006;15:179-200.
Williams HC. How to reply to referees′ comments when submitting manuscripts for publication. J Am Acad Dermatol 2004;51:79-83.
Seals DR, Tanaka H. Manuscript peer review: A helpful checklist for students and novice referees. Adv Physiol Educ 2000;23:52-8.
Baxt WG, Waeckerle JF, Berlin JA, Callaham ML. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med 1998;32:310-7.
Michaels D. Politicizing peer review: The Scientific perspective. In: Wagner W, Steinzor R, editors. Rescuing Science from Politics: Regulation and the Distortion of Scientific Research. New York Cambridge University Press; 2006. p. 219-38.
Hopewell S, Collins GS, Boutron I, Yu LM, Cook J, Shanyinde M, et al.
Impact of peer review on reports of randomised trials published in open peer review journals: Retrospective before and after study. BMJ 2014;349:g4145.
Campanario JM, Acedo E. Rejecting highly cited papers: The views of scientists who encounter resistance to their discoveries from other scientists. J Am Soc Inf Sci Technol 2007;58:734-43.
Arrington P. Some thoughts as changing the review process for academic journals: A personal exploration. Rhetor Rev 1995;13:249-53.
Peters D, Ceci S. Peer review practices of psychological journals: The fate of submitted articles, submitted again.Behavioral and Brain Sciences 1982;5:187-255.
Hwang K. The Editor′s Role as a Harriet Shaw Weaver. Arch Plast Surg 2014;41:109-10.
Kachewar SG, Sankaye SB. Reviewer index: A new proposal of rewarding the reviewer. Mens Sana Monogr 2013;11:274-84.
Rothwell PM, Martyn CN. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain 2000;123:1964-9.
Yankauer A. Who are the peer reviewers and how much do they review? JAMA 1990;263:1338-40.
Shelomi M. Editorial misconduct-definition, cases, and causes. Publications 2014;2:51-60.
Bajwa SJ. Unethical practices in anesthetic research and publication: Clinical impact, consequences and preventive measures. Saudi J Anaesth 2013;7:491-2.
Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: Randomised controlled trial. BMJ 2004;328:673.
Harsoor S, Gangadhar S. Fraud in anaesthetic research and publication. Indian J Anaesth 2012;56:1-3.
[Table 1], [Table 2]