Editorial behaviors in peer review

Wang et al. SpringerPlus (2016) 5:903 DOI 10.1186/s40064-016-2601-y Open Access RESEARCH Editorial behaviors in peer review Wei Wang1, Xiangjie Kong1*, Jun Zhang1, Zhen Chen1, Feng Xia1 and Xianwen Wang2 *Correspondence: 1 School of Software, Dalian University of Technology, Dalian 116621, China Full list of author information is available at the end of the article Abstract Editors play a critical role in the peer review system. How do editorial behaviors affect the performance of peer review? No quantitative model to date allows us to measure the influence of editorial behaviors on different peer review stages such as, manuscript distribution and final decision making. Here, we propose an agent-based model in which the process of peer review is guided mainly by the social interactions among three kinds of agents representing authors, editors and reviewers respectively. We apply this model to analyze a number of editorial behaviors such as decision strategy, number of reviewers and editorial bias on peer review. We find out that peer review outcomes are significantly sensitive to different editorial behaviors. With a small fraction (10 %) of biased editors, the quality of accepted papers declines 11 %, which indicates that effects of editorial biased behavior is worse than that of biased reviewers (7 %). While several peer review models exist, this is the first account for the study of editorial behaviors that is validated on the basis of simulation analysis. Keywords: Referee, Editor, Agent-based model Background The peer review system is a cornerstone of scientific research enterprise. The development of science in the last century is an partial endorsement of the value of peer review (Alberts et al. 2008). Unfortunately, peer review systems have recently been under severe strain potentially contributing to cases of misconduct and fraud, such as the stem cell scandal in Science (Crocker and Cooper 2011) or the more recent open-access journals investigation by Plunk (2013). Some of the potential drawbacks result from the fact that the peer review system is a complex social interaction where scientists interact in various roles as journal editors, authors and reviewers in a decentralised, scarcely transparent and relatively unregulated system. Thus, the different behaviours (e.g., positive and negative) of scientists may result in unpredictable collective outcomes in terms of quality and fairness of the reviewing process (Martins 2013). Meanwhile, peer review is a cooperation dilemma that lacks transparency and reputational incentives/sanctions. This creates conditions for self-interest behaviours (Xiao et al. 2014). The main challenge is how to measure the influence of these various social interactions on peer review and then modify the existing systems in order to minimize the negative and bad effects. Generally, most scientists and journal editors have opinions on how to improve this system, nevertheless it is ambiguous to distinguish which method would be most effective without performing large scale experiments. A large body of research © 2016 The Author(s). This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Wang et al. SpringerPlus (2016) 5:903 have been done on analyzing the validity and reliability of peer review (Benos et al. 2007; Hojat et al. 2003). The pitfalls, bias, and ethics of peer review had been discussed (Lee et al. 2013; Souder 2011). Meanwhile, to study and optimize peer review, many empirical analysis and statistical approaches have been proposed (Bornmann 2011; Gallo et al. 2014; Petchey et al. 2014). Additionally, examining the quality of various social interactions through empirical analysis from a system level is a tedious and context-dependent task (Edmonds et al. 2011). Some researchers begin to simulate the social process of peer review from a modeling perspective (Squazzoni and Takács 2011), so that approximate measures of the phenomenon can be manipulated. Recent articles, such as Paolucci and Grimaldo (2014), demonstrated that peer review should be viewed as a complex social interaction problem which requires simulations of social systems to investigate it. Social simulations can help to explore the relevance of social interactions and scientists’ behaviours to better understand how peer review systems work. More importantly, social simulations, especially the agent-based model in this paper, can be used to test various scenarios under specific circumstances. A typical example of applying agent-based modeling to simulate science was Gilbert’s model (Gilbert 1997) which succeeds in designing a specialty structure with ‘areas’ of science displaying growth and decline. An influential simulation of the social factors of peer review was reported by Thurner and Hanel (2011), where the authors studied the effect of rational reviewers, who might not want to see high quality work better than their own published or promoted, with an agent-based model. They found out that a small fraction of incorrect reviewers is sufficient to drastically lower the quality of the accepted publications. They showed how a simple quality-increasing policy can lead to additional loss in overall scientific quality. The same model was applied in Roebber and Schultz (2011), where the authors focused on funding requests instead of peer review of papers. In Squazzoni and Gandelli (2012), the authors investigated whether the quality and efficiency of peer review is more influenced by scientists’ behaviour or by the type of scientific community structure (homogeneous vs. heterogeneous). They modeled peer review as a process based on knowledge asymmetries and subject to evaluation bias. They also analyzed the reciprocity behaviour in peer review and found out that reciprocity can have a positive effect on peer review only when reviewers are not driven by selfinterest motivation and are inspired by standards of fairness. Based on this work, they further studied the mechanisms of peer review in Squazzoni and Gandelli (2013). Previous modeling approaches on peer review were mostly designed from author– reviewer (Cabotà et al. 2014) or author–reviewer–conference perspective (Allesina 2012), where they simply overlook the impact of editors’ behaviours. However, there is no doubt that editors play an important role during the whole review process. It is known to all that editors are the bridges between authors and reviewers and so play a gate-keeping role. At the very beginning editors decide how to build the reviewer community and how to assign submitted papers to specific reviewers. After reviewing, the final decisions are also made (...truncated)