How to Choose Appropriate Experts for Peer Review: An Intelligent Recommendation Method in a Big Data Context

Proceedings Papers How to Choose Appropriate Experts for Peer Review: An Intelligent Recommendation Method in a Big Data Context Authors: Duanduan Liu , School of Information, Renmin University of China, Beijing, 100872, CN X close Wei Xu, School of Information, Renmin University of China, Beijing, 100872, CN X close Wei Du, School of Information, Renmin University of China, Beijing, 100872 Department of Information Systems, City University of Hong Kong, Kowloon, CN X close Fuyin Wang School of Information, Renmin University of China, Beijing, 100872, CN X close Abstract The rapid development of the internet has led to the accumulation of massive amounts of data, and thus we find ourselves entering the age of big data. Obtaining useful information from these big data is a crucial issue. The aim of this article is to solve the problem of recommending experts to provide peer reviews for universities and other scientific research institutions. Our proposed recommendation method has two stages. An information filtering method is first offered to identify proper experts as a candidate set. Then, an aggregation model with various constraints is suggested to recommend appropriate experts for each applicant. The proposed method has been implemented in an online research community, and the results exhibit that the proposed method is more effective than existing ones. Keywords: Experts recommendation ,   Text mining ,   Integration model ,   Big data analysis   How to Cite: Liu, D. et al., (2015). How to Choose Appropriate Experts for Peer Review: An Intelligent Recommendation Method in a Big Data Context. Data Science Journal. 14, p.16. DOI: http://doi.org/10.5334/dsj-2015-016 988 Views 152 Downloads 2 Citations 5 Twitter   Published on 22 May 2015 Peer Reviewed  CC BY 4.0 1 Introduction Talent introduction is very important in the construction of a university faculty; the quality of the talent reflects the academic level of a university as well as, to some extent, its comprehensive strength. Therefore, universities must take effective measures to strictly control the quality of applicants in the process of their talent introduction. In general, universities use peer review to evaluate applicants. That is, they select a certain number of experts, usually three to five, who have similar research areas as the applicant to review their application documents. In this process, the choice of reviewers has a great impact on the assessment of applicants because appropriate reviewers will help universities to select excellent talent. On the other hand, unsuitable reviewers might result in the loss of talent. Therefore, choosing appropriate reviewers becomes a key step in finding the best talent for an institution. Currently, the most widely used expert selection method is manual. In this process, a university staff first collects applicants’ application documents and personal information and identifies applicants’ features. Then they retrieve a database of experts with high professional levels and try to find those with research areas similar to that of the applicants. Next, they send invitations to the selected experts by phone or e-mail. Finally, they determine whether the applicant is employable or not according to the expert reviewers’ opinions. At present, most colleges and universities use this method to select reviewers. There is no doubt that this is an effective way to select reviewers. However, there are obvious defects in the manual selection method. First, this selection method is time-consuming. The staff needs to retrieve expert databases and find suitable experts with similar research areas. From the list of experts, three to five names are chosen. This consumes a lot of energy and time. In particular, when the number of experts in the expert database is very large, this defect is more obvious. Furthermore, an amount of manpower and resources is needed to maintain and improve the expert database. Second, manual selection has a certain one-sidedness. For example, a staff always searches for relevant experts in terms of the discipline codes provided by the applicant and the experts although only using discipline codes to reflect research areas is not comprehensive. There is much other content that can reflect research areas, such as publications, research projects, and so on. In addition, staffs use title, rewards, and status to weigh the quality of the experts. These indicators only represent a part of their quality, and this method still needs a more comprehensive evaluation standard for the experts’ productivity. Third, in the process of manual expert selection, it is difficult to avoid subjectivity both in the retrieval process and in the weighing process. The manual method also ignores the possible relationships between experts and applicants. For example, if the expert and the applicant have any cooperative relations or other relationships, the expert will not be suitable for reviewing the applicant. In order to solve the above problems, we propose an intelligent recommendation method to recommend appropriate experts for peer review and developed a recommendation system to realize it. The proposed method has two stages in the recommendation process. In the first stage, we collected as much information as possible about experts and applicants. We obtained most of the information about the experts from an expert database provided by the university and also extracted useful information from research social network websites and the experts’ personal home pages. We obtained information about the applicants from their personal information (e.g., discipline codes, work experience, graduate school) and application documents (e.g., publications, research projects, patents). Then we created an expert profile and an applicant profile representing all their characteristics by using an information filtering method. In the second stage, we designed a recommendation model to recommend appropriate reviewers for each applicant. The recommendation model contains three modules: the relevance module, the connectivity module, and the quality module. An aggregation model was constructed to integrate these modules. Through these two stages, the most suitable list of experts was recommended for each applicant. The proposed recommendation system has been implemented, and a real survey has been taken to verify the effectiveness of the proposed intelligent recommendation method. The reminder of this article is as follows. Section 2 reviews the related literature on expert recommendation. Section 3 introduces the proposed recommendation method. In Section 4, we implement the recommendation system and verify the results. In Section 5, we conclude the article and indicate future work. 2 Literature Review The rapid development of the intern (...truncated)