Friend or foe? Exploring the implications of large language models on the science system

AI & SOCIETY https://doi.org/10.1007/s00146-023-01791-1 MAIN PAPER Friend or foe? Exploring the implications of large language models on the science system Benedikt Fecher1,2 · Marcel Hebing1,3 · Melissa Laufer1 · Jörg Pohle1 · Fabian Sofsky1 Received: 3 August 2023 / Accepted: 26 September 2023 © The Author(s) 2023 Abstract The advent of ChatGPT by OpenAI has prompted extensive discourse on its potential implications for science and higher education. While the impact on education has been a primary focus, there is limited empirical research on the effects of large language models (LLMs) and LLM-based chatbots on science and scientific practice. To investigate this further, we conducted a Delphi study involving 72 researchers specializing in AI and digitization. The study focused on applications and limitations of LLMs, their effects on the science system, ethical and legal considerations, and the required competencies for their effective use. Our findings highlight the transformative potential of LLMs in science, particularly in administrative, creative, and analytical tasks. However, risks related to bias, misinformation, and quality assurance need to be addressed through proactive regulation and science education. This research contributes to informed discussions on the impact of generative AI in science and helps identify areas for future action. Keywords Large language models · Science system · Delphi study · Scholarly communication 1 Introduction The release of ChatGPT by OpenAI in November 2022 has sparked a plethora of editorials, position papers and essays, or interviews with experts, as well as some articles and preprints on the potential impacts on science and higher education. While many concerns raised relate to how ChatGPT and large language models (LLMs) will change education * Benedikt Fecher Marcel Hebing Melissa Laufer Jörg Pohle Fabian Sofsky 1 Alexander von Humboldt Institute for Internet and Society, Berlin, Germany 2 Wissenschaft im Dialog, Berlin, Germany 3 DBU Digital Business University of Applied Sciences, Berlin, Germany (e.g., Perkins 2023; Fyfe 2023), there is much less—especially empirical research—on the implications of LLMs as well as LLM-based chatbots or prompts on scholarly practices and the science system, which we understand as a collective body of all academic disciplines, including the sciences and humanities (Ribeiro et al. 2023; Chubb et al. 2022). One can, however, draw inspiration from fields that are also characterized by largely text-based or -focused, creative and knowledge work. For instance, the opinion paper by Dwivedi et al. (2023) provides a viewpoint on the potential impact of generative AI technologies such as ChatGPT in the domains of education, business, and society, based on 43 contributions by AI experts from various disciplines. However, the literature on knowledge work and the transformative effects of AI cannot account for the complexities of specific practices (Jiang et al. 2022). In light of the limited research conducted on LLMs and their impact on the science system and scientific practice, we initiated a Delphi study involving experts who specialize in the intersection of research and AI technology. The purpose of this study was to investigate the following areas: (a) the potential applications and limitations in using LLMs, (b) the positive and negative effects of LLMs on the science system, (c) the regulatory and ethical considerations associated with the use of LLMs in science, and (d) the necessary 13 Vol.:(0123456789) AI & SOCIETY competencies and capacities for effectively utilizing LLMs. Our objective in this study was to gather and structure expert opinions in an initial phase, focusing on the aforementioned categories, and subsequently evaluate and assess them in a second phase. As generative AI continues to advance, it is crucial to gather expert knowledge and informed assessments regarding its potential impact on science. This knowledge will contribute to an informed scholarly debate and help anticipate potential fields of action. Our findings indicate that experts anticipate that the utilization of LLMs will have a transformative and largely positive impact on science and scientific practice. In LLMs, they recognize significant potential for administrative, creative, and analytical tasks. The main risks associated with LLMs pertain to issues of bias, misinformation, and overburdening of the scientific quality assurance system. Despite the perceived advantages of LLMs for science, it is imperative to acknowledge and address the associated risks. This necessitates proactive measures in regulation and science education. 2 Literature review In the following, we provide an overview of the current state of the scholarly discourse along the aforementioned areas. While our aim was to present a comprehensive and contemporary overview of this discourse, it is, however, important to acknowledge that new and pertinent studies may have emerged by the time of the publication of this article. 2.1 Applications and limitations of LLMs in science LLMs and LLM-based tools are expected to have a wide range of applications in scientific practice. Possible uses for researchers identified in the literature range from generating plausible research ideas (Dowling and Lucey 2023), brainstorming (Staiman 2023), transforming notes into text (Buruk 2023), creating a first draft of a paper (Dwivedi et al. 2023), assisting with grammar and language (Flanagin et al. 2023), e.g., to improve clarity (Lund et al. 2023), especially for non-native speakers (Perkins 2023), but also stylistic issues, from formatting references to complying with editing standards (Flanagin et al. 2023; Lund et al. 2023). LLM-based tools like ChatGPT may be used to generate literature reviews (Dowling and Lucey 2023), data crunching (Staiman 2023), data summaries (Lucey and Dowling 2023), even proposing new experiments (Grimaldi and Ehrler 2023). They may support the dissemination of publications and the diffusion of knowledge by helping to create better metadata, indexing, and summaries of research findings (Lund et al. 2023). They are expected to assist editors in screening submission for issues such as plagiarism or image manipulation, triaging, 13 validating references, editing and formatting (Flanagin et al. 2023; Hosseini and Horbach 2023). Beyond scholarly writing, LLM-based tools are expected to assist with code writing, automating simple tasks, and error management (Dwivedi et al. 2023), but also in writing reports, strategy documents, emails as well as cover and rejection letters (Corless 2023). They may even be used as a replacement for human participants in psychological experiments (Dillion et al. 2023). Scientists may also use LLM-based tools for non-scholarly tasks, as a recent Nature poll has shown: while eighty per cent of respondents have used AI chatbots, more than half say they use them for ‘creative (...truncated)