Full title: evaluating AI guidelines in leading family medicine journals: a cross-sectional study

O’Brien et al. BMC Primary Care (2025) 26:368 https://doi.org/10.1186/s12875-025-03044-0 BMC Primary Care Open Access RESEARCH Full title: evaluating AI guidelines in leading family medicine journals: a cross-sectional study Cameron O’Brien1*, Zohaib Thayani1, Tim Smith1, Andrew V. Tran1, Patrick Crotty1, Alec Young1, Alicia Ito Ford1,2 and Matt Vassar1,2 Abstract Background Artificial intelligence (AI) is increasingly integrated into family medicine research and practice, enhancing diagnostics, data analysis, and care delivery. Yet, its rapid adoption has outpaced the development of standardized editorial policies, raising concerns about transparency, ethics, and reproducibility. Clear guidance from journals is urgently needed to ensure responsible use of AI in research and publishing. Objective To evaluate editorial policies and reporting guideline endorsements related to AI across leading FM journals. Methods Using the SCImago Journal Rank database, we conducted a cross-sectional analysis of FM journals. From November 2024 to January 2025, we reviewed publicly available Instructions for Authors for AI-related policies, including authorship, manuscript writing, content/image generation, and disclosure. We also assessed whether journals endorsed AI-specific RGs (e.g., CONSORT-AI, SPIRIT-AI). Data were extracted in duplicate using a standardized form. Reproducibility was supported through protocol registration on Open Science Framework. Results Of 57 FM journals identified, 40 met inclusion criteria. Among these, 82.5% (33/40) referenced AI in their policies. Most (77.5%) prohibited AI authorship and required disclosure of AI use, while 72.5% permitted AI-assisted manuscript writing. Policies on AI-generated content and images varied, with 47.5% and 50.0% of journals allowing their use, respectively. Only 5.0% (2/40) endorsed AI-specific RGs. No correlation was observed between journal characteristics and AI policy adoption. Conclusions Most family medicine journals now address AI use, but notable gaps remain, particularly in endorsing AI-specific reporting guidelines. Without broader adoption of structured guidance, AI-integrated research risks inconsistency, limited reproducibility, and ethical challenges. Strengthening journal policies and endorsing standardized reporting frameworks is essential to ensure high-quality, trustworthy AI research in family medicine. Keywords Artificial intelligence, Family medicine, Editorial policies, Reporting guidelines, AI journal policies *Correspondence: Cameron O’Brien 1 Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17th St., Tulsa, OK 74107, USA 2 Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA © The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creati vecommons.org/licenses/by-nc-nd/4.0/. O’Brien et al. BMC Primary Care (2025) 26:368 Introduction In the field of family medicine (FM), Artificial Intelligence (AI) has become an increasingly valuable resource, driving advancements in both clinical practice and research. Clinically, AI has been used to improve diagnostic accuracy, predict patient outcomes, and personalize treatment plans [1]. Additionally, AI tools are increasingly used in chronic disease management, telemedicine, and preventive care, improving efficiency and outcomes [2]. In FM research, AI aids large dataset analysis, automates data extraction for reviews, and supports predictive modeling of disease risk and treatment outcomes [3, 4]. However, as AI becomes embedded in research, it raises challenges around transparency, ethics, and reproducibility [5, 6]. These challenges include, but are not limited to, inadequate disclosure of AI use in study design or writing, hallucinated or fabricated outputs that risk undermining scientific integrity, algorithmic bias that can perpetuate inequities in research and publishing, and barriers to reproducibility stemming from limited access to code, data, and model parameters [7, 8]. Additionally, ethical tensions remain in balancing accuracy, fairness, explainability, and privacy when implementing AI tools in research workflows [5, 6, 9]. Our study’s primary aim was to examine the Instructions for Authors in leading FM journals to determine how they address key policy areas. Specifically, we assessed whether journals allow AI to be credited as an author, what limitations are placed on AI-assisted writing, content, and image generation, and whether disclosure of AI use is required. This work examines policies addressing the use of generative AI tools (e.g., text, content, and image generation) within research and publishing. While we recognize that ‘AI’ encompasses a broader range of applications, including predictive modeling and decision-support algorithms, our analysis is limited to editorial policies and reporting guidelines relevant to generative AI and AI-integrated research methods. The second focus of our study extends beyond editorial policies to the methodological rigor of AI-integrated studies. Guidelines such as CONSORT-AI (Consolidated Standards of Reporting Trials involving Artificial Intelligence) and SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials involving Artificial Intelligence) promote transparency, reproducibility, and ethical standards by offering structured frameworks for reporting AI methods [10]. However, it is unclear to what extent FM journals reference or require these reporting guidelines. Accordingly, this study was designed to answer two questions: [1] Do FM journals have explicit editorial policies regarding the use of generative AI tools in authorship, manuscript writing, content and image generation, and disclosure? and [2] Do they endorse AI-specific Page 2 of 9 reporting guidelines, such as CONSORT-AI and SPIRITAI, that support the methodological rigor and transparency of AI-integrated studies? Several recent meta-research studies have assessed the ex (...truncated)