Trust, trustworthiness and AI governance

www.nature.com/scientificreports OPEN Trust, trustworthiness and AI governance Christian Lahusen 1,4, Martino Maggetti 2,4 & Marija Slavkovik 3,4* An emerging issue in AI alignment is the use of artificial intelligence (AI) by public authorities, and specifically the integration of algorithmic decision-making (ADM) into core state functions. In this context, the alignment of AI with the values related to the notions of trust and trustworthiness constitutes a particularly sensitive problem from a theoretical, empirical, and normative perspective. In this paper, we offer an interdisciplinary overview of the scholarship on trust in sociology, political science, and computer science anchored in artificial intelligence. On this basis, we argue that only a coherent and comprehensive interdisciplinary approach making sense of the different properties attributed to trust and trustworthiness can convey a proper understanding of complex watchful trust dynamics in a socio-technical context. Ensuring the trustworthiness of AI-Governance ultimately requires an understanding of how to combine trust-related values while addressing machines, humans and institutions at the same time. We offer a road-map of the steps that could be taken to address the challenges identified. Artificial intelligence (AI) is now widely recognised as a socio-technical phenomenon that creates unprecedented opportunities while also possibly posing existential risks1,2. Overly positive or negative outlooks on the impact of AI tend to overestimate both the potential as well as the threat of artificial intelligence, but they are correct in highlighting the progressive incorporation of AI-based systems into all aspects of societal l ife3. This observation also applies to public authorities, which notably adopted AI because they see the merits of investing into technological innovation in order to increase the efficiency, effectiveness, and quality of services. Automation of decisions with the help of AI has thus been increasingly, and sometimes invisibly, introduced into core state activities such as public policy decision-making, internal management, and public service delivery in a wide range of areas4,5. The relationships between public authorities and citizens are particularly affected by the growing reliance on AI automation, namely algorithmic decision-making (ADM)—software tools that use AI to aid and to make automated decisions. ADM is increasingly used to manage citizens’ applications and requests, identify personalized services, predict risks in regard to service provision, specify potential sanctions, and communicate decisions. ADM systems are thus having concrete effects in several domains, especially in—but not limited to—high-income countries. For instance, automation is being applied by public authorities to identify the eligibility for children’s allowances6,7, to select available employment offers for jobseekers8, to calculate social b enefits9, to detect tax fraud10,11, and to forecast and monitor criminal behaviour12,13. The implications of these developments are considerable and need to be examined closely. In fact, public authorities are sole providers of service, particularly in sensitive areas of citizens’ lives, they have privileged access to personal data, take binding decisions, and might thus inescapably and directly affect beneficiaries and society as a whole. The magnitude of the problem is determined by the fact that the growing insertion of AI into public service provision has ushered forms of AI-Governance, i.e., highly complex and dynamic socio-technical configurations that comprise technological, institutional, and regulatory elements. Specifically, they consist of: – ADM applications developed for public authorities; – Institutional practices that define the purposes and forms of usages; and – A regulatory framework that both enables and constrains ADM deployment. Algorithms introduce a strong transformative element into these configurations, because they constitute an inscrutable, opaque system whose operations, choices, and consequences are still poorly understood. They might encourage or facilitate improper or malicious use by individuals and governments, e.g., for criminal or 1 Department of Social Sciences, Universität Siegen, 57068 Siegen, Germany. 2Université de Lausanne, Institute of Political Studies, CH‑1015 Lausanne, Switzerland. 3Information Science and Media Studies, Universitetet I Bergen, 5007 Bergen, Norway. 4These authors contributed equally: Christian Lahusen, Martino Maggetti and Marija Slavkovik. *email: Scientific Reports | (2024) 14:20752 | https://doi.org/10.1038/s41598-024-71761-0 1 Vol.:(0123456789) www.nature.com/scientificreports/ censoring purposes. Furthermore, they may yield unpredictably inaccurate or biased results, as shown by the case of Dutch tax authorities who used a self-learning algorithm for spotting child care benefits fraud, which proved to be biased against lower incomes and ethnic minorities, and violated privacy l aws1. The question then is how aligned is AI-governance with the values of the society that it serves and that sustains it—particularly core democratic values? In this regard, trust and trustworthiness plays a particularly important role, because trust is at the same time a precondition, a product, and a foundational ethical value for functioning human societies. Our claim is that work on AI-governance “trust-alignment” is lacking a profound multi-disciplinary understanding of how trust and trustworthiness should be conceived and they operate in a socio-technical context where the main agent of trust and the main bearer of trustworthiness is not human but an algorithm. In this paper, we propose to focus on AI-Governance as the main object of inquiry. We argue that the question of trust and trustworthiness can only be properly addressed, when considering the interplay between its three components: AI applications, administrative practices, and regulatory systems. Such a research object calls for an interdisciplinary research agenda that transcends the limited insights of previous studies. We argue that wide gaps exist in-between the different research fields when it comes to trustworthy AI. Only an interdisciplinary collaboration, involving sociology, political science, and artificial intelligence, is thus able to generate the necessary synergies to address the challenges associated with the development of value-aligned AI-Governance. The aim of this paper is thus to provide a selective overview of the core literature in research fields involved in the analysis of AI-Governance. It wishes to engage computational science, sociology and political science into an interdisciplinary dialogue aimed at merging available evidence and synchronising research agendas. For this purpose, the paper pursues the following goals: to (1) justify the relevance of a study of trust and trustworthiness of AI-Governance; (2) provide an o (...truncated)