AI Within Online Discussions: Rational, Civil, Privileged?

Minds and Machines (2024) 34:10 https://doi.org/10.1007/s11023-024-09658-0 AI Within Online Discussions: Rational, Civil, Privileged? Ethical Considerations on the Interference of AI in Online Discourse Jonas Aaron Carstens1 · Dennis Friess2 Received: 1 June 2023 / Accepted: 21 January 2024 © The Author(s) 2024 Abstract While early optimists have seen online discussions as potential spaces for deliberation, the reality of many online spaces is characterized by incivility and irrationality. Increasingly, AI tools are considered as a solution to foster deliberative discourse. Against the backdrop of previous research, we show that AI tools for online discussions heavily focus on the deliberative norms of rationality and civility. In the operationalization of those norms for AI tools, the complex deliberative dimensions are simplified, and the focus lies on the detection of argumentative structures in argument mining or verbal markers of supposedly uncivil comments. If the fairness of such tools is considered, the focus lies on data bias and an input–output frame of the problem. We argue that looking beyond bias and analyzing such applications through a sociotechnical frame reveals how they interact with social hierarchies and inequalities, reproducing patterns of exclusion. The current focus on verbal markers of incivility and argument mining risks excluding minority voices and privileges those who have more access to education. Finally, we present a normative argument why examining AI tools for online discourses through a sociotechnical frame is ethically preferable, as ignoring the predicable negative effects we describe would present a form of objectionable indifference. Keywords AI · Discourse · Deliberation · Fairness · Equality · Discrimination * Jonas Aaron Carstens 1 Department of Political Philosophy and Ethics, Heinrich Heine University Düsseldorf, Düsseldorf, Germany 2 Düsseldorf Instute for Internet and Democracy, Heinrich Heine University Düsseldorf, Düsseldorf, Germany 13 Vol.:(0123456789) 10 Page 2 of 25 J. A. Carstens, D. Friess 1 Introduction The advent of the internet has led to the emergence of multiple online publics, where people from many different backgrounds are able to discuss issues of public manner. While some authors have argued that the internet could provide the infrastructure for a more deliberative public sphere (e.g., Coleman & Gøtze, 2001; Dahlberg, 2001), others have raised concerns that online communication could reveal the darkest human abysses (Papacharissi, 2004; Suler, 2004). More than two decades later, one may argue that these pessimistic assessments are empirically evident. In fact, several studies indicate that online discussions suffer in terms of civility and are far away from reasoned and democratic discourse envisioned by the advocates of deliberative democracy (Coe et al., 2014; Kreissel et al., 2018). Against this background, Artificial Intelligence (AI) has entered this research field in recent years, which means that both scholars and commercial organizations develop and employ AI-driven tools in order to maintain democratic discussions online (Rodríguez-Ruiz et al., 2020; Stoll et al., 2020; Wojcieszak et al., 2021). In this paper, we focus on AI that aims to improve the quality of online discussions. Following Hancock et al., (2020, p. 90), AI broadly refers “to computational systems that involve algorithms, machine learning methods, natural language processing [NLP], and other techniques that operate on behalf of an individual to improve a communication outcome”. However, AI aiming to improve communication quality may also trigger ethical issues, which constitute our main point of interest. While scholars and developers may intend to improve the quality of public online discourses when introducing automated hate speech detection tools, argument mining models, and moderating bots, they may also unintentionally increase existing inequalities, exclude certain voices from the discourse, and deepen social hierarchies.1 Since determining what is a solid argument, an appropriate wording or a comment worth to be automatically replied to by a bot has powerful implications; such decisions have to be the subject of ethical reflections. Those reflections are in place for some AI applications—e.g. medical diagnostic tools or credit scoring, but are less developed in the context of AI interfering in public online discourses. In the first part of the paper, we are going to sketch the state of online discourse, arguing that norms of deliberation are still considered to be important normative standards to evaluate the quality of online discussions and that the violation of these norms has paved the way for AI to clean up discussions (1). In the next step, we will provide some orientation on previous AI research in the context of online deliberation to illustrate that AI research already aims to improve certain norms of deliberation while neglecting others (2). Zooming in on rationality and civility, we discuss how those norms are conceptualized and subsequently operationalized for AI tools. 1 We use the term social hierarchies to refer to systematic differences in the power and authority that members of different groups hold within society. Power is understood as the amount of possible actions available as well as the potential to compel others to action, whereas authority refers to the ability to be recognized and listened to as well as the potential to be ascribed expertise (see for power and authority Moreau 2020, pp. 51–52). Inequality, on the other hand, refers to broader patterns of different levels of access to goods and opportunities between groups. 13 AI Within Online Discussions: Rational, Civil, Privileged? Page 3 of 25 10 Furthermore, we discuss how such AI tools are imagined as neutral once the data bias that influences the pipeline from input to output has been addressed (3). We proceed by adopting a sociotechnical perspective, moving beyond data bias, showing that the promotion of deliberative norms through AI can exclude and silence marginalized groups by ignoring the interaction of models with cultural norms and social hierarchies. (4). Finally, we argue that restricting the analysis to the relationship between input and output constitutes an arbitrary choice, which expresses objectionable indifference towards those that are further excluded and marginalized through AI tools. To avoid expressing objectionable indifference, AI tools should be evaluated through a sociotechnical framework that explicitly addresses equality instead of presupposing neutrality (5). Finally, we provide concluding remarks (6). 2 The State of Online Discourse Some early writings have painted the internet as a virtual public space for free-flowing discussions and the respectful exchange of arguments (Dahlberg, 2001; Negroponte, 1995). Particularly, advocates of deliberative democracy have argued that the internet wou (...truncated)