The medium is the message: toxicity declines in structured vs unstructured online deliberations

World Wide Web (2024) 27:31 https://doi.org/10.1007/s11280-024-01269-0 The medium is the message: toxicity declines in structured vs unstructured online deliberations Mark Klein1,2 · Nouhayla Majdoubi2 Received: 12 January 2024 / Revised: 27 March 2024 / Accepted: 9 April 2024 / Published online: 8 May 2024 © The Author(s) 2024 Abstract Humanity needs to deliberate effectively at scale about highly complex and contentious problems. Current online deliberation tools—such as email, chatrooms, and forums—are however plagued by levels of discussion toxicity that deeply undercut the willingness and ability of the participants to engage in thoughtful, meaningful, deliberations. This has led many organizations to either shut down their forums or invest in expensive, frequently unreliable, and ethically fraught moderation of people’s contributions in their forums. This paper includes a comprehensive review on online toxicity, and describes how a structured deliberation process can substantially reduce toxicity compared to current approaches. The key underlying insight is that unstructured conversations create, especially at scale, an “attention wars” dynamic wherein people are often incented to resort to extremified language in order to get visibility for their postings. A structured deliberation process wherein people collaboratively create a compact organized collection of answers and arguments removes this underlying incentive, and results, in our evaluation, in a 50% reduction of high-toxicity posts. Keywords Collective intelligence · Crowd-scale deliberation · Toxicity 1 Introduction Deliberation processes have changed little in centuries, perhaps even millennia. Typically, small groups of powerful stakeholders and designated experts craft solutions behind closed doors. Most people affected by the decisions have limited input, so important ideas and perspectives do not get incorporated, and there is often substantial resistance to implementing the ideas from those who were frozen from the process. * Mark Klein Nouhayla Majdoubi 1 Center for Collective Intelligence, Massachusetts Institute of Technology, Cambridge, MA, USA 2 School of Collective Intelligence, University Mohammed VI Polytechnic, Rabat, Morocco 13 Vol.:(0123456789) 31 Page 2 of 30 World Wide Web (2024) 27:31 Humanity now however needs to deliberate effectively about highly complex, contentious, and existentially important problems – such as climate change, security, and poverty – where a small-circle process is no longer adequate. We need to find a way to effectively integrate the expertise and preferences of tens, hundreds or even thousands of individuals in our most consequential deliberations. This paper addresses one important barrier to creating this capability: toxicity1 in online deliberations. Online technology seems to represent our best hope for scaling up deliberations, but it has been plagued by debilitating levels of toxic comments. How can we fix that? As part of that discussion, we will cover: • • • • • Goal: defining deliberation, and why scale is so important Challenge: the toxicity trap of existing deliberation technologies Solution: an introduction to deliberation mapping, a solution to online toxicity: Assessment: an evaluation of how well deliberation mapping reduces toxicity Conclusions: lessons learned and next steps 2 The goal: effective deliberation at scale Let us define deliberation as the activity where groups of people (1) identify possible solutions for a problem, (2) evaluate these alternatives, and (3) select the solution(s) that best meet their needs (4). Research from the field of collective intelligence has shown that engaging crowds in the way has the potential to unleash such powerful benefits as [1]: • many hands: the advent of cheap digital communication and ubiquitous personal computing has revealed the existence of a massive cognitive surplus: very large numbers of people with deep and diverse skill sets are eager to participate in collective tasks, driven by such non-monetary incentives as contributing to problem or communities they care about [2, 3]. Wikipedia is an excellent example of this. • casting a wide net: frequently, solutions for difficult problems can be found by consulting outside of the usual small circle of conventional experts in that field [4]. Innocentive is one example of a company that has been very successful exploiting this phenomenon. • idea synergy: out-of-the-box solutions can often be achieved by bringing together many individuals and engaging them in combining and refining each other’s ideas. The Matlab Coding Coopetition is a spectacular example of the power of this effect [5] • wisdom of crowds: large numbers of suitably diverse, motivated and independent raters have been shown to produce assessment accuracy—e.g. for prediction and estimation tasks—that exceeds that of experts [6]. Prediction markets are a powerful example of the value of this phenomenon. • many eyes: our ability to detect possible problems in solution ideas increases dramatically by simply engaging more people in the task. This has been one of the key reasons for the success of such volunteer-created open-source software tools as Linux 1 We define toxicity as the presence of rude, disrespectful, or unreasonable comments that are likely to make people leave a discussion. 13 World Wide Web (2024) 27:31 Page 3 of 30 31 (the dominant operating system for supercomputers), Apache (the most widely-used web server), MySQL (the most widely-used relational DB) and the web toolkits used by Chrome, Firefox (the most popular web browsers in the world). These open source tools have decisively out-competed software developed by massive software companies with thousands of highly-paid engineers [7]. Engaging the relevant stakeholders in making decisions also has the great advantage of reducing the resistance and confusion that can occur when trying to actually implement the solutions developed by the deliberation engagement. 3 The challenge: online toxicity in existing deliberation technologies We conducted a systematic literature review, using the PRISMA 2020 methodology, to better understand the phenomenon of online toxicity. We queried 3 major databases (SCOPUS, Sage Journals, JSTOR) for articles available in English, using toxicity and incivility as key terms, and adding terms referring to online mediums and tools (e.g. social media, platform), toxicity/incivility-related terms (e.g. negative, offensive, toxic*, incivil*, uncivil*), and collective deliberation (e.g. debate, collaboration, deliberat*). We excluded articles from unrelated domains like chemistry or biology that potentially conflate toxicity with other terms, as well as toxicity studies in online gaming communities as those represent a different case study than deliberation. Our review included 91 articles meeting these criteria, the majority of which are recent. 3.1 Defining our terms (...truncated)