How to present the analysis of qualitative data within interdisciplinary studies for readers in the life and natural sciences

Quality & Quantity https://doi.org/10.1007/s11135-021-01162-2 How to present the analysis of qualitative data within interdisciplinary studies for readers in the life and natural sciences Gerda Casimir1 · Hilde Tobi1 · Peter Andrew Tamás1 Accepted: 3 May 2021 © The Author(s) 2021 Abstract Research that addresses complex challenges often requires contributions from the social, life and natural sciences. The disciplines that contribute subject response data, and more specifically qualitative analyses of subject response data, to interdisciplinary studies are characterised by low consensus with respect to methods they use a diversity of terms to describe those methods and they often work from assumptions that are foreign to readers in the natural and life sciences. The first contribution this paper makes is to demonstrate that the forms of reporting that may be adequate for communicating quantitative analysis do not provide teams that include members from natural, life and social sciences with useful accounts of qualitative analysis. Our second contribution is to discuss and model how to report four methods appropriate for qualitative contributions to interdisciplinary projects. Keywords Qualitative · Inter-disciplinary · Mixed-methods · Transparency · Reporting 1 Introduction There are strong arguments to combine quantitative analysis and qualitative analysis within the social sciences (Babbie 1989; Creswell and Clark 2000; Johnson Onwuegbuzie and Turner 2007). Research that addresses complex challenges, such as adaptation to the effects of climate change, often involves teams from the social, life and natural sciences. These interdisciplinary studies frequently demand teams to integrate qualitative analysis of subject response data with quantitative analysis of direct measures of natural phenomena. Further, reports of these studies are often presented in journals whose reporting formats anticipate quantitative analysis of direct measurements for natural and life science readers. We have found specific guidance on the design of interdisciplinary research (e.g. Tobi & Kampen 2018), on how to make it meaningful for policy (e.g. Kampen and Tamás 2014) and we have found a large number of guidelines for the reporting of both quantitative and qualitative analysis for both disciplinary researchers and for those times when * Peter Andrew Tamás 1 Wageningen University, Wageningen, Netherlands 13 Vol.:(0123456789) G. Casimir et al. interdisciplinarity is limited to the social sciences. Despite our own and our peers’ efforts, we have not found guidelines for the presentation of the qualitative analysis of subject response data that well serve integration into the reports of interdisciplinary studies published in journals that are read outside of the social sciences. Our purpose with this paper is to strengthen inter-disciplinary science by improving the adequacy of the reports of analysis of qualitative subject response data within reports of interdisciplinary studies. In the next section we demonstrate the need for these guidelines by describing and faulting existing reporting practices. The guidance we then offer is presented through the use of a model case. The analysis methods we present in this model case were selected for their relevance to interdisciplinary research addressing environmental challenges. 2 The transparency of reporting in interdisciplinary research In preparation for this manuscript we downloaded four years of papers that contained both ‘interdisciplinary’ and ‘interview’ in their titles, keywords and abstracts (N = 1160 papers).1 The term ‘interdisciplinary’ was selected as we were certain that authors’ selfidentification would be a strong indicator of interdisciplinarity and the term ‘interview’ was selected as the alternatives we considered, such as ‘qualitative’ produced high falsepositive rates. We recognize that this search strategy likely excluded many studies which compromises the generalisability of our findings. We then used automatic coding in Atlas. ti to identify all paragraphs that contained both words ‘analysis’ and ‘interview’ (n = 1033 paragraphs) to quickly identify those papers that contained a substantial discussion of the methods used to analyse interview data and a location within papers where that discussion is certain to be found. We then used random selection from these paragraphs to identify papers for examination. We continued to randomly select papers for examination until five in a row produced no novel observations (n = 79 papers). In all of the papers examined, researchers reported that they identified and aggregated themes in order to present patterns. The description given these efforts generally mirrored the account given of their analysis of quantitative data. For example, many reported ‘thematic content analysis’ which appears to be as informative as ‘multiple logistic regression.’ These two are neither equivalent nor are they similarly informative. The term ‘multiple logistic regression’ references a specific set of analysis procedures and assumptions about which there is well-known consensus. Thematic content analysis, however, involves two distinct steps neither of which benefits from the consensus supporting interpretation of the term ‘multiple logistic regression’. The first step in thematic content analysis is the attachment of codes to text that capture meaning. This step, coding, is akin to measurement or data processing in the natural and life sciences. The codes applied are the equivalent to the pH value recorded by a researcher when using litmus strips to measure acidity in surface water or the calculation of BMI based on data provided on weight and height. 1 In March of 2019 we executed the following search in Web of Science: • TOPIC: (interdisciplin* or inter-disciplinary). • Refined by: TOPIC: (interview*) AND DOCUMENT TYPES: ( ARTICLE) AND PUBLICATION YEARS: ( 2019 OR 2018 OR 2017 OR 2016). • Timespan: All years. Indexes: SCI-EXPANDED, SSCI, A&HCI, ESCI. 13 How to present the analysis of qualitative data within… Staying with the step of coding, which tended to be far better discussed than synthesis, in the articles we reviewed it was consistently clear that researchers identified themes, but it was not clear where those themes came from. Unlike chemistry, where budding scientists are taught consistently how to read litmus strips so the reader knows what procedures lie behind a stated pH value, there is no consensus in the social sciences that we know of that allows a reader to infer from ‘thematic content analysis’ an unequivocal understanding of how researchers identified units of text as meaningful and then determined what speakers meant by what they said. Certainly, many of the articles we reviewed used multiple raters and negotiation to improve reliability, but interrater agreement does not improve transparency in the manner required to shed light on validity. Turning now to synthesis (...truncated)