Enhancing decision-making in user-centered web development: a methodology for card-sorting analysis

World Wide Web https://doi.org/10.1007/s11280-021-00950-y Enhancing decision‑making in user‑centered web development: a methodology for card‑sorting analysis José A. Macías1 · Alma L. Culén2 Received: 10 May 2021 / Revised: 4 August 2021 / Accepted: 10 September 2021 © The Author(s) 2021 Abstract The World Wide Web has become a common platform for interactive software development. Most web applications feature custom user interfaces used by millions of people every day. Information architecture addresses the structural design of information to build quality web applications with improved usability of content, navigation, and findability. One of the most frequently utilized information architecture methods is card sorting—an affordable, user-centered approach for eliciting and evaluating categories and navigable items. Card sorting facilitates decision-making during the development process based on users’ mental models of a given application domain. However, although the qualitative analysis of card sorts has become common practice in information architecture, the quantitative analysis of card sorting is less widely applied. The reason for this gap is that quantitative analysis often requires the use of customized techniques to extract meaningful information for decision-making. To facilitate this process and support the structuring of information, we propose a methodology for the quantitative analysis of card-sorting results in this paper. The suggested approach can be systematically applied to provide clues and support for decisions. These might significantly impact the design and, thus, the final quality of the web application. Therefore, the approach includes proper goodness values that enable comparisons among the results of the methods and techniques used and ensure the suitability of the analyses performed. Two publicly available datasets were used to demonstrate the key issues related to the interpretation of card sorting results and the overall suitability and validity of the proposed methodology. Keywords User-centered web application development · Information architecture · Quantitative analysis · Card sorting · User experience · Human–computer interaction * José A. Macías Alma L. Culén 1 Computer Engineering Department, Escuela Politécnica Superior, Universidad Autónoma de Madrid, Tomás y Valiente 11, 28049 Madrid, Spain 2 Department of Informatics, University of Oslo, P. Box 1080, 0316 Oslo, Norway 13 Vol.:(0123456789) World Wide Web 1 Introduction The World Wide Web has become the main platform on which interactive applications are developed [1–4]. Initially, most web applications were produced as content blocks without consideration for specific quality issues. However, as the field of human–computer interaction (HCI) has moved toward user experience [5, 6], interactive web application design has changed to include a user-centered and experiential perspective [7]. Content, structure, and navigation are all considered crucial to a website’s success [8]. Therefore, a positive user experience with web browsing is essential since efficient navigation and ease of access to good content increases perceived satisfaction and helps users find information that is of interest to them [9]. In contrast, deficiencies in content or navigation design negatively affect the user experience and may also affect the intelligent extraction algorithms that automatically produce knowledge [10] from content and page structure [9, 11]. Information architecture [9] is a discipline that focuses on the structural design of both content and navigation in web application development [6], aiming to provide positive user experiences with web applications. The latter implies a user-centered approach in which different methods, tools, and techniques are utilized to address usability issues [11]. In this environment, card sorting is a common and easy-to-use method [12]. It facilitates design decisions that align with users’ mental models of an application area [13], thus increasing the possibility of creating positive experiences with the web application. In information architecture, card sorting is used to build and evaluate the navigation structure of a website [14] and to elicit the most convenient content categories, concepts, and information labeling [15]. The approach is particularly advantageous in the early design phases of a web development project [16]. However, card sorting is also utilized in other phases, for example, for summative evaluation and for evaluating and improving the quality of existing web applications [11]. Once the sorting of cards is complete, analysis is performed to gain insights relevant to the information structuring and navigation. While qualitative analysis of card sorting has become commonplace in information architecture, quantitative analysis requires further attention. The reason for this is that quantitative analysis often requires additional means of analysis, for example, clustering and scaling, which are very dependent on the way the data is understood [16]. Usually, quantitative analysis of a card sorts is more complex than qualitative, and care in applying it is essential for obtaining meaningful results that lead to good design. Therefore, in practice, most card sorts are still analyzed using custom spreadsheets to obtain only basic information about raw data [13]. This leaves room for increasing the understanding of the key issues related to interpreting card sorting results and creating a systematic approach, a methodology that supports quantitative analysis of card sorting results. Even though many commercial tools have been made to facilitate the quantitative analysis of card sorting, the decision-making process still needs improvement. The existing tools often include visualizations that show the main outcomes and facilitate reasoning, enabling evaluators and participants to successfully carry out card-sorting tasks in different application domains, providing mechanisms that simplify the process. However, most such tools produce only basic quantitative results, addressing mainly hierarchical clustering and co-occurrence, which greatly predetermines the parameters and statistical techniques to be used and restricts the outcomes and the goodness observed. In addition, without adequate knowledge and correct initial analysis, a 13 World Wide Web wrong selection of statistical techniques might be made when using commercial tools, for example, when proceeding without evaluating the validity of different conditions [17]. The above arguments allow the framing of the main research question discussed in this paper: Is it possible to define a systematic method for carrying out a quantitative analysis of card-sorting data that provides instruments and goodness indicators for decision-making in web application development? A methodology for quantitative analysis of data obtained from card sorting is proposed to answer the above quest (...truncated)