A solution for factorial validity testing of three-item scales: An example of tau-equivalent strict measurement invariance of three-item loneliness scale

Current Psychology https://doi.org/10.1007/s12144-021-01554-5 A solution for factorial validity testing of three-item scales: An example of tau-equivalent strict measurement invariance of three-item loneliness scale Stanisław K. Czerwiński 1 & Paweł Andrzej Atroszko 1 Accepted: 26 February 2021 # The Author(s) 2021 Abstract Ultra-short scales are increasingly popular in surveys. Congeneric model fit of a three-item scale cannot be tested with Confirmatory Factor Analysis (CFA) without additional assumptions because the number of degrees of freedom is equal to zero. A more rigorous tau-equivalent model, assuming equality of factor loadings can be tested instead. The objective of this study was to demonstrate this approach with an example of the psychometric study of the Polish version of the Three-Item Loneliness Scale (TILS), and to discuss the arising problems and possible solutions. There seems to be a high need for such analysis because currently, some properties of CFA make it an approach still predominant over Item Response Theory (IRT) models in the quality of life research. A sample of 3510 students completed TILS together with the questionnaires measuring a variety of indicators of well-being. The results provided evidence for a good fit of a tau-equivalent model. Furthermore, multi-group CFAs provided support for strict measurement invariance of this model. To the Authors’ knowledge, it is the first practical application of a tau-equivalent model to testing the factorial validity of an ultra-short scale and probably the first empirical case of tau-equivalent measurement invariance in psychological literature in general. TILS showed good criterion validity and satisfactory reliability. Unidimensionality of three-item scales can be examined with a tau-equivalent model that has some favorable psychometric properties. However, it might be exceedingly restrictive in certain practical cases. When developing a new short scale, it is recommended to maintain at least four items. Keywords Loneliness . Measurement invariance . Quality of life . Scale . Tau-equivalence . Validity Introduction Loneliness Loneliness is defined by de Jong Gierveld as ‘a situation experienced by a participant as one where there is an unpleasant or unacceptable lack of (quality of) certain social relationships. The extent to which the situation is experienced as serious depends upon the participant’s perception of his or her ability to realize new relationships, or to improve existing ones.’ (de Jong Gierveld, 1989). It is important to emphasize * Paweł Andrzej Atroszko Stanisław K. Czerwiński 1 Institute of Psychology, University of Gdańsk, Bażyńskiego 4, 80-309 Gdańsk, Poland that loneliness is a subjective feeling and to distinguish it from social isolation, as they are two separate constructs (Coyle & Dugan, 2012; Perissinotto & Covinsky, 2014), for one can be in the company of others and still feel alone or live in seclusion and have no negative feelings about it whatsoever. A large number of studies concerning loneliness focus on the elderly population (Holt-Lunstad, Smith, Baker, Harris, & Stephenson, 2015) whilst neglecting younger samples, even though the relationship between age and loneliness has been found to be U-shaped (Pinquart & Sorensen, 2001). Loneliness is not a problem specific only to the elderly. Studies show that younger generations are showing less concern for other people and gravitate away from civic orientation (Twenge, Campbell, & Freeman, 2012), which in turn could explain their increase in loneliness (Cigna U.S. Loneliness Index, 2018; Twenge, Spitzberg, & Campbell, 2019). With social support being one of the fundamental factors in human well-being (Cohen & Wills, 1985), it is not surprising that each successive generation shows more and more signs of mental problems (Twenge et al., 2010). Poland could be exceptionally vulnerable to this effect. For example, it is one of Curr Psychol the few countries in Europe to have increasing suicide rates (Höfer, Rockett, Värnik, Etzersdorfer, & Kapusta, 2012). The rapid social change in post-communist countries is among the postulated causes for a decrease in well-being (DomagałaKrecioch & Majerek, 2014; Höfer et al., 2012). Poland as a country that transited to the market economy and is still in the process of adapting the Western “way of life” is a compelling case for the process of increasing alienation among youth (Zoutewelle-Terovan & Liefbroer, 2017). The existing data seems to support the notion that dynamic economic growth comes with a price. For example, compulsive overworking prevalence evaluated with the same measure and cut-off score seems to be 2 to 3 times higher in Poland than in countries with established and stable economies such as Denmark and Norway (Andreassen, Nielsen, Pallesen, & Gjerstad, 2019; Atroszko, Pallesen, Griffiths, & Andreassen, 2017; Lichtenstein, Malkenes, Sibbersen, & Hinze, 2019), and this trend is already visible among undergraduate students (Lawendowski, Bereznowski, Wróbel, Kierzkowski, & Atroszko, 2019). Having that in mind, further investigation of loneliness in young adults, especially in rapidly economically developing countries, is of great importance. Our study focused on the undergraduate students because they comprise almost half of the population age 19–24 in Poland (Główny Urząd Statystyczny, 2019) and previous studies showed that student populations are highly vulnerable to loneliness (American College Health Association, 2016). Polish studies showed high prevalence of depression and considerable hopelessness among university students (Czerwiński, Mackiewicz, Mytlewska, & Atroszko, 2020; Koryczan, Piotrowski, Roj, Czerwiński, & Atroszko, 2020). The previous studies showed significant relationships of loneliness with a wide range of well-being indicators, such as health (Swami et al., 2007; Tobiasz-Adamczyk et al., 2017), sleep quality (Matthews et al., 2017; Yu, Steptoe, Niu, Ku, & Chen, 2017), self-esteem (Heinrich & Gullone, 2006; Vanhalst, Luyckx, Scholte, Engels, & Goossens, 2013), satisfaction with personal relationships (Mellor, Stokes, Firth, Hayashi, & Cummins, 2008), social support (Segrin & Passalacqua, 2010; Tobiasz-Adamczyk et al., 2017), general quality of life (Fanakidou et al., 2017), satisfaction with life (Buelga, Musitu, Murgui, & Pons, 2008; Huo & Kong, 2014; Liu & Guo, 2008), stress (Hughes, Waite, Hawkley, & Cacioppo, 2004 ; Lee & Goldstein, 2015), anxiety (Fanakidou et al., 2017; Moore & Schultz, 1983), and depression (Cacioppo, Hughes, Waite, Hawkley, & Thisted, 2006; Erzen & Çikrikci, 2018). Issues in Factorial Validity Testing of Three-Item Scales Ultra-short scales are becoming increasingly popular in educational and psychological research due to the convenience of application, often satisfactory psychometric properties and reduction of bias introduced by the excessive burden on participants with long questionnaires. They are often (...truncated)