Choosing an adequate design and analysis in cross-cultural personality research

current issues in personality psychology · volume 5(1), 7 doi: https://doi.org/10.5114/cipp.2017.65824 review article Choosing an adequate design and analysis in cross-cultural personality research Jia He 1,2, Fons J. R. van de Vijver 2,3,4 1: German Institute for International Educational Research, Frankfurt am Main, Germany 2: Tilburg University, Netherlands 3: North-West University, South Africa 4: University of Queensland, Australia The flourishing of cross-cultural personality research requires a keen eye for rigorous methodology in such research. With decades of experience in cross-cultural research methods, we have come to appreciate that methodological aspects of such studies are critical for obtaining valid findings. Ill-designed or -conducted studies may produce results that are difficult to interpret. A careful design and analysis can help to deal with various methodological problems in cross-cultural personality studies. Drawing on the extensive knowledge that has been accumulated in cross-cultural and personality research in the past decades, we describe a framework of bias and equivalence that enables the choice of adequate research methods and the avoidance of pitfalls that endanger valid conclusions in cross-cultural personality research. Specifically, we focus on sampling issues, test adaptations, and the combination of emic and etic approaches in this short review article. We encourage researchers to use the tools and experience that are available to considerably enlarge our insights in cross-cultural differences and similarities in personality research. key words bias; research design; equivalence; etic-emic corresponding author – Jia He, Ph.D., German Institute for International Educational Research, Schlossstrasse 29, 60486 Frankfurt am Main, Germany, e-mail: authors’ contribution – A: Study design · B: Data collection · C: Statistical analysis · D: Data interpretation · E: Manuscript preparation · F: Literature search · G: Funds collection to cite this article – He, J., & van de Vijver, F. J. R. (2017). Choosing an adequate design and analysis in cross-cultural personality research. Current Issues in Personality Psychology, 5(1), 3–10. received 02.09.2016 · reviewed 07.12.2016 · accepted 21.01.2017 · published 20.02.2017 Design and analysis in personality research Background Cross-cultural personality research is burgeoning. The field is however complicated by different traditions and challenged by various methodological pitfalls. On the one hand, proposed personality models have been validated in the sense that similar structures of personality can be found in various cultural contexts (e.g., McCrae & Allik, 2002; Schmitt, Allik, McCrae, & Benet-Martínez, 2007); on the other hand, nuanced and more context-dependent aspects of personality have emerged in both culture-comparative (e.g., Church et al., 2011) and indigenous research on personality (e.g., Behrens, 2004). Personality instruments developed in one culture may not travel well to another culture, which may render results that are hard to interpret (e.g., Fetvadjiev & van de Vijver, 2015). With decades of experience in cross-cultural research methods, we have come to appreciate that methodological aspects of such studies are critical for obtaining valid findings (van de Vijver & Leung, 1997). A careful design and analysis goes a long way to deal with various methodological problems. In this paper, we propose that advancing the field of culture and personality requires a better integration of perspectives and procedures in cross-cultural research methods. Below we describe the use of adequate research methods and the avoidance of pitfalls in cross-cultural personality research, particularly focusing on choosing an adequate research design. We first introduce bias and equivalence as a general framework to deal with methodological challenges. This framework serves as the backbone of cross-cultural research methods that are relevant in all stages of a cross-cultural study. The second part of the paper is more topical. We focus in this part on three methodological aspects, which are salient in current cross-cultural studies of personality, namely sampling issues, instrument adaptation, and the combination of etic and emic approaches. These issues are by no means exhaustive, yet they merit special attention in advancing personality and culture research. Finally, we draw conclusions. Bias and equivalence Taxonomy of bias Bias occurs when score differences on the indicators of a particular construct do not correspond to differences in the underlying trait or ability (van de Vij ver & Leung, 1997). This incomplete correspondence means in practice that whereas a response in one culture represents a target construct (e.g., conscientiousness), responses in another culture reflect other constructs (e.g., social desirability) or additional constructs (e.g., a combination of conscientiousness and social desirability). Based on the source of invalidity, 4 current issues in personality psychology three types of bias are distinguished, namely construct bias, method bias, and item bias. Construct bias indicates that the construct measured is not identical across cultures. It can occur when there is only a partial overlap in definition of the construct across cultures, or when not all relevant behaviors associated with the construct are present and properly sampled in each culture (van de Vijver & Poortinga, 1997). For instance, self-esteem conceptualized in interpersonal contexts is considered to result from fulfillment of desires for love (affiliative quality) and/or status (social dominance) (Zeigler-Hill, 2010), whereas self-esteem in more independent contexts is more related to individual achievements. Consequently, it is important to take multiple aspects of self-esteem into consideration in comparing different cultures where the sources of self-esteem differ. Method bias comprises all nuisance factors that derive from the sampling, structural features of the instrument, or administration processes. Sample bias results from incomparability of samples due to cross-cultural variations in sample characteristics that have a bearing on target measures, such as confounding cross-cultural differences in education levels when testing intelligence, variations in urban or rural residency, or in affiliation to religious groups. Instrument bias involves problems deriving from instrument characteristics, such as stimulus familiarity (in cognitive and educational tests) and response styles (in personality and attitude inventories). Individuals from different cultures tend to have different levels of familiarity with stimulus materials (e.g., pictures taken in one culture may be easier to recognize for people from other cultures than for people from the same culture), response modes (e.g., differences in familiarity with computers in computer-assisted assessment), and response procedures (e.g., working (...truncated)