Clinician educators’ conceptions of assessment in medical education

Advances in Health Sciences Education https://doi.org/10.1007/s10459-022-10197-5 Clinician educators’ conceptions of assessment in medical education D. A. Sims1 · F. J. Cilliers2 Received: 26 September 2022 / Accepted: 11 December 2022 © The Author(s) 2022 Abstract In pursuing assessment excellence, clinician-educators who design and implement assessment are pivotal. The influence of their assessment practice in university-run licensure exams on student learning has direct implications for future patient care. While teaching practice has been shown to parallel conceptions of teaching, we know too little about conceptions of assessment in medical education to know if this is the case for assessment practice and conceptions of assessment. To explore clinician-educators’ conceptions of assessment, a phenomenographic study was undertaken. Phenomenography explores conceptions, the qualitatively different ways of understanding a phenomenon. Data analysis identifies a range of hierarchically inclusive categories of understanding, from simple to more complex, and the dimensions that distinguish each category or conception. Thirtyone clerkship convenors in three diverse Southern settings were interviewed in three cycles of iterative data collection and analysis. Four conceptions of assessment were identified: passive operator, awakening enquirer, active owner and scholarly assessor. Six dimensions were elucidated to describe and distinguish each conception: purpose of assessment; temporal perspective; role and responsibility; accountability; reflexivity and emotional valence. Additionally, three characteristics that appeared to track the progressive nature of the conceptions were identified: professional identity, assessment literacy and self-efficacy. These conceptions encompass and extend previously described conceptions across different educational levels, disciplines and contexts, suggesting applicability to other settings. There is some evidence of a relationship between conceptions and practice, suggesting, together with the hierarchical nature of these conceptions, that targeting conceptions during faculty development may be an effective approach to enhance assessment practice. Keywords Assessment · Assessor · Clerkship · Clinician educator · Conceptions · Assessment for licensure · Phenomenography * D. A. Sims 1 University of the Western Cape, 14 Blanckenberg Street, Bellville, South Africa 2 Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa 13 Vol.:(0123456789) D. A. Sims, F. J. Cilliers Introduction Assessment drives learning, but, as Boud (1995, p.36) writes, “The fundamental question is, ‘What kind of learning?’” Evidence of the undesirable learning effects of assessment is abundant (see Cilliers et al. (2010); Cilliers et al., (2012a, 2012b); Cilliers et al., (2012a, 2012b) for a summary and evidence of the effects of summative assessment), that of desirable learning effects less so. Undesirable learning effects of assessment in medical education, particularly assessment in a system where graduation from medical school leads to supervised internship and subsequent licensure for independent practice as a general practitioner, have consequences not only for students but also patient care and public safety. Adapting assessment practice to ensure more desirable learning effects is a longstanding goal in higher and medical education (Black & Wiliam, 1998; Gibbs & Simpson, 2004; Joughin, 2010; Swan Sein et al., 2020). Programmatic assessment (Schuwirth & van der Vleuten, 2019) is a systematic response to this challenge. Whether such an approach is adopted or, as in many settings, change is undertaken more piecemeal, a key player in the assessment game is the individual involved in designing assessment. Designing assessment is essentially a behaviour and improving assessment practice, a behaviour change challenge. The behaviour of individual assessors during the design of assessment will be influenced by numerous factors, not least their conceptions of assessment. Background Conceptions, the object of study in phenomenography, are defined as, “Experiential descriptions, that is, content-orientated and interpretative descriptions of qualitatively different ways people perceive and understand their reality” (Marton, 1981, p. 177)—in this study, assessment. Conceptions have also been described as ‘ways of conceptualizing’, ‘ways of experiencing’, ‘ways of seeing’, ‘ways of apprehending’, ‘ways of understanding’ (Marton & Pong, 2005). Conceptions of knowledge, of teaching and of learning are widely written about, conceptions of assessment less so. The nature of teaching practice has been shown to be related to conceptions of teaching (Kember, 1997; Trigwell et al., 1999). Support exists for such a link between conceptions of assessment and its practice (Box et al., 2015; DeLuca et al., 2019; Govaerts, et al., 2013; Watkins et al., 2005). Changed conceptions appear to contribute to effective changes to educational practice (Ho, 2000; Ho et al., 2001). Attending to conceptions would thus seem a likely candidate target for successful assessment practice change. The known conceptions of assessment of educators in basic education (school), higher education (university) and medical education settings are summarised in Table 1. Conceptions are typically represented by different categories of description and dimensions that describe and illustrate those categories (Svensson, 1997). Discernible dimensions of existing conceptions include an accountability or quality control purpose (Brown, 2004; de Jonge et al., 2017); and a teacher-centred and knowledge-reproduction or studentcentred and learning-related purposes for assessment (Brown, 2004; de Jonge et al., 2017; Halinen et al., 2013; Postareff et al., 2012; Samuelowicz & Bain, 2002). The relationship between existing descriptions of conceptions of assessment has not, however, been clarified. Whether conceptions of educators hold across contextually different settings has also 13 - Psychometric: tailored vs. standardised assessment - Socio-constructivist: self- vs. externally-regulated learning - Agency (student-regulated learning) - Mutuality (student- and teacher-regulated learning) - Objectivity (externally-regulated assessment) - Adaptivity (externally- and clinically-regulated assessment) - Accountability (clinically-regulated assessment) From de Jonge et al. (2017) - Assessment as “knowledge production” (teachercentred) - Assessment as “knowledge construction/transformation” (learning-centred) From Samuelowicz and Bain (2002) - Assessment as “reproduction” (content-reproduction) - Assessment as “transformation” (student-development) From Postareff et al. (2012) - “Status quo” (teacher-centred; content-reproduction) - “Awareness” (content- and learning-centred) - “Development (shared autonomy; learning-centred) From Halinen et al. (2013) - Assessment as “irrelevant” (little or harmfu (...truncated)