Developing a Whole Child School Screening Instrument: Evaluating Perceived Usability as an Initial Step in Planning for Consequential Validity

School Mental Health https://doi.org/10.1007/s12310-024-09670-w ORIGINAL PAPER Developing a Whole Child School Screening Instrument: Evaluating Perceived Usability as an Initial Step in Planning for Consequential Validity Jessica B. Koslouski1 · Sandra M. Chafouleas1 · Amy Briesch2 · Jacqueline M. Caemmerer1 · Brittany Melo2 Accepted: 11 May 2024 © The Author(s) 2024 Abstract We are developing the Equitable Screening to Support Youth (ESSY) Whole Child Screener to address concerns prevalent in existing school-based screenings that impede goals to advance educational equity using universal screeners. Traditional assessment development does not include end users in the early development phases, instead relying on a psychometric approach. In working to develop the ESSY Whole Child Screener, we are integrating a mixed methods approach with attention to consequential validity from the outset of measure development. This approach includes end users in measure development decisions. In this study, we interviewed a diverse sample of school staff (n = 7), administrators (n = 3), and family caregivers (n = 8) to solicit their perceptions of the usability of the initial draft of the ESSY Whole Child Screener. We identified three overarching themes: (1) paving the road for implementation of a whole child screener, (2) potential roadblocks to use, and (3) suggested paths forward to maximize positive intended consequences. Paving the road for implementation of a whole child screener includes subthemes related to alignment with existing initiatives, comprehensive yet efficient design, and potential positive consequences of assessing the whole child. Potential roadblocks to use includes subthemes of staff buy-in, family comfort with contextual screening items, teacher accuracy, and school capacity to provide indicated supports. Suggested paths forward to maximize positive intended consequences include clear and precise messaging to staff and families, optimizing instrumentation and data collection procedures, and strengthening connections to data interpretation and use. We discuss next steps in the design and testing of the initial measure as well as assessment development more broadly. Keywords Usability · Whole child · Screening · Assessment · Schools · Consequential validity · Equity Introduction Although concerns have long existed in the US regarding educational inequities, the COVID-19 pandemic served to widen existing gaps in ways that are expected to be felt for some time (Bailey et al., 2021; Goldberg, 2021). It has been suggested that educational equity will be achieved when all students have the opportunities they need to reach their full potential (Moore et al., 2023; Osher et al., 2020); however, * Amy Briesch 1 Department of Educational Psychology, University of Connecticut, Storrs, CT, USA 2 Department of Counseling and Applied Educational Psychology, Northeastern University, 404 International Village, Boston, MA 02115, USA this requires multiple coordinated actions and efforts to transform the ways in which schools have historically operated. Among these, universal screening has been suggested as a means to promote equity by ensuring all students’ needs are systematically assessed, and students receive appropriate supports that match the intensity of those needs (Dever et al., 2016). Universal screening for academic or physical health concerns (e.g., reading and scoliosis) has long been employed in US schools, and universal social, emotional, and behavioral (SEB) screening instruments have begun to proliferate in recent years (Kim et al., 2021). Despite increased attention to universal screening assessments, however, several critical limitations of existing measures may impede intended efforts to promote educational equity. First, existing screening instruments tend to take a siloed approach, focusing on one area of child development (e.g., reading, vision, and behavior). A recent survey by Briesch and colleagues (2021) found that 70–80% of school districts Vol.:(0123456789) School Mental Health in the US engage in universal academic (e.g., literacy) and health (e.g., vision) screening, whereas as few as 9% of districts engage in any type of universal SEB screening. Such data suggest that school teams infrequently consider student SEB competencies in a systematic way when looking to proactively identify those students in need of supports. This is concerning given extensive evidence supporting the bidirectional link between academic and SEB challenges (McIntosh & Goodman, 2016). Furthermore, when SEB screening is employed, most available tools focus on SEB competencies to the exclusion of other areas of student functioning. For example, in their review of 26 SEB screening measures for use in schools, Brann et al. (2022) identified three measures that included assessment of academic enablers (i.e., BASC-3 Behavioral and Emotional Screening System; Elementary Social Behavior Assessment; Integrated Screening and Intervention System) and only two that directly asked about student academic performance (i.e., Kindergarten Academic and Behavior Readiness Screener, Social Skills Improvement System-Performance Screening Guide). As such, in order to consider the whole child, school teams must gather and synthesize screening data from multiple sources, and in the absence of such data, may have a more fragmented view of the whole child when making decisions about supports. Second, historically, screening instruments have been characterized by a deficit focus, most often evaluating skills a student is lacking or problematic behavior they are exhibiting. For example, many screeners are designed to identify students demonstrating symptoms or indicators related to a particular diagnosis (e.g., emotional and behavioral disorder) to inform efforts that reduce symptoms. These screeners rely on the pathological medical model, locate and label behavior problems as deficits within the child (Briesch et al., 2016), and do not provide strengths-based data that can be leveraged to support successful interventions. There has, however, been a paradigm shift in recent years toward greater assessment of student strengths and skills, particularly in SEB domains (von der Embse et al., 2023). Consistent with the dual-factor model of mental health (Suldo & Shaffer, 2008), research has shown the assessment of student strengths to contribute meaningful variance in the prediction of student well-being (Kim & Choe, 2022; Kim et al., 2014). Despite this push, however, deficit-focused screeners continue to predominate in school-based settings with relatively fewer strength-based screeners available for use (e.g., Devereaux Student Strengths Assessment-Mini, Naglieri et al., 2011; Social Emotional Health Survey, Furlong et al., 2018). Third, existing screeners miss opportunities to identify the root causes of student challenges by not collecting information about contextual assets or barriers th (...truncated)