Large-scale Assessments in Education

http://link.springer.com/journal/40536

List of Papers (Total 61)

The role of reading motivation, self-efficacy, and home influence in students’ literacy achievement: a preliminary examination of fourth graders in Abu Dhabi

BackgroundThis study aimed to identify motivation and home influence factors that predict reading literacy achievement of grade 4 students in Abu Dhabi It drew on the Progress in International Reading Literacy Study (PIRLS) 2011 data, which placed Abu Dhabi students' results substantially below the international benchmark of 500.MethodsSelected items from the PIRLS 2011 student...

The dependence on mathematical theory in TIMSS, PISA and TIMSS Advanced test items and its relation to student achievement

BackgroundThis paper discusses a framework for analyzing the dependence on mathematical theory in test items, that is, a framework for discussing to what extent knowledge of mathematical theory is helpful for the student in solving the item. The framework can be applied to any test in which some knowledge of mathematical theory may be useful, both within mathematics itself and in...

Personality, cognitive skills and life outcomes: evidence from the Polish follow-up study to PIAAC

BackgroundThere is a growing literature providing evidence on the importance of non-cognitive skills for life outcomes. However, to date there is limited evidence on the gains from incorporating such measures into large-scale competence surveys.MethodsWe investigate the relationship between personality traits and eight important life outcomes: educational attainment, labour...

Teachers’ participation in professional development concerning the implementation of new technologies in class: a latent class analysis of teachers and the relationship with the use of computers, ICT self-efficacy and emphasis on teaching ICT skills

The increasing availability of new technologies in an ever more digitalized world has gained momentum in practically all spheres of life, making technology-related skills a key competence not only in professional settings. Thus, schools assume responsibility for imparting these skills to their students, and hence to future generations of professionals. In so doing, teachers play...

Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics

Background A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals in order to better understand conditioning factors of test-taking disengagement: First, a model-based approach was used to investigate whether item indicators of...

The use of test scores from large-scale assessment surveys: psychometric and statistical considerations

Background Economists are making increasing use of measures of student achievement obtained through large-scale survey assessments such as NAEP, TIMSS, and PISA. The construction of these measures, employing plausible value (PV) methodology, is quite different from that of the more familiar test scores associated with assessments such as the SAT or ACT. These differences have...

Performance decline in low-stakes educational assessments: different mixture modeling approaches

Background In low-stakes educational assessments, test takers might show a performance decline (PD) on end-of-test items. PD is a concern in educational assessments, especially when groups of students are to be compared on the proficiency variable because item responses gathered in the groups could be differently affected by PD. In order to account for PD, mixture item response...

Beyond the mean estimate: a quantile regression analysis of inequalities in educational outcomes using INVALSI survey data

The number of studies addressing issues of inequality in educational outcomes using cognitive achievement tests and variables from large-scale assessment data has increased. Here the value of using a quantile regression approach is compared with a classical regression analysis approach to study the relationships between educational outcomes and likely predictor variables. Italian...

Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups

BackgroundData from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-rich environments skills; skill levels also vary by gender, age, level of education or migration background. Similarly, systematic differences have been documented...

Skills, earnings, and employment: exploring causality in the estimation of returns to skills

Ample evidence indicates that a person’s human capital is important for success on the labor market in terms of both wages and employment prospects. However, unlike the efforts to identify the impact of school attainment on labor-market outcomes, the literature on returns to cognitive skills has not yet provided convincing evidence that the estimated returns can be causally...

PIAAC: a new design for a new era

As the largest and most innovative international assessment of adults, PIAAC marks an inflection point in the evolution of large-scale comparative assessments. PIAAC grew from the foundation laid by surveys that preceded it, and introduced innovations that have shifted the way we conceive and implement large-scale assessments. As the first fully computer-delivered survey of...

Assuming measurement invariance of background indicators in international comparative educational achievement studies: a challenge for the interpretation of achievement differences

Background Large-scale cross-national studies designed to measure student achievement use different social, cultural, economic and other background variables to explain observed differences in that achievement. Prior to their inclusion into a prediction model, these variables are commonly scaled into latent background indices. To allow cross-national comparisons of the latent...

Coverage of the migrant population in large-scale assessment surveys. Experiences from PIAAC in Germany

Background European countries, and especially Germany, are currently very much affected by human migration flows, with the result that the task of integration has become a challenge. Only very little empirical evidence on topics such as labor market participation and processes of social integration of migrant subpopulations is available to date from large-scale population surveys...

A structural equation modeling approach for examining position effects in large-scale assessments

Position effects may occur in both paper–pencil tests and computerized assessments when examinees respond to the same test items located in different positions on the test. To examine position effects in large-scale assessments, previous studies often used multilevel item response models within the generalized linear mixed modeling framework. Using the equivalence of the item...

The relevance of the school socioeconomic composition and school proportion of repeaters on grade repetition in Brazil: a multilevel logistic model of PISA 2012

The paper extends the literature on grade repetition in Brazil by (a) describing and synthesizing the main research findings and contributions since 1940, (b) enlarging the understanding of the inequity mechanism in education, and (c) providing new findings on the effects of the school socioeconomic composition and school proportion of repeaters on the individual probability of...

Degrees of competency: the relationship between educational qualifications and adult skills across countries

Background Educational qualifications and literacy skills are highly related. This is not surprising as it is one aim of educational systems to equip individuals with competencies necessary to take part in society. Because of this relationship educational qualifications are often used as a proxy for “human capital”. However, from a theoretical perspective, there are many reasons...

Evaluating the risk of nonresponse bias in educational large-scale assessments with school nonresponse questionnaires: a theoretical study

Survey participation rates can have a direct impact on the validity of the data collected since nonresponse always holds the risk of bias. Therefore, the International Association for the Evaluation of Educational Achievement (IEA) has set very high standards for minimum survey participation rates. Nonresponse in IEA studies varies between studies and cycles. School participation...

Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study

The present paper investigates the power of personality to predict important life outcomes in the context of the Programme for the International Assessment of Adult Competencies (PIAAC). On the most global level, personality can be described by the Big Five dimensions, extraversion, agreeableness, conscientiousness, emotional stability, and openness to experience. These five...

The PIAAC longitudinal study in Germany: rationale and design

In Germany, the respondents who had participated in the 2012 survey of the Programme for the International Assessment of Adult Competencies (PIAAC) were re-approached for the panel study PIAAC-L. PIAAC-L aims at investigating the longitudinal effects of skill outcomes over the life course and the development of the key skills assessed in PIAAC. Moreover, additional and...

School-level predictors for the use of ICT in schools and students’ CIL in international comparison

The increasing relevance of information and communication technologies (ICT) and society’s transition towards an information or knowledge society have led to the emergence of new challenges for schools and school systems. Thus, the need for students to develop new forms of skills like digital literacy or computer and information literacy (CIL) is constantly gaining in importance...

Home resources as a measure of socio-economic status in Ghana

In large scale international assessment studies, questionnaires are typical used to query students’ home possessions. Composite scores are computed from responses to the home resource questionnaires and are used as a measure of family socioeconomic background in achievement comparison or for statistical control. This paper deals with profiling the socio-economic status (SES) of...

Proficient beyond borders: assessing non-native speakers in a native speakers’ framework

Background English language proficiency is considered a basic skill that students from different language backgrounds are expected to master, independent of whether they are native or non-native speakers. Tests that measure language proficiency in non-native speakers are typically linked to the common European framework of reference for languages. Such tests, however, often lack...

Comparing DIF methods for data with dual dependency

Background The current study compared four differential item functioning (DIF) methods to examine their performances in terms of accounting for dual dependency (i.e., person and item clustering effects) simultaneously by a simulation study, which is not sufficiently studied under the current DIF literature. The four methods compared are logistic regression accounting neither...

The relation between teachers’ emphasis on the development of students’ digital information and communication skills and computer self-efficacy: the moderating roles of age and gender

Teachers’ integration of information and communication technology (ICT) has been widely studied, given that digital competence is considered to be a crucial outcome of twenty first century education. In this context, research highlighted teachers’ computer self-efficacy (CSE) as one of the most important determinants of their ICT integration into teaching practices. Whereas...