Coverage of the migrant population in large-scale assessment surveys. Experiences from PIAAC in Germany
Maehler et al. Large-scale Assess Educ
Coverage of the migrant population in large‑scale assessment surveys. Experiences from PIAAC in Germany
Débora B. Maehler 0 1 2
Silke Martin 0 1 2
Beatrice Rammstedt 0 1 2
Survey Design 0 1 2
Methodology 0 1 2
0 Sciences , PO Box 12 21 55
1 Institute for the Social
2 Department , GESIS-Leibniz
Background: European countries, and especially Germany, are currently very much affected by human migration flows, with the result that the task of integration has become a challenge. Only very little empirical evidence on topics such as labor market participation and processes of social integration of migrant subpopulations is available to date from large-scale population surveys. The present paper provides an overview of the representation of the migrant population in the German Programme for the International Assessment of Adult Competencies (PIAAC) sample and evaluates reasons for the under-coverage of this population. Methods: We examine outcome rates and reasons for nonresponse among the migrant population based on sampling frame data, and we also examine para data from the interviewers' contact protocols to evaluate time patterns for the successful contacting of migrants. Results and Conclusions: This is the first time that results of this kind have been presented for a large-scale assessment in educational research. These results are also discussed in the context of future PIAAC cycles. Overall, they confirm the expectations in the literature that factors such as language problems result in lower contact and response rates among migrants.
Migrant; Nonresponse; PIAAC; Germany; Paradata
European countries, and especially Germany, are currently very much affected by human
migration flows, in particular refugee flows. At present, the major drivers of migration
are economic factors and armed conflicts. Although the refugees are seeking first and
foremost temporary refuge, they may also wish to make a new home for themselves.
European countries have long been confronted with the task of integrating migrants.1
However, because of the somewhat unexpected extent of the current flows, the
integration task is a challenge for most countries. Researchers have already developed several
1 The operationalization of a migrant varies greatly in the interdisciplinary literature on migration. It is dependent on
the research question or on the information available in the datasets used for secondary analyses. Several indicators,
such as citizenship, place of birth, or first language, could be considered individually or jointly. Arguments for the
appropriate usage of the different indicators for operationalization purposes are given, for example, in Maehler et al. (2015).
In the analyses in the present article, we use the indicator citizenship to operationalize a migrant, as this was the only
key indicator available in the gross sample dataset. In what follows, the terms migrant and non-German citizen are used
© The Author(s) 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License
(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and
indicate if changes were made.
assumptions and models of how the integration process takes place. However, the
empirical underpinning of these assumptions is based mostly on a small number of
observed cases. This is due to several reasons, for example problems in reaching the
migrant population and gaining their cooperation. Reaching more members of this
population allows for more reliable verification of assumptions and more accurate derivation
of action plans. Therefore, we need to improve the share of migrants in large-scale
population surveys and thus enable policymakers to answer the integration questions that
Europe is dealing with today. This paper aims to contribute to increasing the coverage of
the migrant population in surveys (e.g., social surveys and large scale assessments) by
investigating the reachability of adult migrants using level-of-effort paradata and by
exploring reasons for migrant nonresponse.
As Font and Méndez (2013) pointed out, there are, on the one hand, surveys that are
specifically designed to measure and capture the realities of migrants. On the other
hand, there are surveys that are designed to cover the general resident population in a
given country, which includes individuals with a migration background. Several
largescale assessments, such as the Programme for the International Assessment of Adult
Competencies (PIAAC), are examples of the latter approach. PIAAC is an international
survey conducted under the auspices of the OECD that assesses basic skills of adults
aged 16–65 years in the areas of literacy, numeracy, and problem solving in
technologyrich environments in the official language or languages of the respective participating
countries. Information on social and language background, time in country, education,
and labor force status, among other topics, was collected in a personal interview. As
proficiency in the language of the host country is one central factor for the successful social
and economic integration of migrants, the PIAAC data are an excellent source with
which these questions can be further explored (OECD 2013a). For integration-related
research, however, it is crucial that the core migrant population be suitably covered. For
migrants, the basic skills assessed in PIAAC, such as literacy and numeracy, can serve
as indicators of the extent to which they have achieved key prerequisites for social
participation or structural integration in the host country. Literacy, for instance, is assessed
through tasks such as reading and understanding text passages of varying length and
difficulty, for example a medical package insert, a short newspaper article, or an online
job advertisement (OECD 2013a). The tasks used to measure literacy are related to
everyday life situations and are comparable for individuals from different countries as well
as from various migrant subgroups within these countries (Zabal et al. 2014). However,
if we take a look at the skills of migrants in the PIAAC countries overall (e.g., OECD
2013a; Maehler et al. 2014), we can observe a literacy gap between natives and migrants:
The proportion of adults classified as individuals with a low literacy level (i.e., level I and
below) is, on average, twice as high in the subpopulation with a migration background
(operationalized here as first-generation migrants). Considering literacy as the target
variable, and assuming that persons with low literacy skills are less likely to participate
in surveys, the results might even be further tilted against migrants’ literacy. In this
context, it is important to look at effects that are potentially induced by nonresponse. In the
case of a registry-based sample design (e.g., PIAAC Germany; Zabal et al. 2014), sample
units—in this case, individuals—are selected from a population register. These registers
might be affected to some extent by incomplete or out-of-date information. If, for
example, migrants were selected into the gross sample on the basis of the current information
available in the population register, but they have since moved abroad without
de-registering with their local registration office (Martin et al. 2015; Salentin 2014), it would
not have been possible to contact them and they would have become nonrespondents. If
migrants with low literacy skills happened to be more likely to have moved abroad than
those with high literacy levels—for example, because, being low-skilled, they had fewer
job opportunities—this could lead to an overestimation of the literacy level in the
resident migrant subpopulation.
In addition, a comparison with the majority population (i.e., natives) can provide
information as to whether a given nonresponse behavior is specific to persons with a
migration background. Thus, the question that arises is whether, and why, the response
rate of the migrant population differs from that of the majority population. There could
be several reasons why nonresponse occurs, for example incomplete address
information, refusal to participate, inability to participate because of absence, or even inability
to communicate because of language barriers. Information about structural differences
in response rates could contribute to improving future large-scale assessments. PIAAC
offers a unique opportunity to pursue this question on the basis of a high-quality,
largescale survey in educational research, thus covering a broad range of the adult population
in the participating countries.
However, the comparability of the country-specific sample designs and selection
procedures in PIAAC is limited. First, there is variation in sampling designs and sampling
frames across countries (Mohadjer et al. 2013). On the one hand, there is a
distinction between countries using household samples (e.g., Canada, England, or the United
States) and countries using registry-based samples (e.g., Austria, Spain, or Sweden).
On the other hand, within the group of countries who use registry-based samples, the
sampling frames are sometimes decentralized registers (e.g., in Germany) and
sometimes centralized registers (e.g., in Sweden). Second, the differences in the sampling
frame information across PIAAC countries have an impact on both the identification
and the classification of migrants. Countries with household samples do not usually have
information on migration background (such as citizenship and country of birth) in their
gross sample. By contrast, registry countries have access to some migration-related data.
However, this information is not harmonized across countries (e.g., country of birth in
Sweden, citizenship in Austria and Germany). Beyond that, the coverage of the migrant
subpopulation across countries is subject to other moderating factors, such as different
fieldwork strategies. These measures include, for instance, the translation of invitation
letters (e.g., Germany; Zabal et al. 2014, p. 70), the use of bilingual interviewers (e.g.,
United States; Hogan et al. 2016, p. 5–9) or translators to administer the questionnaire
(e.g., Austria; Statistik Austria 2013, p. 31), the translation of questionnaires into the
languages of selected migrant groups (e.g., Austria, United States), or even the exclusion
of the migrant population altogether (e.g., Japan; OECD 2016, p. 52). Finally, the largest
obstacle is the restricted access to the data required for nonresponse analyses, such as
gross sample and outcome information (e.g., disposition codes). Thus, our analyses will
focus on German PIAAC data only, as Germany is currently one of the EU countries
most affected by the flow of refugees and is in need of more information in order to
make decisions about future integration measures.
Sampling frame and the coverage of the migrant population in Germany
There are several possible approaches to sampling an adult migrant population, such as
name-based selection from telephone directories, use of person-centered networks (e.g.,
snowballing), or selection from population registers on the basis of distinguishing
personal characteristics. In his overview for Germany, Salentin (2014) compared the
advantages and disadvantages of sampling frames and concluded that a combination of a
registry-based sample design and a name-based procedure (onomastics), or name-based
sampling in a population register, would be the most appropriate approach to achieve a
representative sample of the migrant population. At present, the implementation of a
registry-based sample design in Germany for such a representative sample of the
migrant population is subject to potential restrictions (Salentin 2014): first,
registrybased samples can be drawn only if the survey is of public interest; second, not all
existing information can be used as a selection criterion. For sample selection in large-scale
surveys of public interest in Germany, information obtainable from population registers
is limited to specific variables, namely age, gender, and citizenship. Although place of
birth is generally recorded, the current registration legislation does not permit the
distribution of these data.2 However, citizenship as the sole criterion leads to an
underestimation of individuals with a migration background and to a potential distortion of the
social structure (Salentin 2014). For example, naturalized migrants (i.e., migrants who
have acquired the citizenship of the host country) who were born abroad are classified as
natives. In addition, in the case of Germany, it is not possible to identify for sampling
purposes one large migrant sub-population, namely the ethnic German resettlers
The German 2012 PIAAC sample is a registry-based sample and is representative of
Germany’s adult population aged between 16 and 65 years. The core population includes
all individuals who were resident in Germany at the time of data collection and who were
not living in institutions, such as prisons, nursing homes, etc. (Zabal et al. 2014). The
German population registers hold information on all individuals who are permanently
resident in Germany (mandatory registration) and on individuals who enter Germany
(legally) and expect to stay in the country for at least three months. Hence, to be part
of the German PIAAC core population, it was crucial that the usual place of residence
(principal residence) was Germany, while citizenship, legal status, or first language were
not critical in this case (Mohadjer et al. 2013; OECD 2010). In Germany, the realized net
sample comprised 5465 respondents (Zabal et al. 2014, p. 9). With regard to the available
sampling frame criterion citizenship, for example, 395 (unweighted) were non-Germans.
As mentioned earlier, a registry-based random sample has limitations that are relevant
for the selection of the migrant population. For example, persons who have recently
moved and have not yet registered at their new place of residence cannot be covered by
the register. This is particularly relevant for migrants in Germany, who are most likely to
2 See, for example, § 31 (5) of the Bavarian Registration Law (http://www.gesetze-bayern.de, retrieved October 18, 2014).
be unfamiliar with the practice of registration (Salentin 2014). In certain circumstances,
moves from one municipality to another are reported only with considerable delay, or
only if proof of current residence is required for other purposes. In addition, some
migrants fail to deregister with their local registration office when they permanently
return to their country of origin. In consequence, in some cases, the addresses of
selected individuals are no longer current (Martin et al. 2015). Moreover, migrants may
be residing in the host country illegally. According to an estimate provided by the
Hamburgisches Weltwirtschaftsinstitut (2010),3 the proportion of illegal immigrants in the
target population in Germany at the time of sample selection in PIAAC Germany was
approximately .5%. However, this figure is subject to change due the current refugee
Furthermore, an additional particularity of the PIAAC study should be mentioned.
Following international standards, the procedure for weighting the German PIAAC data
consisted of several steps (Martin et al. 2013; Zabal et al. 2014). While age and gender
had to be included in the set of variables for the final weighting step, there was no
guideline regarding the inclusion of further variables, such as citizenship or country of birth
(OECD 2010). Weighting adjustment for nonresponse, however, included the variables
citizenship (German vs. non-German), age, and municipality size. In the final weighting
step (benchmarking to external data), the German PIAAC data were adjusted to data
from the 2010 Microcensus4 for age, gender, region, and education level (Zabal et al.
2014). Citizenship was not considered in this step in order to ensure both the inclusion
of the most essential variables and the minimization of the number of weighting cells.
Even though citizenship was used at some point during the weighting process, (similarly
to other countries), it was not benchmarked to population totals in the final step, so that
an almost perfect alignment could not be achieved. As a result, the weighted proportion
of non-German individuals in the PIAAC sample was 9.4%, compared to 10.7% in the
2010 Microcensus data (Zabal et al. 2014, p. 88).
In summary, the target population in PIAAC was the non-institutionalized population
aged 16–65 years residing in the country at the time of data collection. In Germany, this
core population was successfully covered, and the appropriate sampling frame was used
(Zabal et al. 2014). However, it cannot be assumed that the target migrant
subpopulation was fully covered and sampled without survey errors (e.g., because migrant-specific
information was not used for stratification in the sampling procedure). This is also likely
to be the case in other countries participating in PIAAC. Hence, a registry-based
sampling frame can be a limiting factor for answering specific migration-related questions
and for subsequent analyses. For integration-related questions, the extracted survey
data are quite appropriate when migrant is operationalized through foreign citizenship
(Maehler et al. 2015; Salentin 2014). As noted earlier, the German population registers
provide only selected information, such as age, gender, and citizenship. To
operationalize migrant on the basis of place of birth in order to analyze different generations of
migrants, this information must be requested directly during the survey. Subsequently,
3 Information derived from the total stocks of irregular foreign residents in Germany retrieved from the Database on
4 The Microcensus is a mandatory representative survey of one percent of households in Germany.
the sample population can then be compared, for example, with Microcensus data to
retrospectively check the representativeness of the sample, and weighting procedures
can be used post hoc to adjust for deviations.
In the next section, we will summarize the relevant literature on nonresponse, which is
related to outcome rates such as contact and cooperation rates, and the reasons for
nonresponse among the migrant population, particularly in Germany.
Nonresponse among the migrant population: previous findings and theoretical framework
One of the main aspects addressed in this paper is nonresponse error. In the reference
literature, it is specifically classified as a “unit nonresponse error” (Groves and Couper
1998), which occurs when a sampled unit—such as an individual from the migrant
subpopulation—refuses to participate in a face-to-face survey or when an eligible sample
member cannot be contacted (Biemer 2010). As Groves (2006) pointed out,
undercoverage problems resulting from the sampling frame and from nonresponse lead to
underrepresentation of population (sub-)groups. Consequently, parts of the core population
are not adequately represented.
The consequences of the registry-based sampling frame for the coverage of the migrant
population in PIAAC Germany have been discussed above. Nonresponse, on the other
hand, is related to non-contact or to non-cooperation once contacted. In the context of
contact rates, several authors (e.g., Baykara-Krumme 2010; Blohm and Diehl 2001; Koch
1997; Feskens et al. 2006) have reported a low accessibility of migrants (operationalized
by the criterion citizenship) in Germany because of higher mobility (e.g., longer visits to
the country of origin) or due to specific work schedules (e.g., shift work) or
self-employment. It transpires that the probability of making contact with the sample persons, in
particular migrants, is related to the time spent at home. Feskens et al. (2006), for
example, discovered that in several European countries the non-contact rates were higher for
migrants than for non-migrants and that these substantially lower contact rates still held
true when socio-economic status, urbanization, and several other demographic
variables were controlled for. In Germany, it appears that especially older migrants and male
migrants are more difficult to reach (Feskens et al. 2006): Older migrants, and Turkish
migrants in particular, often visit their country of origin for a longer period of time.
Based on nonresponse analyses using German ALLBUS data (1996), Blohm and Diehl
(2001) reported that incorrect addresses were also a reason for non-contact among the
migrant population. However, in more complex analyses based on ALLBUS data from
the year 2000, migration status (citizenship) had no effect on the probability of contact
(e.g., Blohm et al. 2007).
Throughout the literature, there is no consistent evidence that non-cooperation rates
are higher for individuals with a migration background (Font and Méndez 2013). While
Blohm and Diehl (2001) found evidence of lower refusal rates for migrants, Deding et al.
(2013) showed that non-cooperation rates were higher for the migrant groups observed
in their survey. Their investigation of Iranian, Turkish, and Pakistani migrants in
Denmark revealed, that, for migrants, indirect refusal (i.e., other persons refuse contact with
the person in question) occurred more often in the case of women from patriarchal
cultures, for example. Feskens et al. (2006) compared outcome rates from surveys in six
different countries and found that cooperation rates were higher for migrants (in the
authors’ terminology ethnic minorities) than natives. However, they assumed that these
results could have been masked by language problems, as migrants may have had
problems communicating a refusal and were instead coded as not able to participate.
Nonparticipation due to inability was found to be always higher for migrant populations.
This finding is supported by Baykara-Krumme (2010), who observed a high
non-cooperation rate among migrants in Germany, due mainly to language-related issues.
The literature about the probability of cooperation in surveys with migrants is based
on the social isolation hypothesis (e.g., Font and Méndez 2013; Helmschrott and Martin
2014). According to this hypothesis, socially isolated individuals are out of touch with
mainstream society and behave in line with subgroup norms, or rather reject the norms
of the majority. It is assumed that socially isolated individuals will be less likely to accede
to a survey request than non-isolated individuals (Groves and Couper 1998). This might
be the case for individuals who have immigrated to a new country and are not (yet)
integrated into the host society. Thus, it is also associated with the length of stay in a country
and with the question of whether migrants have acquired the citizenship of that country
(by naturalization). These factors may have an effect on the survey cooperation rate as
they are related to different dimensions of integration (cultural, economic, social, and
emotional) in the host country (e.g., Esser 2008; Maehler 2012).
As mentioned above, the non-cooperation rate among migrants in Germany is strongly
related to language issues (Baykara-Krumme 2010; Feskens et al. 2006). The
implementation of surveys in the German language leads to systematic nonresponse among
migrants, and especially among those migrants with a shorter length of stay in the
country (e.g., Salentin 2014). Hence, to ensure a high response rate in PIAAC, strict standards
were established by the international PIAAC Consortium (e.g., Rammstedt et al. 2014).
These requirements were not only essential for sampling the core population, but were,
in part, also suggested in the literature on the surveying of migrant populations (Font
and Méndez 2013). Therefore, we will address related steps aimed at enhancing survey
response in PIAAC.
Enhancing survey response in PIAAC
Fieldwork procedures can have an effect on the response rate of migrants (Feskens et al.
2006; Font and Méndez 2013). Font and Méndez (2013) recommended tailoring
fieldwork procedures to the considerably different survey response behaviors of migrants
and non-migrants. Méndez et al. (2013), for example, proposed strategies such as the
alignment of interviewing times to better suit the schedules of migrants or the provision
of special training to interviewers in order to enable them to adapt to different types of
non-national respondents. It is assumed that these activities could contribute to
achieving higher response rates among migrants and better coverage of that population.
In Germany—as in many other countries—a constant decline in response rates has
been observed in large-scale face-to-face surveys over the last decades (European Social
Survey 2012, 2013; Wasmer et al. 2012). Thus, in PIAAC Germany, great efforts were
made to increase participation of the core population, thereby yielding a strikingly high
overall response rate of 55% (Zabal et al. 2014). The international PIAAC Consortium
defined a detailed set of high quality standards, such as the requirement to achieve a
high response rate (OECD 2010). However, no general recommendations were made to
put specific effort into reaching and including migrants in the sample.
Some of the measures taken, such as introductory material (e.g., advance letter,
brochure) or the use of incentives (Martin et al. 2014; Zabal et al. 2014), addressed the
respondent directly. Other measures, such as an intensive five-day training workshop
prior to fieldwork or thorough quality control and monitoring throughout fieldwork,
were aimed at improving interviewer performance. Special attention was given to
contacting target persons and gaining their cooperation. Interviewers were instructed
to make at least four in-person contact attempts before closing a case. With a view to
increasing the contact rate, interviewers were instructed to contact target persons on
different days of the week and at different times of the day.
Fieldwork was organized into main working phases and several re-issue phases.
Approximately one-third of the sample were considered for re-issuing and were followed
up in one of the re-issue phases. The re-issued cases were mainly soft refusals,5
noncontacts, or sampled persons who had moved to another municipality or who had an
invalid address. For the latter group, procedures were employed to trace the new
addresses of these individuals. In some cases, interviewer reassignments were made. In
order to address more of the population with non-German citizenship, special
documents were developed for the re-issue phase: (a) an endorsement letter from the
German Federal Ministry of Education and Research and (b) advance letters and FAQ
documents in the languages of the major migrant groups in Germany (among others,
Turkish, Polish, and Russian).6
Hypotheses about nonresponse among the migrant subpopulation in PIAAC
In this paper, three main research questions will be addressed using data from PIAAC
Germany. In our first, three-part, research question, we will investigate whether
migrants and non-migrants differ in terms of response rates. When doing so, we will
focus, first, on differences in the outcome rates of migrants and non-migrants and,
second, on differences in the outcome rates of migrants by gender and age group. After
examining the overall outcomes (contact, able to be interviewed, cooperation, and
participation), we will focus in the third part of our first research question on one specific
outcome, namely contact (see the aforementioned findings by, e.g., Baykara-Krumme
2010; Feskens et al. 2006), and compare, in particular, the contact rates among migrants
and non-migrants by age group and gender. As explained above, the only key indicator in
the PIAAC gross sample that can be used to operationalize migration background is
citizenship. Thus, for our analyses migrants are defined as non-German citizens and
nonmigrants as holders of German citizenship. Consequently, our second research question
explores reasons for possible differences in the response rates. And finally, to increase
participation of migrants in future surveys, it is important to know when the target
subgroups are reachable and whether they differ from non-migrants in this regard. This is
the subject of our third research question.
5 Reasons for a refusal are divided into “hard” and “soft”: Hard refusals include reasons that do not allow the re-approach
of a target person by an interviewer (e.g., data confidentiality), whereas cases with soft refusals (such as “no time”) may
legally be recontacted.
6 For more details, see the technical report for PIAAC Germany (Zabal et al. 2014).
As discussed in the literature, response rates of persons with a migration background
are lower than those of natives. We will investigate, first, whether migrants and
nonmigrants differ in their response behavior, and we will test the following hypothesis:
(1.1) The overall outcome rates for migrants are lower than for non-migrants.
Focusing only on migrants’ outcome rates, it is assumed that, as proposed in the
literature (e.g., Feskens et al. 2006), older migrants and male migrants were more difficult
to reach. Thus, transferring these findings to PIAAC Germany, we will test the following
(1.2) The outcome rates for older migrants are lower than for younger migrants.
(1.3) The outcome rates for migrant males are lower than for migrant females.
After investigating the overall outcome rates of migrants, we will compare the
contact rates of migrants and non-migrants. As proposed in the literature, we assume that
the contact rates of migrants are lower than those of non-migrants. We will investigate
whether this assumption is valid for different age groups. Due to the high mobility and
specific work schedules of migrants (Feskens et al. 2006), and their higher rate of
selfemployment, the third part of our first research question asks whether the proportion of
non-contacted males was higher among migrants than among non-migrants. Thus, the
following hypotheses will be tested:
(1.5) The contact rate for migrant males is lower than for non-migrant males.
Our second research question focuses on the main reasons for nonresponse among
migrants in PIAAC Germany. Helmschrott and Martin (2014) investigated the potential
for nonresponse bias in the PIAAC Germany data and found that being a migrant
correlated with nonresponse: Migrants were indeed significantly less likely to participate
than non-migrants. Thus, we ask: How do migrants and non-migrants differ in their
response behavior and what are the reasons for nonresponse? In accordance with the
literature (i.a., Blohm and Diehl 2001), it is expected that nonresponse among migrants
is due mainly to (1) language problems, (2) refusals (direct, or indirect through other
persons), and (3) address-related reasons (e.g., invalid address or the person has moved).
As the survey language is usually the official language of the country in question, it could
be expected that language problems are the major cause of nonresponse. Consequently,
we are interested in testing whether refusals, language problems, and address-related
reasons are the main causes of nonresponse among migrants, and whether migrants
differ from their non-migrant counterparts because of address-related and literacy-related
reasons. The following hypothesis will be tested:
(2) A low response rate of migrants compared to non-migrants is related mainly to
refusals, language problems, and address-related reasons.
Our third research question asks: When are migrants reachable? According to a study
of households in the Netherlands conducted by Stoop (2004), the chances of contacting
the total population were higher in the evening. Similarly, Blohm et al. (2007) stated
that interviewers working primarily in the afternoon were more successful in
contacting target persons from the German population compared to other times of the day,
and that the contact rate was lower on the weekend. To help achieve a higher response
rate of migrants and a better coverage of the migrant population, Méndez, Ferreras, and
Cuesta (2013) proposed strategies such as the alignment of interviewing times to better
suit the needs of the foreign population. In order to meet these requirements in PIAAC
Germany, interviewers were—among other measures—instructed to establish contact
with the sampled persons on different days of the week and at different times of the day
(Zabal et al. 2014). The PIAAC data offer a vast dataset with which to explore contact
rates by time. Thus, we will explore whether the contact rate is related to the
indicators for contact time (namely, the time of day, the day of the week, and the period of the
year). Based on the few findings in the literature, we will investigate if the contact rates
of migrants and non-migrants are correlated with contact times, in particular we
investigate the following hypothesis:
(3.1) The probability to contact both migrants and non-migrants is higher during the
evening than during other times of the day.
(3.2) The probability to contact both migrants and non-migrants is higher during the
week than during the weekend.
(3.3) The probability to contact both migrants and non-migrants during holidays is
lower than during the other periods of the year.
Methods and data
For our analyses, we need data that are available for both respondents and
nonrespondents. Thus, we use frame information (such as first citizenship, age, and gender) as
well as auxiliary variables and paradata such as disposition codes or contact data from
interviewers collected in Germany (Rammstedt et al. 2014) during the PIAAC
fieldwork phase (August 2011–March 2012). PIAAC is designed to provide representative
measures of cognitive skills of adults aged 16–65 years. In PIAAC, a sampled person
is defined as a completed case if the person completed an adequate proportion of the
background questionnaire and at least some basic part of the cognitive assessment, or
if he or she was classified as a literacy-related nonrespondent for whom age and gender
were collected (OECD 2010, 2013b). Literacy-related reasons for nonresponse include
language problems, reading and writing difficulties, and learning or mental disabilities.
According to the OECD (2013a), these respondents tended to have lower proficiency
levels. In the German PIAAC net sample, approximately 1.6% were literacy-related
nonrespondents (.8% non-migrants). As stated above, we have to use the information about
citizenship as a proxy for the migration status. Hence, in what follows, persons who are
not holders of German citizenship are defined as migrants and persons who hold
German citizenship are defined as non-migrants.
The unweighted gross sample consists of N = 10,240 cases. Based on the frame
information first citizenship, n = 931 target persons are classified as migrants and n = 9049 as
non-migrants.7 51.7% of the migrants are males, compared to 50.4% of the non-migrants.
The average age of migrants (38 years) is slightly lower than that of non-migrants
(41 years). Regarding citizenship, the largest proportion of the migrants hold a Turkish
passport (22.1%), followed by Italian (7.8%), Polish (7.3%), Greek (5.2%), former
Yugoslavian (4.3%), Russian (3.4%), and Croatian (3.1%) passport holders.
To answer the first (three-part) research question, and to test the hypotheses derived
from it (1.1 to 1.5) regarding the outcome rates in general and the contact rates in
particular, we used PIAAC disposition codes and computed outcome rates according to
AAPOR standards (The American Association for Public Opinion Research 2016)8:
• Contact (following AAPOR CON1: I + P + R + O/I + P + R + O + NC + U)
• Able to be interviewed (I + P + R/I + P + R + O)
• Cooperation (following AAPOR COOP4: I + P/I + P + R
• Participation (following AAPOR RR2: I + P/I + P + R + O + NC + U
To investigate the reasons for non-participation (our second research question), we
focused on the disposition codes used in PIAAC Germany, as these differentiate several
literacy- and address-related reasons for nonresponse. We used the final distribution of
disposition codes for the unweighted German gross sample, separated by citizenship.9
For comparison purposes, a differentiation similar to that in the technical report for the
overall population (Zabal et al. 2014, p. 76) was chosen.
To analyze contact rates by time as outlined in our third research question, we used
the paradata, that is, the data provided by interviewers in their contact protocols (in
PIAAC, these protocols are called case folders). The PIAAC case folder is a document
that is available for each sampled person and is used by the interviewer to record all
contact activities (such as date, number and time of contact attempts, and the result of each
contact or contact attempt). The majority of the sampled individuals were successfully
contacted in one of the first two contact attempts. For example, among migrants the first
contact attempt was successful in 37.1% of the cases and among Germans in 36.2% of the
Hence, to test the third hypothesis (that the contact rates of migrants and of
nonmigrants are correlated with contact time), we used three time indicators from the
PIAAC paradata for the first contact (attempt),10 namely the time of day, the day of the
week, and the period of the year. The time of the day was categorized into three ranges:
before lunch (12 am), after lunch (12 pm to 5 pm) and in the evening (after 5 pm). The
days of the week were grouped into four periods: (1) Monday/Tuesday, (2) Wednesday,
(3) Thursday/Friday, and (4) Saturday/Sunday. And finally, the period of the year was
categorized into school holidays (no/yes) and religious holidays (no/yes). To address school
holidays, we used the information about school holidays in the respective German
7 The citizenship status of 260 persons was either not provided by the population registers or it was not recorded in the
register. One hundred and thirty-six of these respondents participated in the PIAAC interview, 38 of whom reported
that they had non-German citizenship.
8 I = interviews, P = partials, R = refusals, O = other, NC = non-contacts, U = unknown.
9 See also footnote 7.
10 Unfortunately, the evaluation of subsequent contact attempts had to be omitted because it could not be ensured that
the timing for subsequent attempts occurred at random and independently of previous attempts (e.g., no appointments
between interviewer and sample unit were made).
federal states during the assessment time (summer, winter, and autumn holidays). To
address religious holidays, we categorized Easter and Christmas time in the respective
years of assessment. And finally, we also controlled for gender and age. To predict the
contact probability, we performed separate regression analyses for migrants and
nonmigrants. The dichotomous variable contact (yes/no) was used as an independent
Do migrants and non‑migrants differ in their response behavior?
To provide an overview on the response rate of migrants in PIAAC Germany, and to
verify the assumptions in literature, we coded survey outcomes in accordance with the
AAPOR standards into four groups: (a) contact, (b) able to be interviewed,11 (c)
cooperation, and (d) participation. Table 1 provides a descriptive overview categorizing migrants
and non-migrants by gender and age group.
First, it can be noted that, for all four fields, migrants’ outcome rates are lower
than the outcome rates of non-migrants (hypothesis 1.1). There is a difference of
11.7 percentage points [χ2 (1) = 138.915, p < .001] for the contact rate, 10.4
percentage points [χ2 (1) = 209.023, p < .001] for the ability to be interviewed, 4.5 percentage
points [χ2 (1) = 4.869, p < .05] for the cooperation rate, and 15.2 percentage points
[χ2 (1) = 74.712, p < .001] for the response rate.
Looking at migrants only, it can be observed that some outcome rates vary significantly
across age groups (hypothesis 1.2). While the contact rates are fairly closely distributed
around 80% across age groups [χ2 (4) = 7.980, p < .10], there is a steady decline in the
cooperation rate [χ2 (4) = 40.928, p < .001] and the participation rate [χ2 (4) = 46.951,
p < .001] from the youngest to the oldest age group.
The evaluation of the outcome rates by gender (hypothesis 1.3) show that male
migrants are significantly more difficult to contact than female migrants [χ2 (1) = 5.339,
p < .021]. However, contrary to the expectation, a significant difference between genders
could not be confirmed with regard to migrants’ cooperation and participation rates.
The next question of interest is whether there are differences in contact rates between
migrants and non-migrants across age groups (hypothesis 1.4). While, for the
nonmigrant population, the highest contact rate (95.9%) can be observed for the
oldest age group (55–65 years), migrants in the middle working-age group (35–44 years)
show the highest contact rate (86.1%). For both migrants and non-migrants, the contact
rate for individuals between the ages of 25 and 34 is the lowest compared to other age
groups. Comparing contact rates of migrants and non-migrants by age shows that, for
each inspected age group, the contact rate for migrants is significantly lower than for
non-migrants (16–24: χ2 (1) = 15.933, p < .001; 25–34: χ2 (1) = 21.415, p < .001; 35–44:
χ2 (1) = 9.862, p < .01; 45–54: χ2 (1) = 43.035, p < .001; 55–65: χ2 (1) = 66.687, p < .001).
Table 1 shows that male migrants have the lowest contact rate overall (77.6%)
compared to female migrants (83.7%), non-migrant males (91.2%), and non-migrant females
(93.5%). In accordance with the hypothesis (1.5), the probability of being contacted
11 This category is not specified in the AAPOR standards.
G A N m % 9 9 9 9 9 9 9 9 8 a
l 5 A
in ta .9 .7 .4 .3 .7 .1 .8 .3 04 A
su To % 98 29 19 68 19 29 49 19 01 teh
n ts iifc
o n e
it ra sp
irag igM % .776 .837 .823 .768 .861 .781 .784 .806 880 tson
How do migrants and non‑migrants differ in their response behavior and what are the
reasons for nonresponse?
Table 2 shows the final distribution of disposition codes for the unweighted non-migrant
gross sample, separated by migration status (operationalized by citizenship). In general,
the main reason for the nonresponse of migrants (28.8%) and non-migrants (34.4%) was
refusal by the sample person. Refusals were more common among non-migrants than
migrants (z = 1.95, p = .05). It can also be seen that an address-related issue, for
example an invalid address, was apparently more often the reason for nonresponse among
migrants than among non-migrants (z = 2.36, p = .02). In line with the hypothesis,
migrants differed from non-migrants with regard to nonresponse due to literacy-related
Table 2 Disposition codes for PIAAC Germany by migration status
Sorted by most frequently cited reasons by migrants
reasons, such as language problems or reading or writing difficulties (.6% vs. 7.7%;
z = 2.16, p = .02). Furthermore, Table 2 shows that non-contact with the household
or the sample person seems to be more likely in the case of migrants (7.6%) than
nonmigrants (5.3%), even though this difference does not reach significance. Migrants also
appear to be more often unreachable because they have left Germany (3.5% vs. .5%, n.s.).
In summary, migrants and non-migrants differed in their response behavior,
particularly with regard to address-related and literacy-related reasons. Thus, in line with the
hypothesis (2), a low response rate of migrants was related mainly to refusals, language
problems, and address-related reasons.
If we look at reasons for nonresponse between genders, Table 3 shows a similar picture
for migrants and non-migrants: There are no significant differences. However, among
migrants, it appears that literacy-related reasons were more often a cause of
non-participation on the part of females than males (10.2 vs. 5.4%, n.s.). By contrast, male migrants
appear to have been harder to reach due to address-related issues than female migrants
(12.1 vs. 7.3%, n.s.).
Looking closer at response behavior by age, Table 4 shows the results for hypothesis
1.2 that across all age groups the rate of completed interviews is higher for non-migrants
than for migrants (z ranges between 2.03 and 4.08, ps = .00). This difference is
particularly visible for the older age groups of migrants.
Across all age groups, nonresponse due to address-related issues was slightly
more pronounced for migrants than for non-migrants. In addition, among migrants,
nonresponse due to literacy-related reasons slightly increases across age groups and
was higher for older migrants than for non-migrants in the comparable age group (12.3
vs. .9%, n.s.).
When are migrants reachable?
In the next step, the contact rate for the first attempt was predicted using the time
indicators day of the week, time of the day, and time of the year (school and religious
holidays) as independent variables, and controlling for gender and age (see Table 5).
Table 3 Disposition codes for PIAAC Germany by migration status and gender
For 29 cases we have no gender information. These cases were excluded from the analyses
5 M 2 .8% .39 1 3 7 5 3 7 1
m r s
r a se
e e ts a
G y n c
C 24 ra % % see
IAA –61 igM .055 .7% .616 .62% .33% .68% .0% .33% .99% 151 .Thn
Day: evening vs. before lunch .435* .227 .793** .083
Day: evening vs. after lunch .176 .144 .465** .050
Day of the week: Sat./Sun. vs. Mon./Tues. −.212 .193 .466** .071
Day of the week: Sat./Sun. vs. Wed. .439 .235 .496** .079
Day of the week: Sat./Sun vs. Thurs./Fri. .062 .205 .506** .074
Year: school holidays (reference = no) −.405** .138 −.023 .048
Year: religious holidays (reference = no) −.291 .664 −.465 .307
Gender (reference = male) .345** .133 .336** .047
Age .013* .005 .017** .002
Migrant subpopulation: n = 1176. X2 = 35 (df = 9); −2 log likelihood = 1.341; Cox and Snell R2 = .03; Nagelkerke R2 = .04.
RN2o =n‑m .0i5g.rTahntesWubalpdosptautliasttiiocnh:ans =a C8h9i4s7q. uXa2r e=d 3is0t9rib(duft =io n9)w;−ith2 1lodgf.liRkeefliehroenocde =c a1t0e.g7o0r8y; CcoodxeadndasS0nell R2 = .03; Nagelkerke
* Significant at the .01 level
** Significant at the .001 level
As can be seen in Table 5, the results show that, among migrants, controlling for
gender and age, the time of the day, and the period of the year had a significant impact on
the probability of being contacted. The probability of contacting a migrant individual
was higher in the evening (in line with hypothesis 3.1) compared to before lunch time
(p = .05). However, contrary to the assumption, for migrants the probability of contact
is not related to the time within the week (hypothesis 3.2). The criterion time of
contact within the year (hypothesis 3.3) indicates that migrants were not equally reachable
during the school holidays (summer, winter, and autumn). During school holidays, for
instance, the probability of contacting people with a migration background was lower
(p = .001) than outside the school holidays. By contrast, the contact rate of migrants is
not correlated with periods of the year that are religious holidays.
For the non-migrant subpopulation, the time of the day and the day of the week had
a significant effect on the probability of being contacted. The evening was (in line with
hypothesis 3.1) the best time to reach non-migrants in comparison to times before or
after lunch (ps = .001). In addition, the probability of being contacted was (contrary to
hypothesis 3.2) significantly higher on Saturdays and Sundays, compared to the other
periods of the week, Monday/Tuesday, Wednesday, or Thursday/Friday. In contrast to
the migrant subpopulation, the period of the year (school holidays) does (as assumed in
hypothesis 3.3) not seem to have had an influence on the probability of being contacted.
Global migration has considerably increased in the last decade and has been
overwhelming for many European countries. Within a short time, the number of refugees and
asylum seekers has risen considerably in many countries. In the coming months and years,
these people will become part of the societies of the European countries concerned.
Large-scale social surveys, such as future cycles of PIAAC, will be confronted with this
fact and will have to assess whether and how to cover this group of people in the survey
design. Using data from Germany, a country that is strongly affected by migration flows,
as a case example, the present article addresses substantial questions related to the
nonresponse behavior of migrants.
With the aim of examining the migrant subpopulation, the first objective of the
present contribution was to find out more about response behavior differences between
the migrant subpopulation and the majority population and differences in the outcome
rates within the migrant subpopulation by age and gender. As the results confirm, the
overall outcome rates were lower for migrants than for non-migrants. Looking only at
the migrant subpopulation, it was revealed, as expected, that outcome rates—in
particular the cooperation rate and the response rate—were lower for older than for younger
migrants. With regard to outcome rates by gender, the only effect was found for the
contact rates. In line with previous findings, male migrants were more difficult to contact
than female migrants. By further investigating contact rate differences between migrants
and non-migrants, it could be observed that contact rates were lower for migrants across
all age groups. It should be emphasized that the highest contact rate among migrants
was observed in the middle working-age group (35–44 years), whereas for the
nonmigrant subpopulation the highest contact rate was found for the oldest age group
(55–65 years). In addition, the data revealed that male migrants were the most difficult
to contact. Overall, the results confirm the expectations in the reference literature. It
is noteworthy, however, that migrants in the middle working-age group (35–44 years)
were the most accessible group. We assume that this fact may be related to their specific
Addressing the second main research question, reasons for differences in the response
behavior of migrants and non-migrants were further investigated. The PIAAC results are
in line with the literature. Migrants and non-migrants differed in their response
behavior, particularly due to address- and literacy-related reasons. It could be observed that
especially literacy-related reasons (including language problems), refusals, and
addressrelated reasons (unavailability during the field period) had an impact on the response
behavior of migrants.
Finally, we looked for the best time to contact the migrant and non-migrant
subpopulations. We did not find any evidence relating to the migrant population in the previous
literature. As our results revealed, contact attempts for both groups were most
successful in the evening, compared to before and after lunch. The day of the week (e.g.,
Monday/Tuesday vs. Saturday/Sunday) of the contact attempt does not appear to affect the
reachability of migrants. However, compared to other periods of the week, the weekend
was the most appropriate time to contact the non-migrant subpopulation. In line with
the literature, school holiday time led to a lower contact rate, but only for the migrant
subpopulation. As the present analyses focused on the first contact attempt only, future
research should investigate whether interviewers varied contact day or contact time for
subsequent contact attempts and whether this had an effect on the contact rate of the
more reluctant migrant subpopulation.
Limitations when surveying migrants and suggestions for future research
Addressing limitations of large-scale surveys in general, it must be noted that, on the
one hand, they aim to cover the whole resident population, but, on the other hand, they
exclude de facto quite a large proportion of the population if survey instruments are
administered in only one language version. The core resident population is restricted to
those who have sufficient proficiency in the respective country’s official language. Not
only new migrants, whose social inclusion process would be interesting to investigate,
but also long-resident migrants are more likely to be excluded because they tend to
refuse to participate in the survey.
The results from PIAAC Germany indicate that, to some extent, the migrant
subpopulation was not covered by the survey. This was due most probably to language
barriers. Assuming that this problem also exists in several other OECD countries, it could
be discussed whether measures could be taken to better address the migrant
subpopulation in future cycles of PIAAC. These measures could include recruiting bilingual
interviewers (Méndez et al. 2013) or translating questionnaires into several languages
within countries. In some countries, such as Austria or the United States, for example,
the PIAAC background questionnaires were translated for a substantial percentage of
the migrant population. However, it must be considered that implementing such
measures to account for diverse cultural backgrounds can involve some constraints, such as a
significant increase in costs.
Besides the above-mentioned potential measures to better address the migrant
population, the selection of a separate migrant sample could be considered (e.g., as an add-on
study). With the increasing number of refugees and related illegal immigrants not only
in Germany but also across Europe, it appears to be even more advisable to consider
this approach. A separate migrant sample would allow researchers to focus on specific
analyses with regard to labor market and social integration processes. To find out more
about integration processes and acculturation, it would, for example, be advisable for
future research to compare migrants from the same country of origin (e.g., Syria, Turkey,
Eritrea, Ghana) across different PIAAC countries. However, if the survey focus were on
refugees only, PIAAC countries vary considerably regarding the accessibility of refugees
for interviewing (e.g., because they live in refugee housing facilities). This is due mainly
to differences in the legal regulation systems (from arrival to application for asylum) and
the speed with which migrants are structurally and socially integrated. Furthermore,
comparisons across countries will be limited because (a) the countries of origin of the
current refugees vary across the different host countries, and (b) the proportion of
refugees per country may vary substantially (see, e.g., EUROSTAT 2016). In Germany, for
instance, some refugees who migrated to Germany in the last few years or months might
get a residence permit and thereby become part of the resident population (and thus the
target population) and might be selected and interviewed in a future cycle of PIAAC.
For others without a residence permit, or for illegal immigrants, for example, the use of
population registers as a sampling frame may lead to exclusion and thus render it
necessary to consider alternative sampling approaches. These are precisely the challenges
in survey research that researchers and survey institutes in Germany are dealing with
at present (see, e.g., the feasibility study conducted by the Expert Council of German
Foundations on Integration and Migration; Schiefer 2016). Implementing a cluster
sample of local registration offices might be an option. However, these data do not contain
information about the residency status, for example (one of several possible selection
criteria). In addition, the data are not equally distributed with regard to the countries of
origin, or the address quality is poor (e.g., strong fluctuation; Schiefer 2016).
Returning to limitations regarding the registry-based sampling frame that were
illustrated exemplarily for Germany in this article, and to the definition of the term migrant,
a major challenge appears to persist. In the case of Germany, a large percentage of
individuals are classified as non-migrants (according to their citizenship) for the purposes
of analyzing the contact rate, but, according to their birthplace, they have a migration
background. The information about the place of birth would allow for more accurate
nonresponse analyses and thus the planning of precise measures to increase outcome
rates, in particular the contact rate.
The aim in large-scale surveys such as PIAAC could be to obtain more reliable data
(e.g., large N) for the migrant population that can be compared with the general
population, and to survey a sufficient proportion of different migrant groups (at least the
largest ones). Non-contacts or refusals are not the only main obstacle to achieving high
response rates of migrants. As Font and Méndez (2013) pointed out, a further obstacle
is the fact that some migrant groups are difficult to locate. For random samples of
population registers, the authors suggested obtaining samples that are much larger than the
final sample one wants to achieve. Due to different patterns of response rates between
migrants and non-migrants, Morales and Ros (2013) recommended designing fieldwork
procedures tailored to the considerably different survey response behaviors of these
different groups. Méndez et al. (2013), for example, endorsed strategies such as
providing training to interviewers to enable them to adapt to different types of non-national
respondents, which could help to achieve a higher response rate by migrants and a
better coverage of the population.
In conclusion, for the next PIAAC cycle, it would be important to put more effort into
reaching and including the migrant subpopulation in the sample. In addition to gaining
representativeness in terms of their proportion in general, a more accurate coverage of
migrants should also be pursued, for example with regard to length of residency, skills
in the host country’s language, country of origin, and other sociodemographic
characteristics. It can be assumed that the composition of the migrant population in a country
is diverse and that migrants differ with regard to their profiles and nonresponse
behavior. For example, recently arrived migrants, in particular, presumably have lower skills
in the language of the host country and are thus more likely to achieve comparatively
low results in literacy and to be more prone to nonresponse. Addressing these
circumstances, thought should first be given to whether the definition of the PIAAC target
population should, for example, deliberately include or exclude recently arrived migrants. If
a re-definition of the target population is not a suitable option, it would be interesting
to further pursue measures to overcome language barriers as reasons for
non-participation. Results presented in this paper show language to be a barrier to contact and
participation of migrants in PIAAC, so that translation of the background questionnaire,
for the largest migrant groups at least, seems reasonable. Last but not least, it is
important to evaluate other PIAAC countries’ experiences in surveying migrants and to derive
therefrom specific recommendations for better coverage of the migrant subpopulations.
In a further step, this would enable the formulation of specific guidelines for
addressing migrants in the sample (e.g., specific contact strategies, use of bilingual
interviewers, etc.) and thus facilitate the international comparability of the outcomes for migrant
Work on this paper was supported by the College for Interdisciplinary Educational Research (CIDER), a joint initiative of
the German Federal Ministry of Education and Research (BMBF), the Jacobs Foundation, and the Leibniz Association. The
data used were provided by GESIS-Leibniz Institute for the Social Sciences.
DM had the lead for the manuscript and is expert in migration research. SM is expert for PIAAC and survey operations, in
particular nonresponse analyses. BR was national project manger for PIAAC Germany. DMand SM wrote the manuscript
and performed the analyses. DM, SM and BR revised the manuscript. AllAuthors read and approved the final manuscript.
Baykara-Krumme , H. ( 2010 ). Interviewereffekte in Bevölkerungsumfragen. Ein Beitrag zur Erklärung des Teilnahme- und Antwortverhaltens von Migranten . Working Paper of the Germany Family Panel No . 19: TU Chemnitz . http://www. pairfam.de/uploads/tx_sibibtex/workingpaper_19p.pdf. Accessed 12 March 2014 .
Biemer , P. P. ( 2010 ). Total survey error . Design , implementation, and evaluation. Public Opinion Quarterly , 74 ( 5 ), 817 - 848 . doi:10.1093/poq/nfq058.
Blohm , M. , & Diehl , C. ( 2001 ). Wenn Migranten Migranten befragen . Zeitschrift für Soziologie , 30 ( 3 ), 223 - 242 . doi:10.1515/ zfsoz- 2001 - 0304 .
Blohm , M. , Hox , J. , & Koch , A. ( 2007 ). The influence of interviewers' contact behavior on the contact and cooperation rate in face-to-face household surveys . International Journal of Public Opinion Research , 19 ( 1 ), 97 - 111 . doi:10.1093/ijpor/ edh120.
Deding , M. , Fridberg , T. , & Jakobsen , V. ( 2013 ). Non-response among immigrants in Denmark . In J. Font & M. Méndez (Eds.), Surveying ethnic minorities and immigrant populations (pp . 173 - 194 ). Amsterdam: Amsterdam University Press.
Esser , H. ( 2008 ). Assimilation, ethnische Schichtung oder selektive Akkulturation? Neuere Theorien der Eingliederung von Migranten und das Modell der intergenerationalen Integration . Kölner Zeitschrift für Soziologie und Sozialpsychologie , 48 , 81 - 107 .
European Social Survey . ( 2012 ). ESS5-2010 Documentation Report . The ESS Data Archive. ( 3 .0 ed.). Bergen: Norwegian Social Science Data Services . http://www.europeansocialsurvey.org/docs/round5/survey/ESS5_data_ documentation_report_e03 _0.pdf. Accessed 24 March 2014 .
European Social Survey ( 2013 ). ESS6-2012 Documentation Report . The ESS Data Archive ( 1 .3 ed.). Bergen: Norwegian Social Science Data Services . http://www.europeansocialsurvey.org/docs/round6/survey/ESS6_data_ documentation_ report_e01 _3.pdf. Accessed 24 March 2014 .
Feskens , R. , Hox , J. , Lensvelt-Mulders , G. , & Schmeets , H. ( 2006 ). Collecting data among ethnic minorities in an international perspective . Field Methods , 18 ( 3 ), 284 - 304 . doi:10.1177/1525822X06288756.
Font , J. , & Méndez , M. (Eds.). ( 2013 ). Surveying ethnic minorities and immigrant populations . Amsterdam: Amsterdam University Press.
Groves , R. M. ( 2006 ). Nonresponse rates and nonresponse bias in household surveys . Public Opinion Quarterly , 70 ( 5 ), 646 - 675 . doi:10.1093/poq/nfl033.
Groves , R. M. , & Couper , M. P. ( 1998 ). Nonresponse in household interview surveys . New York : John Wiley. doi:10.1002/9781118490082.
Hamburgisches Weltwirtschaftsinstitut ( 2010 ). Stocks of irregular migrants: Estimates for Germany . http://irregular-migration.net//typo3_upload/groups/31/3.Database_on_IrregMig/3.2. Stock _ Tables/GeGerma_Estimates_IrregularMigration_Dec10 .pdf. Accessed 6 Sept 2011 .
Helmschrott , S. , & Martin , S. ( 2014 ). Nonresponse in PIAAC Germany . Methods, Data, Analyses, 8 ( 1 ), 243 - 266 . doi:10.12758/ mda.2014.010.
Hogan , J. , Thornton , N. , Diaz-Hoffmann , L. , Mohadjer , L. , Krenzke, T. , Li , J. , VanDeKerckhove , W., Yamamoto , K. , and Khorramdel , L. ( 2016 ). U.S. Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014: Main study and national supplement technical report (NCES 2016 -036). U.S. Department of Education. Washington, DC: National Center for Education Statistics . http://nces.ed.gov/pubs2016/2016036.pdf. Accessed 1 Sept 2016 .
Koch , A. ( 1997 ). ADM-Design und Einwohnermeldeamtsregister-Stichprobe: Stichprobenverfahren bei mündlichen Bevölkerungsumfragen . In S. Gabler & J. H. P. Hoffmeyer-Zlotnik (Eds.), Stichproben in der Umfragepraxis [Sampling in survey practice] (pp. 99 - 116 ). Opladen: Westdeutscher Verlag.
Maehler , D. B. ( 2012 ). Akkulturation und Identifikation bei eingebürgerten Migranten in Deutschland [Acculturation and identification among naturalized immigrants in Germany] . Münster: Waxmann.
Maehler , D. B. , Massing , N. , & Rammstedt , B. ( 2014 ). Grundlegende Kompetenzen Erwachsener mit Migrationshintergrund im internationalen Vergleich: PIAAC 2012 [Basic skills of adults with a migration background in international comparison : Results of PIAAC 2012 ]. Münster : Waxmann.
Maehler , D. B. , Teltemann , J. , Rauch , D. & Hachfeld , A. ( 2015 ). Die Operationalisierung des Migrationshintergrunds [The operationalization of migration background] . In Maehler, D.B. & Brinkmann , U. H. (Eds.), Methoden der Migrationsforschung . Ein interdisziplinärer Forschungsleitfaden [Methods of migration research] (pp. 261 - 282 ). Wiesbaden: Springer VS. doi:10.1007/978- 3 - 658 - 10394 -1.
Martin , S. , Helmschrott , S. , & Rammstedt , B. ( 2014 ). The use of respondent incentives in PIAAC: The field test experiment in Germany . methods, data, analyses, 8 ( 2 ), 223 - 242 . doi:10.12758/mda.2014.009
Martin , S. , Maehler , D. B. , Behr , B. & Pötzschke , S. ( 2015 ). Methodische Grundlagen der quantitativen Migrationsforschung [Methodological foundations of quantitative migration research] . In D. B. Maehler & U.H. Brinkmann (Eds.), Methoden der Migrationsforschung . Ein interdisziplinärer Forschungsleitfaden [Methods of migration research] (pp. 17 - 60 ). Wiesbaden: Springer VS. doi:10.1007/978- 3 - 658 - 10394 -1.
Martin , S. , Zabal , A. , Helmschrott , S. , Ackermann , D. , Massing , N. , & Rammstedt , B. ( 2013 ). Qualitätssicherung, Design und Datenqualität . In B. Rammstedt (Ed.), Grundlegende Kompetenzen Erwachsener im internationalen Vergleich-Ergebnisse von PIAAC 2012 [Basic skills of adults in international comparison : Results of PIAAC 2012 ] (pp. 167 - 182 ). Münster: Waxmann.
Méndez , M. , Ferreras , M. , & Cuesta , M. ( 2013 ). Immigration and general population surveys in Spain: The CIS Surveys . In J. Font & M. Méndez (Eds.), Surveying ethnic minorities and immigrant populations (pp . 196 - 217 ). Amsterdam: Amsterdam University Press.
Mohadjer , L. , Krenzke, T & Van de Kerckhove , W. ( 2013 ). Sampling design . In OECD (Ed.), Technical Report of the Survey of Adult Skills (PIAAC) (Chapter 14; pp. 1 - 31 ). Paris: OECD. http://www.oecd.org/site/piaac/surveyofadultskills.htm. Accessed 16 April 2014 .
Morales , L. & Ros , V. ( 2013 ). Comparing the response rates of autochthonous and migrant populations in nominal sampling surveys: The LOCALMULTIDEM study in Madrid . In. J. Font & M. Méndez (Eds.), Surveying ethnic minorities and immigrant populations (pp . 148 - 172 ). Amsterdam: Amsterdam University Press.
OECD. ( 2010 ). PIAAC technical standards and guidelines . http://www.oecd. org/site/piaac/PIAACNPM%282014_06%29PIAAC_Technical_Standards_and_Guidelines.pdf. Accessed 06 Sept 2011 .
OECD. ( 2013a ). OECD skills outlook 2013 : First results from the survey of adult skills . Paris: OECD Publishing . doi:10.1787/9789264204256-en.
OECD. ( 2013b ). Technical report of the Survey of Adults Skills (PIAAC) . Paris: OECD.
OECD. ( 2016 ). The survey of adult skills: Reader's companion (Second edition) . Paris: OECD Publishing. doi:10.1787/9789264204027-en.
Rammstedt , B. , Martin , S. , Zabal , A. , Massing , N. , Ackermann-Piek , D. , Helmschrott , S. , Perry , A. , & Maehler , D. ( 2014 ). Programme for the International Assessment of Adult Competencies (PIAAC), Germany-Master Dataset . Unpublished data . Mannheim: GESIS-Leibniz Institute for the Social Science .
Salentin , K. ( 2014 ). Sampling the ethnic minority population in Germany . The background to “ migration background” . Methods, Data, Analyses, 8 ( 1 ), 25 - 52 . doi:10.12758/mda.2014.002.
Schiefer , D. ( 2016 , February). Studying life circumstances of refugees in Germany: A feasibility analysis . Paper presented at the GESIS Vortragsreihe , Mannheim.
Statistik Austria ( 2013 ). Schlüsselkompetenzen von Erwachsenen . Erste Ergebnisse der PIAAC-Erhebung 2011 /12. Wien: Statistik Austria.
Stoop , I. A. L. ( 2004 ). Surveying nonrespondents . Field Methods , 16 ( 1 ), 23 - 54 . doi:10.1177/1525822X03259479.
The American Association for Public Opinion Research . ( 2016 ). Standard definitions: Final dispositions of case codes and outcome rates for surveys. 9th edition . AAPOR.
Wasmer , M. , Scholz , E. , Blohm , M. , Walter , J. , & Jutz , R. ( 2012 ). Konzeption und Durchführung der “Allgemeinen Bevölkerungsumfrage der Sozialwissenschaften” (ALLBUS) 2010 . GESIS Technical Reports. Cologne: GESIS-Leibniz Institute for the Social Sciences.
Zabal , A. , Martin , S. , Massing , N. , Ackermann , D. , Helmschrott , S. , Barkow, I. , et al. ( 2014 ). PIAAC Germany 2012 : Technical report. Münster: Waxmann.