The Autism Family Experience Questionnaire (AFEQ): An Ecologically-Valid, Parent-Nominated Measure of Family Experience, Quality of Life and Prioritised Outcomes for Early Intervention
Journal of Autism and Developmental Disorders
The Autism Family Experience Questionnaire (AFEQ): An Ecologically- Valid, Parent-Nominated Measure of Family Experience, Quality of Life and Prioritised Outcomes for Early Intervention
Kathy Leadbitter 0 1 2 3 4 5 6 7
Catherine Aldred 0 1 2 3 4 5 6 7
Helen McConachie 0 1 2 3 4 5 6 7
Ann Le Couteur 0 1 2 3 4 5 6 7
Dharmi Kapadia 0 1 2 3 4 5 6 7
Tony Charman 0 1 2 3 4 5 6 7
Wendy Macdonald 0 1 2 3 4 5 6 7
Erica Salomone 0 1 2 3 4 5 6 7
Richard Emsley 0 1 2 3 4 5 6 7
Jonathan Green 0 1 2 3 4 5 6 7
The PACT Consortium 0 1 2 3 4 5 6 7
0 Institute of Psychiatry, Psychology & Neuroscience, King's College London , London , UK
1 Northumberland, Tyne & Wear NHS Foundation Trust , Newcastle upon Tyne , UK
2 Newcastle University , Newcastle upon Tyne , UK
3 Social Development Research Group, University of Manchester , Room 3.312, Jean McFarlane Building, Oxford Rd., Manchester M13 9PL , UK
4 Manchester Academic Health Sciences Centre, Royal Manchester Children's Hospital , Manchester , UK
5 University of Turin , Turin , Italy
6 Greater Manchester Mental Health NHS Foundation Trust , Manchester , UK
7 Collaborators for The PACT Consortium Barbara Barrett , Sam Barron, Karen Beggs, Laura Blazey, Katy Bourne, Sarah Byford, Rachel Cole- Fletcher, Julia Collino, Ruth Colmer, Anna Cutress, Isobel Gammer, Clare Harrop, Tori Houghton, Pat Howlin, Kristelle Hudry, Sue Leach, Jessica Maxwell, Jeremy Parr, Andrew Pickles, Sarah Randles, Vicky Slonims, Carol Taylor, Kathryn Temple, Hannah Tobin, George Vamvakas, Lydia White
There is a lack of measures that reflect the intervention priorities of parents of children with autism spectrum disorder (ASD) and that assess the impact of interventions on family experience and quality of life. The Autism Family Experience Questionnaire (AFEQ) was developed through focus groups and online consultation with parents, and reflected parental priorities. It was then administered to the parents of children enrolled in the Pre-school Autism Communication Trial and its 6-year follow-up study. The AFEQ showed good convergent validity with well-established measures of child adaptive functioning, parental mental health and parental wellbeing. It was sensitive to change in response to a parent-mediated intervention for young children with autism, showing treatment effect at treatment endpoint which increased at six-year follow-up.
Autism Spectrum Disorder; Family experience; Parent; Intervention; Quality of Life; Wellbeing
Collaborators of “The PACT Consortium” are listed in
University of Manchester, Manchester, UK
There is a notable paucity of parent-nominated measures
designed to assess intervention outcomes for children with
neuro-developmental disabilities, such as autism spectrum
disorder (ASD), and their families
(McConachie et al. 2015;
Morris et al. 2014, 2015)
. Parents of children with ASD are
often best placed to advocate for the interests and opinions
of their children and to observe and report on their child’s
progress, particularly when those children have significant
intellectual and communicative disabilities and are unable
(Morris et al. 2014)
. Many autism
interventions, particularly in the pre-school years, are
(e.g., Kasari et al. 2010; Oono et al. 2013; Pickles et al.
2016; Rahman et al. 2016)
and effective services for the
child with ASD can have “spillover effects” for the rest of
the family (Payakachat et al. 2012). Parents therefore have
a vital contribution to make on two levels: (1) in specifying
which intervention outcomes are important for their child,
for themselves and for their family and thereby setting the
success criteria for interventions, and (2) in reporting on the
measures of those outcomes for their individual children.
The measurement of patient- and parent/carer-centred
outcomes has become an integral part of the evaluation of
treatment effectiveness and cost-effectiveness in the UK and
(see, for example, the Promis System of the
US National Institutes of Health 2017, and the NHS
Outcomes Framework of the UK Department of Health 2017)
This gives the clear mandate that patients and their families/
carers should be central to the process of prioritising which
outcomes matter most within any particular health domain
and therefore determining the criteria by which
interventions are judged
(Marshall et al. 2005; Morris et al. 2015;
National Institute for Health and Care Excellence 2013;
Tait and Lester 2005)
. This mandate extends to paediatric
research, with an emphasis on the crucial role of parents
and carers in nominating and reporting when their children
are not developmentally able to do so themselves (Morris
et al. 2009).
Alongside the policy drive for patient- and
parent/carernominated outcomes, there has recently been increased
attention more generally to the research priorities of
individuals with ASD themselves and their parents and carers
(Iemmi et al. 2017; McConachie et al. 2015; Pellicano et al.
. Interventions to improve individual and family
wellbeing and quality of life are frequently cited as a priority by
people with ASD and their parents and carers
et al. 2015; Pellicano et al. 2014)
. Despite this, the inclusion
of patient- or parent-nominated measures or, indeed, any
outcome measure of quality of life in autism intervention
research, and particularly within randomised controlled
trials, is still uncommon
(Burgess and Gutstein 2007;
McConachie et al. 2015; Pellicano et al. 2014)
, with the emphasis
firmly on child developmental outcomes such as intellectual
ability, communication and language skills, play or autism
(e.g., Estes et al. 2015; Kasari et al. 2012;
Kasari 2015; Pickles et al. 2016)
The measurement of quality of life in preschool children
(Eiser and Morse 2001; Grange et al. 2007)
Standardised health-related quality of life indicators have
been developed for childhood chronic conditions
Morse 2001; Solans et al. 2008)
, such as the Pediatric
Quality of Life Inventory
(Varni et al. 2001)
, the EuroQol—5
(Wille et al. 2010)
the Child Health Utility—9 Dimensions
Such measures are psychometrically strong and effective in
providing a metric of general wellbeing in children with a
range of conditions, particularly physical illness. They have
less utility for measuring condition-specific dimensions.
They do not, for example, capture the unique and complex
practical, social and emotional intricacies of living with
(Eapen et al. 2014)
. The World Health Organisation
has identified this gap and is currently developing a
comprehensive Core Set for Autism Spectrum Disorder (ASD)
within its International Classification of Functioning,
Disability and Health (ICF), which aims to describe the lived
experience of individuals with ASD across various health,
wellbeing and developmental domains
(Bölte et al. 2014; de
Schipper et al. 2016)
Whilst child-specific quality of life is an important
outcome within autism intervention work, a wider focus that
encompasses the wellbeing of parents and the whole family
system has been recommended, particularly within
intervention aimed at young children
(Hastings et al. 2014;
Payakachat et al. 2012; Tint and Weiss 2016)
. The raised levels of
fatigue, stress, anxiety and depression in parents of children
with ASD are well documented
(e.g., Dykens et al. 2014;
Giallo et al. 2011; Quintero and McIntyre 2010)
have also been shown to be at risk of higher levels of
emotional and behavioural difficulties
(e.g., Rodrigue et al. 1993;
Ross and Cuskelly 2006)
. These effects are, of course, often
multi-directional: sibling wellbeing can be related to
parental coping and stress (Tsai et al. 2016) and parental stress
and distress can affect child adjustment and behaviour, as
well as vice versa
(Hastings et al. 2014; Totsika et al. 2013;
Zaidman-Zait et al. 2014)
. However, importantly, parents
of children with ASD emphasise that there are many
positive aspects to their family life
(McConachie et al. 2015)
and several beneficial effects of having a sibling with ASD
have been reported empirically, such as reduced conflict in
the sibling relationship
(Kaminsky and Dewey 2002)
increased general resilience
. A strengths-based
approach to ASD has also been called for by researchers
(Burnham Riosa et al. 2017; McCrimmon and Montgomery
. It would therefore be valuable to document family
strengths, as well as difficulties.
Mental health constructs like parenting stress have been
commonly utilised to tap parental wellbeing
(e.g., Hayes and
Watson 2013; Zaidman-Zait et al. 2014)
and, more recently,
quality of life indicators specific to parents of children with
ASD have been developed, such as the Quality of Life in
Autism Questionnaire (Eapen et al. 2014). However there
is, to our knowledge, no parent-generated measure of family
experience in the autism literature that can be used to assess
the impact of an intervention. In this paper we describe
the development of a parent-nominated measure of
family experience, quality of life and prioritised outcomes for
early intervention designed to address this measurement gap.
This measure was developed and tested within the context
of a large randomised trial of a pre-school parent-mediated
Background: The PACT Intervention, Trial and Follow‑Up
Pre-school Autism Communication Therapy (PACT
Therapy) is a parent-mediated video-aided
communicationfocussed intervention for pre-school children with autism
and their parents. The UK Medical Research Council
Preschool Autism Communication Trial (PACT Trial) was a two
arm parallel group randomised controlled trial (RCT) of 152
pre-school children with core autism. It ran between 2006
and 2009 across three UK centres (Manchester, London and
Newcastle) and evaluated the effectiveness of PACT therapy
plus treatment-as-usual (TAU) versus TAU alone
et al. 2010; Pickles et al. 2015)
. A subsequent follow-up
phase assessed outcomes at 6 years after the end of the
treatment phase (Pickles et al. 2016). The trial was registered on
International Standard Randomised Controlled Trial,
number ISRCTN58133827, and the protocol is available at http://
At trial entry, all children had a clinical diagnosis of
autism and met criteria for core autism on the
researcheradministered Autism Diagnostic Observation Schedule
(Lord et al. 2000)
and two of the three domains
of the Autism Diagnostic Interview—Revised
(Lord et al.
. 77% of the sample was not yet using regular,
spontaneous phrase speech. The sample was ethnically and
socioeconomically diverse (57% white; 23% single parents; 74%
with at least one parent with post-16 qualifications). The
PACT treatment group consisted of 77 children (6 girls)
aged 26–60 months (mean age = 45 months) with a mean
non-verbal IQ age equivalence of 27 months. The TAU
consisted of 75 children (8 girls) aged 24–60 months (mean
age = 45 months) with a mean non-verbal IQ age
equivalence of 25 months. Other demographic details of families
were balanced across the arms at baseline. The trial showed
a strong treatment effect in favour of the PACT intervention
in observed parental synchronous responsive behaviours and
child communicative behaviours in dyadic communication,
and in parent-reported outcome ratings on child language
and social communication. However, there were more
modest effects on child social affective autism symptoms and on
standardised objective tests of child language
(Green et al.
. At follow up 80% of the sample was assessed (mean
age 10.5 years; SD = 0.8; n = 121). Intention-to-treat
analysis based on initial randomisation showed a reduction of
autism symptom severity at treatment endpoint which was
sustained to 6-year follow-up and represented a 17% relative
reduction in symptom severity in the treatment arm
et al. 2016)
. The improvement in child dyadic
communicative behaviours seen in the intervention group at trial
endpoint was sustained into follow-up, but standardised
measures of language development continued not to show group
The commissioning and protocol of the PACT trial
included a strategy to promote involvement of service users
in pre-trial research design, and to address the
measurement gap in relation to family experience outcome
measures through the development of a parent-generated
measure of child and family wellbeing for use in the trial. This
new measure aimed to reflect both parental report of their
personal and family experience, and their autistic child’s
development and adjustment. The specific focus was on
whether the parental priorities for change were achieved
through the pre-school intervention. It was hypothesised
that, as the PACT intervention works partly through
changing the quality of parental responsiveness to child
communication, it could result in more general positive changes in
family life, and improved parental confidence and morale.
Since the assessment had not been developed prior to the
trial, data from the resulting instrument would not be used in
the formal outcome analysis, but secondary post hoc
analyses could contribute to the evaluation of this prototype of
an autism specific, parent-nominated change measure that
could be suitable for future intervention and other research.
Developing the Measure
A two-phase process was used. Firstly, a series of focus
groups was held with parents of children with autism to
generate a core set of outcome parameters. Secondly, we
implemented a larger web-based consultation using the resources
of the UK National Autistic Society (www.autism.org.uk) in
order to subject the initial parameters to wider review,
comment and refinement. Such a method is an extension of
recommended methods for developing measures of this kind
(e.g., Eiser and Morse 2001, p. 90)
Focus Group Phase
Thirty-one parents of pre-school or school-aged children
with autism were recruited from local clinical services and
parent-support groups and attended one of five focus groups
(four held in Manchester UK and one in Newcastle UK;
4–11 participants in each group). The groups were convened
and led by members of the PACT Principal Investigator team
and an independent qualitative researcher (WM). The aims
of the focus groups were to explore the specific parameters
that parents identified as the most important outcomes from
a pre-school communication intervention for autism.
Following consultation with parents and stakeholders at the
National Autistic Society, four key topics of enquiry were
pre-specified: (a) parental personal life and relationships;
(b) general family functioning (including sibling needs); (c)
specific outcomes for the child with autism related to general
development, communication and learning; (d) the child’s
symptoms in terms of emotional wellbeing and behaviour.
The initial orientating question for the groups was: ‘Given a
communication-focused intervention for a child with autism
in the preschool years, what would be the key aspects of
family life, child behavior or development that—if they
changed—would make you feel the intervention had been a
success?’ Focus group leaders structured the groups to cover
the pre-specified domains without directing the content.
Audio-tapes of each focus group meeting were transcribed
for analysis using NUDIST software in a constant
(Glaser and Strauss 1967)
. An initial thematic
coding framework was devised and the transcripts analysed
line-by-line and coded against the framework. Themes
obtained were then discussed with the wider team and
constant comparison was used to refine a final set of themes.
Individual statements were then abstracted from these
themes to serve as response items within the questionnaire.
This process resulted in 78 questionnaire items. Examples of
the themes and the abstracted questionnaire items are shown
in Table 1.
The 78 items derived from these focus group themes
were uploaded in the form of a draft questionnaire onto
an online survey facility within the website of the UK
National Autistic Society. This website facility allowed the
presentation of the items of the draft questionnaire,
contextualised with an introduction about its aims.
Respondents rated each question against two criteria: (1) Clarity:
comprehensibility and lack of ambiguity; (2) Usefulness
of the question for evaluating the effectiveness of a
communication treatment for autism. The website consultation
was live over a 10-day period in February/March 2006.
Over this period, 35 completed questionnaire responses by
parents of children with autism were received and collated
by staff at the National Autistic Society.
Realistic expectations: For many parents having realistic expec- “I have realistic milestones for my child’s development”
tations about the rate of improvement they could expect from
an intervention was important. There was general
understanding that they needed to adjust to a new set of milestones for
their child with autism. For some parents the slow rate of
change was very difficult to come to terms with, whilst for
other parents (particularly of older children) there was an
appreciation of the slower rate of change and improvement
they had observed in their child, often over years
Ability to communicate illness A number of parents described
the difficulty associated with their child’s inability to
understand physical symptoms, what is going on in their body and
to communicate physical symptoms to others
Emotional development: Parents were looking for strategies that “My child is happy”
could help their child to understand and express their feelings
and as a parent to understand their child’s feelings in order to
help them to be happier and more relaxed
Family outings: A recurring theme was the desire to be able to
go out as a family without being overwhelmed by feelings of
“My child is embarrassing when going out”
Wider family attitudes: Some parents talked about the lack of
acceptance and understanding of their child in the wider
family and this lack of understanding making it difficult for them
to attend extended family events and occasions
Routines and structure at home: There was discussion around
the problems around transition between school and home and
the problems because of a perceived lack of structure and
‘routine’ in the home environment
Child development Changes in child’s social experience: All parents wanted their
child to be able to play with other children, to develop and
keep friendships, and to be socially accepted and included.
One key consensus marker of improved social experiences and
relationships mentioned by several parents was if their child
was invited to birthday parties
“I feel confident to go out to family events with my child”
“I feel confident in making routines at home more
ageable for my child”
“My child gets invited to birthday parties”
“I know when my child feels poorly”
Analysis of Ratings
The web consultation resulted in weighted ranks against
both criteria (i.e., clarity and usefulness) for each of the
items presented. Items rated in the bottom quintile for
relevance were discarded. Items weighted highly for relevance
but low for clarity were retained but re-worded, or discarded
if it was difficult to rephrase the item into a short and simple
statement that was clearer.
Data Preparation Within the PACT Trial and Follow‑up
The resulting questionnaire had 56 items. Items included
both positively and negatively worded statements and were
scored on an order scale: 1 = always to 5 = never, with an
option for “Not Applicable”. Items were organised into
domains which matched onto the pre-specified focus-group
topics: (1) experience of being a parent of a child with
autism; (2) family life, (3) child development (development,
understanding and social relationships); and (4) child
symptoms (feelings and behaviour). This questionnaire was then
rated by parents, with researcher support if requested, as
part of the baseline and endpoint assessments of the PACT
trial. Prior to group unblinding and analysis, this data was
then subject to initial data cleaning. Eight items that were
rarely endorsed were excluded, including items on language
(when many children were non-verbal), and siblings (when
there often were none). The resulting 48 item questionnaire
was named the Autism Family Experience Questionnaire
(AFEQ; Online Appendix I). Items were organised into the
four pre-specified domains; Table 2 shows how items map
onto these domains. The AFEQ included both positively
and negatively worded statements and was scored on an
order scale: 1 = always to 5 = never, with an option for “Not
Applicable”. The 48-item AFEQ was then used in the PACT
6-year follow-up study.
We planned an initial descriptive analysis of the pattern of
AFEQ scores across the three time-points and an
examination of the internal consistency of the domain scores
and total score. A pre-specified analysis plan of the AFEQ
data (included in the original trial proposal and protocol)
included a study against an external criterion referent for the
child development domain of the AFEQ, using the parental
Vineland Adaptive Behavior Scales, Second Edition
Sparrow et al. 2006)
, a well-validated parent-rated scale
of child adaptive functioning. An additional analysis was
planned within the follow-up phase of the trial to assess the
convergent validity of the parent domain of the AFEQ with
a study of the association of the parent domain with a
wellestablished measures of adult mental health—the General
Health Questionnaire-12 Items
(GHQ-12; Goldberg 1992)
and of adult wellbeing—the Warwick-Edinburgh Mental
(WEMWBS; Tennant et al. 2007)
The initial pre-specified analysis plan included the
examination of the change in AFEQ over time between treatment
and TAU groups. For this treatment effect estimate we
postulated a priori hypotheses based on the nature of the
intervention and potential effects on aspects of family experience
contained within the AFEQ. Thus we predicted that,
compared to TAU, the PACT intervention would result in relative
improvements at trial endpoint and at 6-year follow-up in:
(1) the total AFEQ score; (2) each of the four domain scores
(parent, family life, child development and child symptoms).
We compared baseline, endpoint and follow-up AFEQ
total and domain scores between groups using an
intentionto-treat approach, using the same analytic model as in the
(Green et al. 2010; Pickles et al. 2016)
outcome was analysed separately using linear regression
(analysis of covariance) including baseline measures of
the outcome as covariate. In line with the main analyses
we adjusted for the same set of baseline covariates as fixed
effects, namely: age group (≤42 months, > 42 months), sex,
centre (Manchester, London, Newcastle), non-verbal
ability (mean non-verbal age-equivalent on the Mullen Scales
of Early Learning; Mullen 1995), socioeconomic status
(dichotomised as at least one parent with post-16
qualifications versus all others) and parental education (dichotomised
as at least one parent in professional or administrative
occupation versus all others). These were variables pre-specified
as potentially having an effect on outcome scores
independent of treatment.
Descriptive Analysis of AFEQ and Internal
Of the total sample of 152 participants, AFEQ data were
obtained from 145 parents at baseline assessment (7
participants had completely missing data), from 140/145 (97%)
parents at trial endpoint (13 months later; 12 completely
missing data), and from 105/145 (72%) parents at
followup, 6 years from end of treatment. There were no significant
demographic differences between the baseline sample and
those for whom we had follow-up data. Total AFEQ scores
were derived from these data. Individual items that were
missing or entered as NA were pro-rated with mean scores
of all items. Items that were negatively worded were reverse
scored, so that throughout the data a lower score indicates
a positive outcome, and a higher score is a poor outcome
(minimum possible score is 48, maximum possible is 240).
At baseline, the total score for the sample as a whole had
mean 141.0 (SD = 21.3), median 141.0 (range 81.8–188.0).
At endpoint, the whole sample total score had mean 133.0
(SD = 22.8), median 134.4 (range 64.0–180.0). At follow-up,
the total score had a mean 132.5 (SD = 24.6), median 133
(range 78.1–196.2). Therefore there was a slight reduction in
scores between baseline and endpoint and stability on scores
between endpoint and follow-up.
In order to assess the internal consistency of the 48 item
questionnaire, we examined the scale reliability based on
Cronbach’s alpha for the domain scores and the AFEQ total
score, calculated from baseline data. All domains and the
total score demonstrated excellent reliability at baseline:
parent (alpha = 0.85), family (0.83), child development (0.81),
child symptoms (0.79), AFEQ total (0.92).
Comparative Analysis Against Parental VABS
For comparison and to assess the external criterion
validity of the AFEQ, we correlated the scores on the AFEQ
child development domain (items 23–36; 14 items) with
the VABS total score at baseline, endpoint and follow-up.
Both AFEQ and VABS were completed with 143 families
at baseline (2 missing VABS in addition to the 7 missing
AFEQs); 134 at endpoint (6 missing VABS in addition
to the 12 missing AFEQs); and 102 at follow-up (3
missing VABS in addition to the 47 missing AFEQs).
Recalling that a positive outcome is indicated by a low score
on the AFEQ and a high score on the VABS, we would
anticipate a significant negative correlation to indicate that
the scales were producing similar results. The correlation
between the VABS and child developmental AFEQ
subtotal at baseline was r = − 0.478 (p < .001, n = 143), at
endpoint was r = − 0.575 (p < .001, n = 134), and at follow-up
was r = − 0.710 (p < .001, n = 102), indicating a moderate to
strong association between the two measures at each of the
Comparative Analysis Against Parental Mental
Health and Wellbeing Measures
To assess the external criterion validity of the parent domain
of the AFEQ, we tested for an association between the parent
domain score (items 1–13) and the GHQ-12 and WEMWBS
total scores at trial follow-up. These adult mental health and
wellbeing scales were not used as part of the trial baseline
or endpoint assessment battery, so these analyses could not
be conducted. The GHQ-12 total score was computed using
the ‘GHQ scoring method’
(items scored 0-0-1-1; Goldberg
, with missing items recoded as low scores, resulting
in a total score between 0 and 12, where a higher score
indicates poorer mental health. Both the AFEQ and the GHQ-12
were completed by 101 parents (4 missing GHQ-12
questionnaires in addition to the 47 missing AFEQs). The
correlation between the parent domain score and GHQ-12 was
Spearman’s Rho = 0.408 (p < .001, n = 101; a Spearman’s
rank correlation was conducted as the GHQ-12
distribution was highly positively skewed). The WEMWBS total
score was the sum of scores on all 14 questionnaire items,
with missing data on individual items pro-rated, resulting
in a total score ranging from 14 to 70 where a higher score
indicates increased wellbeing
(Tennant et al. 2007)
the WEMWBS and the AFEQ were completed by 103
parents (2 missing WEMWBS questionnaires on top of the 47
missing AFEQs). The correlation between the AFEQ parent
domain score and the WEMWBS total score was r = − 0.528
(p < .001, n = 103).
Treatment Effect Estimation
The results of the treatment effect estimation analysis (mean
differences and effect sizes [Cohen’s d]), along with
summary statistics of the total and domain scores, are shown in
Tables 3 and 4.
On the 48 item AFEQ total score, there was a statistically
significant improvement of PACT over TAU at both trial
endpoint and at 6-year follow-up, with a stronger effect at
follow-up than at endpoint (Cohen’s d = − 0.49 and − 0.29,
respectively). The treatment effects on the family domain
score were non-significant at both endpoint and follow-up.
On the parent domain there was a significant and moderately
strong treatment effect at follow-up (d= − 0.53), which was
not seen at trial endpoint. On the child development domain
we found a significant effect of the treatment at endpoint (d
= − 0.28), which was not sustained six years later and on
the child symptoms domain we found a marginal effect at
endpoint (d = − 0.31), also not sustained at follow-up.
The aim of the study was to develop a parent-generated
measure of family outcomes in the context of a RCT of
a parent-mediated video-aided pre-school
communication-focused intervention for young children with autism
(the PACT trial). In keeping with the patient- and parent/
carer-centred outcome agenda and the research priorities
of people with ASD and their families
(Iemmi et al. 2017;
McConachie et al. 2015; Pellicano et al. 2014)
, the AFEQ
is truly user-generated. Our procedure started prior to the
trial itself with focus groups and an online consultation with
parents of children with ASD to generate parents’ own
formulations of the priorities for early intervention outcomes.
Participating parents were asked to consider the key
outcomes under four domains: for themselves as parents, for
their family as a whole, for their child’s development, and
for their child feelings and behaviour (child symptoms).
Thematic analysis generated themes which were formed into
statements to serve as questionnaire items. The final
selection of items was made using parental ratings of clarity and
usefulness collected in the online consultation.
We conclude that such a process is feasible and
productive and has culminated in a viable, ecologically-valid
instrument with good internal consistency. Parents within the trial
reported that they found the measure easy to complete and
that they valued the opportunity to report real-life
experiences for their children and other family members on metrics
nominated by other parents of a child with ASD. These
anecdotal reports are consistent with the findings that individuals
with ASD and their parents and carers value wellbeing and
quality of life as outcomes measures within intervention
(McConachie et al. 2015; Pellicano et al. 2014)
The importance of considering external validity in trials
has been highlighted in the field of autism
(Jonsson et al.
and includes the extent to which new skills carry over
into everyday life and valued outcomes. As an initial test of
external validity we undertook a planned a priori analysis
of the relation to the parent rated Vineland Adaptive
(VABS; Sparrow et al. 2006)
. This widely-used
measure relates to child adaptive behaviour in a number of
domains and thus was an obvious comparator for the AFEQ
child development domain. There was a high correlation
between the AFEQ and the VABS at each of the three
timepoints of the PACT trial, in keeping with our hypothesis in
the trial protocol that the parental VABS would act as an
initial external validation of the child development domain
of the newly developed instrument. A further test of the
convergent validity of the AFEQ was the analysis of the
association of the AFEQ parent domain and well-established
measures of adult mental health - the General Health
(GHQ-12; Goldberg 1992)
Warwick-Edinburgh Mental Wellbeing Scale
Tennant et al. 2007)
, collected during the follow-up phase.
A significant correlation with these measures suggested that
the newly developed domain was tapping a construct that is
linked to parental mental health and wellbeing.
An interesting point of note from the descriptive analysis
was that the AFEQ total score taken across the two groups
showed a slight reduction (improvement) between baseline
and endpoint and then remained fairly stable to follow-up.
The pattern of domain scores across timepoints shown in
Table 3 suggests that this pattern is driven mainly by: (1)
reductions in the parent and family domain scores in the
PACT group only, and (2) decreases in scores in the child
development domain across both groups, the latter perhaps
reflecting developmental change across childhood.
A planned analysis within the trial protocol was to test the
estimation of treatment effect based on the AFEQ. We found
that the AFEQ total score showed a significant treatment
effect at the 13-month trial endpoint and at 6-year
followup, with a larger effect size at follow-up than at endpoint.
These findings provide evidence that the AFEQ total score
is sensitive to change in the expected direction in response
to a parent-mediated intervention for young children with
autism. They suggest that the AFEQ is tapping short and
longer term benefits from the PACT intervention that extend
beyond, and complement, the standardised observational and
parent-report measures of child development and
functioning reported in our main papers
(Green et al. 2010; Pickles
et al. 2016)
. A treatment effect was also seen on the child
development domain at endpoint, consistent with the other
positive parental report of outcomes on child social
communication and language development at endpoint reported
in our main paper (Green et al. 2010).
At follow-up, the parent domain showed a significant and
moderately strong treatment effect; the family, child
development and child symptoms domains showed non-significant
changes over this time period. The six-year improvement
in the parent domain seen in the treatment group is
particularly illuminating, as this domain does not directly address
aspects of the child’s functioning and wellbeing, but instead
aspects of the parent’s coping, confidence and self-efficacy.
We interpret this as showing that early parent-mediated
intervention of this kind with relatively low intensity can
bring about such sustained reported benefits on parental
experience and morale years later. This adds weight to the
notion that effective early intervention may lead to
spillover effects for the wider family
(Payakachat et al. 2012)
is also intriguing that the AFEQ parent domain showed a
treatment effect given that the PACT therapy did not have
a protective effect on parental mental health and wellbeing,
over and above other child and family risk factors, when
measured with established instruments (GHQ-12 and
WEMWBS) and given that the AFEQ is well-correlated with these
measures. As intended by its method of development, it may
well be that the AFEQ items taps constructs that are more
directly and specifically influenced by a parent-mediated
intervention (e.g., items such as “I feel listened to by
professionals” and “I feel I know how to help my child progress”),
compared to the more general mental health and wellbeing
constructs assessed by the GHQ-12 and the WEMWBS. The
AFEQ may therefore be more sensitive to change
resulting from an autism intervention, compared to more general
To balance its ecological validity as a parent-nominated
measure, the AFEQ is completed unblind to treatment
allocation (as is inevitably the case in psychosocial trials of this
kind) and thus is subject to expectation effects that can
substantially inflate estimates of treatment effect, compared to
trials which use blind-rated outcomes
(Sonuga Barke et al.
. The strength of the correlations between AFEQ and
other questionnaire measures (GHQ-12 and WEMWBS)
may be subject to common rater biasing, as they were often
completed at the same session; the VABS, on the other hand,
was completed as an interview and often in a different
session, and this correlation may be less subject to bias. These
biases must be borne in mind in the interpretation of the
results, but are an intrinsic part of self-reports of lived
experience and it is precisely this subjectivity that is valuable in
this context, particularly when it is able to complement more
objective blind-rated measures.
The AFEQ was designed to measure parental priorities
for key outcomes of an early intervention for their autistic
child and family. An interesting question is whether the final
questionnaire measures what we, as its developers, hoped
it would. As researchers, we are confident that the items
do indeed reflect parental priorities: all the questionnaire
items were generated directly from discussions by parents in
response to open questions designed to tap these priorities.
However, there are two issues which we feel were
underrepresented within the questionnaire: the issue of sibling
relationships and that of parental couple relationships. A
number of themes related to siblings were raised within the
focus groups. One item specifically about siblings had to
be removed in the data preparation phase as it generated
large amounts of missing data in families with only one
child. Other items (e.g., “I feel guilty about not giving other
members of the family enough attention”) were phrased in
such a way that they could be interpreted as being about
siblings in families who had more than one child, but did not
ask specifically about sibling relationships. Future research
might seek to develop a “sibling relationship” questionnaire
which could be used alongside the AFEQ by families with
more than one child. Another issue that may be
under-represented within the questionnaire is that of parental couple
relationships and how these are affected, both positively and
negatively, by having a child with autism within the family.
Parents in the focus groups did not provide any discussion
around this issue, perhaps due to inhibitions around
discussing this personal topic in a group of relative strangers. This
is something that could also be addressed in future research;
possibly one-to-one interviews with parents or written
questionnaires would be more likely to elicit reflection on this
In summary, the AFEQ holds promise as a viable,
parent-nominated measure of family life, prioritised
intervention outcomes and change indicators. It was developed in
a sequential manner to be truly user-generated, shows
ecological validity and is sensitive to change. It is not in itself
intended as a formal Quality of Life Measure but might be
used in future work alongside generic quality of life
measures, to evaluate family experience and to test associations
between family experience and other factors, such as
demographic variables and parental mental health. It could be
applied in a number of research, health care and
developmental settings to quantify the experience of families of
children with autism and similar neurodevelopmental disorders.
Acknowledgments We would like to thank the parents who kindly
gave their time and lived experience in the focus groups and website
consultation, our collaborators at the UK National Autistic Society for
hosting the web-based survey, and the parents and children in the PACT
cohort. The PACT trial and its follow-up study were sponsored by the
University of Manchester and Manchester Biomedical Research centre.
PACT was principally funded by the UK Medical Research Council
(G0401546), the UK Department for Children, Schools and Families
and the UK Department of Health. The PACT 7-11 Follow-Up Study
was funded by the Medical Research Council (MR/K005863/1).
Authors contribution JG, CRA, ALC, HM designed the development
process for the AFEQ measure. CRA, WM, JG, ALC carried out focus
groups with parents to generate the measure and WM led on the
qualitative analysis with support from DK. RE and KL carried out data
analysis, with input from ES. JG led the PACT trial and follow-up study.
JG, TC, ALC, and HM led the research teams at each study site. KL
and JG led the interpretation and writing with input from all authors.
Compliance with Ethical Standards
Conflict of interest The authors declare that they have no conflict of
Ethical Approval All procedures performed in studies involving human
participants were in accordance with the ethical standards of the
institutional and/or national research committee and with the 1964 Helsinki
declaration and its later amendments or comparable ethical standards.
Informed Consent Informed consent was obtained from all individual
participants included in the study.
Open Access This article is distributed under the terms of the Creative
Commons Attribution 4.0 International License
(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided you give appropriate
credit to the original author(s) and the source, provide a link to the
Creative Commons license, and indicate if changes were made.
Bayat , M. ( 2007 ). Evidence of resilience in families of children with autism . Journal of Intellectual Disability Research , 51 , 702 - 714 . doi: 10 .1111/j.1365- 2788 . 2007 . 00960 .x.
Bölte , S. , de Schipper , E., Robison , J. E. , Wong , V. C. , Selb , M. , Singhal , N. , et al. ( 2014 ). Classification of functioning and impairment: the development of ICF core sets for autism spectrum disorder . Autism Research , 7 ( 1 ), 167 - 171 , doi:10.1002/aur.1335.
Burgess , A. F. , & Gutstein , S. E. ( 2007 ). Quality of life for people with autism: Raising the standard for evaluating successful outcomes . Child and Adolescent Mental Health , 12 ( 2 ), 80 - 86 . doi: 10 .1111/j.1475- 3588 . 2006 . 00432 .x.
Burnham Riosa , P. , Chan , V. , Maughan , A. , Stables , V. , Albaum , C. , & Weiss , J. A. ( 2017 ). Remediating deficits or increasing strengths in autism spectrum disorder research: A content analysis . Advances in neurodevelopmental Disorders. doi: 10.1007/ s41252-017-0027-3.
De Schipper , E. , Mahdi , S., de Vries, P. , Granlund , M. , Holmann , M. , Karande , S. , et al. ( 2016 ). Functioning and disability in Autism Spectrum Disorder: A worldwide survey of experts . Autism Research , 9 ( 9 ), 959 - 969 . doi: 10 .1002/aur.1592.
Department of Health, UK ( 2017 ). The NHS outcomes framework 2016-2017 . Retrieved from: https://www.gov.uk/government/ publications/nhs-outcomes-framework-2016-to-2017.
Dykens , E. M. , Fisher, M. H. , Lounds Taylor , J., Lambert , W. , & Miodrag , N. ( 2014 ). Reducing distress in mothers of children with autism and other disabilities: A randomized trial . Pediatrics , 134 , e454 - e463 . doi: 10 .1542/peds.2013- 3164 .
Eapen , V. , Crncec , R. , Walter , A. , & Ping Tay , K. ( 2014 ). Conceptualisation and development of a quality of life measure for parents of children with autism spectrum disorder . Autism Research and Treatment. doi: 10 .1155/ 2014 /160783.
Eiser , C. , & Morse , R. ( 2001 ). Quality-of-life measures in chronic diseases of childhood . Health Technology Assessment , 5 ( 4 ), 1 - 157 .
Estes , A. , Munson , J. , Rogers , S. J. , Greenson , J. , Winter , J. , & Dawson , G. ( 2015 ). Long-term outcomes of early intervention in 6-year-old children with Autism Spectrum Disorder . Journal of American Academy Child Adolescent Psychiatry , 54 ( 7 ), 580 - 587 . doi: 10 .1016/j.jaac. 2015 . 04 .005.
Giallo , R. , Wood , C. E. , Jellett , R. , & Porter , R. ( 2011 ). Fatigue, wellbeing and parental self-efficacy in mothers of children with an Autism Spectrum Disorder . Autism: The International Journal of Research and Practice , 17 ( 4 ), 465 - 480 . doi: 10 .1177/1362361311416830.
Glaser , B. G. , & Strauss , A. L. ( 1967 ). The Discovery of Grounded Theory: Strategies for Qualitative Research . New York: Aldine De Gruyter.
Goldberg , D. ( 1992 ). General health questionnaire (GHQ-12) . Windsor: Nfer-Nelson.
Grange , A. , Bekker , H. , Noyes , J. , & Langley , P. ( 2007 ). Adequacy of health-related quality of life measures in children under 5 years old: systematic review . Journal of Advanced Nursing , 59 ( 3 ), 197 - 220 .
Green , J. , Charman , T. , McConachie , H. , Aldred , C. , Slonims , V. , Howlin , P. , et al. ( 2010 ). Parent-mediated communicationfocused treatment in children with autism (PACT): a randomised controlled trial . The Lancet , 375 ( 9732 ), 2152 - 2160 . doi: 10 .1016/ S0140- 6736 ( 10 ) 60587 - 9 .
Hastings , R. P. , Petalas , M. A. , Jones , L. , & Totsika , V. ( 2014 ). Systems analysis of associations over time between maternal and sibling well-being and behavioral and emotional problems of children with autism . Research in Autism Spectrum Disorders , 8 , 1516 - 1520 .
Hayes , S. A. , & Watson , S. L. ( 2013 ). The impact of parenting stress: A meta-analysis of studies comparing the experience of parenting stress in parents of children with and without autism spectrum disorder . Journal of Autism and Developmental Disorders , 43 , 629 - 642 . doi: 10 .1007/s10803-012-1604-y.
Iemmi , V. , Knapp , M. , & Ragan , I. ( 2017 ). The Autism Dividend: Reaping the Rewards of Better Investment . London, UK: London School of Economics and Political Science. Retrieved from http:// nationalautismproject.org.uk/wp-content/uploads/2017/01/autismdividend-report.pdf.
Jonsson , U. , Olsson , N. C. , & Bölte , S. ( 2016 ). Can findings from randomized controlled trials of social skills training in autism spectrum disorder be generalized? The neglected dimension of external validity . Autism: The International Journal of Research and Practice , 2 ( 3 ), 295 - 305 . doi: 10 .1177/1362361315583817.
Kaminsky , L. , & Dewey , D. ( 2002 ). Psychosocial adjustment in siblings with autism . Journal of Child Psychology & Psychiatry , 43 ( 2 ), 225 - 232 .
Kasari , C. ( 2015 ). Update on behavioral interventions for autism and developmental disabilities . Current Opinion in Neurology , 28 ( 2 ), 124 - 129 . doi: 10 .1097/WCO.0000000000000185.
Kasari , C. , Gulsrud , A. , Freeman , S. , Paparella , T. , & Helleman , G. ( 2012 ). Longitudinal follow up of children with autism receiving targeted interventions on joint attention and play . Journal of American Academy of Child & Adolescent Psychiatry , 51 ( 5 ), 487 - 495 . doi: 10 .1016/j.jaac. 2012 . 02 .019.
Kasari , C. , Gulsrud , A. C. , Wong , C. S. , Kwon , S. , & Locke , J. ( 2010 ). Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism . Journal of Autism & Developmental Disorders , 40 , 1045 - 1056 . doi.10.1007/ s10803-010-0955-5.
Lord , C. , Risi , S. , Lambrecht , L. , Cook , E. H. , Leventhal , B. L. , DiLavore , P. C., et al. ( 2000 ). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism . Journal of Autism and Developmental Disorders , 30 ( 3 ), 205 - 223 .
Lord , C. , Rutter , M. , & Le Couteur , A. ( 1994 ). Autism diagnostic interview-revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive development disorders . Journal of Autism & Developmental Disorders , 24 , 659 - 685 .
Marshall , S. S. , Haywood , K. L. , & Fitzpatrick , R. ( 2005 ). Patient involvement and collaboration in shared decision-making: a structured review to inform chronic disease management Report from the Patient-reported Health Instruments Group to the Department of Health . Retrieved from http://phi.uhce.ox.ac.uk/pdf/ phig_patient_ involvement_report.pdf.
McConachie , H. , Parr , J. R. , Glod , M. , Hanratty , J. , Livingstone , N. , Oono , I. P. , et al. ( 2015 ). Systematic review of tools to measure outcomes for young children with autism spectrum disorder . Health Technology Assessment , 19 ( 41 ), 1 - 506 . 10 .3310/hta19410.
McCrimmon , A. W. , & Montgomery , J. M. ( 2014 ). Resilience-based perspectives for autism spectrum disorder . In S. Prince-Embury & D. H. Saklofske (Eds.), Resilience interventions for youth in diverse populations (pp. 375 - 396 ). New York: Springer New York.
Morris , C. , Gibbons , E. , & Fitzpatrick , R. ( 2009 ). Child and Parent Reported Outcome Measures: A Scoping Report Focusing on Feasibility for Routine Use in the NHS . Report to the Department of Health . Oxford: University of Oxford.
Morris , C. , Janssens , A. , Allard , A. , Thompson-Coon , J. , Schilling , V. , Tomlinson , R. , et al. ( 2014 ). Informing the NHS Outcomes Framework: Evaluating meaningful health outcomes for children with neurodisability using multiple methods including systematic review, qualitative research, Delphi survey and consensus meeting . Health Services & Delivery Research. doi: 10 .3310/hsdr02150.
Morris , C. , Janssens , A. , Shilling , V. , Allard , A. , Fellowes , A. , Tomlinson , R. , et al. ( 2015 ). Meaningful health outcomes for paediatric neurodisability: Stakeholder prioritisation and appropriateness of patient reported outcome measures . Health and Quality of Life Outcomes , 13 , 87, doi:10.1186/s12955-015-0284-7.
Mullen , E. M. ( 1995 ). Mullen scales of early learning . Circle Pines , MN: American Guidance Service.
National Institute for Health and Care Excellence ( 2013 ). Guide to the methods of technology appraisal 2013 . Retrieved (2017, July 2) from : http://www.nice.org.uk/process/pmg9.
National Institutes of Health ( 2017 ). Promis: Dynamic Tools to Measure Health Outcomes from the Patient Perspective . Retrieved from: http://www.nihpromis.com/.
Oono , I. P. , Honey , E. J. , & McConachie , H. ( 2013 ). Parent-mediated early intervention for young children with autism spectrum disorders (ASD) . Cochrane Database of Systematic Reviews. doi:10.1002/14651858.CD009774 . pub2 .
Payakachat , N. , Tilford , J. M. , Kovacs , E. , & Kuhlthau , K. ( 2012 ). Autism spectrum disorders: A review of measures for clinical, health services and cost-effectiveness applications . Expert Review of Pharmacoeconomics & Outcomes Research , 12 ( 4 ), 485 - 503 . doi: 10 .1586/erp.12.29.
Pellicano , E. , Dinsmore , A. , & Charman , T. ( 2014 ). What should autism research focus upon? Community views and priorities from the United Kingdom . Autism: The International Journal of Research and Practice , 18 ( 7 ), 756 - 770 . doi: 10 .1177/1362361314529627.
Pickles , A. , Harris , V. , Green , J. , Aldred , C. , McConachie , H. , Slonims , V. , et al. ( 2015 ). Treatment mechanism in the MRC Pre-school Autism Communication Trial: Implications for study design and parent-focussed therapy for children . Journal of Child Psychology and Psychiatry , 56 ( 2 ), 162 - 170 .
Pickles , A. , Le Couteur , A. , Leadbitter , K. , Salomone , E. , ColeFletcher , R., Tobin , H. , et al. ( 2016 ). Parent-mediated social communication therapy for young children with autism (PACT): long-term follow-up of a randomised controlled trial . The Lancet , 388 , 2501 - 2509 . doi: 10 .1016/S0140- 6736 ( 16 ) 31229 - 6 .
Quintero , N. , & McIntyre , L. L. ( 2010 ). Sibling adjustment and maternal well-being: An examination of families with and without a child with an autism spectrum disorder . Focus on Autism and Other Developmental Disabilities , 25 ( 1 ), 37 - 46 .
Rahman , A. , Divan , G. , Hamdani , S. U. , Vajaraktar , V. , Taylor , C., Leadbitter , K. , et al. ( 2016 ). Effectiveness of the parent-mediated intervention for children with autism spectrum disorder in south Asia in India and Pakistan (PASS): A randomised controlled trial . Lancet Psychiatry , 3 ( 2 ), 128 - 136 . doi: 10 .1016/ S2215- 0366 ( 15 ) 00388 - 0 .
Rodrigue , J. R. , Geffken , G. R. , & Morgan , S. B. ( 1993 ). Perceived competence and behavioral adjustment of siblings of children with autism . Journal of Autism and Developmental Disorders , 23 ( 4 ), 665 - 674 .
Ross , P. , & Cuskelly , M. ( 2006 ). Adjustment, sibling problems and coping strategies of brothers and sisters of children with autistic spectrum disorder . Journal of Intellectual & Developmental Disability , 31 ( 2 ), 77 - 86 .
Solans , M. , Pane , S. , Estrada , M.-D., Serra-Sutton , V. , Berra , S. , Herdman , M. , et al. ( 2008 ). Health-related quality of life measurement in children and adolescents: a systematic review of generic and disease-specific instruments . Value in Health, 11 ( 4 ), 742 - 764 .
Sonuga Barke , E. J. , Brandeis , D. , Cortese , S. , Daley , D. , Ferrin , M. , Holtman , M. , et al. ( 2013 ). Nonpharmacological interventions for ADHD: systematic review and meta-analyses of randomized controlled trials of dietary and psychological treatments . American Journal of Psychiatry , 170 ( 3 ), 275 - 289 . doi: 10 .1176/appi. ajp. 2012 . 12070991 .
Sparrow , S. S. , Cicchetti , D. V. , & Balla , D. A. ( 2006 ). Vineland Adaptive Behavior Scales: Second Edition . Livonia, MN: Pearson Assessments.
Stevens , K. ( 2012 ). Valuation of the Child Health Utility 9D Index . Phar macoEconomics, 30 ( 8 ), 729 - 747 , doi:10.2165/ 11599120 -000000000-00000.
Tait , L. , & Lester , H. ( 2005 ). Encouraging user involvement in mental health services . Advances in Psychiatric Treatment , 11 , 168 - 175 .
Tennant , R. , Hiller , L. , Fishwick , R. , Platt , S. , Joseph , S. , Weich , S. , et al. ( 2007 ). The Warwick-Edinburgh mental well-being scale (WEMWBS): Development and UK validation . Health and Quality of Life Outcomes , 5 ( 1 ), 63 .
Tint , A. , & Weiss , J. A. ( 2016 ). Family wellbeing of individuals with autism spectrum disorder: A scoping review . Autism: The International Journal of Research and Practice , 20 ( 3 ), 262 - 275 .
Totsika , V. , Hastings , R. P. , Emerson , E. , Lancaster , G. A. , Berridge , G. M. , & Vagenas , D. ( 2013 ). Is there a bidirectional relationship between maternal well-being and child behavior problems in autism spectrum disorders? Longitudinal analysis of a population-defined sample of young children . Autism Research , 6 ( 3 ), 201 - 211 , doi: 10.1002/aur.1279.
Tsai , H.-W.J., Cebula , K. , & Fletcher-Watson , S. ( 2016 ). Influences on the psychosocial adjustment of siblings of children with Autism Spectrum Disorder in Taiwan and the United Kingdom . Research in Autism Spectrum Disorders, 32 , 115 - 129 . doi: 10 .1016/j. rasd. 2016 . 09 .007.
Varni , J. W. , Seid , M. , & Kurtin , P. S. ( 2001 ). PedsQL 4.0: reliability and validity of the pediatric quality of life inventory version 4.0 generic core scales in healthy and patient populations . Medical Care , 39 ( 8 ), 800 - 812 .
Wille , N. , Badia , X. , Bonsel , G. , Burstrom , K. , Cavrini , G. , Devlin , N. , et al. ( 2010 ). Development of the EQ-5D-Y: a child-friendly version of the EQ-5D . Quality of Life Research , 19 , 875 - 886 . doi: 10 .1007/s11136-010-9648-y.
Zaidman-Zait , A. , Mirenda , P. , Duku , E. , Szatmari , P. , Georgiades , S. , Volden , J. , et al. ( 2014 ). Examination of bidirectional relationships between parent stress and two types of problem behaviour in children with autism spectrum disorder . Journal of Autism & Developmental Disorders , 44 ( 8 ), 1908 - 1917 . doi: 10 .1007/ s10803-014-2064-3.