Caregiving in severe mental illness: the psychometric properties of the Involvement Evaluation Questionnaire in Portugal
Annals of General Psychiatry
Caregiving in severe mental illness: the psychometric properties of the Involvement Evaluation Questionnaire in Portugal
Manuel Gonalves-Pereira 0 3 4
Bob van Wijngaarden 2 3
Miguel Xavier 0 3 4
Ana L Papoila 1 3
Jos M Caldas-de-Almeida 0 3 4
Aart H Schene 3 5
0 Department of Mental Health, CEDOC, Faculdade de Ciencias Medicas, Universidade Nova de Lisboa , Lisbon , Portugal
1 Department of Biostatistics and Informatics, Faculdade de Ciencias Medicas, Universidade Nova de Lisboa , and CEAUL, Lisbon , Portugal
2 The Netherlands Institute of Mental Health and Addiction , Utrecht , The Netherlands
3 Authors' information MG-P, MD PhD, is a psychiatrist and family therapist, and Professor of Medical Psychology in the Department of Mental Health, FCM, NOVA University, Lisbon. His interests include family issues in mental health. BvW, PhD, is senior research associate at the Trimbos-instituut, The Netherlands Institute of Mental Health and Addiction. He is one of the developers of the Involvement Evaluation Questionnaire. MX, MD, PhD, is Professor of Psychiatry and Mental Health in the Department of Mental Health, FCM, NOVA University , Lisbon. ALP, PhD , is a statistician in the Department of Biostatistics and Informatics, FCM, NOVA University , Lisbon. JMCdA, MD, PhD , is Full Professor of Psychiatry and Mental Health in the Department of Mental Health, FCM, NOVA University, Lisbon. He is also the Dean of the faculty. AHS, MD, PhD, is Professor of Psychiatry and head of the Program for Mood Disorders in the Department of Psychiatry, Academic Medical Center, University of Amsterdam , Amsterdam , The Netherlands
4 Department of Mental Health, CEDOC, Faculdade de Ciencias Medicas, Universidade Nova de Lisboa , Lisbon , Portugal
5 Department of Psychiatry, Academic Medical Center, University of Amsterdam , Amsterdam , The Netherlands
Background: Despite the achievements of previous research, caregiving assessments in severe mental illness should be crossculturally validated in order to define risk groups or to evaluate family work. This study reports on the psychometric properties of the European version of the Involvement Evaluation Questionnaire (IEQ-EU) in Portugal. Methods: A Portuguese translation of the IEQ-EU was developed according to the 'European Psychiatric Services: Inputs Linked to Outcome Domains and Needs' (EPSILON) group guidelines. We then studied 194 caregivers who were related to patients with schizophrenia spectrum disorders in psychiatric outpatient services. All relatives were assessed using the IEQ-EU. In order to describe the corresponding patients' sample, the majority (n = 162) was evaluated with the World Health Organization Disability Assessment Schedule (WHO-DAS II); 108 patients were also assessed with the Brief Psychiatric Rating Scale (BPRS) and the Global Assessment of Functioning (GAF). Results: The factor structure of the Portuguese version of the questionnaire was similar to the original; internal consistency was good, with Cronbach's a ranging from 0.71 to 0.87 in the IEQ-EU scales (total score and domains: tension, supervision, worrying, urging); test-retest reliability yielded intraclass correlation coefficients (ICCs) from 0.80 to 0.94, concerning the same scores. Ecological validity was confirmed. Most caregiving consequences were reported on the worrying domain of the IEQ-EU. Conclusions: Validity and reliability of the Portuguese IEQ-EU translation were established. Specifically the four IEQEU subscale domains seem to be valid in Portugal.
Caregiver; crosscultural psychiatry; family member; measurement validation; questionnaires; schizophrenia
Despite huge amounts of high-quality research related
to caregiving in severe mental illness (see for example
[1-4]), a need exists for the crosscultural validation of
caregiver instruments, and for the development of
national norms regarding assessments. Without better
knowledge of the intercultural validity of measures and
local standards, international comparisons will be
undermined by doubts regarding the origin of differences
between scorings (for example, sampling vs real cultural
The European Psychiatric Services: Inputs Linked to
Outcome Domains and Needs (EPSILON) study was a
crossnational, cross-sectional survey [6,7], which
compared characteristics, needs and quality of life of people
with schizophrenia and their caregivers in five European
countries (Denmark, England, Italy, The Netherlands
and Spain). Standardized versions of related key research
instruments were produced, including the Involvement
Evaluation Questionnaire (IEQ) for the assessment of
caregiving consequences .
The IEQ measures the consequences of psychiatric
disorders for relatives, friends or other significant persons
involved, being a well established tool for assessments of
caregiving impact in psychotic, mood or mixed disorders
[5,9]. In the EPSILON study the English, Danish, Italian
and Spanish translations were validated . Over the last
decade ten more translations were made. Validation data
on the German, Swedish, Malawi, Chinese, Polish and
Arabic versions are available [10-15]. The Finnish, French
and Greek versions have not been validated yet . A
previous version of the IEQ was used in Portugal , but
to date the IEQ-EU has not been adequately validated in
Therefore, our aim was to validate and test the reliability
of a Portuguese translation of the IEQ-EU, according to
the EPSILON study methodology, in Portuguese caregivers
of psychotic outpatients. In this paper, we document the
validity of the IEQ-EU scales in Portugal (focusing on
ecological, face value and content aspects), along with their
test-retest reliability and internal consistency.
Study design and participants
After the development of the IEQ-EU Portuguese
translation and pretests, factor structure and internal
consistency were determined.
We studied a non-randomized sample of primary
caregivers of outpatients with chronic psychosis, who were in
contact with mental health services in Lisbon (n = 194).
These services were Hospital S. Francisco Xavier, a public
hospital, and Clnica Psiquitrica de S Jos, a non-profit
facility. The majority of participants (n = 108) formed the
baseline sample of the FAmilies of people with PSychotic
disorders (FAPS) survey, for which preliminary results
have been presented . In this paper, only the IEQ-EU
data will be used. Results for the FAPS baseline study are
reported in detail elsewhere  (Gonalves-Pereira M,
Xavier M, van Wijngaarden B, Papoila AL, Schene AH,
Caldas de Almeida JM: Impact of psychosis in a
Portuguese population: a cross-cultural exploration of burden,
distress, positive aspects and clinical-functional
correlates, unpublished). Test-retest reliability was evaluated
in those caregivers who agreed to collaborate (n = 50), by
asking them to complete the IEQ-EU a second time
within a 2 to 3 week period. The overall sample size was
not defined a priori, yet it should allow for the study of
the IEQ-EU psychometrics, including factorial validity.
For that purpose, the FAPS sample was enlarged with
In the FAPS survey, 108 patients with International
Classification of Diseases, 10th edition (ICD-10)
Diagnostic Criteria for Research (DCR) schizophrenia (F20),
schizoaffective (F25) or delusional disorder (F22), were
included. Regarding all non-FAPS patients in this study
(n = 86), a simple ICD-10 clinical diagnosis of one the
above-mentioned disorders was the condition to be
included. Cases with coexisting learning disability,
organic disorders, or inpatient treatment episodes in the
previous 2 months were excluded.
Primary caregivers were then approached. All of them
were family to the patient, but relatives will be referred
to as caregivers for the remainder of this article. The
protocol was approved by local ethical committees.
Informed consent was obtained from all participants.
The IEQ-EU and its Portuguese translation
The IEQ-EU is an 81-item questionnaire to be completed
by any caregiver who, during the past 4 weeks, had at
least 1 h a week contact with the patient. The
questionnaire consists of seven modules: (1) demographics of
patient and family, and general clinical data concerning
the patient; (2) caregiving consequences of psychiatric
disorders; (3) extra financial expenses; (4) the General
Health Questionnaire, 12-item version (GHQ-12), used
as a general measure of distress ; (5) professional
help for patients relative; (6) consequences for patients
children; (7) open question for additional remarks.
Module 2 is the core module (31 items), where items refer to
all kinds of encouragement and care the caregiver has to
provide to the patient, to supervision of the patients
dangerous behaviors, to interpersonal problems between
patient and caregiver, and to the caregivers worrying (for
example, how often have you worried about your
relatives future?). These items are scored on a five-point
Likert scale (0 = never, 1 = sometimes, 2 = regularly, 3 =
often, 4 = (almost) always), and the time frame is the
prior 4 weeks. The caregivers coping and subjective
burden are also assessed in individual items. In research use,
27 of the 31 core module item scores can be summarized
in a total or sum score, and 4 subscales or domains:
tension (9 items), supervision (6 items), worrying (6 items),
and urging (8 items). Two items (no. 29 and no. 43) load
on two subscales [5,21]. Tension refers to the strained
interpersonal atmosphere between patient and relatives;
supervision to the caregiving tasks of ensuring and
guarding related to, for example, the patients intake of
medicine or dangerous behaviors; worrying to painful
cognitions and concerns about patients safety or future;
and urging to issues related to activating and motivating
the patient. An overview of the core items of the IEQ-EU
is given in the Results section.
The IEQ Portuguese translation was developed in
accordance with international conventions and EPSILON
guidelines, in order to maintain face and content validity
. All modules had to be translated except the
GHQ12, for which a Portuguese translation was available from
the publisher . A first draft concerned the translation
of the English IEQ-EU by one of the authors (MGP), who
also was involved in the Portuguese version of the
original IEQ . This translation was refined by a Dutch
native Portuguese-fluent lay contributor using the Dutch
IEQ-EU. A focus group involving caregivers, and a
discussion group involving mental health professionals and
researchers, were both conducted in order to check the
quality and acceptability of the translation. Drafts were
subsequently refined and a back translation was
performed by a Dutch native Portuguese-fluent professional
translator. When revised by one of the authors of the
original tool (BvW), only minor corrections were made.
We chose not to report the GHQ-12 results here, as
only FAPS participants completed this questionnaire.
Moreover, these assessments are relevant to construct
validation but not to define the psychometric properties
related to the core IEQ-EU.
Patient clinical data not covered by the IEQ-EU, such as
number of previous admissions, were collected in
Regarding the FAPS baseline sample, patients
symptoms were assessed with the extended (24 item) Brief
Psychiatric Rating Scale (BPRS) . BPRS items are
coded into seven categories (1 = no symptoms, to 7 =
extremely severe). Patients disability was assessed
through interviewing caregivers and collecting other
sources of information with the World Health
Organization Disability Assessment Schedule II (DAS-II) ,
producing global evaluations from 0 (excellent or very
good adjustment) to 5 (severe maladjustment). Finally,
the Global Assessment of Functioning (GAF) scale 
was applied, in a continuum from 0 to 100. The DAS
interview was also used for 54 patients in the non-FAPS
sample, so that data on the global evaluation section
were available for 162 cases. These clinical and functional
assessments were made by trained research assistants, all
of them mental health professionals.
Descriptive statistics and non-parametric tests were used
as required. Patient and caregiver characteristics are
presented as frequencies and percentages for categorical
data, and as mean or median, standard deviation (SD),
range and minimum/maximum values for continuous
variables. The 95% confidence intervals (CI) for the mean
value were calculated whenever appropriate.
Principal component analysis was used to conduct an
exploratory factor analysis on the IEQ scores. To verify
the appropriateness of the factor analysis, three techniques
were used for the assessment of the psychometric
adequacy of the correlation matrix: (1) Bartletts test of
sphericity, evaluating the hypothesis that the correlation matrix
is an identity matrix (that is, there is no correlation among
the items); rejection of this hypothesis suggests that data
are appropriate for factor analysis; (2) inspection of the
off-diagonal elements of the anti-image correlation matrix
(that contains the negatives of the partial correlation
coefficients), enabling us to quantify individual measures of
sampling adequacy (MSA) and to conclude that the
correlation matrix is factorable whenever the absolute values of
those elements are small; (3) Kaiser-Meyer-Olkin (KMO)
statistic, an overall MSA that varies between 0 and 1 (high
values for this statistic indicate that the data are unsuitable
for factor analysis). Four factors were chosen a priori in
order to provide comparable results with similar studies.
The Varimax rotation technique with Kaiser normalization
was used. It was postulated that factor loadings should be
> 0.40. In case of several loadings > 0.40 in a particular
item, this item would be used for more than one factor
For IEQ scores, test-retest reliability was assessed
calculating intraclass correlation coefficients (ICCs) and
95% CI; Cronbachs a values were computed for internal
consistency. Regarding correlation studies, Spearmans
coefficients were used when a linear association was
present between continuous variables. The significance
level of a = 5% was considered. All data were entered
and analyzed using SPSS for Windows V.15.0 (SPSS
Inc., Chicago, IL, USA).
Demographics and clinical data regarding the patients
and their primary caregivers are first described. The
mean age of the patients in the total sample (n = 194)
was 35.6 (SD 9.5) years. Most were male (63.4%), had a
diagnosis of schizophrenia (70.1%) or other chronic
psychosis (29.9%), a number of psychiatric admissions
ranging from none to 16 (median 2.0), and a duration of
illness ranging from 1 to 63 years (median 10.0). DAS
mean (SD) scores were 2.6 (0.9) (n = 162). In the FAPS
baseline sample (n = 108), BPRS mean (SD) scores were:
1.8 (0.5); and GAF mean (SD) scores were: 52.6 (13.8).
These FAPS patients had ICD-10 DCR schizophrenia in
the majority of cases (88.9%), mainly of the paranoid
type, F20.0 (76.9%).
The corresponding caregivers characteristics (n = 194)
are shown in Table 1. Most were female (67.5%), parent
of the patient (71.1%), and living with him/her (89.7%).
All completed the IEQ-EU.
General data on the psychometrics of the IEQ Portuguese
Factor analysis was conducted on this sample.
According to Bartletts test (value = 1,811.66 and P < 0.001) the
correlation matrix was not an identity matrix and was
therefore suitable for analysis. The KMO statistic was >
0.70 (0.798), the MSA values for all the individual items
Table 1 Caregivers demographics and caregiving arrangements
n = 194. Monthly net income is coded into six range scores in item 14 of the IEQ (1, minimum, to 6, maximum); hours of personal contact are coded into six
range scores (1 = less than 1 h a week to 6 = more than 32 h a week).
were > 0.60 (89% were > 0.70) and the absolute values
of the off diagonal values were very low. Therefore, the
analysis seemed appropriate. The factor analysis resulted
in a very similar solution to the original one . This
solution is presented in Table 2 and it accounted for
47% of the total variance. The value of lost expected
variance for IEQ-EU totals and subscales was acceptable
(13%) if we complied with the original subscales, so we
decided to use these in the remainder of the analysis.
The distribution of IEQ-EU scales sum item scores
and their internal consistency are presented in Table 3.
Mean scores (SD), for each IEQ domain and total score,
were: tension 0.88 (0.57); supervision 0.45 (0.58);
worrying 2.3 (0.73); urging 1.07 (0.75); IEQ total score 1.12
(0.54). Overall, Cronbach a values were substantial ,
being higher for the IEQ sum score (0.87) than for
IEQEU subscales (0.71 to 0.74).
Tension and worrying, the two interpersonal domains,
had a substantial correlation (rS = 0.71; P < 0.001), and
there was a moderate coefficient between tension and
urging (rS = 0.53; P < 0.001). All other correlations
between domains were lower than 0.49, favoring the
adopted original factor structure.
Test-retest reliability was assessed for 50 participants.
The ICC for the IEQ-EU total score was 0.93 (95% CI
0.88 to 0.96, P < 0.001), while for subscales ICCs were:
0.88 (95% CI 0.79 to 0.93, P < 0.001) for tension, 0.79
(95% CI 0.63 to 0.88, P < 0.001) for supervision, 0.94
(95% CI 0.89 to 0.97, P < 0.001) for worrying, and 0.89
(95% CI 0.80 to 0.94, P < 0.001) for urging.
Three IEQ-EU core module items which are not used to
compute domain or total scores provided specific results
on being able to pursue own activities and interests,
getting used to mental illness, and self-perception of lack of
coping ability. These three items were also scored from 0
(lower level) to 4 (higher level). Results (mean (SD)) were
as follows: 1.5 (1.5) for pursuing own activities (median 1,
meaning sometimes); 1.9 (1.3) for getting used to
problems (median 2, meaning fairly well); and coping ability
1.9 (1.2) (median 2, meaning regularly felt able to cope).
Ecological validity and acceptability of the IEQ-EU
The ecological validity of the IEQ Portuguese version is
sufficient. Considering the FAPS baseline sample, the
overall response rate was 100%, no questionnaires had to
be discarded from analysis, and the applicability was also
sufficient. However, caregivers varied in their ability to
complete the IEQ-EU on their own, reflecting
heterogeneity of literacy levels. In total 61 respondents (56.5%)
were able to fully self-complete questionnaires, 32
(29.6%) required significant help, and 15 (13.9%) had to
be interviewed due to limited literacy. According to
interviewers impressions, assessments were easily conducted
and item formulation and understandability seemed
adequate, even for illiterate respondents. All but five
caregivers fully completed the IEQ.
This study replicates a part of the EPSILON study
methodology for adequate testing of validity and reliability of
the IEQ-EU in a South European setting (Portugal), with
a considerable sample size. The IEQ had been originally
developed in the north of Europe (The Netherlands).
Regarding patients characteristics, they are typical of a
clinical sample of chronic psychotic users in mental
health outpatient public practices in Portugal [1,17]. The
levels of psychopathological symptoms and global
functioning of these users are similar to the ones described in
previous studies on caregiving . Mean disability levels
are around the poor adjustment DAS category, again in
Table 2 Principal component analysis on Involvement Evaluation Questionnaire (IEQ) items
Bold type indicates the factor to which the item was allocated in cases where it loaded 0.4 in two or more factors; items loading < 0.4 in any factor are not
shown (20: accompanying on outside activities; 28: encouraging to get up in the morning; 29: sleep disturbance due to relatives behavior).
agreement with our assumption that the sample is
probably representative of populations with chronic psychosis
in Portuguese services . Moreover, the corresponding
caregivers characteristics and caregiving arrangements
are close to the usual pattern in South Mediterranean
Overall, validity of the Portuguese IEQ-EU translation
has been established. In this study, the ecological, face
and content validity of the questionnaire were grounded
on quality assurance of the translation, and on replication
of the EPSILON methodology. Criterion and construct
validity, and sensibility to change, were not directly
approached here. However, the FAPS survey contributed
to their specific testing in Portugal by comprehensively
assessing 108 of these 194 patient-caregiver dyads, as
reported elsewhere [18,19]. Construct validity is
particularly sound  (Gonalves-Pereira M, Xavier M, van
Wijngaarden B, Papoila AL, Schene AH, Caldas de
Almeida JM: Impact of psychosis in a Portuguese
population: a cross-cultural exploration of burden, distress,
Table 3 Involvement Evaluation Questionnaire (IEQ) totals and domains: distribution of scores and internal
Sum item scores (0 to 4)
Minimum to maximum
Mean (95% CI) SEM
n = 194; minimum and maximum possible scores are given in brackets.
positive aspects and clinical-functional correlates,
Our data on psychometric properties ensure
considerable reliability. Concerning internal consistency, alphas
were slightly lower than in the EPSILON global sample,
but some of them were more satisfactory than in some of
the EPSILON centers . Regardless of comparisons, they
were substantial according to the EPSILON convention
 and good, according to Streiner and Norman, for this
kind of scales . ICCs on test-retest were invariably
substantial to high . We emphasize that the 50
respondents constituted a larger subsample for this kind
of testing than most in the EPSILON study .
Our results also suggest that the original IEQ-EU factor
structure can be preserved, as its use in our sample did
not lead to too much loss of explained variance. Rank
order of the four IEQ-EU domains was the same as in the
EPSILON and previous Dutch studies: the mean score of
worrying is the highest, followed by urging, interpersonal
tension and supervision [5,21]. We consider this finding as
another sign of validity.
Therefore, in Portugal, the IEQ-EU seems to cover the
same caregiving domains that have been described for
other countries, and instrument bias in the assessment of
differences in caregiving consequences appears unlikely. In
the FAPS baseline assessments, ecological validity of the
IEQ-EU was evident, with high response rates and very
few missing data.
Limitations of the study
We did not use randomization procedures so sampling
bias cannot be fully discarded. Therefore, despite the
strong impression that our participants represent a typical
sample of caregivers of chronic psychotic patients in
Portugal, one must theoretically recognize limitations to the
generalizability of our findings. We also did not use a semi
structured psychiatric interview, although a robust clinical
diagnosis ascertainment was ensured for the majority of
All assessments were conducted in clinical settings,
and postal means were not used. This may have
positively influenced response rates and questionnaires
A heavy negative impact of caregiving in severe mental
illness has been acknowledged once more in a regional
sample . There is a need to continuously address
vulnerable caregivers, and feasible risk assessment
routines must be pursued.
The Portuguese version of the IEQ-EU is valid and
reliable for research use, but its clinical usefulness
remains a challenging topic. Regardless of this, the
available evidence sustains the maintenance of the
original IEQ factor structure in Portugal.
We thank all patients and caregivers for participating in our study. The
Portuguese translation of the IEQ-EU was developed in 2005 by the first
author. It was partially based on a previous Portuguese version of the IEQ in
collaboration with JMCdA. L Caenen was responsible for back translations.
Members of the FAPS Workgroup contributing to data collection: A Cardoso,
A Ramires, A Alves, C Conceio, I Landeiro, JA Silva, J Grcio, M Gomes, N
Ventura, S Andrade, T Ribeiro. A Melo conducted part of non-FAPS
assessments. The FAPS study was supported by a clinical research grant
from the Portuguese AstraZeneca Foundation, Fundao AstraZeneca
MG-P conceived the present study, which was designed together with BvW,
MX, JMCdA and AHS. MG-P coordinated the fieldwork and data analysis, and
drafted the initial manuscript together with BvW. ALP supervised statistical
procedures. All authors contributed significantly to the critical revision of the
1. Magliano L , Fadden G , Madianos M , de Almeida JM , Held T , Guarneri M , Marasco C , Tosini P , Maj M : Burden on the families of patients with schizophrenia: results of the BIOMED I study . Soc Psychiatry Psychiatr Epidemiol 1998 , 33 : 405 - 412 .
2. Barrowclough C : Families of people with schizophrenia . In Families and Mental Disorders: From Burden to Empowerment. Edited by: Sartorius N, Leff J , Lpez-Ibor JJ , Maj M , Okasha A. Chichester , UK: Wiley; 2005 : 1 - 25 .
3. Kuipers E , Watson P , Onwumere J , Bebbington P , Dunn G , Weinman J , Fowler D , Freeman D , Hardy A , Garety P : Discrepant illness perceptions, affect and expressed emotion in people with psychosis and their carers . Soc Psychiatry Psychiatr Epidemiol 2007 , 42 : 277 - 283 .
4. Ochoa S , Vilaplana M , Haro JM , Villalta-Gil V , Martinez F , Negredo MC , Casacuberta P , Paniego E , Usall J , Dolz M , Autonell J , NEDES Group: Do needs, symptoms or disability of outpatients with schizophrenia influence family burden ? Soc Psychiatry Psychiatr Epidemiol 2008 , 43 : 612 - 618 .
5. van Wijngaarden B , Schene A , Koeter M , Becker T , Knapp M , Knudsen HC , Tansella M , Thornicroft G , Vsquez-Barquero JL , Lasalvia A , Leese M , EPSILON Study Group: People with schizophrenia in five countries: conceptual similarities and intercultural differences in family caregiving . Schizophr Bull 2003 , 29 : 573 - 586 .
6. Becker T , Knapp M , Knudsen HC , Schene AH , Tansella M , Thornicroft G , Vsquez-Barquero JL : Aims, outcome measures, study sites and patient sample . EPSILON Study 1. European Psychiatric Services: Inputs Linked to Outcome Domains and Needs. Br J Psychiatry Suppl 2000 , 39 : s1 - 7 .
7. Schene AH , Koeter M , van Wijngaarden B , Knudsen HC , Leese M , Ruggeri M , White IR , Vsquez-Barquero JL : Methodology of a multi-site reliability study . EPSILON Study 3. European Psychiatric Services: Inputs Linked to Outcome Domains and Needs. Br J Psychiatry Suppl 2000 , 39 : s15 - 20 .
8. van Wijngaarden B , Schene AH , Koeter M , Vazquez-Barquero JL , Knudsen HC , Lasalvia A , McCrone P : Caregiving in schizophrenia: development, internal consistency and reliability of the Involvement Evaluation Questionnaire-European Version . EPSILON Study 4. European Psychiatric Services: Inputs Linked to Outcome Domains and Needs. Br J Psychiatry Suppl 2000 , 39 : s21 - 27 .
9. van Wijngaarden B , Schene AH , Koeter MW : Family caregiving in depression: impact on caregivers' daily life, distress, and help seeking . J Affect Disord 2004 , 81 : 211 - 222 .
10. Bernert S , Kilian R , Matschinger H , Mory C , Roick C , Angermeyer MC : The assessment of burden on relatives of mentally ill people: the German version of the involvement evaluation questionnaire (IEQ-EU) . Psychiatr Prax 2001 , 28 (Suppl 2): S97 - 101 .
11. Magne-Ingvar U , Ojehagen A : Significant others of persons with mental health problems: the testing of a questionnaire on the burden of significant others . Nord J Psychiatry 2005 , 59 : 441 - 447 .
12. Sefasi A , Crumlish N , Samalani P , Kinsella A , O'Callaghan E , Chilale H: A little knowledge: caregiver burden in schizophrenia in Malawi . Soc Psychiatry Psychiatr Epidemiol 2008 , 43 : 160 - 164 .
13. Tang VW , Leung SK , Lam LC : Clinical correlates of the caregiving experience for Chinese caregivers of patients with schizophrenia . Soc Psychiatry Psychiatr Epidemiol 2008 , 43 : 720 - 726 .
14. Hadrys T , Adamowski T , Kiejna A : Mental disorder in Polish families: is diagnosis a predictor of caregiver's burden ? Soc Psychiatry Psychiatr Epidemiol 2011 , 46 : 363 - 72 .
15. Zahid MA , Ohaeri JU : Relationship of family caregiver burden with quality of care and psychopathology in a sample of Arab subjects with schizhophrenia . BMC Psychiatry 2010 , 10 : 71 .
16. van Wijngaarden B , Schene AH : Measuring family or carer burden in severe mental illness: the instruments . In Mental Health Outcome Measures .. 3 edition. Edited by: Tansella M, Thornicroft G. London , UK: Gaskell; 2010 : 116 - 135 .
17. Gonalves-Pereira M , de Almeida JM : The repercussions of mental disease in the family . Acta Med Port 1999 , 12 : 161 - 168 .
18. Gonalves-Pereira M , Papoila A , Xavier M : The FAPS study on severe mental illness: Views on the implementation of psychoeducational family work in Portuguese settings . European Psychiatry 2009 , 24 (Suppl 1): s232 - s232 .
19. Gonalves-Pereira M : Families of People with Psychotic Disorders: Dimensional Analysis and Effectiveness of Groups for Relatives PhD thesis , Universidade Nova de Lisboa, Faculdade de Cincias Mdicas; 2009 .
20. Goldberg D : The Detection of Psychiatric Illness by Questionnaire London , UK: Oxford University Press ; 1972 .
21. Schene AH , van Wijngaarden B , Koeter MW : Family caregiving in schizophrenia: domains and distress . Schizophr Bull 1998 , 24 : 609 - 618 .
22. Knudsen HC , Vazquez-Barquero JL , Welcher B , Gaite L , Becker T , Chisholm D , Ruggeri M , Schene AH , Thornicroft G : Translation and crosscultural adaptation of outcome measurements for schizophrenia . EPSILON Study 2. European Psychiatric Services: Inputs Linked to Outcome Domains and Needs. Br J Psychiatry Suppl 2000 , 39 : s8 - 14 .
23. Goldberg D , Williams P : A User's Guide to the General Health Questionnaire: GHQ London, UK: GL Assessment ; 1998 .
24. Ventura J , Green MF , Shaner A , Liberman RP : Training and quality assurance with the Brief Psychiatric Rating Scale: 'The drift busters' . Int J Methods Psychiatr Res 1993 , 3 : 221 - 244 .
25. World Health Organization: Psychiatric Disability Assessment Schedule. 2 edition . Geneva, Switzerland: World Health Organization ; 1988 .
26. Endicott J , Spitzer RL , Fleiss JL , Cohen J : The Global Assessment Scale. A procedure for measuring overall severity of psychiatric disturbance . Arch Gen Psychiatry 1976 , 33 : 766 - 771 .
27. Streiner DL , Norman GR : Health Measurement Scales: A Practical Guide to their Development and Use. 4 edition. Oxford, UK: Oxford University Press ; 2008 .
28. Nasr T , Kausar R : Psychoeducation and the family burden in schizophrenia: a randomized controlled trial . Ann Gen Psychiatry 2009 , 8 : 17 .