Response: Re: Promoting Regular Mammography Screening I. A Systematic Assessment of Validity in a Randomized Trial
We appreciate the opportunity to respond to the three comments of McCambridge et al. regarding our article. Our specific responses follow this general observation: Ever since the landmark efforts of Campbell and Stanley (1) to develop a classification system for characteristics of human research studies that can weaken causal inferences, a few characteristics have defied consensus. Some study characteristics classified originally as threats to external validity (1) have been reclassified as threats to construct validity (2,3). However, not all disciplines have adopted the revised classification scheme, and disagreements remain (4,5). In response to the first comment, we believe that our comparison of the control group with the two groups that were sent delayed baseline surveys assessed external rather than internal validity based on the following logic. As we explained, our tailored intervention required participant responses to a baseline survey, whereas the targeted intervention did not. To avoid a potential threat to internal validity due to pretesting (1,3), we administered the same baseline survey to all three groups of interest in the randomized trial, including the control group (group 3). Although such nondifferential pretesting across the study groups eliminated the threat to internal validity, the threat to external validity (1) or, more specifically, to construct validity (2,3) remained as the pretest may have conferred its own independent effect and modified (ie, interacted with) the targeted intervention effect. We resolved this problem by delaying administration of the baseline surveys in the posttest-only control groups (groups 4 and 5) to assess what would have occurred without pretesting (2,3). Although we can understand why McCambridge et al. may have construed this assessment as a test of internal valid-
-
ity at first blush, we hope this explanation
clarifies the rationale for our external
validity classification.
In response to their second comment,
as stated in our discussion, between-group
equivalence was not our sole criterion for
internal validity. For evidence of internal
validity, we examined the flow diagrams
that traced the passage of study candidates
through the critical sequence of
randomization, intervention delivery, and outcome
measurement as well as the between-group
equivalence of the participants remaining
at each time point. Although McCambridge
et al. view unintended differences in how
study groups are treated as threats to
internal validity, others consider them threats
to external validity (1) or to construct
validity (2,3).
In response to their third comment, we
note that the Hawthorne effect, one type
of reactivity to the experimental
situation, is considered a threat to external
validity (1,4) or construct validity (3).
However, there are dissenting views (5).
In any case, a Hawthorne effect would
have predicted increases in mammography
screening even in our posttest-only
control groups due to the perceived novelty of
being included in a research trial (2).
However, mammography rates in our
study population decreased over the study
period, when comparing either baseline
rates across control groups 3, 4, and 5 [see
Table 4] or preintervention rates with
postintervention rates in groups 1, 2, and
3 [see Table 4 and Table 1 per-protocol
results (6)]. Moreover, these findings
mirrored the declines in mammography rates
in the general US female population over
the same time interval.
DEBORAH J. del JUNCO
SALLY W. VERNON
SHARON P. COAN
(...truncated)