Evaluating Diabetes Education: Are we measuring the most important outcomes?

B E H A V I O R A L D I A B E T E S S E R I E S A R T I C L E Evaluating Diabetes Education Are we measuring the most important outcomes? RUSSELL E. GLASGOW, PHD VIRGINIA L. OSTEEN, BA This article reviews the published literature on diabetes education evaluations and makes recommendations for outcome measures to be used in future research. We conclude that program evaluations to date have focused too narrowly on assessing knowledge and GHb outcomes to the exclusion of other important variables. To reflect the changing emphasis and conceptual basis of diabetes education, we recommend that future evaluations do the following: I) report on the program's target population, recruitment methods, and representativeness of participants; 2) collect measures of self-efficacy and patient-provider interaction; 3) include quality of life and patient-functioning outcomes; and 4) use more standardized and objective measures of diabetes management behaviors. We close by providing practical examples of feasible collection measures for most settings and references to studies that have done so. M ore than 100 evaluations of diabetes education have been published, and there are probably more than twice that number of dissertations, masters theses, and unpublished reports on this topic. Provocative debates on the efficacy and cost-effectiveness of diabetes education have appeared (1,2). These issues are especially important, given the current crisis in health care reimbursement and financing, and increasing national emphasis on medical outcomes research. Useful meta-analyses of the results of diabetes education programs have been published (3,4), and these reviewers have concluded, as does Peyrot (5), that, in general, diabetes education is effective. Even these much needed reviews have not been able to provide conclusive answers to several important questions about diabetes education, such as the classic outcome research issue: "What treatment, by whom, is most effective for this individual with that specific problem, under which set of circumstances, and how does it occur ( 6 ) " . . . to which we might add: and how cost-effective is it? A major reason for the lack of consensus on the value of diabetes edu- cation is that data relevant to many key issues typically are not available. The thesis of this article is that evaluations of diabetes education have been too narrowly focused and have not assessed the most important outcomes. To our knowledge, no thorough review of measures for use in evaluating diabetes selfmanagement programs has been conducted since the classic report by the Rand Corporation in 1979 (7). This article attempts to fill this void and to substantiate the above thesis. Using the published studies reviewed in Brown's recent meta-analysis as a basis (3), we discuss the frequency of use and quality of the measures that have been collected within each of six conceptual categories: J) environmental and social context of a program, 2) characteristics of participants, 3) process and mediating variables, 4) diabetes management, 5) short-term health outcomes, and 6) long-term health outcomes. We then discuss what these studies have measured thoroughly, and what they have not, and provide examples of model studies within each of the above categories. Evaluations of diabetes education range from tightly controlled, and often highly selective clinical research trials to broad scale, often uncontrolled program evaluations of ongoing programs. This article and the Brown review include both types of evaluations. It is acknowledged that most of the measures recommended come from the research end of this continuum. Part of the purpose of this article is to identify measures that are feasible to use in both settings. In conclusion, we note the complexity of the challenges inherent in evaluating diabetes education and make recommendations for future research. FROM THE OREGON RESEARCH INSTITUTE, EUGENE, OREGON. ADDRESS CORRESPONDENCE AND REPRINT REQUESTS TO RUSSELL E. GLASGOW, PHD, OREGON RESEARCH INSTITUTE, 1899 WILLAMETTE STREET, EUGENE, OR 9 7 4 0 1 . RECEIVED FOR PUBLICATION 26 AUGUST 1991 AND ACCEPTED IN REVISED FORM 20 APRIL 1992. DCCT, DIABETES CONTROL AND COMPLICATIONS TRIAL. THIS ARTICLE IS ONE OF A SERIES PRESENTED AT THE MEETING ON THE BEHAVIORAL ASPECTS OF DIABETES MELLITUS. DIABETES CARE, VOLUME 15, NUMBER 10, OCTOBER 1992 REVIEW OF THE RESEARCH— Outcome measures for a diabetes education program can be conceptualized along a temporal continuum, ranging from those associated with initial contact with potential participants 1423 Measuring the most important outcomes IMPORTANT VARIABLES TO ASSESS SOCIAL & ENVIRONMENTAL CONTEXT PATIENT CHARACTERISTICS DURING PROGRAM: PROCESS & MEDIATING VARIABLES DIABETES MANAGEMENT - LIFESTYLE CHANGE - MEDICAL SELF-CARE - PATIENT - PROVIDER INTERACTION AFTER PROGRAM: SHORT - TERM HEALTH OUTCOMES - PHYSIOLOGIC -QUALITY OF LIFE Studies often include, however, only a single measure in each of two categories: a measure of patient knowledge in the process/mediating variable category and a measure of GHb in the short-term health outcomes category. We recommend that greater attention be focused on assessing outcomes in all results categories, especially characteristics of participants and long-term health outcomes; and collecting objective, standardized, specific, and, if possible, multiple measures within each category. Table 1 expands the six categories outlined above by listing the types of variables within each of them. Asterisks indicate variables that have been underresearched. The following sections briefly review the status of research within each of these categories. Social and environmental context Diabetes education does not occur in a vacuum. We hypothesize that much of the variance in the outcomes of diabetes education programs eventually can be understood by careful analyses of socialenvironmental factors (8,9). In Table 1, most of the measures listed in this category have asterisks, indicating that they have been underresearched. Substantial literature on the relationship of social support (especially from family members) to diabetes adherence, psychosocial adjustment, and glycemic control is available (10,11). But LONG-TERM HEALTH OUTCOMES Table 1—Relevant variables within each assessment category Figure 1—Temporal sequence of program results. to follow-up status many years after the program (Fig. 1). The first two categories of social- environmental context in which a program occurs and characteristics of participants usually are not considered outcomes. However, these factors provide important information about the public health impact of an education program and provide a context within which to interpret other program results. During a program, participants presumably change through improvement in underlying process or mediating variables, such as knowledge or self-effi (...truncated)