A measuring instrument for evaluation of quality systems

CORDULA WAGNER 0 DINNY H. DE BAKKER 0 PETER P. GROENEWEGEN 0 Setting. The Netherlands. 0 0 NIVEL, Netherlands Institute of Primary Health Care , Utrecht, The Netherlands Objective. To develop an instrument for provider organizations, consumers, purchasers, and policy makers to measure and compare the development of quality systems in provider organizations. Design. Cross-sectional study of provider organizations using a structured questionnaire to survey managers. Study participants. Provider organizations of six health care fields: primary health care, care for the disabled, mental health care, care for the elderly, hospital care and welfare care. Main measures. Existence of quality assurance and quality improvement activities. Results. The study presents a survey instrument for assessing the quality assurance and improvement activities of health care provider organizations and the developmental stage of quality systems. The survey instrument distinguishes five focal areas for quality improvement activities and four developmental stages. The study also reports data on the reliability and validity of the survey instrument. Conclusion. The instrument is reliable, easy to administer, and useful across health care fields as well as different kinds of organizations. Developing quality systems provide a common language across all parts of the health care sector. By assigning the activities to focal areas and developmental stages the instrument gives insight into the implementation of quality systems in health care. Comparable information on quality assurance activities increases the accountability of providers. Because of the efficient (not time consuming) approach, the instrument complements existing accreditation reviews. - Since the time that the question changed from whether quality can be measured to how best to measure quality, interest has been focused upon the selection of measurement sets which reliably and credibly inform about quality of health care service [1]. The complexities of health care demand a balance between structure, process and outcome measures in quality monitoring. Quality systems that influence the structure and processes in provider organizations are one approach used to avoid poor quality. Advocates of quality systems suggest that they have significant potential to enable provider organizations to improve quality without increasing costs. In this study we define a quality system as the organizational structure, procedures, processes and activities that are mutually dependent and directed at the improvement of health care services. By measuring the developmental stage of a quality system, purchasers, consumers and regulators can more easily compare provider organizations. On the other hand, provider organizations can also compare themselves with other organizations and can show patients and purchasers what improvements have been made in the service delivery process. Although in the USA, states have an array of regulations designed to strengthen the position of patients, questions arise as to whether the states or the Federal Government have the resources to monitor and properly enforce the regulations. In a study done by the USA General Accounting Office on state oversight, the conclusion was that states needed to institute a set of safeguards to protect consumers, including better quality assurance mechanisms [2]. Therefore, an efficient and routine examination of the organizations arrangements to control and assure the quality of care is required. Different organizational audit frameworks exist that highlight areas of an organization that experts believe to be essential to the organizations ability to provide consistently good quality of care [36]. Examples are the European ISO 9000 standards, the Malcolm Baldrige USA National Quality Award, the UK Kings Fund Accreditation, the European Quality Award (EFQM) and the Dutch Quality Award. Moreover, organization theory describes developmental stages that organizations follow during the implementation of innovations. The four stages most frequently distinguished are: (i) orientation and awareness that change is necessary; (ii) planning and preparation for change; (iii) implementation of projects; and (iv) organization-wide implementation and establishment of the innovation [710]. The rationale for developing an instrument to measure focal areas and the developmental stage of quality systems was provided by the need to obtain information on how provider organizations assure the quality of care, and how many have actually developed a quality system. Until now quality systems have been evaluated by voluntary accreditation processes; in general such information is not available for research. These evaluations are very time-consuming and for that reason not suitable for gathering comparable data from many provider organizations. In the literature only a few studies have been found that assess, on a wider scale, the development of quality management against a set of criteria [1115]. All studies have taken place in a hospital setting with different questionnaires. The literature as yet describes no sustained approach to assess the developmental stages of quality systems in health care across sectors of care. The purpose of this study is to assess the internal consistency, reliability and construct validity of a survey instrument measuring the developmental stage of quality systems in provider organizations. The instrument presented in this article has been used to measure the developmental stage of quality systems across health care sectors in a nationwide inquiry. Premises in the inquiry were the management perspective and a total quality management approach. Data used in the analyses were survey-data collected in a large nationwide study within different health care sectors and health care-related social service sectors in The Netherlands. Almost all provider organizations are registered as members of one national umbrella organization; all members of this organization were included in the study and received a postal questionnaire. Only for the organizations for the elderly did we take a random sample (10% of the homes for the elderly and 50% of the nursing homes). A total of 1594 provider organizations were approached; 315 organizations of primary health care, 372 organizations for disabled people, 248 mental health care organizations, 316 organizations for the elderly, 143 hospitals and 200 organizations of health care-related social services. The questionnaire was sent to the management of the organization; the professionals were not involved in the study. Therefore, the data show the perspective of the management. The instrument was part of a larger survey. Survey instrument The questionnaire (Appendix) was developed by the researchers in co-operation with experts on quality improvement from different health care fields, and partly derived from the Dutch Quality Award, which is a translat (...truncated)