Impact of Hospital and Community Provider Based Clinical Audit Programmes: Perceptions of Doctors, Nurses and Other Health Professionals

International Journal for Quality in Health Care, Jan 1996


A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

Impact of Hospital and Community Provider Based Clinical Audit Programmes: Perceptions of Doctors, Nurses and Other Health Professionals

International Journalfor Quality in Health Care Impact of Hospital and Community Provider Based Clinical Audit Programmes: Perceptions of Doctors, Nurses and Other Health Professionals 0 Health Care Evaluation Unit, tDepartment of Public Health Sciences, St. George's Hospital Medical School , Cranmer Terrace, London SW17 ORE , UK JOANNE LORD* and PETER LITTLEJOHNSf A postal survey of staff (doctors, nurses, midwives, health visitors, therapy professionals, senior managers and clinical audit staff) was conducted in three Fnglish hospital and community healthcare providers. The aim was to assess staff perceptions of the impact of local clinical audit programmes and to investigate differences between staff groups. The questionnaire contained a 24 item opinion scale with a summary total: 371 out of 566 questionnaires were returned completed (66%). The majority of respondents were positive about the impact of clinical audit in their organizations, which is encouraging. However, there was a wide range of responses and significant variations between professional groups. Medical staff were significantly less positive than non-medics (p = 0.0007), and junior staff significantly less positive than seniors (p = 0.0306 for doctors and p = 0.0013 for other health care professions). After over five years experience of clinical audit in UK hospitals and community providers, many local staff remain sceptical about its real value. Copyright © 1996 Elsevier Science Ltd. INTRODUCTION Clinical audit was introduced by the UK government and widely welcomed as a profes sionally controlled system of peer review throughout the National Health Service (NHS) [ 1 ]. An extension of existing voluntary projects, it was to be a confidential process through which clinical teams could learn together and improve patient care. At the same time, managerial interests were recognised [ 2 ]. Most commentator? welcomed the national initiative, but there were warnings and hostile voices. Some questioned the effectiveness and cost-effectiveness of audit as means of changing clinical behaviour [ 3 ]. Others were suspicious over the government's motives, fearing that audit would be misused to attack clinical freedom in the interests of cost-containment rather than genuine quality improvement [ 4 ]. Since 1989, a lot of time and money has been spent on audit in the NHS [ 5,6 ]. The initial concept of "medical" audit has been extended to "clinical" audit, involving the whole range of health professionals, not just doctors. Government funding for clinical audit has been devolved to local Health Authorities, the "purchasers", who have been given responsibility for monitoring audit in their local hospital, community and primary care providers [7,8]. Much effort has gone into evaluating the national clinical audit programme [ 9-25 ]. This is an inherently difficult task, because of the diverse nature of the programme, the multitude of confounding factors and the lack of sufficiently broad outcome measures. The quantitative analysis of costs and outcomes that is really needed to assess whether the programme has been an appropriate use of scarce healthcare resources, is still lacking [ 26,27 ]. Taking a step back, one evaluative approach has been to consider the opinions of participat- geographical spread, and a mix of acute and ing staff. Good attitudes to audit are likely to be community units. A letter was sent to the Chief a necessary, if not a sufficient, condition for Executives describing the project and asking if "good" audit. Quantitative surveys of staff they would like to participate. This was followed attitudes to clinical audit have been conducted by a phone call to their audit co-ordinators two in several localities in the UK: they have weeks later. It was considered important to included primary care practitioners [ 28-30 ], > obtain the agreement of the audit co-ordinators, hospital and community consultants [ 31-35 ], as we wanted the exercise to be of use to them. junior doctors [ 9,35,36 ], non-clinical managers Three wanted to participate—two from mixed [ 31 ] and audit support staff [ 9 ]. However, none acute/community trusts and one from a smaller have included nurses, midwives, health visitors community trust. The level of audit activity in or therapy professionals—a serious omission the non-participating trusts was similar to that in given the current emphasis on multi-professional the participating trusts (Table 1). clinical audit. The studies report positive general attitudes to audit and belief in the value of specific local audit programmes, but with certain common reservations and differences between staff groups. Qualitative studies reveal more detail about the nature of staff attitudes and perceptions [ 37-39 ]. The above studies were conducted before or shortly after the introduction of local clinical audit programmes except for two recent studies in primary care [ 29,30 ]. As part of our monitoring role for the South Thames NHS region, we wanted to assess the views of hospital and community staff on their organizations' audit programmes now, after four or five years experience. Do staff think these programmes have a positive impact? Have the concerns highlighted in earlier studies been allayed? We wanted to include a wide range of staff, from all of the healthcare professions, as well as managers and audit staff, and to investigate differences within and between professional groups. METHODS Six provider organizations, NHS "trusts", in the South Thames area were selected to give a The two trusts who provide both acute and community services (Trust A and B) have central clinical audit departments, evolved from medical audit departments, and give help and advice to clinicians largely on a project basis. The trust that provides only community services (Trust Q has an audit programme based on a quality improvement philosophy, focusing on the dissemination of quality skills and agreement of outcome targets with clinical teams. The survey was targeted at all staff who might, or should, have had first hand experience of clinical audit in the trusts. This included: all consultants, senior registrars, registrars and career grades; senior nursing and therapy professionals (Grades G, H and I or equivalent); selected managerial posts; and clinical audit coordinators and support staff. Lists of staff were obtained by the trust audit co-ordinators from their personnel departments. Questionnaires were posted between March and June 1995, and a reminder sent to non-responders after a month. The questionnaires were anonymised, but number coded to allow follow-up by us (the trust audit departments did not have access to these codes). The questionnaire contained 24 statements Source: South Thames Regional Health Authority, Clinical Audit Annual Report, 1994/95. relating to the perceived effects of clinical audit in the provider organization (see Appendix). These items were derived from a qualitative analysis [ 40 ] of policy documents on medical and clinical audit from the Department of Health and other major national organisations [ 1,5,6,41-46 ]. Subjects were asked to assess each statement against a standard five-point Likert scale—1 "strongly disagree", 2 "disagree", 3 "uncertain", 4 "agree", or 5 "strongly agree". A scale total was calculated by adding up the individual item scores (having reversed the scoring of negatively worded items), and transforming to a scale from 0 to 100. For simplicity each item was given an equal weight. Streiner and Norman argue that more complicated weighting systems make little difference in practice with scales comprising more than about 20 items [ 47 ]. Tests of internal consistency were performed, and all items, except two, were found to have a significant positive association with the sum of the other items at the 1% level (Kendalls' r-b statistic) [ 48 ]. The two items that failed this test (C3 and HI) were omitted from the scale total. We took a scale total of 0-50 to represent negative overall feelings about the audit programme, and 50-100 to represent positive overall feelings. Individual items and the scale total were treated as ordinal data and analysed using non-parametric statistics [ 48 ]. In addition to the 24 item opinion survey, the questionnaire asked for an overall assessment of the value of the provider audit programme on a seven point scale from 1 for "not at all worthwhile" to 7 for "extremely worthwhile". There was a fair level of association between this overall assessment and the scale total described above—Kendall's t-b 0.53. Finally, the questionnaire asked for comments or suggestions on the way in which clinical audit works in their organization. Content analysis [ 49 ] was performed on this qualitative data using a classification system developed from that used in an earlier study [ 36 ]. RESULTS Questionnaires were sent to 583 people, 17 of these had retired or left the trust, seven refused to participate, and 371 returned completed questionnaires—an overall response rate of 66%. There were differences in response rate by trust and profession, but these were not significant (Chi-squared tests at 5%). 96 out of 155 staff (62%) responded in Trust A, 206 out of 311 (66%) in Trust B and 69 out of 100 (69%) in Trust C. 229 out of 303 non-medics (76%) responded compared to 141 out of 231 medical staff (61 %). There were no significant differences in response rate by grade or department. The results did not differ significantly between early responders (before reminders) and late responders (afterreminders),except for one item (IMP) for which late responders were more positive (Mann-Whitney U test, p = 0.0157). There was no significant difference in the frequency with which early and late responders used the five different categories of response (t-tests at 5%). More respondents were positive than negative about their experience of clinical audit, but there was a wide spread of responses (Fig. 1). The scale total median was 52.3, just on the positive side of neutral, and it ranged from 13.6 to 84.1. For most items the median response was 3, "uncertain", but for some the median was 4, showing more positive perceptions about: • the focus of audit on important topics (A2); • its effectiveness in changing clinical practice and improving patient care (B3 and Bl); • its educational role in improving professional knowledge and helping clinicians to learn together (Cl and C4); and • equal expectations of involvement from different professional groups (G2). However, the median response was 2 for the following items, showing that most respondents agreed that: • professionals found audit threatening (C3); • there was no real sharing of information about audit between professionals and managers (D3); • time spent on audit disrupted patient care (El); and • that there was unequal access to resources to support audit activity (Gl). Results varied between the trusts (Table 2). The scale total for Trust C, the smaller community trust, was significantly higher than that for the other two trusts (Mann-Whitney U test, p = 0.0014). There was no significant difference between Trust A and B. There were significant differences by profes5 Negative Total Total Scale total Number of valid responses * Median Range than their senior colleagues (Grade H and I or equivalent) (Mann-Whitney U test, p = 0.0013). Differences were also observed between clinical departments within the hospitals (Table 2). Staff working in women and child health were most positive and surgical staff were most negative. Except in Trust B, community staff were also quite positive. Comments from health visitors, district nurses and community nurses at Trust B pointed to particular problems. They had had little experience of clinical audit other than a single study led by management. This appeared to be a time-and-motion study rather than clinical audit Staff were highly suspicious of the motivation behind this project, and very critical about its conduct. District nurse "Lack of communication between management and staff makes audit threatening. As far as staff are aware only money seems important!... results are not fully explained or acted upon in a beneficial way to staff and patients." Community nursing staff were much less positive in their assessment of audit than other staff in Trust B (scale total median 44.3 compared to 53.4 respectively, n = 55 and 134, Mann-Whitney U test, p = 0.0000). For one of the trusts in the survey (Trust B) there was information available on the audit experience of staff over the last year: attendance at audit meetings and participation in audit projects. There was a higher level of audit activity amongst doctors than amongst other staff. Only four out of 63 doctors (6.3%) reported that they had not attended an audit meeting or participated in an audit project over the last year, this compared to 66 out of 142 other staff (46.5%) (Chi-squared test, p = 0.0000). Staff who reported that they had attended at least one audit meeting or participated in at least one audit project in the last year were significantly more positive about audit than those who had not (median scale total 53.4 compared to 47.7, n = 121 and 65, MannWhitney U test;? = 0.0017). A total of 197 out of the 371 respondents (53%) added written comments or suggestions (Table 3). The most common concern was the shortage of clinical time in which to undertake clinical audit. Number of comments Resources Resources for audit Resources to implement change Audit process Organisation, strategy and co-ordination Selection of topics Data collection Methods of audit Setting of standards/guidelines Implementation of change Cooperation Between specialities/departments/ professions Between clinicians/managers Between secondary and primary care Between providers Ownership Commitment of clinicians Feedback to clinicians Encourage clinician involvement Senior Registrar, psychiatry DISCUSSION Staff from only a small number of hospital and community provider trusts were included in this be proposed: that they reflect different levels of resources available to support clinical staff in their audit activity; that they reflect different experiences of clinical audit; and/or that they reflect differences in professional culture. There is insufficient evidence to support any one of these theories alone, it appears much more likely that they all make a contribution. Differential access to audit resources does not appear to be a very satisfactory explanation of the observed differences: doctors were more negative about audit, though they have received the bulk of the national audit funds, roughly ten times the money that the, much larger, nursing and therapy professions have received [ 5 ]. As mentioned above, shortage of clinical time to conduct audit is seen as a major problem. It is possible that this is more of a problem for some groups of staff than for others. Junior staff, for example, may well feel more pressured than senior staff, and as they are often responsible for collection and analysis of data, this may be one reason for their more negative views of audit. This is certainlyreflectedin the qualitative comments in our survey and in the more detailed analysis of Black and Thompson [ 37 ]. survey, and it can not be claimed that they are representative nationally. Their decision to participate probably indicates selection bias in favour of audit. The problems discussed below are likely to arise in other provider trusts, possibly to a greater extent The purpose of the survey was to elicit opinions related to specific experience of clinical audit within the organisations, not general beliefs about whether audit is, or is not, a "good thing". Of course, the two are not always separable, and some responses clearly reflected general concerns about clinical audit, or wider concerns about developments within the health service. Overall the opinions expressed in this survey are similar to those reported in other studies of medical staff (9,28-36). The majority of respondents were positive about their experience of audit, they agreed that it leads to changes in practice and improves patient care. They also felt that it is an educational process. This is encouraging as most people would agree that these are the most important aspects of audit. Indeed many staff in all three trusts made enthusiastic comments about the achievements and the potential of clinical audit. However, certain concerns highlighted in ear- Differences in the amount and type of audit lier surveys were seen to persist. Many staff in our experience may well have an effect. It has been survey were worried over the resource require- suggested [ 32 ] that the reason for surgeons' less ments of audit, particularly its demand on scarce positive opinions on audit compared to other clinical time. There were calls for greater com- doctors is their greater experience of audit, that munication and co-operation between profes- this greater familiarity has bred contempt, sions and departments over clinical audit. In demonstrating how time-consuming an activity contrast to earlier studies, however, there audit is, and how difficult it is to implement appeared to be little concern over confidentiality changes and bring about improvements as a in audit or its possible impact on clinical freedom. result of audit. Surgeons also have more experiA small minority of staff clearly did not believe in ence of large national comparative audits, by clinical audit, and resented being forced to comparison with which local audit might appear participate. These "pockets of discontent" must trivial and unproductive. represent a problem for the development of Higher levels of audit activity amongst docorganizational quality strategies. tors compared to other professions, might Considerable differences in opinion within actually be used to explain their greater sceptiand between staff groups were observed. Medi- cism about audit. Of course, this theory rests on cal staff were significantly less positive about the assumption that audit activity is greater or their experience of audit than non medical staff. more widespread amongst doctors than in the Junior staff (both medical and non-medical) other health care professions, for which there is were significantly less positive than their seniors. no hard evidence. Certainly, the medical profesStaff working in surgical departments were sion has a long history of clinical audit, but a significantly less positive, and those in women's national survey in 1989 actually found more and child health and community departments audit and quality assurance projects in the nonsignificantly more positive than others. Three medical health professions [ 50 ]. The governpossible explanations for these differences might ment, has stated that all doctors should participate in regular audit, but has not placed a similar requirement on other healthcare professionals [ 1 ]. This suggests that doctors participating in audit will be less self-selected than nurses and other professionals. The idea of a negative relationship between audit activity and opinions on the value of audit was not supported by the evidence from our survey. For one of the trusts, we had information on respondents' level of experience of audit over the last year. There was evidence that more doctors had attended audit meetings and participated in audit projects over the last year than nurses and other healthcare professionals. But respondents with experience of audit over the last year were actually more positive than others about the value of audit. Finally, cultural differences between professions and clinical departments might be used to explain their different opinions of audit. For instance, community based departments might be more used to multi-professional working and so more open to clinical audit. Earlier surveys showed high levels of suspicion amongst doctors over the government's motives for introducing clinical audit [ 37 ]. Many feared that it would be used as a "Trojan horse" to challenge their traditional clinical freedom. This is less likely to be a problem amongst the nursing and therapy professions who have never had the same autonomy. There was no evidence of such differences between the professions in the written comments from our survey. In fact, very few respondents objected to the basic principle of increased scrutiny of their work by themselves and others that auditrepresents.However, it can not be ruled out that they might be offering tactical answers, complaining about the organisation and resourcing of audit, to cover more fundamental problems related to professional power, that they find difficult articulate openly, even in a confidential survey. Acknowledgements: We thank the staff of the participating trusts, particularly the audit co-ordinators who helped in the conduct of the survey. Source of funding: South Thames Regional Health Authority. Conflict of interest: none. APPENDIX Relevance Items for the subjective appraisal of local clinical audit programme Al The clinical audit programme covers the totality of health care from prevention to treatment, care and rehabilitation. A2 Clinical audit does not really focus on the important topics • Effectiveness Bl Clinical audit improves patient care. B2 Audit increases the sensitivity of provider staff to the needs and wishes of users. B3 Audit rarely leads to changes in clinical practice.* B4 When appropriate, management act on recommendations from clinical audit Development Cl Audit helps clinicians to enhance their professional knowledge. C2 Health care professionals have confidence in audit as an educational process. C3 Clinicians often feel very threatened by audit* C4 Audit helps clinical teams to learn together. Collaboration Dl Audit brings together health care staff from different professions. D2 There is little co-operation across departmental boundaries over audit* D3 There is no real sharing ofinformation about audit between health care professionals and managers.* D4 Many clinicians take part in collaborative audits with colleagues from other acute or community trusts. D5 Purchasers make a positive contribution to the audit programme. D6 Audit helps to improve links with general practice. Efficiency El The amount of time and money spent on audit in this trust is about right E2 Time spent on clinical audit does disrupt patient care.* Inctusiveness Fl Patients have the opportunity to make a meaningful contribution to the audit programme. F2 Patient participation in audit is not really encouraged.* Gl Some groups of staff have less access to audit training, funding and support than others.* G2 Equal involvement in audit is expected from different professional groups. HI Clinical audit support staff need more job security to be able to work effectively.* H2 There is a clear strategy for the long term development of clinical audit in this trust. •Negatively phrased items for which the scoring was reversed. 1. Department of Health. Medical audit . NHS review working paper 6 . HMSO, London, 1989 . 2. Lord J and Littlejohns P , Links between clinical audit and contracting systems . International Journal of Health Care Quality Assurance 1995 ; 8 : 15 - 24 . 3. Maynard A , Case for Auditing Audit . Health Service Journal 1991 ; 18 (July): 26 . 4. Jessop J , Audit ; all talk and no action? Health Service Journal 1989 ; 31 ( August ): 1072 . 5. Department of Health. Clinical audit: meeting and improving standards in healthcare , 1993 . 6. Department of Health. The evolution of clinical audit , 1994 . 7. Department of Health. Clinical Audit: 1994 /95 and beyond, 1994 8. Department of Health. The new health authorities and the clinical audit initiative: outline of planned monitoring arrangements , 1995 . 9. Kerrison S , Packwood T and Buxton M J , Medical audit: taking stock . King's Fund Centre , London, 1993 . 10. Kerrison S , Packwood T and Buxton M J , Monitoring medical audit . In Robinson R and Le Grand J. (eds) . Evaluating the NHS reforms . King's Fund Institute , London, pp. 155 - 77 , 1993 . 11. Harman D and Martin G , Medical audit and the manager a discussion document . Health Services Management Centre, Birmingham , 1991 . 12. Robinson M B, Evaluation of medical audit . Journal of Epidemiology & Community Health 1994 ; 48 : 435 - 440 . 13. Walshe K , editor. Evaluating clinical audit past lessons, future directions . The Royal Society of Medicine Press, London , 1995 . 14. Walshe K and Coles J , Evaluating audit a review of initiatives . CASPE Research , London, 1993 . 15. Buttery Y , Walshe K , Coles J , et al., Evaluating medical audit: the development of audit Findings of a national survey of healthcare provider units in England . CASPE Research , London, 1994 . 16. Rumsey M , Walshe K , Bennett J , et al., Evaluating medical audit: the role of the commissioner in audit. Findings of a national survey of commissioning authorities in England . CASPE, Research, London, 1994 . 17. Walshe K , Bennett J , Buttery Y , et al., Evaluating audit: effective audit? Results of a detailed review of healthcare providers' audit programmes . CASPE Research , London, 1994 . 18. Willmot M , Foster J , Walshe K , et al., Evaluating audit. A review of audit activity in the nursing and therapy professions: findings of a national survey . CASPE Research , London, 1995 . 19. Foster J , Willmot M , Walshe K , et al., Evaluating audit. Nursing and therapy audit: a review of the regions' role . CASPE Research , London, 1995 . 20. Department of Health. Medical audit in the hospital and community health service , 1994 . 41. Department of Health, Framework of audit for nursing services: guidance document , 1991 . 42. Standing Committee on Postgraduate Medical Education. Medical audit the educational . implications. SCOPME , London, 1989 . 43. Scottish Office . Guidance on the development of clinical audit , 1993 . 44. Royal College of Physicians. Medical Audit: a second report. London. The Royal College of Physicians of London , 1993 . 45. Royal College of Surgeons. Guidelines to clinical audit in surgical practice . London: The Royal College of Surgeons of England , 1989 . 46. Royal College of Psychiatrists. Preliminary report on medical audit. Psychiatric Bulletin 13 : 577 - 80 , 1989 . 47. Streiner D L and Norman G R , Health measurement scales: a practical guide to their development and use . Oxford University Press, 1995 . 48. Siegel S and Castellan N J , Nonparametric statistics for the behavioural sciences . 2nd ed. McGraw-Hill , New York, 1988 . 49. Weber R P, Basic content analysis . 2nd ed. Sage, California , 1990 . 50. Dailey G , Baldwin S , Carr-Hill R A , et al., Quality management initiatives in the NHS: strategic approaches to improving quality . York. Centre for Health Economics . QMI Series No. 3 , 1991 . 21. Hopkins A (ed), Medical audit activities of the Royal Colleges and their faculties in the UK . Department of Health, 1994 . 22. von Degenberg K , Clinical audit in the nursing and therapy professions . Department of Health, 1994 . 23. Department of Health. Medical audit in primary care , 1994 . 24. National Audit Office. Auditing clinical care in Scotland . HMSO, London, 1994 . 25. National Audit Office. Clinical audit in England. Report by the Comptroller and Auditor General. HMSO , London, 1995 . 26. Buxton M J , Achievements of audit in the NHS . Quality in Health Care 1994 ; 3{Supplement): S31 - S34 . 27. Barton A G , Thonison R G and Bhopal R S, Clinical audit: more research is required . Journal of Epidemiology & Community Health 1995 ; 49 : 445 - 447 28. Webb S J , Dowell A C and Heywood P , Survey of general practice audit in Leeds . BMJ 1991 ; 302 : 390 - 392 . 29. Baker R , Robertson N and Farooqi A , Audit in general practice: factors influencing participation . BMJ 1995 ; 311 : 31 - 34 . 30. Lough J R , Mckay J and Murray T S , Audit: trainers' and trainees' attitudes and experiences . Medical Education 1995 ; 29 : 85 - 90 . 31. Smith H E , Russell G I , Frew A J and Dawes , P T , Medical audit: the differing perspectives of managers and clinicians . Journal of the Royal College of Physicians of London , 26 , 177 - 180 ( 1992 ). 32. Barton A G and Spencer J , Differences in attitudes towards audit among specialties in the Northern region . Medical Audit News 1992 ; 4 : 78 - 79 . 33. Webb M D and Harvey I M, Taking stock of medical audit: a questionnaire survey . Medical Audit News 2 : 1992 . 34. Webb M D and Harvey I M, Auditing the introduction of audit . Medical Audit News 1994 ; 4 : 19 - 20 . 35. Davison K , Time spent by doctors on medical audit . Psychiatric Bulletin 1993 ; 17 : 41 &- 419 . 36. Firth-Cozens J and Storer D , Registrars' and senior registrars' perceptions of their audit activities . Quality in Health Care 1992 ; 1 : 161 - 164 . 37. Black N and Thompson E , Obstacles to medical audit: British doctors speak . Soc Sd Med 1993 ; 36 : 849 - 856 . 38. Harvey G , An evaluation of approaches to assessing the quality of nursing care using (predetermined) quality assurance tools . Journal of Advanced Nursing 1991 ; 16 : 277 - 286 . 39. Dawson J , The role of quality assurance in future midwifery practice . Journal of Advanced Nursing 1993 ; 18 : 1251 - 1258 . 40. Fitzpatrick R and Boulton M , Qualitative methods for assessing health care . Quality in Health Care 1994 ; 3 : 107 - 113 .

This is a preview of a remote PDF:

LORD, JOANNE, LITTLEJOHNS, PETER. Impact of Hospital and Community Provider Based Clinical Audit Programmes: Perceptions of Doctors, Nurses and Other Health Professionals, International Journal for Quality in Health Care, 1996, 527-535, DOI: 10.1093/intqhc/8.6.527