Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers

Australian Journal of Teacher Education, Dec 2018

This paper posits that almost all inspectorates are now following, if to varying degrees, a similar overarching ideology and methodology for school accountability and improvement. The first part of the paper provides an analysis of recent changes to school inspection policies across Frontiers. Using Ireland as a case example, the next part of the paper provides an analysis of Irish school inspection policies and practices that appear to mirror other school inspection systems. To test these assumptions, the paper then provides an analysis of a key informant interview with the Chief Inspector of schools in Ireland. The evidence suggests that there is an increasingly convergent approach to school evaluation discernible across all inspection frontiers. Among the many aspects of this changing landscape is the rapid pace at which schools have accepted school inspection frameworks and the emergence of a genuinely co-professional as opposed to co-existent mode of evaluation between the inspectorate and schools.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://ro.ecu.edu.au/cgi/viewcontent.cgi?article=4163&context=ajte

Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers

Vol Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers Martin Brown 0 Shivaun O'Brien 0 0 EQI, School of Policy and Practice, DCU Institute of Education - Article 6 Integrated co-Professional Evaluation? Converging Approaches to School Evaluation across Frontiers Martin Brown Gerry McNamara Joe Ohara Shivaun O?Brien Cornelius Young DCU Institute of Education, Ireland Jerich Faddar Universiteit Antwerpen, Belgium Abstract: This paper posits that almost all inspectorates are now following, if to varying degrees, a similar overarching ideology and methodology for school accountability and improvement. The first part of the paper provides an analysis of recent changes to school inspection policies across Frontiers. Using Ireland as a case example, the next part of the paper provides an analysis of Irish school inspection policies and practices that appear to mirror other school inspection systems. To test these assumptions, the paper then provides an analysis of a key informant interview with the Chief Inspector of schools in Ireland. The evidence suggests that there is an increasingly convergent approach to school evaluation discernible across all inspection frontiers. Among the many aspects of this changing landscape is the rapid pace at which schools have accepted school inspection frameworks and the emergence of a genuinely co-professional as opposed to co-existent mode of evaluation between the inspectorate and schools. Introduction During the last two decades, many European education systems have attempted to modernize the control and monitoring of schools by establishing some form of what is often described as evidence-based governance (Altrichter et al. 2016, Schildkamp, and Ehren, 2012) . Such regimes are usually built on two dominant arrangements which often exist side by side, achievement standards and tests of student performance on the one hand, and school inspections on the other (Ehren et al., 2016). Both of these features are evident in Ireland with increasing emphasis on standardized tests and achievement in state examinations together with what might be described as the reconceptualization of school evaluation to include both inspection and school self-evaluation (Brown, McNamara, O?Hara and O?Brien, 2016) . Testing is for another day. The paper at hand aims to describe the main features of recent changes in inspections both internationally and in Ireland based, firstly, on an analysis of the extensive literature on school evaluation including policy documents and subsequently on an interview with the Chief Inspector of Irish schools. The paper commences by suggesting that there is evidence of some convergence between the very widely differing school inspection approaches to be found across Europe and by identifying some of the common themes or features of this shared notion of inspection. Ireland is then offered as a good illustrative example of the way in which school inspection appears to be evolving internationally. This is done through an analysis of policy documents governing inspection and by presenting the thoughts of the Chief Inspector of Irish schools Dr Harold Hislop. The paper concludes with some reflections on changing school inspection modalities both in Ireland and internationally. Methodology The methodology used in this study is an evidence-based mixed methods analysis of school evaluation policies and practices. Following the ?policy-scientific? approach (Leeuw, 2003), the first phase of the study deconstructs the literature relating to the changing conception of school inspection. Using Ireland as a case study example, the next step provides an analysis of literature relating to the extent to which this integrated mode of school evaluation has permeated Ireland?s education system. Finally, the latter part of the paper, further examines these global and local trends, by providing a key informant interview (Gilchrist, 1993) with the Chief Inspector of Schools in Ireland. A Converging Model of School Inspection? In a climate of globalization and international comparisons, evaluation and accountability have become key educational issues in most developed countries (European Commission, 2004; OECD, 2013) . As a consequence, many education systems have introduced varieties of evidence-based governance regimes. School inspections are one of the major incarnations of this governance concept. In line with the logic of evidence-based governance (see Ehren et al., 2013, 4) inspectorates of education (1) set expectations by virtue of their inspection standards and procedures. They (2) collect evidence during inspection visits and use information produced by other evaluation instruments, such as student testing, to assess the quality of education and hold schools accountable for a broad range of goals related to student achievement, teaching, organization, and leadership. They (3) aim to stimulate school and system improvement, sometimes by the deployment of sanctions, but more usually by producing reports which point to strengths and weaknesses of individual schools. These reports also include or imply recommendations for action to be undertaken by the inspected schools or the authorities in charge of them (Scheerens and Ehren, 2015) . Finally, they (4), to lesser or greater extent, require or encourage schools to engage in school self ? evaluation both to complement external inspection and to provide an evidential base for school improvement (Brown, McNamara & O?Hara, 2016a; Brown, McNamara O?Hara, & O?Brien, 2016b; Brown, 2013) . However not everything which was labelled ?inspection? operated in identical ways. According to Donaldson (2013), ?inspection is often associated in the public mind with a rather narrow set of activities which involve notions of compliance and audit. In fact, it is a very plastic concept which takes and has taken many forms, and which can serve many different purposes? (8). For example, some of the European Inspectorates of Education (e.g. HMI in England and the Irish Inspectorate of Education), have existed since the mid nineteenth century, have always been an arm of central government and oversaw both compliance with nationally mandated policies and mainly judged the performance of individual teachers rather than schools as organizations per se (Brown, McNamara, O?Hara, 2016) . In contrast the traditional inspection role in centralist-bureaucratic states (such as Austria or the Czech Republic) was based on an intermediate position in the hierarchical line located above individual schools but beneath the central administration. Several supervisory functions for schools were amalgamated in this role. Heads of schools were directly accountable to these inspectors who were also responsible both for distributing staff to individual schools and for inspecting and controlling the ?quality? of the schools in their region. Criteria for assessing this quality may have varied in time but always included some mixture between educational effectiveness on the one hand and legal and administrative compliance on the other (Altrichter et al. 2014) . In recent decades these historical differences came to be substantially subsumed into divergent inspection policies based more on ideological stance as opposed to tradition. Scholars of school inspection identified two fairly distinct camps, as it were, with very different concepts of inspection. On the one hand, certain inspectorates, notably OFSTED in England, other parts of the English speaking world such as New Zealand, and some European countries including the Netherlands and to an extent Sweden were characterized as being driven by reliance on hard data and concerned primarily with monitoring and accountability (see for example, Nelson and Ehren, 2014; Ryan and Timmer, 2013; Perryman, 2009) . In contrast an alternative model, associated with Scotland and the seminal work of John MacBeath, and adopted widely in Central and Eastern Europe was perceived to be more focussed on a variety of data sources, interested in school self-evaluation and more concerned with collaborative improvement than accountability (MacBeath ,1999, 2006; Nevo, 2010) . While significant disparities still remain, in recent years a noticeable convergence between these different traditions, ideologies and approaches has become perceptible. Due to a range of influences including membership of international bodies such as the EU, the OECD and SICI, the Standing International Conference of Inspectorates, globalization and international comparisons of education systems such as PISA and the drive for reforms such as greater accountability and decentralization of decision making in public services, there has been a significant coming together of school inspection theory and practice (Ehren and Shildkamp, 2012) . This point is further illustrated by the remarkable similarities across countries displayed in the OECD?s Synergies for Better Learning report (2013), and the types of school inspection found among SICI member countries. As van Bruggen (2010 ) stated, ?the content of the various frameworks in use by 18 inspectorates in Europe does not differ widely? (91). So marked has this process been that it is possible we suggest to speak of coprofessional integrated inspection systems with common features across much of Europe. We propose the term co-professional to indicate a high level of professional co-operation and respect between inspectorates and inspectors on the one hand and schools and teachers on the other. This is operationalised through, for example, negotiated and agreed standards and frameworks and acceptance of school self-evaluation as a valid element of inspection. We propose the term integrated in the sense of integrating ideologies and approaches which previously tended to be associated with very differing conceptualizations of inspection. The most notable of these would be a more widely accepted focus on the importance of data including student achievement data in evaluating schools and the increasingly central role accorded to school self- evaluation as a component of the evaluation process. (Brown, McNamara, O?Hara and O?Brien, 2016, McNamara and O?Hara, 2012; Ehren, Perryman, & Shackleton, 2014) . These ?co- professional integrated? inspection systems differ from their predecessors in that they, ? tend to concentrate more on evaluative functions and place the focus on evidence and less on administrative/supervisory/compliance functions; ? aim to professionalize these evaluative functions by formalizing them through developing sets of standards/criteria/ indicators and by the use of instruments taken from social science to collect data both quantitative and qualitative; ? utilise a range of inspection treatments driven by risk assessments based on data or follow up to previous inspections rather than the more traditional cyclical model; ? are concerned with both accountability/public reporting and school improvement but with a greater emphasis on the latter; ? focus on the performance of schools as organizations as much, or perhaps more, than on the work of individual teachers; ? perceive collaboration and co-professional dialogue between inspectors and schools/teachers as fundamental; ? look to involve stakeholders including parents and students in the inspection process; ? encourage schools to self-evaluate in a formal way as part of the inspection process. By virtue of these features ?co- professional integrated inspection? fits well in to the framework of ?rationalized control? propagated by the proponents of evidence-based governance. In this conceptualization inspectors are not simply bureaucrats who are supervising schools according to old-fashioned and easily controllable administrative criteria; rather they aspire to form a new breed of professionals in close contact with schools and teachers and with the latest developments in educational research. Ireland provides a very interesting example of the growth of what one might begin to describe as an increasingly typical inspection model and this paper hopes to clarify both what this looks like and the thinking driving the emerging policy and processes. School Inspection in Ireland With the introduction of a centrally administered education system in Ireland in the 1830s, in a letter to the Duke of Leinster, the Chief Secretary, Lord Stanley, proposed a structure for the ?superintendence? of education that would be overseen by a board of commissioners. According to what is frequently referred to as the ?Stanley Letter? (1831), within this centralized administration, exchequer funding was to be used for six particular purposes, such as ?granting aid for the erection of schools? (Stanley, 1831, 4). The second of these purposes resulted in the commencement of school inspection in Ireland, by allowing exchequer funding to be used for the purpose of ?paying Inspectors for visiting and reporting upon schools? (ibid, p.4). Although the specific duties of the inspectorate consisted of various legal and administrative activities, Akenson (1970) notes that ?as far as routine inspection visits were concerned, the inspectors? tasks were just what one would expect? (146). For example, inspectors were required to ?observe the teacher conducting the class and suggest any improvements that might occur to him? (ibid). Inspection continued along these lines until the foundation of an independent Irish state in 1922 (Brown, McNamara and O?Hara, 2016) . After this date a strong focus of inspection became the implementation of policies around the restoration of the Irish language to be accomplished largely through the school system. The figure of the ?cigire? (Irish for inspector) became a respected, if rather feared, part of the educational landscape. The visit of the ?cigire? was a big event for schools, teachers and indeed pupils. However in the final decades of the twentieth century the role of inspection greatly diminished, partly due to the power of the teacher unions but also because much of the resourcing of the Inspectorate was directed to other tasks particularly the management of the annual state examinations. Interestingly, it is suggested by scholars of inspection in Ireland that the impetus for the revival of inspection in the 1990?s came not from domestic political demands but largely from international developments and the influence of the OECD. (See: McNamara and O?Hara., 2008, 2002) . In the Education Act (Government of Ireland, 1998) a statutory framework relating to the duties of school inspectors came into force. As stated in Section 13(3), ?the functions of an Inspector shall be: to support and advise recognised schools, centres for education and teachers on matters relating to the provision of education??. The Act is clear that a dual system of internal and external school evaluations was envisioned, resulting in the development of a multi-mode inspection framework and concomitant government-devised school self-evaluation instruments. Thus, it was in this period that the forerunners of ?new? forms of school inspections were put into place. Until 2010, school inspection was typically conducted using four evaluation instruments, namely thematic, subject, whole school evaluation (WSE), and at Post-Primary level, programme inspections. Although the frequency of inspections varied depending on the evaluation instrument used, inspection schedules were infrequent and centred on a cyclical mode of inspection (See: Ehren et al., 2013). According to the Standing International Conference of Inspectorates (SICI) profile of the Irish inspectorate in 2009, ?at present, primary level inspections are carried out approximately once every nine to ten years ? At Post-Primary level, schools may expect a Whole-School Evaluation with a similar frequency? (SICI, 2009, 14) . However, as subject, thematic, and programme inspections were conducted along with whole school evaluations (WSEs), the inspections were generally more frequent. Recent Changes in Inspections in Ireland Since 2010 Ireland has experienced a profound restructuring of its school inspection process, including significant changes to its schedule for inspections and the types of inspection instruments that are used. As previously stated until this point school inspections had typically used four evaluation types, and for the most part, internal planning activities had primarily consisted of developing school policies and plans. Moreover, there was little evidence to suggest that schools were collecting and analysing the data necessary to develop and implement actions plans for improvement (See: McNamara and O?Hara, 2008, 2006, 2005) . In fact, akin to other European countries, school self-evaluations were rarely mentioned, if at all, in the majority of school inspection reports. However during the 2009/2010 academic year the inspectorate began to experiment with new modes of inspection, including the introduction of a codicil version of WSE referred to as WSE-MLL (Whole School Evaluation ? Management, Leadership and Learning) . Significant changes compared to the WSE model are evident and primarily relate to: (1) a greater emphasis being placed on internal evaluation where the ?onus is placed on schools to engage in the self-evaluation process? (Egan, 2010, 53) (2) the structure and composition of the final WSE report, and (3) the inclusion of parent/student opinions in the form of questionnaires. According to the Chief Inspector WSE-MLL ?will provide a shorter and more focused report on the school? with less time on school planning and even more time in classrooms? (Hislop, 2010, 20). The most notable change in this new model of the WSE was the introduction of confidential, anonymous questionnaires that are given to parents and students prior to the external evaluation. The purpose of this initiative is to acknowledge that ?schools exist to serve the learner and so one of the more important changes that we are making in this new model is to give voice to both students and their parents? (ibid). A significant emphasis was also placed on ensuring that schools become proactive in carrying out their own internal evaluations. As stated by the Chief Inspector ?at the beginning of the inspection we are asking boards of management for their assessment of the strengths and weaknesses of the school so that we get some sense of the capability of the school to conduct self-evaluation? (ibid). However, this new focus highlighted another issue, namely the alignment of the school inspections with school self-evaluation activities. Prior to 2010 there was no statutory requirement for schools to conduct selfevaluations, although it was assumed that self-evaluations were an intrinsic part of a school?s development planning (which was and is a legislative requirement of all schools in Ireland). However, to ensure that all schools would engage in the self-evaluation process and subsequent action planning for school improvement, Circular Nos. 0040/2012 (DES, 2012a) and 0039/2012 (DES, 2012b) of the Department of Education and Skills (DES) required all schools to conduct self-evaluations during the 2012/2013 academic year. These selfevaluation activities have to be performed in accordance with the inspectorate-devised school self-evaluation guidelines (DES:2012a;2012b) that were published in November 2012. According to the then Minister for Education ?the School Self-Evaluation Guidelines will support schools to evaluate their own work and to set targets to improve teaching and learning. This will help to achieve the targets set out in the Programme for Government and in the National Literacy and Numeracy Strategy, launched by the Minister last year? (Quinn, 2012) . Furthermore, for a number of reasons, such as Ireland?s ?PISA shock? in 2010, Circular Nos. 0040/2012 and 0039/2012 also required school self-evaluations to focus on literacy, numeracy, or an aspect of teaching and learning, and as stated by the DES, ?in subsequent years, schools should select again from the above options so that, within the fouryear period, a School Self-Evaluation report and a three-year school improvement plan for literacy, for numeracy and for one aspect of teaching and learning across all subjects will be completed? (DES,2012a, 3). Other inspection instruments were also introduced during this time of inspection reform. For example, from 2011 onwards, unannounced incidental 1-day inspections were introduced into the Irish education system. According to the DES, ?because it is an unannounced inspection, an incidental inspection can provide a more authentic quality assurance process than inspection models that provide schools with advance notification? (DES,2012c, 2). These surprise inspections were a drastic change as up to this point all schools were always given significant notice as to when an inspection would occur. Another issue with the previous approach to inspection was, that the inspectors were unable to ascertain the extent to which schools implemented the recommendations detailed in previous inspection reports. Although follow-up inspections were performed for schools with very serious weaknesses, it was not until 2012 that follow-up visits, referred to as followthrough inspections were implemented for all schools in the system. According to the DES, ?these inspection visits seek to establish how well the school community has responded to the recommendations made in the previous inspection? (DES, 2013, 6). Finally, as a result of the rapid development of various inspection instruments, coupled with a more efficient use of inspection resources and system-level data, in the 2011/2012 academic year, the inspectorate abandoned its traditional cyclical model of inspections and moved towards a proportionate, risk-based model (Figure 1). As stated by DES (2013): ?Having a range of inspection models available has allowed us to target a proportion of our inspection activity where the risk to students? learning is greatest?. For example, information acquired during short, unannounced inspections can now be used to highlight where further, more intensive inspections are needed... (DES, 2013, 24). We would argue that in these post 2010 developments we can see Ireland moving to an inspection regime which mirrors the general direction of school evaluation accross Europe and shares the key features of what one might suggest is, to an increasing extent, a common model which we have defined as integrated co-professional inspection. Key features of this model include, a range of inspection types usually not cyclical but driven by other factors such as risk assessment and follow up to previous inspection, a set of negotiated standards/criteria for schools to meet, professional dialogue between schools and inspectorates, a balance between accountability and improvement as the primary goal of inspection, emphasis on evidence informed evaluation, and a combination of external inspection and internal self evaluation. The next sections explore the views of the Chief Inspector (CI) of Ireland Dr Harold Hislop with respect to these common features of integrated co-professional inspection. The Changing Contours of Inspection Internationally The (CI) began by outlining what he sees as two rather differing philosophies and approaches to the evaluation of schools accross Europe up to relatively recently: I think you see a tradition which was very influenced by the Scottish experience I think all over Europe which very much combined professional engagement with teachers. One that was very much focused on improvement. It had an accountability function and a public reporting function but it was very much concerned with good self-evaluation, a form of external evaluation that complemented that and one that was less concerned with naming and shaming and control and much more concerned with a professional development and engagement model. You can see that in Scotland and you can see it in some of the eastern European countries that have been shepherded by Scotland in the development of their systems. I think the other group really can be characterised by the Ofsted approach which is very heavily data dependant and I think data based on state or state run examinations or more tests of one type or another. You see elements of that obviously in England and The Netherlands where it?s very strong and very highly developed, Sweden to an extent has adopted elements of it, and has a lot of value in its own way. However, the (CI) goes on to support the theory that these differing approaches may be in the process of coalescing into a model with more in common: Neither model is perfect, you know, there are strengths in both approaches and you can see countries striving to take on the best of both, finding a balance between them. For example you have the Dutch then who have in a sense also been pioneers in a way, equally data driven as the Ofsted people would be but I think the effect, when it?s transacted on the ground is less strongly data driven. They seem to have made huge investments in data, in its analysis and so on like Ofsted but had a different approach on the ground or attempted to have a different approach on the ground, more collaborative and improvement focussed, more supportive of schools. The Accountability/Improvement Spectrum The CI acknowledges that the interplay between inspection as an accountability process and the enabling of school and teacher improvement is a dilemma not just in Ireland but is common to inspection regimes in general. Interestingly he makes it quite clear that the Irish inspectorate prioritise improvement and that is, in his view, the direction in which inspection in other jurisdictions is also going: All inspectorates have this problem which of course depends a lot on current government policy, etc and can chop and change. Basically, we say the primary aim is to improve the schools and if we?re not getting that aim then the other ones just fall by the wayside. If we overemphasise say the accountability function you are likely to impair the improvement function within the schools so that?s where we?ve ended up. We see ourselves as on a continuum with accountability and improvement on either end but we are more towards the improvement one than the accountability and I think this is not unusual in the international context. The CI goes on to give an example of the extent to which the Irish inspectorate are careful not to be overly critical and risk damaging school morale and presumably the relationship between schools and inspectors: As I said there are multi aims but clearly we?ve put an emphasis on the improvement function within the school and we?ve said there is an accountability function of course, there is a public function that we perform as well, much of that through publication of inspection reports. There are things that are not in our inspection reports because we think that would go too much towards the accountability emphasis so there are no standardised test results for instance as you know. Inspection Becoming More Selective, ?Risk Based? and ?Follow Up? The gradual evolution of inspection in Ireland and elsewhere from a repetitive cyclical structure to a more selective model is described by the CI: First of all, like most inspectorates we abandoned the notion of a cycle of inspections of just one type of inspection model. We decided instead that you certainly needed a much greater presence of inspectors in a greater proportion of schools, so that was a key aim for us. We did it through a series of what we would call proportionate models of inspection but they?re not simply proportionate in terms of length or intensity but they also do slightly different jobs. So the unannounced short one day type of inspection has primarily a focus on teaching and learning only in the school. It also has primarily a focus for us on gathering baseline data whereas a subject inspection, which is well in at the post primary level, we?re only developing it at the primary level, is very much a much more intense professional discussion between an expert in the subject and the professional teachers in the subject about the intricacies of teaching mathematics or geography or whatever it is. Equally significant is the determination, again in line with changing international practice, to follow through both with further inspection and other supports (but decreasingly with sanctions) to make sure that improvement is following inspection. The weakness in the single external inspection model is that it becomes a single event, you know it is an event that is done to you which you survive as a school, the end of which you celebrate and you say well we?re not going to see them for another 5,6,7,8 whatever number of years it is. What we?re finding now and what we?re doing you know there?s a percentage of schools that we take for follow through and we do them within a 6-12 month period. That?s enough time for something to have happened and to have seen the effect of it yet. Not so long ago I mean data would show you know after about 18 months to 2 years really if the effect hasn?t been seen then you?re not going to see it. The supportive as opposed to confrontational approach taken to schools requiring further intervention is another hall mark of many, ?integrated co-professional? inspection regimes: We worked with putting a wraparound model in place which meant that the different parts of the department (of education) coordinated to recognise when those problems existed and to make sure that resources could be targeted on those schools. Now we haven?t gotten it perfect but it does mean that during most bi monthly or monthly meetings of a group would sit down here with a caseload of schools and say well where are each of these schools at. External Inspection and School Self-Evaluation Perhaps the most common and striking feature of ?integrated? and ?co-professional inspection? is the role increasingly accorded to school self-evaluation as an element of the inspection process. The CI speaks about this development: The big changes are the increasing frequency with which we?re in schools and the philosophical decision that you need a binary approach, external balanced by an internal. Our initial emphasis was getting the external one right. We did put more emphasis on that to begin with but in the sure and certain knowledge that we would end up with both working parallel with each other. The impact of thinking internationally on the emergence of school self ? evaluation is acknowledged: There was and is lots of international evidence that clearly shows that external inspection of itself won?t work as effectively as it might but when combined with self- evaluation both of them can be together a reasonably powerful force for good. Now you need other things as well, you need things like resources, continuing professional development of teachers to support it and other things as well. I think the other influence that struck us at the time was I think a realisation that many countries came to that you will never ever have enough resources to be in schools with an external inspection model as frequently as you would want to be. If you?re relying on external inspection as the driver of your improvement solely that?s going to be an episodic experience so what you need is a continuous experience. They were a very practical approach that drove us to it. The CI goes to expound a vision of what self- evaluation might be able to achieve and how it can and should interact with external inspection: If you can manage to have a self-evaluation culture within schools that already has placed teachers and school leaders in the space of thinking about that improvement cycle, of questioning practice of looking at how we?re already doing it can change everything. And if you have a parallel process you hope that there?s a good deal of congruence between the two and that post inspection the improvement work continues but has been informed by an external pair of eyes almost like a critical friend would. I think the experience European wide is that the two need to work together. What we?re really interested in making sure of is that they are engaging in a process. The quality of the judgements they make, no we?re not making judgements of the quality of that. Self- evaluation is in too early a stage in the process here in its development for us to be publicly saying the quality of the self- evaluation in the school or the judgements made in the self- evaluation of the school are not aligned with standards or whatever. When you mention trust, there must be and there is a challenge in getting selfevaluation, there?s a trust challenge and there?s a trust challenge in a number of ways. One, that parental communities actually trust the self-evaluation process as being a real one and a beneficial one for the school. That will be a challenge in the long term. There?s a trust on the schools point of view that any external evaluators will not misuse the results of self-evaluation. I mean a lot of systems struggled with that and if you make them, that?s why we didn?t but if you do seek to publish them too much or provide too much access to it they?re not likely to be as honest in the process. Inspection Standards/Frameworks/Criteria A key feature in the role and function of all inspectorates is the development of evaluation criteria not only against which performance will be judged but to provide clarity for schools and stakeholders on priority areas on which to focus improvement. The CI, focuses on the importance of the collaborative, ?co-professional? development and therefore acceptance of these frameworks and notes that this is now common internationally: I?d like to think that we certainly shared the development of the criteria and I think most inspectorates recognise that you need to do this, it cannot be just top down??An external perspective on the quality of the school with a reporting and accountability function to the taxpayer balanced with really good professional conversations about improving practice in schools, that that works best if you have a partnerships development model, partnership is perhaps not the right word for it but it is one which I prefer co-professional, one that involves the stakeholders in the process and that allows them to build the trust in the system. Now, it means it?s uncomfortable, it?s messy, it?s slow I mean this is the disadvantage and it?s not neat and clean. It just doesn?t allow you to sit in a capital and say right, this is the inspection framework. Of course a government could say give me an inspection framework and I could write an inspection framework within a week, of course or you could or any of us could and here are the models and this is how it?s going to happen. But, would it have any real long term impact on what the student is doing in the school. In fact it could have perverse influences on it sometimes. I do think that if a system has a choice it should aim to be on where there is a co-professional working out of the inspection and quality, I do think that. The Role of Data in Inspection In terms of the theme of this paper around the emergence of an integrated inspection model which includes a greater emphasis on data for evaluation purposes than would have been the case in the past for many inspectorates, including Ireland, the CI is very clear: I think the other weakness in the Irish system is the ready accessibility of data for schools, there?s no doubt about that. Other systems are far ahead in data use and have been prioritising data for a long time. We could have waited until you had perfect systems to collect that data and supply the data back to schools like a proper pod or a proper p pod. Both of which are still in their development stages, but we chose not to wait. I mean second level schools have a considerable amount of exam data supplied to the main frame so they have that. The primary schools have the assessment data for literacy and numeracy but they?re at the early stages of learning to work that data and there are limitations in that data and you wouldn?t want that to be the sole data they use or collect and we made that was a pretty strong point to them. In terms of the data that?s collected at school level, the vast bulk is collected through observation by inspectors. Now, one consequence of self- evaluation is that schools will be increasingly creating more data for analysis by themselves. At present we?ve made a decision that that is data, we can contribute to it but it is data that?s owned by the school. If it wants to share it with us that?s fine but we haven?t made that a requirement that they actually give us every inch of paper that they have. That was a deliberate decision take to allow selfevaluation to get properly established and to get it off the ground in schools. We felt that if we inspected or appeared to inspect self- evaluation too quickly it would then become a process for the inspectorate rather than a process for schools. I think the data and the expertise in handling data within the school system is a weakness but that is the same in most countries where data was not taken all that seriously and it will take time to catch up. We know that even expertise in assessment itself is generally weak across the Irish system anyway so you are trying to have a system deal with handling data when the capacity to do so is limited enough. But by getting people starting in self- evaluation there?s lots of good you can get out of self- evaluation without a lot of very sophisticated data. Looking to the Future The final comments of the CI, in terms of future directions again add strength to the notion of an increasing convergence across inspection systems. He outlines similarities and future challenges which, significantly, he perceives to be common across inspection systems: Well I know from conferences, visits, talking to colleagues etc that there is a much closer relationship between what we are all doing now than there used to be. The language, ideas, plans, policy and so on are pretty similar although there and always will be limits to that. A challenge that we all still have to solve is the challenge of getting the right balance between the internal and the external because we know now that we will have to start adjusting external inspection in ways to reflect good internal evaluation when it happens and there?ll be a challenge to build confidence in that sort of an approach. So that you?re much more likely to end up with a graduated approach that says certain schools, that school clearly has a selfevaluation process which would mean that external evaluation is less frequently needed or less intensely needed. It doesn?t obviate the need for it I think at all and there was a fear I mean if you?re asking me about challenges I think there was a fear expressed by some people that introducing self- evaluation meant the end of inspection and funnily enough that wasn?t a fear expressed by inspectors let?s say. I mean some did say that but does this mean there?s an end to external inspection? It was more frequently expressed funnily enough by schools or perhaps not by schools but certainly school leaders who said actually inspection is really important for us and we wouldn?t want to see the end of it. On the external thing I?ve said earlier the need to make sure that external inspection adjusts to complement the growth of the comprehensiveness and the thoroughness of school self- evaluation is going to be a big theme for us over the next number of years and to align properly external and internal inspections so that they genuinely do complement one another. If truth be told most inspection systems are still a strongly external inspection model with a growing school self -evaluation model. Now, we need to make sure we balance that properly. Something else, like data, where some systems like ours are behind is peer to peer evaluation of teachers or principal to teacher evaluations. You can call it, some systems would, teacher appraisal but whatever that is it?s way underdeveloped in the Irish system but I think it is valuable and is a common feature of most inspection. I think another challenge across Europe is the matter of dealing with data, public accessibility of data to get the balance right between a genuine need for public accessibility but one that isn?t damaging to the long term educational interests. The way it seeks to drive behaviours or can inadvertently drive behaviours, needs to be constantly looked at. Conclusion This paper attempted to provide an overview of the emergence of a binary mode of integrated co-professional school inspection which has permeated many education systems where school evaluation exists. Starting with a documentary analysis of the literature on school inspection systems in Europe and elsewhere, the paper began by positing that almost all inspectorates are now following to varying degrees of evaluation maturity, the same overarching trajectory for school accountability and improvement. Next, using Ireland as a case example, the paper provided an overview of the rapid changes that have occurred in school inspection policies and practice whose underlying principles seem to mirror other school inspection systems. To test these assumptions, the paper then provided an analysis of a key informant interview conducted with the Chief inspector of schools in Ireland. The study of policy documents and literature on the changing face of school inspection, coupled with an analysis of the interview data, confirm the view that convergent approaches to school evaluation appear to be visible accross national frontiers. These approaches we have described as integrated in that they contain elements of inspection ideology and methodology previously considered to be very different and even inimical. For example a strong focus on quantitative data is now widely combined with the use of much qualitative data in making judgements about schools. Similarly dialogue for improvement and negotiation about criteria and acceptable standards sits side by side with accounability and compliance with regulation. We have also described the convergent model as coprofessional in that in many instances it allows schools a significant voice through dialogue and self-evaluation, shitfing the balance of evaluation power and making the role of inspectorates more about helping schools to improve quality through robust internal processes. The responses of Chief Inspector Hislop are illustrative of these developments in the Irish context and also convey his belief that developments in Ireland are very much in line with those in other countries. The drivers of these developments vary but include political ideology, which of course can change rapidly, the high cost of extensive external inspection and the perceived need to empower schools through decentralising decision making and yet make democratic accountability an accepted part of everyday school life. Of particular influence in our view has been the work of supranational bodies which has come to favour a compromise, as it were, between heavy handed accounability which can stifle innovation and forms of evaluation systems which are not robust enough to drive improvement. Many of these elements are and will remain in a state of flux. For example the balance between school inspection and school self-evaluation is crucial to the further development of integrated coprofessional inspection and this in turn is dependent on the extent to which school selfevaluation becomes credible both in terms of improvement and accountability. All that can be said for now is that the main features of a common approach to driving school improvement through evaluation and inspection can be found in many education systems. Department of Education and Skills. 2012c. A Guide to Incidental Inspection in Second-Level Schools and Centres for Education, Dublin: Inspectorate of the Department of Education and Skills. Donaldson, G. 2013. The SICI Bratislava memorandum on inspection and innovation.http://www.sici-inspectorates.eu/About-us/Vision-mission/The-BratislavaMemorandum-is-on-the-Website Egan, E. 2010. School improvement through external inspection and school self-evaluation. SDPI Summer School NUI Galway: 22 June 2010. http://www.sdpi.ie/summer_schools_NUIG_2010.html Ehren, M. Altrichter, H., McNamara G. et al. 2013. ?Impact of school inspections on improvement of schools?describing assumptions on causal mechanisms in six European countries.? Educational Assessment, Evaluation and Accountability, 25(1): 343. https://doi.org/10.1007/s11092-012-9156-4 Ehren,M.C.M., Perryman, J. and Shackleton, N. 2015. ?Setting expectations for good education: how Dutch school inspections drive improvement.? School effectiveness and school improvement 26 (2): 296 - 327. https://doi.org/10.1080/09243453.2014.936472 Ehren, M.C.M., and Schildkamp, K. 2012. ?From ?Intuition?- to ?Data?-based Decision Making in Dutch Secondary Schools?? In Data-based Decision Making in Education: Challenges and Opportunities, edited by. K. Schildkamp, M. Lai, and L. Earl, 49-67. Dordrecht: Springer. European Commission. 2004. Evaluation of schools providing compulsory education. Eurydice: Brussels. Eddy-Spicer D, Ehren M, Bangpan M, Khatwa M, Perrone F .2016. Under what conditions do inspection, monitoring and assessment improve system efficiency, service delivery and learning outcomes for the poorest and most marginalised? A realist synthesis of school accountability in low- and middle-income countries. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London. Gilchrist V. 1993. ?Key informant interviews.? In Doing Qualitative Research, edited by Crabtree, B. and W. Miller, 359-381. Newbury Park: Sage Publications. Government of Ireland. 1998. Education Act. Dublin: Stationery Office. Hislop, H. 2010. Parents, school evaluation and school improvement. National Parents Council?Post-Primary 25th Anniversary Conference, 1 May 2010. http://www.npcpp.ie/fileadmin/files/conference2010/NPCpp_Conference_2010__Harold_Hislop_Address.pdf Leeuw, F.L., 2003. ?Reconstructing program theories: Methods available and problems to be solved.? American journal of evaluation, 24(1): 5-20. https://doi.org/10.1177/109821400302400102 McNamara, G. and O?Hara, J. 2012. ?From looking at our schools (LAOS) to whole school evaluation-management, leadership and learning (WSE-MLL): the evolution of inspection in irish schools over the past decade.? Educational Assessment, Evaluation and Accountability 24 (2): 79-97. https://doi.org/10.1007/s11092-012-9143-9 McNamara, G. and O Hara, J. 2008. ?Contexts and Constraints: An Analysis of the Evolution of Evalaution in Ireland with particular reference to the Education System.? Journal of multidisciplinary evaluaton, (4) 75-83 McNamara, G. and O'Hara, J. 2006. ?Workable Compromise or Pointless Exercise? School Based Evaluation in the Irish Context?. Educational management administration and leadership 34 (4): 564-582. https://doi.org/10.1177/1741143206068218 Akenson , D. H. 1970 . The Irish education experiment: The National System of Education in Ireland in the nineteenth century . London: Routledge. Altrichter , H. , Ehren , M.C.M. , McNamara , G. and O'Hara , J. 2014 . ? The impact of school inspection? Analysis of assumptions about mechanisms of school inspection in six European countries ? In Schule - Qualit?t und Entwicklung, edited by M. Pfeifer, 184 - 207 . M?nster: Waxmann. Brown , M. 2013 . Deconstructing Evaluation in Education: The case of Ireland . PhD Thesis , Dublin City University, Ireland. Brown , M. , McNamara , G. , and O Hara , J ( 2016a ) ? Teacher Accountability in Education - The Irish Experiment.? In Essays in the History of Irish Education, edited by B .Walsh, 359 - 381 .United Kingdom: Palgrave Macmillan. https://doi.org/10.1057/978-1- 137 -51482-0_ 14 Brown , M. , McNamara , G. , O 'Hara , J. ( 2016b ) ? Quality and the rise of value-added in education: The case of Ireland? . Policy Futures in Education , 14 ( 6 ): 810 - 829 . https://doi.org/10.1177%2F1478210316656506 Brown , M. , McNamara , G. & O Hara , J. 2013 . Nijesh Solk Mwil Start Gyamyath (Save us from the time of trial) : From Austerity to Prosperity and Back Again . [Conference Organising Committee Chairperson], Educational Policy in Changing Times: Consultation, Implementation and Impact - 38th Annual Conference of the Educational Studies Association of Ireland (ESAI) , University of Limerick, 21 - MAR13 - 23 - MAR-13. Brown , M. , McNamara , G. , O 'Hara , J. and O'Brien , S. 2016 . ?Exploring the changing face of school inspections? . Eurasian journal of educational research , 16 ( 66 ): 1 - 35 . http://dx.doi.org/10.14689/ejer. 2016 . 66 .1 Department of Education and Skills . 2013. Chief Inspector's Report: 2010 - 2012 .http://www.education.ie/en/Publications/Inspection-ReportsPublications/ Evaluation-Reports- Guidelines/ Chief-Inspector% E2 % 80 % 99s-Report2010- 2012 -Main-Report.pdf Department of Education and Skills Republic of Ireland. 2012a. Circular 0040 / 2012 , Implementation of school self-evaluation- Post Primary . http://www.education.ie/en/Circulars-and - Forms /Active-Circulars/ cl0040 _ 2012 . pdf [Accessed 12 March 2015 ]. Department of Education and Skills. 2012b. Circular 0039 / 2012 , Implementation of school self-evaluation- Primary . http://www.education.ie/en/Circulars-and-Forms/ActiveCirculars/cl0039_ 2012 .pdf McNamara , G. and O'Hara , J. 2005 . ? Internal Review and Self-Evaluation - The Chosen Route to School Improvement in Ireland?? . Studies in Educational Evaluation, ( 31 ) 267 - 282 . https://doi.org/10.1016/j.stueduc. 2005 . 11 .003 McNamara , G. and O'Hara , J. 2002 . ? Whole School Evaluation and School Development Planning; an Analysis of recent Initiatives in Ireland? . Educational management administration and leadership ( 30 ): 201 - 212 . https://doi.org/10.1177/02611X02030002510 MacBeath , J. 2006 . School inspection and self-evaluation: working with the new relationship . London: Taylor and Francis. https://doi.org/10.4324/9780203967102 MacBeath , J. 1999 . Schools must speak for themselves: the case for school self evaluation . London: Routledge. Nelson , R. , and Ehren , M.C.M. 2014 . Review and synthesis of evidence on the (mechanisms of) impact of school inspections . http://schoolinspections.eu/wp-content/uploads/downloads/2014/02/Review-and -synthesisof-evidence-on-the-mechanisms-of-impact-of-school-inspections .pdf Nevo , D. 2010 . ?Internal and external evaluation.? In International Encyclopaedia of Education (3rd edition) , edited by Peterson, P. , Baker , E. and B. McGaw , 781 - 785 . London: Elsevier. https://doi.org/10.1016/B978-0 -08-044894-7 . 01597 - 9 Organisation for Economic Co-operation and Development - OECD . 2013 . Synergies for better learning: An international perspective on evaluation and assessment . Paris: OECD. Perryman , J. 2009 . ?Inspection and the fabrication of professional and performative processes . ? Journal of Education Policy , 24 ( 5 ): 611 - 631 . https://doi.org/10.1080/02680930903125129 Quinn , R. 2012 . Irish Government News Service: Minister Ruair? Quinn launches Guidelines on School Self-evaluation [press release], 19 November . Available from: http://www.merrionstreet.ie/en/category-index/education/schools/minister-ruairiquinn -launches-guidelines-on-school-self-evaluation . html [Accessed November 2016 ] Ryan , K. and Timmer , J. 2013 . ? New Zealand style school review: A view from outside.? In A developmental and negotiated approach to school and curriculum evaluation, edited by Lai , M. and S. Kushner , 195 - 206 . Bindley: Emerald Publishers https://doi.org/10.1108/S1474-7863( 2013 ) 0000014012 Scheerens , J. , and Ehren , M.C.M. 2015 . ? The evidence base for school inspection frameworks . ? Journal of Educational, Cultural and Psychological Studies , ( 12 ): 43 - 76 . SICI. 2009 . Inspectorate profile , Republic of Ireland . Brussels: Standing International Conference of Inspectorates. Van Bruggen , J.C. 2010 . ? The role of school inspection in ensuring quality in education: past, present and future .? In Beyond Lisbon 2010: Perspectives from research and development for education policy in Europe (CIDREE Yearbook 2010 ), edited by S.Stoney, 85 - 119 . Slough: National Foundation for Educational Research.


This is a preview of a remote PDF: https://ro.ecu.edu.au/cgi/viewcontent.cgi?article=4163&context=ajte

Martin Brown, Gerry McNamara, Joe Ohara, Shivaun O'Brien, Jerich Faddar, Cornelius Young. Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers, Australian Journal of Teacher Education, 2018,