Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers
Integrated Co-Professional Evaluation? Converging Approaches to School Evaluation Across Frontiers
Martin Brown 0
Shivaun O'Brien 0
0 EQI, School of Policy and Practice, DCU Institute of Education
Integrated co-Professional Evaluation?
Converging Approaches to School Evaluation across Frontiers
DCU Institute of Education, Ireland
Universiteit Antwerpen, Belgium
Abstract: This paper posits that almost all inspectorates are now
following, if to varying degrees, a similar overarching ideology and
methodology for school accountability and improvement. The first
part of the paper provides an analysis of recent changes to school
inspection policies across Frontiers. Using Ireland as a case example,
the next part of the paper provides an analysis of Irish school
inspection policies and practices that appear to mirror other school
inspection systems. To test these assumptions, the paper then provides
an analysis of a key informant interview with the Chief Inspector of
schools in Ireland.
The evidence suggests that there is an increasingly convergent
approach to school evaluation discernible across all inspection
frontiers. Among the many aspects of this changing landscape is the
rapid pace at which schools have accepted school inspection
frameworks and the emergence of a genuinely co-professional as
opposed to co-existent mode of evaluation between the inspectorate
During the last two decades, many European education systems have attempted to
modernize the control and monitoring of schools by establishing some form of what is often
described as evidence-based governance
(Altrichter et al. 2016, Schildkamp, and Ehren,
. Such regimes are usually built on two dominant arrangements which often exist side
by side, achievement standards and tests of student performance on the one hand, and school
inspections on the other (Ehren et al., 2016). Both of these features are evident in Ireland
with increasing emphasis on standardized tests and achievement in state examinations
together with what might be described as the reconceptualization of school evaluation to
include both inspection and school self-evaluation
(Brown, McNamara, O?Hara and O?Brien,
. Testing is for another day. The paper at hand aims to describe the main features of
recent changes in inspections both internationally and in Ireland based, firstly, on an analysis
of the extensive literature on school evaluation including policy documents and subsequently
on an interview with the Chief Inspector of Irish schools.
The paper commences by suggesting that there is evidence of some convergence
between the very widely differing school inspection approaches to be found across Europe
and by identifying some of the common themes or features of this shared notion of
inspection. Ireland is then offered as a good illustrative example of the way in which school
inspection appears to be evolving internationally. This is done through an analysis of policy
documents governing inspection and by presenting the thoughts of the Chief Inspector of
Irish schools Dr Harold Hislop. The paper concludes with some reflections on changing
school inspection modalities both in Ireland and internationally.
The methodology used in this study is an evidence-based mixed methods analysis of
school evaluation policies and practices. Following the ?policy-scientific? approach (Leeuw,
2003), the first phase of the study deconstructs the literature relating to the changing
conception of school inspection. Using Ireland as a case study example, the next step
provides an analysis of literature relating to the extent to which this integrated mode of
school evaluation has permeated Ireland?s education system. Finally, the latter part of the
paper, further examines these global and local trends, by providing a key informant interview
(Gilchrist, 1993) with the Chief Inspector of Schools in Ireland.
A Converging Model of School Inspection?
In a climate of globalization and international comparisons, evaluation and
accountability have become key educational issues in most developed countries
Commission, 2004; OECD, 2013)
. As a consequence, many education systems have
introduced varieties of evidence-based governance regimes. School inspections are one of the
major incarnations of this governance concept. In line with the logic of evidence-based
governance (see Ehren et al., 2013, 4) inspectorates of education (1) set expectations by
virtue of their inspection standards and procedures. They (2) collect evidence during
inspection visits and use information produced by other evaluation instruments, such as
student testing, to assess the quality of education and hold schools accountable for a broad
range of goals related to student achievement, teaching, organization, and leadership. They
(3) aim to stimulate school and system improvement, sometimes by the deployment of
sanctions, but more usually by producing reports which point to strengths and weaknesses of
individual schools. These reports also include or imply recommendations for action to be
undertaken by the inspected schools or the authorities in charge of them
. Finally, they (4), to lesser or greater extent, require or encourage schools to
engage in school self ? evaluation both to complement external inspection and to provide an
evidential base for school improvement
(Brown, McNamara & O?Hara, 2016a; Brown,
McNamara O?Hara, & O?Brien, 2016b; Brown, 2013)
However not everything which was labelled ?inspection? operated in identical ways.
According to Donaldson (2013), ?inspection is often associated in the public mind with a
rather narrow set of activities which involve notions of compliance and audit. In fact, it is a
very plastic concept which takes and has taken many forms, and which can serve many
different purposes? (8). For example, some of the European Inspectorates of Education (e.g.
HMI in England and the Irish Inspectorate of Education), have existed since the mid
nineteenth century, have always been an arm of central government and oversaw both
compliance with nationally mandated policies and mainly judged the performance of
individual teachers rather than schools as organizations per se
(Brown, McNamara, O?Hara,
. In contrast the traditional inspection role in centralist-bureaucratic states (such as
Austria or the Czech Republic) was based on an intermediate position in the hierarchical line
located above individual schools but beneath the central administration. Several supervisory
functions for schools were amalgamated in this role. Heads of schools were directly
accountable to these inspectors who were also responsible both for distributing staff to
individual schools and for inspecting and controlling the ?quality? of the schools in their
region. Criteria for assessing this quality may have varied in time but always included some
mixture between educational effectiveness on the one hand and legal and administrative
compliance on the other
(Altrichter et al. 2014)
In recent decades these historical differences came to be substantially subsumed into
divergent inspection policies based more on ideological stance as opposed to tradition.
Scholars of school inspection identified two fairly distinct camps, as it were, with very
different concepts of inspection. On the one hand, certain inspectorates, notably OFSTED in
England, other parts of the English speaking world such as New Zealand, and some European
countries including the Netherlands and to an extent Sweden were characterized as being
driven by reliance on hard data and concerned primarily with monitoring and accountability
(see for example, Nelson and Ehren, 2014; Ryan and Timmer, 2013; Perryman, 2009)
contrast an alternative model, associated with Scotland and the seminal work of John
MacBeath, and adopted widely in Central and Eastern Europe was perceived to be more
focussed on a variety of data sources, interested in school self-evaluation and more concerned
with collaborative improvement than accountability
(MacBeath ,1999, 2006; Nevo, 2010)
While significant disparities still remain, in recent years a noticeable convergence
between these different traditions, ideologies and approaches has become perceptible. Due to
a range of influences including membership of international bodies such as the EU, the
OECD and SICI, the Standing International Conference of Inspectorates, globalization and
international comparisons of education systems such as PISA and the drive for reforms such
as greater accountability and decentralization of decision making in public services, there has
been a significant coming together of school inspection theory and practice
. This point is further illustrated by the remarkable similarities across
countries displayed in the OECD?s Synergies for Better Learning report (2013), and the types
of school inspection found among SICI member countries. As
van Bruggen (2010
?the content of the various frameworks in use by 18 inspectorates in Europe does not differ
So marked has this process been that it is possible we suggest to speak of
coprofessional integrated inspection systems with common features across much of Europe. We
propose the term co-professional to indicate a high level of professional co-operation and
respect between inspectorates and inspectors on the one hand and schools and teachers on the
other. This is operationalised through, for example, negotiated and agreed standards and
frameworks and acceptance of school self-evaluation as a valid element of inspection. We
propose the term integrated in the sense of integrating ideologies and approaches which
previously tended to be associated with very differing conceptualizations of inspection. The
most notable of these would be a more widely accepted focus on the importance of data
including student achievement data in evaluating schools and the increasingly central role
accorded to school self- evaluation as a component of the evaluation process.
McNamara, O?Hara and O?Brien, 2016, McNamara and O?Hara, 2012; Ehren, Perryman, &
These ?co- professional integrated? inspection systems differ from their predecessors
in that they,
? tend to concentrate more on evaluative functions and place the focus on evidence and
less on administrative/supervisory/compliance functions;
? aim to professionalize these evaluative functions by formalizing them through
developing sets of standards/criteria/ indicators and by the use of instruments taken
from social science to collect data both quantitative and qualitative;
? utilise a range of inspection treatments driven by risk assessments based on data or
follow up to previous inspections rather than the more traditional cyclical model;
? are concerned with both accountability/public reporting and school improvement but
with a greater emphasis on the latter;
? focus on the performance of schools as organizations as much, or perhaps more, than
on the work of individual teachers;
? perceive collaboration and co-professional dialogue between inspectors and
schools/teachers as fundamental;
? look to involve stakeholders including parents and students in the inspection process;
? encourage schools to self-evaluate in a formal way as part of the inspection process.
By virtue of these features ?co- professional integrated inspection? fits well in to the
framework of ?rationalized control? propagated by the proponents of evidence-based
governance. In this conceptualization inspectors are not simply bureaucrats who are
supervising schools according to old-fashioned and easily controllable administrative criteria;
rather they aspire to form a new breed of professionals in close contact with schools and
teachers and with the latest developments in educational research. Ireland provides a very
interesting example of the growth of what one might begin to describe as an increasingly
typical inspection model and this paper hopes to clarify both what this looks like and the
thinking driving the emerging policy and processes.
School Inspection in Ireland
With the introduction of a centrally administered education system in Ireland in the
1830s, in a letter to the Duke of Leinster, the Chief Secretary, Lord Stanley, proposed a
structure for the ?superintendence? of education that would be overseen by a board of
commissioners. According to what is frequently referred to as the ?Stanley Letter? (1831),
within this centralized administration, exchequer funding was to be used for six particular
purposes, such as ?granting aid for the erection of schools? (Stanley, 1831, 4). The second of
these purposes resulted in the commencement of school inspection in Ireland, by allowing
exchequer funding to be used for the purpose of ?paying Inspectors for visiting and reporting
upon schools? (ibid, p.4). Although the specific duties of the inspectorate consisted of various
legal and administrative activities,
notes that ?as far as routine inspection
visits were concerned, the inspectors? tasks were just what one would expect? (146). For
example, inspectors were required to ?observe the teacher conducting the class and suggest
any improvements that might occur to him? (ibid).
Inspection continued along these lines until the foundation of an independent Irish
state in 1922
(Brown, McNamara and O?Hara, 2016)
. After this date a strong focus of
inspection became the implementation of policies around the restoration of the Irish language
to be accomplished largely through the school system. The figure of the ?cigire? (Irish for
inspector) became a respected, if rather feared, part of the educational landscape. The visit of
the ?cigire? was a big event for schools, teachers and indeed pupils. However in the final
decades of the twentieth century the role of inspection greatly diminished, partly due to the
power of the teacher unions but also because much of the resourcing of the Inspectorate was
directed to other tasks particularly the management of the annual state examinations.
Interestingly, it is suggested by scholars of inspection in Ireland that the impetus for the
revival of inspection in the 1990?s came not from domestic political demands but largely
from international developments and the influence of the OECD.
(See: McNamara and
O?Hara., 2008, 2002)
In the Education Act (Government of Ireland, 1998) a statutory framework relating to
the duties of school inspectors came into force. As stated in Section 13(3), ?the functions of
an Inspector shall be: to support and advise recognised schools, centres for education and
teachers on matters relating to the provision of education??. The Act is clear that a dual
system of internal and external school evaluations was envisioned, resulting in the
development of a multi-mode inspection framework and concomitant government-devised
school self-evaluation instruments. Thus, it was in this period that the forerunners of ?new?
forms of school inspections were put into place.
Until 2010, school inspection was typically conducted using four evaluation
instruments, namely thematic, subject, whole school evaluation (WSE), and at Post-Primary
level, programme inspections. Although the frequency of inspections varied depending on the
evaluation instrument used, inspection schedules were infrequent and centred on a cyclical
mode of inspection (See: Ehren et al., 2013). According to the Standing International
Conference of Inspectorates (SICI) profile of the Irish inspectorate in 2009, ?at present,
primary level inspections are carried out approximately once every nine to ten years ? At
Post-Primary level, schools may expect a Whole-School Evaluation with a similar frequency?
(SICI, 2009, 14)
. However, as subject, thematic, and programme inspections were conducted
along with whole school evaluations (WSEs), the inspections were generally more frequent.
Recent Changes in Inspections in Ireland
Since 2010 Ireland has experienced a profound restructuring of its school inspection
process, including significant changes to its schedule for inspections and the types of
inspection instruments that are used. As previously stated until this point school inspections
had typically used four evaluation types, and for the most part, internal planning activities
had primarily consisted of developing school policies and plans. Moreover, there was little
evidence to suggest that schools were collecting and analysing the data necessary to develop
and implement actions plans for improvement
(See: McNamara and O?Hara, 2008, 2006,
. In fact, akin to other European countries, school self-evaluations were rarely
mentioned, if at all, in the majority of school inspection reports.
However during the 2009/2010 academic year the inspectorate began to experiment
with new modes of inspection, including the introduction of a codicil version of WSE
referred to as WSE-MLL (Whole School Evaluation ? Management, Leadership and
Learning) . Significant changes compared to the WSE model are evident and primarily relate
a greater emphasis being placed on internal evaluation where the ?onus is placed on
schools to engage in the self-evaluation process? (Egan, 2010, 53)
(2) the structure and composition of the final WSE report, and (3) the inclusion of
parent/student opinions in the form of questionnaires.
According to the Chief Inspector WSE-MLL ?will provide a shorter and more focused
report on the school? with less time on school planning and even more time in classrooms?
(Hislop, 2010, 20). The most notable change in this new model of the WSE was the
introduction of confidential, anonymous questionnaires that are given to parents and students
prior to the external evaluation. The purpose of this initiative is to acknowledge that ?schools
exist to serve the learner and so one of the more important changes that we are making in this
new model is to give voice to both students and their parents? (ibid).
A significant emphasis was also placed on ensuring that schools become proactive in
carrying out their own internal evaluations. As stated by the Chief Inspector ?at the beginning
of the inspection we are asking boards of management for their assessment of the strengths
and weaknesses of the school so that we get some sense of the capability of the school to
conduct self-evaluation? (ibid). However, this new focus highlighted another issue, namely
the alignment of the school inspections with school self-evaluation activities.
Prior to 2010 there was no statutory requirement for schools to conduct
selfevaluations, although it was assumed that self-evaluations were an intrinsic part of a school?s
development planning (which was and is a legislative requirement of all schools in Ireland).
However, to ensure that all schools would engage in the self-evaluation process and
subsequent action planning for school improvement, Circular Nos. 0040/2012 (DES, 2012a)
and 0039/2012 (DES, 2012b) of the Department of Education and Skills (DES) required all
schools to conduct self-evaluations during the 2012/2013 academic year. These
selfevaluation activities have to be performed in accordance with the inspectorate-devised school
self-evaluation guidelines (DES:2012a;2012b) that were published in November 2012.
According to the then Minister for Education ?the School Self-Evaluation Guidelines will
support schools to evaluate their own work and to set targets to improve teaching and
learning. This will help to achieve the targets set out in the Programme for Government and
in the National Literacy and Numeracy Strategy, launched by the Minister last year?
Furthermore, for a number of reasons, such as Ireland?s ?PISA shock? in 2010,
Circular Nos. 0040/2012 and 0039/2012 also required school self-evaluations to focus on
literacy, numeracy, or an aspect of teaching and learning, and as stated by the DES, ?in
subsequent years, schools should select again from the above options so that, within the
fouryear period, a School Self-Evaluation report and a three-year school improvement plan for
literacy, for numeracy and for one aspect of teaching and learning across all subjects will be
completed? (DES,2012a, 3).
Other inspection instruments were also introduced during this time of inspection
reform. For example, from 2011 onwards, unannounced incidental 1-day inspections were
introduced into the Irish education system. According to the DES, ?because it is an
unannounced inspection, an incidental inspection can provide a more authentic quality
assurance process than inspection models that provide schools with advance notification?
(DES,2012c, 2). These surprise inspections were a drastic change as up to this point all
schools were always given significant notice as to when an inspection would occur.
Another issue with the previous approach to inspection was, that the inspectors were
unable to ascertain the extent to which schools implemented the recommendations detailed in
previous inspection reports. Although follow-up inspections were performed for schools with
very serious weaknesses, it was not until 2012 that follow-up visits, referred to as
followthrough inspections were implemented for all schools in the system. According to the DES,
?these inspection visits seek to establish how well the school community has responded to the
recommendations made in the previous inspection? (DES, 2013, 6).
Finally, as a result of the rapid development of various inspection instruments,
coupled with a more efficient use of inspection resources and system-level data, in the
2011/2012 academic year, the inspectorate abandoned its traditional cyclical model of
inspections and moved towards a proportionate, risk-based model (Figure 1). As stated by
DES (2013): ?Having a range of inspection models available has allowed us to target a
proportion of our inspection activity where the risk to students? learning is greatest?. For
example, information acquired during short, unannounced inspections can now be used to
highlight where further, more intensive inspections are needed... (DES, 2013, 24).
We would argue that in these post 2010 developments we can see Ireland moving to
an inspection regime which mirrors the general direction of school evaluation accross Europe
and shares the key features of what one might suggest is, to an increasing extent, a common
model which we have defined as integrated co-professional inspection. Key features of this
model include, a range of inspection types usually not cyclical but driven by other factors
such as risk assessment and follow up to previous inspection, a set of negotiated
standards/criteria for schools to meet, professional dialogue between schools and
inspectorates, a balance between accountability and improvement as the primary goal of
inspection, emphasis on evidence informed evaluation, and a combination of external
inspection and internal self evaluation. The next sections explore the views of the Chief
Inspector (CI) of Ireland Dr Harold Hislop with respect to these common features of
integrated co-professional inspection.
The Changing Contours of Inspection Internationally
The (CI) began by outlining what he sees as two rather differing philosophies and
approaches to the evaluation of schools accross Europe up to relatively recently:
I think you see a tradition which was very influenced by the Scottish experience I
think all over Europe which very much combined professional engagement with
teachers. One that was very much focused on improvement. It had an
accountability function and a public reporting function but it was very much
concerned with good self-evaluation, a form of external evaluation that
complemented that and one that was less concerned with naming and shaming
and control and much more concerned with a professional development and
engagement model. You can see that in Scotland and you can see it in some of
the eastern European countries that have been shepherded by Scotland in the
development of their systems. I think the other group really can be
characterised by the Ofsted approach which is very heavily data dependant and
I think data based on state or state run examinations or more tests of one type or
another. You see elements of that obviously in England and The Netherlands
where it?s very strong and very highly developed, Sweden to an extent has
adopted elements of it, and has a lot of value in its own way.
However, the (CI) goes on to support the theory that these differing approaches
may be in the process of coalescing into a model with more in common:
Neither model is perfect, you know, there are strengths in both approaches and
you can see countries striving to take on the best of both, finding a balance
between them. For example you have the Dutch then who have in a sense also
been pioneers in a way, equally data driven as the Ofsted people would be but I
think the effect, when it?s transacted on the ground is less strongly data driven.
They seem to have made huge investments in data, in its analysis and so on like
Ofsted but had a different approach on the ground or attempted to have a
different approach on the ground, more collaborative and improvement
focussed, more supportive of schools.
The Accountability/Improvement Spectrum
The CI acknowledges that the interplay between inspection as an accountability
process and the enabling of school and teacher improvement is a dilemma not just in Ireland
but is common to inspection regimes in general. Interestingly he makes it quite clear that the
Irish inspectorate prioritise improvement and that is, in his view, the direction in which
inspection in other jurisdictions is also going:
All inspectorates have this problem which of course depends a lot on current
government policy, etc and can chop and change. Basically, we say the primary
aim is to improve the schools and if we?re not getting that aim then the other
ones just fall by the wayside. If we overemphasise say the accountability function
you are likely to impair the improvement function within the schools so that?s
where we?ve ended up. We see ourselves as on a continuum with accountability
and improvement on either end but we are more towards the improvement one
than the accountability and I think this is not unusual in the international
The CI goes on to give an example of the extent to which the Irish inspectorate are
careful not to be overly critical and risk damaging school morale and presumably the
relationship between schools and inspectors:
As I said there are multi aims but clearly we?ve put an emphasis on the
improvement function within the school and we?ve said there is an
accountability function of course, there is a public function that we perform as
well, much of that through publication of inspection reports. There are things
that are not in our inspection reports because we think that would go too much
towards the accountability emphasis so there are no standardised test results for
instance as you know.
Inspection Becoming More Selective, ?Risk Based? and ?Follow Up?
The gradual evolution of inspection in Ireland and elsewhere from a repetitive cyclical
structure to a more selective model is described by the CI:
First of all, like most inspectorates we abandoned the notion of a cycle of inspections
of just one type of inspection model. We decided instead that you certainly needed a
much greater presence of inspectors in a greater proportion of schools, so that was a
key aim for us. We did it through a series of what we would call proportionate models
of inspection but they?re not simply proportionate in terms of length or intensity but
they also do slightly different jobs. So the unannounced short one day type of
inspection has primarily a focus on teaching and learning only in the school. It also
has primarily a focus for us on gathering baseline data whereas a subject inspection,
which is well in at the post primary level, we?re only developing it at the primary
level, is very much a much more intense professional discussion between an expert in
the subject and the professional teachers in the subject about the intricacies of
teaching mathematics or geography or whatever it is.
Equally significant is the determination, again in line with changing international
practice, to follow through both with further inspection and other supports (but decreasingly
with sanctions) to make sure that improvement is following inspection.
The weakness in the single external inspection model is that it becomes a single
event, you know it is an event that is done to you which you survive as a school,
the end of which you celebrate and you say well we?re not going to see them for
another 5,6,7,8 whatever number of years it is. What we?re finding now and
what we?re doing you know there?s a percentage of schools that we take for
follow through and we do them within a 6-12 month period. That?s enough time
for something to have happened and to have seen the effect of it yet. Not so long
ago I mean data would show you know after about 18 months to 2 years really if
the effect hasn?t been seen then you?re not going to see it.
The supportive as opposed to confrontational approach taken to schools requiring
further intervention is another hall mark of many, ?integrated co-professional? inspection
We worked with putting a wraparound model in place which meant that the
different parts of the department (of education) coordinated to recognise when
those problems existed and to make sure that resources could be targeted on
those schools. Now we haven?t gotten it perfect but it does mean that during
most bi monthly or monthly meetings of a group would sit down here with a
caseload of schools and say well where are each of these schools at.
External Inspection and School Self-Evaluation
Perhaps the most common and striking feature of ?integrated? and ?co-professional
inspection? is the role increasingly accorded to school self-evaluation as an element of the
inspection process. The CI speaks about this development:
The big changes are the increasing frequency with which we?re in schools and
the philosophical decision that you need a binary approach, external balanced
by an internal. Our initial emphasis was getting the external one right. We did
put more emphasis on that to begin with but in the sure and certain knowledge
that we would end up with both working parallel with each other.
The impact of thinking internationally on the emergence of school self ? evaluation is
There was and is lots of international evidence that clearly shows that external
inspection of itself won?t work as effectively as it might but when combined with
self- evaluation both of them can be together a reasonably powerful force for
good. Now you need other things as well, you need things like resources,
continuing professional development of teachers to support it and other things
as well. I think the other influence that struck us at the time was I think a
realisation that many countries came to that you will never ever have enough
resources to be in schools with an external inspection model as frequently as you
would want to be. If you?re relying on external inspection as the driver of your
improvement solely that?s going to be an episodic experience so what you need
is a continuous experience. They were a very practical approach that drove us to
The CI goes to expound a vision of what self- evaluation might be able to achieve and
how it can and should interact with external inspection:
If you can manage to have a self-evaluation culture within schools that already
has placed teachers and school leaders in the space of thinking about that
improvement cycle, of questioning practice of looking at how we?re already
doing it can change everything. And if you have a parallel process you hope that
there?s a good deal of congruence between the two and that post inspection the
improvement work continues but has been informed by an external pair of eyes
almost like a critical friend would. I think the experience European wide is that
the two need to work together.
What we?re really interested in making sure of is that they are engaging in a
process. The quality of the judgements they make, no we?re not making
judgements of the quality of that. Self- evaluation is in too early a stage in the
process here in its development for us to be publicly saying the quality of the
self- evaluation in the school or the judgements made in the self- evaluation of
the school are not aligned with standards or whatever.
When you mention trust, there must be and there is a challenge in getting
selfevaluation, there?s a trust challenge and there?s a trust challenge in a number of
ways. One, that parental communities actually trust the self-evaluation process
as being a real one and a beneficial one for the school. That will be a challenge
in the long term. There?s a trust on the schools point of view that any external
evaluators will not misuse the results of self-evaluation. I mean a lot of systems
struggled with that and if you make them, that?s why we didn?t but if you do seek
to publish them too much or provide too much access to it they?re not likely to be
as honest in the process.
A key feature in the role and function of all inspectorates is the development of
evaluation criteria not only against which performance will be judged but to provide clarity
for schools and stakeholders on priority areas on which to focus improvement. The CI,
focuses on the importance of the collaborative, ?co-professional? development and therefore
acceptance of these frameworks and notes that this is now common internationally:
I?d like to think that we certainly shared the development of the criteria and I
think most inspectorates recognise that you need to do this, it cannot be just top
down??An external perspective on the quality of the school with a reporting
and accountability function to the taxpayer balanced with really good
professional conversations about improving practice in schools, that that works
best if you have a partnerships development model, partnership is perhaps not the
right word for it but it is one which I prefer co-professional, one that involves the
stakeholders in the process and that allows them to build the trust in the system.
Now, it means it?s uncomfortable, it?s messy, it?s slow I mean this is the
disadvantage and it?s not neat and clean. It just doesn?t allow you to sit in a
capital and say right, this is the inspection framework. Of course a government
could say give me an inspection framework and I could write an inspection
framework within a week, of course or you could or any of us could and here are
the models and this is how it?s going to happen. But, would it have any real long
term impact on what the student is doing in the school. In fact it could have
perverse influences on it sometimes. I do think that if a system has a choice it
should aim to be on where there is a co-professional working out of the
inspection and quality, I do think that.
The Role of Data in Inspection
In terms of the theme of this paper around the emergence of an integrated inspection
model which includes a greater emphasis on data for evaluation purposes than would have
been the case in the past for many inspectorates, including Ireland, the CI is very clear:
I think the other weakness in the Irish system is the ready accessibility of data
for schools, there?s no doubt about that. Other systems are far ahead in data use
and have been prioritising data for a long time. We could have waited until you
had perfect systems to collect that data and supply the data back to schools like
a proper pod or a proper p pod. Both of which are still in their development
stages, but we chose not to wait. I mean second level schools have a
considerable amount of exam data supplied to the main frame so they have that.
The primary schools have the assessment data for literacy and numeracy but
they?re at the early stages of learning to work that data and there are limitations
in that data and you wouldn?t want that to be the sole data they use or collect
and we made that was a pretty strong point to them.
In terms of the data that?s collected at school level, the vast bulk is collected
through observation by inspectors. Now, one consequence of self- evaluation is
that schools will be increasingly creating more data for analysis by themselves.
At present we?ve made a decision that that is data, we can contribute to it but it
is data that?s owned by the school. If it wants to share it with us that?s fine but
we haven?t made that a requirement that they actually give us every inch of
paper that they have. That was a deliberate decision take to allow
selfevaluation to get properly established and to get it off the ground in schools. We
felt that if we inspected or appeared to inspect self- evaluation too quickly it
would then become a process for the inspectorate rather than a process for
I think the data and the expertise in handling data within the school system is a
weakness but that is the same in most countries where data was not taken all
that seriously and it will take time to catch up. We know that even expertise in
assessment itself is generally weak across the Irish system anyway so you are
trying to have a system deal with handling data when the capacity to do so is
limited enough. But by getting people starting in self- evaluation there?s lots of
good you can get out of self- evaluation without a lot of very sophisticated data.
Looking to the Future
The final comments of the CI, in terms of future directions again add strength to the
notion of an increasing convergence across inspection systems. He outlines similarities and
future challenges which, significantly, he perceives to be common across inspection systems:
Well I know from conferences, visits, talking to colleagues etc that there is a
much closer relationship between what we are all doing now than there used to
be. The language, ideas, plans, policy and so on are pretty similar although
there and always will be limits to that.
A challenge that we all still have to solve is the challenge of getting the right
balance between the internal and the external because we know now that we will
have to start adjusting external inspection in ways to reflect good internal
evaluation when it happens and there?ll be a challenge to build confidence in
that sort of an approach. So that you?re much more likely to end up with a
graduated approach that says certain schools, that school clearly has a
selfevaluation process which would mean that external evaluation is less frequently
needed or less intensely needed. It doesn?t obviate the need for it I think at all
and there was a fear I mean if you?re asking me about challenges I think there
was a fear expressed by some people that introducing self- evaluation meant the
end of inspection and funnily enough that wasn?t a fear expressed by inspectors
let?s say. I mean some did say that but does this mean there?s an end to external
inspection? It was more frequently expressed funnily enough by schools or
perhaps not by schools but certainly school leaders who said actually inspection
is really important for us and we wouldn?t want to see the end of it.
On the external thing I?ve said earlier the need to make sure that external
inspection adjusts to complement the growth of the comprehensiveness and the
thoroughness of school self- evaluation is going to be a big theme for us over the
next number of years and to align properly external and internal inspections so
that they genuinely do complement one another. If truth be told most inspection
systems are still a strongly external inspection model with a growing school self
-evaluation model. Now, we need to make sure we balance that properly.
Something else, like data, where some systems like ours are behind is peer to
peer evaluation of teachers or principal to teacher evaluations. You can call it,
some systems would, teacher appraisal but whatever that is it?s way
underdeveloped in the Irish system but I think it is valuable and is a common
feature of most inspection.
I think another challenge across Europe is the matter of dealing with data,
public accessibility of data to get the balance right between a genuine need for
public accessibility but one that isn?t damaging to the long term educational
interests. The way it seeks to drive behaviours or can inadvertently drive
behaviours, needs to be constantly looked at.
This paper attempted to provide an overview of the emergence of a binary mode of
integrated co-professional school inspection which has permeated many education systems
where school evaluation exists. Starting with a documentary analysis of the literature on
school inspection systems in Europe and elsewhere, the paper began by positing that almost
all inspectorates are now following to varying degrees of evaluation maturity, the same
overarching trajectory for school accountability and improvement. Next, using Ireland as a
case example, the paper provided an overview of the rapid changes that have occurred in
school inspection policies and practice whose underlying principles seem to mirror other
school inspection systems. To test these assumptions, the paper then provided an analysis of a
key informant interview conducted with the Chief inspector of schools in Ireland.
The study of policy documents and literature on the changing face of school
inspection, coupled with an analysis of the interview data, confirm the view that convergent
approaches to school evaluation appear to be visible accross national frontiers. These
approaches we have described as integrated in that they contain elements of inspection
ideology and methodology previously considered to be very different and even inimical. For
example a strong focus on quantitative data is now widely combined with the use of much
qualitative data in making judgements about schools. Similarly dialogue for improvement
and negotiation about criteria and acceptable standards sits side by side with accounability
and compliance with regulation. We have also described the convergent model as
coprofessional in that in many instances it allows schools a significant voice through dialogue
and self-evaluation, shitfing the balance of evaluation power and making the role of
inspectorates more about helping schools to improve quality through robust internal
processes. The responses of Chief Inspector Hislop are illustrative of these developments in
the Irish context and also convey his belief that developments in Ireland are very much in line
with those in other countries.
The drivers of these developments vary but include political ideology, which of
course can change rapidly, the high cost of extensive external inspection and the perceived
need to empower schools through decentralising decision making and yet make democratic
accountability an accepted part of everyday school life. Of particular influence in our view
has been the work of supranational bodies which has come to favour a compromise, as it
were, between heavy handed accounability which can stifle innovation and forms of
evaluation systems which are not robust enough to drive improvement. Many of these
elements are and will remain in a state of flux. For example the balance between school
inspection and school self-evaluation is crucial to the further development of integrated
coprofessional inspection and this in turn is dependent on the extent to which school
selfevaluation becomes credible both in terms of improvement and accountability. All that can be
said for now is that the main features of a common approach to driving school improvement
through evaluation and inspection can be found in many education systems.
Department of Education and Skills. 2012c. A Guide to Incidental Inspection in Second-Level
Schools and Centres for Education, Dublin: Inspectorate of the Department of
Education and Skills.
Donaldson, G. 2013. The SICI Bratislava memorandum on inspection and
Egan, E. 2010. School improvement through external inspection and school self-evaluation.
SDPI Summer School NUI Galway: 22 June 2010.
Ehren, M. Altrichter, H., McNamara G. et al. 2013. ?Impact of school inspections on
improvement of schools?describing assumptions on causal mechanisms in six
European countries.? Educational Assessment, Evaluation and Accountability, 25(1):
Ehren,M.C.M., Perryman, J. and Shackleton, N. 2015. ?Setting expectations for good
education: how Dutch school inspections drive improvement.? School effectiveness
and school improvement 26 (2): 296 - 327.
Ehren, M.C.M., and Schildkamp, K. 2012. ?From ?Intuition?- to ?Data?-based Decision
Making in Dutch Secondary Schools?? In Data-based Decision Making in Education:
Challenges and Opportunities, edited by. K. Schildkamp, M. Lai, and L. Earl, 49-67.
European Commission. 2004. Evaluation of schools providing compulsory education.
Eddy-Spicer D, Ehren M, Bangpan M, Khatwa M, Perrone F .2016. Under what conditions
do inspection, monitoring and assessment improve system efficiency, service delivery
and learning outcomes for the poorest and most marginalised? A realist synthesis of
school accountability in low- and middle-income countries. London: EPPI-Centre,
Social Science Research Unit, UCL Institute of Education, University College
Gilchrist V. 1993. ?Key informant interviews.? In Doing Qualitative Research, edited by
Crabtree, B. and W. Miller, 359-381. Newbury Park: Sage Publications.
Government of Ireland. 1998. Education Act. Dublin: Stationery Office.
Hislop, H. 2010. Parents, school evaluation and school improvement. National Parents
Council?Post-Primary 25th Anniversary Conference, 1 May 2010.
Leeuw, F.L., 2003. ?Reconstructing program theories: Methods available and problems to be
solved.? American journal of evaluation, 24(1): 5-20.
McNamara, G. and O?Hara, J. 2012. ?From looking at our schools (LAOS) to whole school
evaluation-management, leadership and learning (WSE-MLL): the evolution of
inspection in irish schools over the past decade.? Educational Assessment, Evaluation
and Accountability 24 (2): 79-97. https://doi.org/10.1007/s11092-012-9143-9
McNamara, G. and O Hara, J. 2008. ?Contexts and Constraints: An Analysis of the Evolution
of Evalaution in Ireland with particular reference to the Education System.? Journal
of multidisciplinary evaluaton, (4) 75-83
McNamara, G. and O'Hara, J. 2006. ?Workable Compromise or Pointless Exercise? School
Based Evaluation in the Irish Context?. Educational management administration and
leadership 34 (4): 564-582. https://doi.org/10.1177/1741143206068218
Akenson , D. H. 1970 . The Irish education experiment: The National System of Education in Ireland in the nineteenth century . London: Routledge.
Altrichter , H. , Ehren , M.C.M. , McNamara , G. and O'Hara , J. 2014 . ? The impact of school inspection? Analysis of assumptions about mechanisms of school inspection in six European countries ? In Schule - Qualit?t und Entwicklung, edited by M. Pfeifer, 184 - 207 . M?nster: Waxmann.
Brown , M. 2013 . Deconstructing Evaluation in Education: The case of Ireland . PhD Thesis , Dublin City University, Ireland.
Brown , M. , McNamara , G. , and O Hara , J ( 2016a ) ? Teacher Accountability in Education - The Irish Experiment.? In Essays in the History of Irish Education, edited by B .Walsh, 359 - 381 .United Kingdom: Palgrave Macmillan. https://doi.org/10.1057/978-1- 137 -51482-0_ 14
Brown , M. , McNamara , G. , O 'Hara , J. ( 2016b ) ? Quality and the rise of value-added in education: The case of Ireland? . Policy Futures in Education , 14 ( 6 ): 810 - 829 . https://doi.org/10.1177%2F1478210316656506
Brown , M. , McNamara , G. & O Hara , J. 2013 . Nijesh Solk Mwil Start Gyamyath (Save us from the time of trial) : From Austerity to Prosperity and Back Again . [Conference Organising Committee Chairperson], Educational Policy in Changing Times: Consultation, Implementation and Impact - 38th Annual Conference of the Educational Studies Association of Ireland (ESAI) , University of Limerick, 21 - MAR13 - 23 - MAR-13.
Brown , M. , McNamara , G. , O 'Hara , J. and O'Brien , S. 2016 . ?Exploring the changing face of school inspections? . Eurasian journal of educational research , 16 ( 66 ): 1 - 35 . http://dx.doi.org/10.14689/ejer. 2016 . 66 .1
Department of Education and Skills . 2013. Chief Inspector's Report: 2010 - 2012 .http://www.education.ie/en/Publications/Inspection-ReportsPublications/ Evaluation-Reports- Guidelines/ Chief-Inspector% E2 % 80 % 99s-Report2010- 2012 -Main-Report.pdf
Department of Education and Skills Republic of Ireland. 2012a. Circular 0040 / 2012 , Implementation of school self-evaluation- Post Primary . http://www.education.ie/en/Circulars-and - Forms /Active-Circulars/ cl0040 _ 2012 . pdf [Accessed 12 March 2015 ].
Department of Education and Skills. 2012b. Circular 0039 / 2012 , Implementation of school self-evaluation- Primary . http://www.education.ie/en/Circulars-and-Forms/ActiveCirculars/cl0039_ 2012 .pdf
McNamara , G. and O'Hara , J. 2005 . ? Internal Review and Self-Evaluation - The Chosen Route to School Improvement in Ireland?? . Studies in Educational Evaluation, ( 31 ) 267 - 282 . https://doi.org/10.1016/j.stueduc. 2005 . 11 .003
McNamara , G. and O'Hara , J. 2002 . ? Whole School Evaluation and School Development Planning; an Analysis of recent Initiatives in Ireland? . Educational management administration and leadership ( 30 ): 201 - 212 . https://doi.org/10.1177/02611X02030002510
MacBeath , J. 2006 . School inspection and self-evaluation: working with the new relationship . London: Taylor and Francis. https://doi.org/10.4324/9780203967102
MacBeath , J. 1999 . Schools must speak for themselves: the case for school self evaluation . London: Routledge.
Nelson , R. , and Ehren , M.C.M. 2014 . Review and synthesis of evidence on the (mechanisms of) impact of school inspections .
http://schoolinspections.eu/wp-content/uploads/downloads/2014/02/Review-and -synthesisof-evidence-on-the-mechanisms-of-impact-of-school-inspections .pdf
Nevo , D. 2010 . ?Internal and external evaluation.? In International Encyclopaedia of Education (3rd edition) , edited by Peterson, P. , Baker , E. and B. McGaw , 781 - 785 . London: Elsevier. https://doi.org/10.1016/B978-0 -08-044894-7 . 01597 - 9
Organisation for Economic Co-operation and Development - OECD . 2013 . Synergies for better learning: An international perspective on evaluation and assessment . Paris: OECD.
Perryman , J. 2009 . ?Inspection and the fabrication of professional and performative processes . ? Journal of Education Policy , 24 ( 5 ): 611 - 631 . https://doi.org/10.1080/02680930903125129
Quinn , R. 2012 . Irish Government News Service: Minister Ruair? Quinn launches Guidelines on School Self-evaluation [press release], 19 November . Available from: http://www.merrionstreet.ie/en/category-index/education/schools/minister-ruairiquinn -launches-guidelines-on-school-self-evaluation . html [Accessed November 2016 ]
Ryan , K. and Timmer , J. 2013 . ? New Zealand style school review: A view from outside.? In A developmental and negotiated approach to school and curriculum evaluation, edited by Lai , M. and S. Kushner , 195 - 206 . Bindley: Emerald Publishers https://doi.org/10.1108/S1474-7863( 2013 ) 0000014012
Scheerens , J. , and Ehren , M.C.M. 2015 . ? The evidence base for school inspection frameworks . ? Journal of Educational, Cultural and Psychological Studies , ( 12 ): 43 - 76 .
SICI. 2009 . Inspectorate profile , Republic of Ireland . Brussels: Standing International Conference of Inspectorates.
Van Bruggen , J.C. 2010 . ? The role of school inspection in ensuring quality in education: past, present and future .? In Beyond Lisbon 2010: Perspectives from research and development for education policy in Europe (CIDREE Yearbook 2010 ), edited by S.Stoney, 85 - 119 . Slough: National Foundation for Educational Research.