Looking in the Heads of Experienced Teachers – Do they use the Wide Range of Principles of Effective Teaching when Analysing Lessons?

Australian Journal of Teacher Education, Feb 2019

This study aimed to examine whether principles of effective teaching constitute essential criteria for a systematic and successful analysis of lessons. After watching a video of a complete lesson, the participants (each of nine experienced and pre-service teachers) were asked to analyse this lesson in terms of effectiveness for pupils’ learning in the form of an open dialogue. Their comments were analysed by means of a qualitative content analysis and revealed that the experienced teachers independently used the wide range of principles of effective teaching and differed significantly from the pre-service teachers in this regard. Particularly striking were the large differences in the activation of knowledge about these five principles: goal orientation, relating cognitive activities to prior knowledge, classroom climate/learning atmosphere, clarity, and using appropriate examples. These differences point to specific development tasks, in order to improve the analytical skills of student teachers within teacher education.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://ro.ecu.edu.au/cgi/viewcontent.cgi?article=3927&context=ajte

Looking in the Heads of Experienced Teachers – Do they use the Wide Range of Principles of Effective Teaching when Analysing Lessons?

Vol Looking in the Heads of Experienced Teachers - Do they use the Wide Range of Principles of Effective Teaching when Analysing Lessons? Wilfried Pl?ger 0 0 University of Cologne - Article 2 Looking in the Heads of Experienced Teachers ? Do They Use the Wide Range of Principles of Effective Teaching when Analysing Lessons? Wilfried Pl?ger Matthias Krepf University of Cologne, Germany Daniel Scholl University of Vechta, Germany Andreas Seifert University of Paderborn, Germany Abstract: This study aimed to examine whether principles of effective teaching constitute essential criteria for a systematic and successful analysis of lessons. After watching a video of a complete lesson, the participants (each of nine experienced and pre-service teachers) were asked to analyse this lesson in terms of effectiveness for pupils? learning in the form of an open dialogue. Their comments were analysed by means of a qualitative content analysis and revealed that the experienced teachers independently used the wide range of principles of effective teaching and differed significantly from the preservice teachers in this regard. Particularly striking were the large differences in the activation of knowledge about these five principles: goal orientation, relating cognitive activities to prior knowledge, classroom climate/learning atmosphere, clarity, and using appropriate examples. These differences point to specific development tasks, in order to improve the analytical skills of student teachers within teacher education. Introduction In order to facilitate the learning of students and to maximize their achievements, the core activities of teachers (planning, implementing and analysing lessons) must be oriented towards principles of effective teaching (e.g., goal orientation, feedback, cognitive activation etc.), which are positively connected to students? outcomes. Literature reports and meta-studies, in which several hundred of individual studies have been combined, support the empirical evidence for the effectiveness of these principles (e.g., Brophy, 2000; Hattie, 2009; Muijs et al., 2014) . Therefore, it is to be important that teachers should have sufficient knowledge on principles of effective teaching. This applies not only to planning and implementing a lesson, but also for its subsequent analysis (Hiebert, Morris, Berk, & Jansen, 2007) , because these principles constitute important criteria for a systematic analysis in order to identify the factors that have determined the effectiveness of the pedagogical interactions. The significance of some of these principles for an appropriate analysis of lessons has already been verified in various studies (e.g., Jamil, Sabol, Hamre, & Pianta, 2015; Krull, Oras, & Pikksaar, 2010; Wiens, Hessberg, Lo-Casale-Crouch, & DeCoster, 2013) , in which videos were used as stimulus and the participants were asked to assess the observed teaching behaviour. The clips were of short duration (often only 3?5 min.) and selected in such a way that they represented a limited number of principles. This procedure has two crucial advantages: Short videos show a small cutout of the whole teaching process and, therefore, limit the observers? cognitive load. Additionally, predetermined observation instructions channel the attention to few selected principles. However, videos that are of short duration and only focused on a few principles have the disadvantage that they do not represent the complexity of the whole teaching process. Therefore, the participants will not be given sufficient opportunity to show that they know the wide range of these principles and can apply them to concrete teaching scenarios. This disadvantage can be compensated, if videos of a complete lesson are chosen and if the participants have the opportunity to independently activate their knowledge on principles of effective teaching in the form of an open dialogue. Until now there are no studies following this option (complete videotaped lesson as stimulus combined with the analysis in the form of an open dialogue regarding the wide range of principles of effective teaching). The study reported here addresses this desideratum. The participants were shown a video of a complete lesson on optics in which the Snell?s law of refraction was being studied. In a subsequent interview they were asked to analyse this lesson in terms of effectiveness for learning. This special stimulus and the open dialogue encouraged the participants to give detailed statements, which were analysed by means of a qualitative content analysis (Mayring, 2014) . In order to examine whether principles of effective teaching constitute essential criteria for a systematic and successful analysis of lessons, it seems suitable to compare well-known groups (Cronbach & Mehl, 1955) that are expected to differ in their analytical performances. Therefore, our sampling comprised two groups, experienced teachers and pre-service teachers as novices. The analysis of lessons is a challenge for pre-service teachers, because they have great difficulties to distinguish between important and unimportant information (Berliner, 2001) . Additionally, they perceive lessons as a chronological but disconnected sequence of events (Star & Strickland, 2008) . As a result, they cannot recognize and assess the functions of lesson elements for the whole teaching process. In contrast, experienced teachers consider elements of lessons not in isolation, but think systematically about their interrelations (Rosaen, Carlisle, Mihocko, Melnick, & Johnson, 2013) . They identify single events as meaningful sub-activities within a larger lesson (Berliner, 2001) . Therefore, they are able to link various single situations and actions to broader didactic units and to evaluate the effectiveness of such units against the background of the whole teaching process. Such differences in analytical performances might be traced back to differences in the content and extent of knowledge regarding the principles of effective teaching, because these principles allow to recognise what is important or unimportant, why single actions have an impact on others, or which alternative actions would have been more effective. Against the background of this assumption, the study at hand was guided by two research questions. (1) Do the experienced teachers independently use the wide range of principles of effective teaching? (2) Do the experienced teachers differ significantly from the pre-service teachers in this regard? Principles of Effective Teaching as Criteria for Assessing Teaching Quality and as a Tool for Lesson Analysis In order to examine whether teachers use the wide range of principles of effective teaching when analysing lessons, one needs a language to capture these principles. Such principles have been discussed in educational research for more than three decades. In this section, we give an overview of some influential conceptualizations used in video studies for identifying and promoting teachers? observational skills. Despite the minor or major differences between these conceptualizations, the overview will show that there is a certain degree of conformity regarding principles of teaching linked to positive students outcomes. Based on this consensus, we established a category system (s. Table 1). The categories contained therein are used to determine whether the statements of the interviewed participants can be seen as indicators for an independent activation of their knowledge on principles of effective teaching. First, we refer to the prominent work of Gagn? (1985) . He developed a theory of teacher actions and characteristics, which was centered on the idea of supporting students? learning processes. As criteria for the assessment of the quality of instruction, Gagn? proposed nine features of teaching behaviour: gaining attention, informing learners of the learning objectives, stimulating recall of prior learning, presenting material for learning, providing learning guidance, eliciting performance, providing feedback, assessing performance, and enhancing retention and transfer. The thorough review of process-product research by Brophy and Good (1986) was also very influential. They emphasized a set of 12 principles describing generic patterns of successful teaching behaviour (supportive classroom climate, opportunity to learn and time on task, curricular alignment, establishing learning orientations, coherent content, thoughtful discourse, practice and application activities, scaffolding students? task engagement, strategy teaching, co-operative learning, goal-oriented assessment, and achievement expectations). The conceptualizations of effective teaching behaviour developed by Gagn? and Brophy and Good serve until today as theoretical frameworks for the implementation of video studies. For example, Krull et al. (2010) conducted a study for promoting pre-service teachers? lesson analysis and observational skills. Contrary to the control group, the experimental group was provided with special training on Gagn??s nine principles. The effects of this training (measured by pre- and post-test) showed that the experimental group progressed more than the control group, especially regarding the two principles providing learning guidance and feedback. More recent conceptualizations of teaching quality rely on these former approaches, but set the focus less on elements of concrete teaching behaviour rather than on providing opportunities to foster pupils? meaningful (versus rote) learning. For example, Schoenfeld (2013) developed the Teaching for Robust Understanding (TRU) Framework, which supports teachers and coaches in planning, conducting and reflecting on classroom observations. TRU comprises five dimensions: (1) content (the extent to which central disciplinary ideas are present in instruction), (2) cognitive demand (the extent to which classroom activities initiate and maintain an environment of intellectual challenge), (3) equitable access to content (the extent to which a teacher supports all students taking in consideration their diverse abilities), (4) agency, ownership, and identity (the extent to which students are the source of ideas and discussions), (5) and uses of assessments (the extent to which feedback and assessments ensure pupil?s current state of understanding). Based on these dimensions, Schoenfeld and co-workers developed a rubric for classroom observations. ?Using the rubric involves parsing classroom activities into a sequence of episodes of no more than 5 minutes each in duration, assigning scores to each episode using the relevant subrubric.? (Schoenfeld, 2014, p. 406) The rubric was primarily developed as a research tool in order to measure and validate the dimensions of ?powerful classrooms?. This tool is also used to facilitate the professional development of teachers by engaging them in productive activities and discussions regarding the five dimensions. For this purpose, observation guides were developed and are available in mathematics-specific and domain-general versions. While the focus in TRU is on observing how teachers foster students? meaningful learning, the Classroom Assessment Scoring System (CLASS) is a framework for the observation of effective teacher-child interactions (Pianta & Hamre, 2009) . CLASS comprises three domains (subdivided in dimensions): (1) emotional support (positive/negative climate, teacher sensitivity, regard for student perspectives), (2) classroom organization (behaviour management, productivity, instructional learning formats, and (3) instructional support (concept development, quality of feedback, language modelling, literacy focus). A concrete application of CLASS is the Video Assessment of Interaction and Learning (VAIL) aimed at assessing teachers? skills in detecting and identifying effective interactions in the classroom (Jamil et al., 2015) . VAIL targets three components of the third dimension ?instructional support?; instructional learning formats (providing interesting materials, instructional clarity, describing objectives), quality of feedback (promoting development of metacognitive skills, asking students to explain their thinking), and language and literacy interactions (engaging in frequent conversations, asking open-ended questions, using advanced vocabulary). VAIL provides a sound psychometrical measure of teachers? observation skills. In order to identify such skills, teachers were shown two short videos and had to respond to two prompts asking them to identify strategies the teacher in the video is using for facilitating effective instructional interaction. For example, a prompt regarding the feature feedback was: ?Name up to five specific, observed behaviors that the teacher uses to effectively provide feedback and extend students? learning, skills, and persistence?. (Jamil et al., 2015, p. 415) Such prompts were also used in the study of Wiens et al. (2013) . They used VAIL as video-based assessment in a university teacher education program to examine pre-service teachers? knowledge related to effective teaching. The participants watched three videos (2? 3 min.) which ?each focused on one dimension of the CLASS framework: quality of feedback, instructional learning formats, and regard for student perspectives?. (p. 27) The results showed that the instrument provided standardized measures and a basis for evaluating respective components in pre-service teacher education programs aiming at improving observational skills. The Observer Research Tool developed by St?rmer and Seidel (2017) is a further example of a standardized instrument for measuring prospective teachers? ability to notice and interpret classroom events relevant for students? learning. Based on a meta-analysis (Seidel & Shavelson, 2007) , St?rmer and Seidel focused on three generic teaching and learning (TL) components (goal clarity, teacher support, and learning climate) as principles of effective teaching. In order to test the tool, 12 videos were selected (each of 2?4 min.) covering the three TL components and five different subjects. As a result the instrument has been proven to measure prospective teachers ability to analyse the videotaped scenarios in a reliable and valid way. Additionally, St?rmer and Seidel could show in a pre- and post-test design that the acquisition of knowledge on the three Short definition of this principle Principles of effective teaching (1) opportunity to The time a teacher actively engages learn and time on students in learning during lesson task (2) goal orientation Throughout the teaching process teachers must concentrate on the intended goals so that the students can take on these intentions as the goals of their own learning processes. (3) structuring To avoid cognitive overload teachers have to structure the entire learning process, i.e. to arrange it into several sub-processes or phases. (4) clarity Teachers have to communicate in a way that is characterised by precise language, whole but not too complicated sentences and with an appropriate level of information. (5) cognitive The teacher must create a learning activation/ environment to foster the cognitive motivation activities students need to build their knowledge. (6) relating In addition to prior knowledge, new cognitive information always needs to be activities to absorbed, processed and integrated prior knowledge with the existing prior knowledge in an appropriate way. (7) feedback/ In order to establish whether and to evaluation/ what extent the intended goals have assessment already been achieved, teachers and pupils need feedback in the form of learning-relevant information. (8) adapting/ Teachers have to take into account differentiating the individual abilities and sociocultural backgrounds of students by adopting appropriate differentiation strategies. (9) application/ Teachers have to provide transfer opportunities in which the newly acquired cognitive structures can be applied as a mental tool that enables future situations to be understood. (10) classroom Teachers should show respect climate/ towards children and transmit their learning positive expectations to their atmosphere students. 1 X X X X 3 X X X X X X 2 X X X X X 4 X X X X 5 X X X 6 X Further references Hattie, 2009 Bolhuis, 2003 ; Kyriakides, Christoforou, & Charalambous, 2013; Seidel & Shavelson, 2007 Kyriakides, Christoforou & Charalambous, 2013; Muijs et al., 2014) Kington, Sammons, Day, & Regan, 2011 ; Scheerens & Bosker, 1997 Baumert & Kunter, 2013 ; Seidel & Shavelson, 2007 Ausubel et al., 1980; Hattie, 2009; Walberg & Paik, 2000 ; Wang, Haertel, & Walberg, 1993 Brophy, 2000; Fraser, Walberg, Welch, & Hattie, 1987 ; Hattie, 2009 Kington, Sammons, Day, & Regan, 2011 ; Walberg & Paik, 2000 ; Waxman, Wang, & Anderson, 1985 Chi & VanLehn, 2012 ; Schwartz, Chase, Oppezzo, & Chin, 2011 X X X Seidel & Shavelson, 2007; Muijs & Reynolds, 2011 principles of teaching in university seminars led to positive changes in pre-service teachers? observation skills. Note: 1 = Gagn?, 1985; 2 = Brophy & Good, 1986; 3 = Schoenfeld, 2013 [TRU]); 4 = Hamre et al., 2012 [CLASS]; 5 = Jamil et al., 2015 [VAIL]; 6 = St?rmer & Seidel, 2017 [Observer] In view of our research questions and regarding the necessity to determine a wide range of principles of effective teaching as tool for lesson analysis, we summarize the overview of the conceptualizations reported and the video studies based on them as follows. The conceptualizations represent a great variety in terms of methodology and content, each of which is aligned to a central perspective (e.g., teaching behaviour in Gagn??s approach, fostering meaningful learning in TRU, effective teacher-child interactions in CLASS). In order to concretize the chosen perspective, several dimensions and subdimensions were determined and operationalized by specific indicators. Notwithstanding this variety (especially in the operationalization of the subdimensions), one can identify a certain degree of conformity: Taken together, all six approaches cover a wide range of principles of effective teaching as shown in Table 1: (1) opportunity to learn and time on task; (2) goal orientation; (3) structuring; (4) clarity; (5) cognitive activation/motivation; (6) relating cognitive activities to prior knowledge; (7) feedback/evaluation/assessment; (8) adapting/differentiating instruction; (9) application/transfer; (10) classroom climate/learning atmosphere. However, the table also shows that each of the unique approach focuses on a limited number of these ten principles and, therefore, does not include the others. Additionally, we provide corresponding references to relevant meta-analyses and literature reports in the last table column, which consistently support the empirical evidence and theoretical plausibility of these principles. Method Sampling Design The sampling comprised nine experienced and nine pre-service teachers who would be or were already teaching in gymnasium schools (academic track). In Germany there are three different types of secondary schools: Gymnasium, Realschule and Hauptschule. Students at Gymnasium graduate after grade 12 with the Abitur, the highest school-leaving certificate, which is also required to enter the university-based teacher education. Students at Realschule and Hauptschule graduate after grade 10 and then they usually undergo a 3-year apprenticeship program combined with instruction in a part-time vocational school. Teachers at gymnasium schools are specialised in two subjects that can be combined differently. The experienced teachers were required to meet two criteria: extensive teaching experience in gymnasium schools (at least 15 years), and qualification recognized by the school administration through promotion to teacher training personnel. These persons work in special institutions, called teacher-training seminars, in which future teachers undergo an internship as trainee teachers over a period of 18 months. The tasks of teacher training personnel include the analysis of show case lessons of future teachers and the advice on how they could optimize the effectiveness of their teaching. Contact with these people was made through the institutions at which they worked. Ultimately, nine experienced teachers agreed (three female, six male) to voluntarily take part in an interview. Three of them were teaching physics and three other science subjects (chemistry or biology). The further three persons were teaching neither physics nor other science subjects. In addition to these experienced teachers, the sample included a similar-sized group of preservice teachers. We found nine teacher candidates who were studying in their fourth semester at our own university and participated voluntarily. The selection of these nine candidates (five female, four male) was made in analogy with the distribution of the subjects of the experienced teachers (physics, another science subject, none of these subjects). Their teaching experience comprised a maximum of 15 hours. Videotaped Lesson as Stimulus for Conducting Interviews The stimulus used to conduct interviews with the participants was a recording that showed a complete physics lesson (45 min.) on optics in which the Snell?s law of refraction was being studied. The overarching aim of this physics lesson was to rediscover the law of refraction. For this purpose, the teacher showed phenomena of refraction (via pictures) in the introduction phase and requested the pupils to construct an experiment in order to discover the law of refraction. Later, the experiment was performed and the obtained data were analysed. After the Snell?s law had been formulated, the pupils performed an example calculation (calculating the refraction index from given angles). In the final phase of the lesson the teacher explained the function of a Fresnel lens as an application of the law of refraction. In terms of form, this lesson comprised several crucial elements of teaching. But the actual implementation of the lesson showed a number of serious shortcomings. First of all, the lesson was entirely overloaded by too many subgoals that cannot be reached in the given amount of time (45 min.). Thus, the discourse and the activities ran hastily. Secondly, the teachers? behaviour corresponded only to a limited extent to: engaging students in investigations, facilitating classroom discourse, eliciting student thinking, providing feedback, constructing models or connecting new concepts to application. Regarding the poor implementation of such core teaching practices, we assumed that the participants had enough opportunities to use the wide range of principles of effective teaching (s. Table 1). In the subsequent interview, subjects were asked to comment on the lesson they had observed. In order to encourage the comments to be as open as possible, the definition of conversation?s structure in terms of both time and content was kept to a minimum. After the video was shown, the interview was initiated with an open question (e.g., ?What did you observe in this section of the lesson??). If the conversation came to a halt, further encouragement for discussion was given (e.g., ?How could the teacher?s behaviour that you just described affect the pupils??). At the end of the interview the participants were asked to once again summarise what they believed the most important aspects of the lesson observed were. Data Analysis All 18 interviews were recorded, fully transcribed, and then analysed using the qualitative content analysis method (Mayring, 2014) . This method allowed extensive text material to be reduced down to essential structures and statements. A first trial run-through made it clear that the ten characteristics of effective teacher behaviour described above (see Table 1) could be used as meaningful categories for the analysis of the transcripts. The first and ninth categories referred to in Table 1 (time on task; application/transfer) were hardly mentioned, so we decided these two categories should not be used. However, in the first trial run-through, several statements were identified that could not be coded within the remaining eight categories. These statements were related on the one hand to the conciseness of examples and illustrations selected by the teacher to demonstrate the refraction of light and on the other hand to the need to use concrete contexts from the pupils? known environment as a starting point for learning processes and then to reach general findings inductively. This led us to take two further categories into account: (a) using appropriate examples and (b) situated and inductive learning. These two characteristics were not taken into account in the prominent studies as in the meta studies reported above, but in the literature evidence can be found for the effectiveness of learning through using suitable examples (see Durkin & RittleJohnson, 2012; Renkl, Atkinson, Maier, & Stanley, 2002) and through using situated and inductive learning (Lave & Wenger, 1991). Overall, therefore, it was shown that these two categories together with the other eight tested categories (s. Table 1) were suitable for answering our research questions. Each category was precisely defined, as well as being made concrete by using a prototypical text passage (anchor example). Such anchor examples show analogously how a text passage should be formulated for it to be included in the appropriate category. These definitions and anchor examples together formed a category system and specified the guidelines for performing the content analysis. The performance of the content analysis, which was carried out in part by the first author of this article and two students, required the proper and precise handling of the individual categories. Therefore, it was necessary to prepare the two students for this task with an intensive training course. After these two people had acquired sufficient certainty in the use of the ten categories, the actual coding process began. Over a certain period, each of the three people coded all 18 interviews separately. In the meantime, meetings were held in order to determine where there were similarities or differences in the codings. Ultimately, only the statements on which the three coders reached a consensus were accepted. The category system used was not only an important prerequisite for the process of coding itself, but also for the assessment of the stability and the reproducibility of the codings, which needed to show an acceptable level of inter-coder reliability and intra-coder reliability. After the coding process had finished two other people were involved in the process in order to determine the inter-coder reliability. For this process to be carried out using the time as efficiently as possible, these people did not code all the interviews, instead taking on just a selection of the text passages. This selection covered 10% of the codings that had previously been identified per consensus that was reached between the original three people. In accordance with Fleiss and Cohen (1973) a kappa of between 0.60 and 0.75 was set as a sufficient level for the inter-coder reliability. Five weeks later these two people once again coded the text passages that they alone had worked on to see if they reproduced the same codings as the first time. This intra-coder reliability was also allocated a Cohen?s kappa of between 0.60 and 0.75. In addition to this qualitative content analysis, we used the Mann-Whitney test (because of the small sample and the fact that the data were not normally distributed) to check whether the experienced and pre-service teachers differed significantly in their analytical performance. Results There were high values for the repeatability and stability of the codings. Indeed, the intercoder reliability achieved a kappa of 0.73 and the intra-coder reliability was 0.77. Table 2 shows the number of codings for each category calculated separately for the experienced and pre-service teachers. Overall, 277 text passages representing the respondents? independent statements were coded, 180 for the experienced and 97 for the pre-service teachers. This high number of coded text passages provided us with a positive response to our first research question: The experienced teachers independently activate the wide range of principles of effective teaching when analysing the lesson. Additionally, it became clear that the pre-service teachers are ?on their way?, because they also used these principles of effective teaching, albeit much less. Categories goal orientation structuring clarity cognitive activation/ motivation relating cognitive activities to prior knowledge feedback/ evaluation/ assessment adapting/differentiating instruction classroom climate/ learning atmosphere situated and inductive learning using appropriate examples Experienced teachers (number of codings) 25 19 15 31 13 13 4 34 There were approximately twice as many codings for the experienced teachers as there were for the pre-service teachers. These differences proved to be significant (research question 2) in the Mann-Whitney test (CK: U = 12.5, z = ?2.48, p = .011). As proposed by Fritz, Morris and Richler (2012 ), the effect size r of the Mann-Whitney test can be calculated from the z-values in relation to the size of the sample (N = 18) by the formula: r = z/sqrt[N], whereby r is comparable with the Pearson?s correlation coefficient. The calculated effect size was r = 0.58. The differences between the experienced and pre-service teachers were therefore also of practical importance, because according to Cohen (1992 , p. 157) values of r ? 0.5 represent a large effect. In order to make it clear what sort of statements were behind the bare figures given in Table 2, we provide here a selection of concrete statements from various interviews regarding those five principles the experienced and pre-service teachers differed considerably. E: ?He seemed to me to be very vague in his whole approach to teaching. Personally, I had the impression that the teacher himself did not have a structured approach in his head with which to direct the pupils where he wanted them to go. He did actually verbalise a goal, but the transparency of how the goal would be reached was not easily distinguishable for me.? N: ?It was not clear what he is driving at ? It is difficult to invent an experimental setup and to find out the law of refraction if one does not know at all that the angles of incidence and refraction of the rays and their connections are important. He could have said that. Then the students would have been able to come up with it, but in this way it remained entirely unclear.? Clarity E: ?This also relates to his conceptual impreciseness, which leads me to believe that he himself does not have such profound expertise. Because someone who has really internalised that would be much clearer and more precise with the terminology, and would also call for clarity on the part of the pupils.? N: ?He always jumps from one foreign word to another, since in one sentence, in which he introduces a foreign word, he uses three new ones. I would have been entirely confused and would not know exactly what he meant.? Relating Cognitive Activities to Prior Knowledge E: ?This creates frustration among the students, such that they have the feeling: we do not know enough, we have forgotten a lot. And they are ashamed. This creates strange situations which should be avoided at the outset by taking professional action, in the process creating clarity. And that would be easy to arrange: Once again we need prior knowledge about optical phenomena, here about refraction. Now you have three minutes as small groups, with your neighbours to find out about this, and then we will present the prior knowledge again in the full group. This creates a sound basis for further work. And the students know: We need prior knowledge and the teacher works with what is there. He makes it unnecessarily difficult here.? N: ?You have to be very careful with pupil?s misconceptions, ? you cannot simply ignore them and then move on with what you deem to be correct, rather you must actually include these in the lesson and in advance think about what ideas might already exist, and then use them.? Classroom Climate/Learning Atmosphere E: ?If I think again about the teacher-pupil interaction. I think that they do not have the impression that they, as pupils, are really taken seriously with their level of knowledge. Everyday knowledge will be ridiculed. We cannot actually use it, for science is an entirely different level. This means: As a man he is very kind, but the pupils certainly do not feel competent, as far as the subject is concerned.? N: ?So, what bothered me the most, I would like to emphasize once again. I felt that the teacher was making fun of the students? lack of knowledge. I was frightened by this. As a student, I would have participated once, but then no longer. This is not very encouraging.? Using Appropriate Examples E: ?I feel that he uses too many examples, without discussing the particular example again and clarifying the phenomenon. The pupils are easily overburdened because they see many phenomena, but they are not worked on in a structured way.? N: ?I think that this example is not comprehensible for the pupils. What has this got to do with light refraction? Even though one can see, that there is something distorted, it is not yet clear what this has to do with light refraction ? A layperson who does not know anything about this concept, has no idea what to do with it.? Summary and Discussion In comparison to the above reported video studies, using videos of short duration and focussing only on a few principles of effective teaching, the participants in our study had the opportunity to analyse a complete lesson in form of an open dialogue without predetermined instruction. Regarding our research questions, two key results can be taken from our study: (1) The experienced teachers independently used the wide range of principles of effective teaching. This suggests the assumption, that these principles together inform a homogeneous body of knowledge necessary for the analysis of complete lessons. (2) The pre-service teachers participating in our study also used these principles of effective teaching. However, they differed significantly from the experienced teachers across the most categories. Particularly striking were the large differences in the activation of knowledge about these five principles: goal orientation, relating cognitive activities to prior knowledge, classroom climate/learning atmosphere, clarity, and using appropriate examples (s. the quoted statements above). While the first three principles refer rather to the overall teaching process, the last two principles focus on important details. These quantitative differences could point to a different quality in the perception of experienced and pre-service teachers. As shown in past and recent studies, it is a major challenge for pre-service teachers to distinguish between important and unimportant details (Barnhart & van Es, 2015) . Furthermore, they tend to notice classroom situations step by step and perceive lessons as a chronological sequence of disconnected events (Berliner, 2001; Star & Strickland, 2008) . As a result, they cannot recognize and assess the functions of lesson elements for the whole teaching process. On the contrary, experienced teachers are able to identify single situations and actions as meaningful sub-activities of broader didactic units (phases) and to evaluate the learning effectiveness of such units against the background of the whole teaching process (Rosaen et al., 2013; Wolff, van den Bogert, Jarodzka & Boshuizen, 2015) . In line with such studies it can be explained that the experienced teachers in our study were, compared to the pre-service teachers, in a far better position to turn their attention both to the overall teaching process (e.g., goal orientation, relating cognitive activities to prior knowledge, classroom climate/learning atmosphere), as well as to the precise registering of important details (e.g. confusing explanations, identifying inappropriate examples). These significant differences between the experienced and pre-service teachers point to specific development tasks, in order to improve the analytical skills of pre-service teachers within teacher education. Because it remains a long way to become an experienced teacher with high analytical performances (Berliner, 2001) , the learning opportunities provided should aim at three goals: (1) learning to notice what is significant, (2) increase of complexity of lesson events to be analysed, (3) connecting theory (i.e., knowledge on principles of effective teaching) and application (analysing videotaped lessons). (1) In order to pick up pre-service teachers where they are, it is necessary to support them ?in learning to first notice what is significant in a classroom interaction, than interpret that event, and then use those interpretations to inform pedagogical decisions.? (van Es & Sherin, 2002, p. 575) For this purpose, priority should be given to short videos in combination with predetermined observation tasks focusing on a limited number of principles of effective teaching in order to limit the observers? cognitive load. The intervention studies mentioned above (e.g., Krull et al., 2010; Wiens et al., 2013) have proved such measures to be effective. (2) As the certainty has increased to identify and assess single events, more complex situations should be chosen for making connections visible between single teaching events. In this case, videos of longer duration are suitable, because they capture more complex situations similar to real life teaching scenarios. The analysis of such situations can then be arranged, for instance, in line with the model of the Lesson Analysis Framework (Santagata & Guarino, 2011; Hiebert et al., 2007) characterised by the following steps: identify lesson goals, assess whether the goals are being achieved, construct hypotheses about cause?effect relationships between teaching and students? learning, use analysis to propose improvements in teaching. (3) Both goals (noticing what is important; increase of complexity of lesson events) can only be achieved if pre-service teachers have the necessary theoretical knowledge about principles of effective teaching (s. Table 1), because this knowledge enables them to understand why the teaching behaviour must correspond to these principles. In order to avoid this knowledge remaining idle or being lost when entering the profession, its acquisition should be based on concrete contexts according to the theory of situated learning (Korthagen, 2010) and its application should be decontextualised through a variety of applications (especially through analysing videotaped scenarios). These procedures could strengthen the relationship between theory and practice (Zeichner, 2012) . These proposals are formulated with caution, because we are aware that our study does not allow generalisations to be made due to the small cohort of 18 participants. Additionally, the video stimulus used was focused on a specific topic in a specific subject. Further studies have to investigate, whether these generic principles of effective teaching are also a suitable tool for the analysis of lessons in other subjects. Finally, it must be taken into account the fact that the number of codings given in Table 2 is not produced through a standardised test procedure, but by means of a qualitative content analysis with appropriate interpretive leeway. Taking these limitations in consideration, we see our study as a useful addition to previous studies reported above in which the participants had to watch video clips of only short duration and their attention was channelled to just a few predetermined principles of effective teaching. Our qualitative study provides evidence that experienced teachers use independently the wide range of these principles as a homogeneous body of knowledge when analysing a complete lesson. Therefore, in the course of teacher education these principles should be treated as a coherent whole for an appropriate analysis of lessons. Teachers must be prepared to carry out such analyses, because it is ?hard to imagine teachers becoming more effective over time without being able to analyse teaching in terms of its effects on student learning.? (Hiebert et al., 2007, p. 48) Acknowledgements This work was supported by the German Research Foundation (grant number PL 272/3-1). Ausubel , D. P. , Novak , J. , & Hanesian , H. ( 1978 ). Educational Psychology. A cognitive view (2nd ed .). Weinheim: Beltz. Barnhart , T. , & van Es , E. A. ( 2015 ). Learning to analyze teaching: developing pre-service science teachers' abilities to notice, analyze and respond to student thinking . Teaching and Teacher Education , 45 , 83 - 93 . https://doi.org/10.1016/j.tate. 2014 . 09 .005 Baumert , J. , & Kunter , M. ( 2013 ). The COACTIV Model of teachers' professional competence . In M. Kunter, J. Baumert , W. Blum , U. Klusmann , S. Kraus , & M. Neubrand (Eds.), Cognitive Activation in the mathematics classroom and professional competence of teachers. Results from the COACTIV Project (pp . 25 - 48 ). New York: Springer. https://doi.org/10.1007/978-1- 4614 -5149- 5 _ 2 Berliner , D. C. ( 2001 ). Learning about and learning from expert teachers . International Journal of Educational Research , 35 , 463 - 482 . https://doi.org/10.1016/S0883- 0355 ( 02 ) 00004 - 6 Bolhuis , S. ( 2003 ). Towards process-oriented teaching for self-directed lifelong learning: a multidimensional perspective . Learning and Instruction , 13 , 327 - 347 . https://doi.org/10.1016/S0959- 4752 ( 02 ) 00008 - 7 Brophy , J. ( 2000 ). Teaching. Educational Practices Series 1 . Published by the International Academy of Education . Br?ssel. http://www.ibe.unesco.org. Brophy , J. , & Good , T. L. ( 1986 ). Teacher behavior and student achievement . In Wittrock, M. C. (Ed.), Handbook of research on teaching (pp. 328 - 375 ). New York: Macmillan. Cronbach , L. J. , & Mehl , P. E. ( 1955 ). Construct validity in psychological tests . Psychological Bulletin , 52 , 281 - 302 . https://doi.org/10.1037/h0040957 Chi , M. T. H. , & VanLehn , K. A. ( 2012 ). Seeing deep structure from the interactions of surface features . Educational Psychologist , 47 , 177 - 188 . https://doi.org/10.1080/00461520. 2012 .695709 Cohen , J. ( 1992 ). A power primer . Psychological Bulletin , 112 , 155 - 159 . https://doi.org/10.1037/ 0033 - 2909 . 112 .1. 155 Durkin , K. , & Rittle-Johnson , B. ( 2012 ). The effectiveness of using incorrect examples to support learning about decimal magnitude . Learning and Instruction , 22 , 206 - 214 . https://doi.org/10.1016/j.learninstruc. 2011 . 11 .001 Fleiss , J. L. , & Cohen , J. ( 1973 ). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability . Educational and Psychological Measurement , 33 , 613 - 619 . https://doi.org/10.1177/001316447303300309 Fraser , B. J. , Walberg , H. J. , Welch , W. W. , & Hattie , B. J. ( 1987 ). Syntheses of educational productivity research . International Journal of Educational Research , 11 , 145 - 252 . https://doi.org/10.1016/ 0883 - 0355 ( 87 ) 90035 - 8 Fritz , C. , Morris , P. , & Richler , J. ( 2012 ). Effect size estimates: Current use, calculations, and interpretation . Journal of Experimental Psychology , 141 , 2 - 18 . https://doi.org/10.1037/a0024338 Gagn? , R. M. ( 1985 ). The conditions of learning (4th ed) . New York: Holt, Rinchart & Winston. Hattie , J. ( 2009 ). Visible learning: A synthesis of over 800 meta-analyses relating to achievement . Abingdon: Routledge. Hiebert , J. , Morris , A. , Berk , D. , & Jansen , A. ( 2007 ). Preparing teachers to learn from teaching . Journal of Teacher Education , 58 , 47 - 61 . https://doi.org/10.1177/0022487106295726 Jamil , F. , Sabol , T. , Hamre , B. , & Pianta , R. ( 2015 ). Assessing teachers' skills in detecting and identifying effective interactions in the classroom . The Elementary School Journal , 115 , 407 - 432 . https://doi.org/10.1086/680353 Kington , A. , Sammons , P. , Day , C. , & Regan , E. ( 2011 ). Stories and statistics: Describing a mixed methods study of effective classroom practice . Journal of Mixed Methods Research , 5 , 103 - 125 . https://doi.org/10.1177/1558689810396092 Korthagen , F. A. J. ( 2010 ). Situated learning theory and the pedagogy of teacher education: Towards an integrative view of teacher behavior and teacher learning . Teaching and Teacher Education , 26 , 98 - 106 . https://doi.org/10.1016/j.tate. 2009 . 05 .001 Krull , E. , Oras , K. , & Pikksaar , E. ( 2010 ). Promoting student teachers' lesson analysis and observation skills by using Gang?'s model of an instructional unit . Journal of Education for Teaching , 36 , 197 - 210 . https://doi.org/10.1080/02607471003651789 Kyriakides , L. , Christoforou , C. , & Charalambous , Y. ( 2013 ). What matters for student learning outcomes: A meta-analysis of studies exploring factors of effective teaching . Teaching and Teacher Education , 36 , 143 - 152 . https://doi.org/10.1016/j.tate. 2013 . 07 .010 Lave , J. , & Wenger , E. ( 1991 ). Situated learning: Legitimate peripheral participation . Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511815355 Mayring , P. ( 2014 ). Qualitative content analysis: theoretical foundation, basic procedures and software solution . Klagenfurt . http://nbn-resolving.de/urn:nbn:de: 0168 -ssoar-395173. Muijs , D. , Kyriakides , L., van der Werf , G., Creemers , B. , Timperley , H. , & Earl , L. ( 2014 ). State of the art - teacher effectiveness and professional learning . School Effectiveness and School Improvement , 25 , 231 - 256 . https://doi.org/10.1080/09243453. 2014 .885451 Muijs , D. , & Reynolds , D. ( 2011 ). Effective teaching: evidence and practice (3rd ed .). London: Sage. Pianta , R. C. , & Hamre , B. K. ( 2009 ). Conceptualization, measurement, and improvement of classroom processes: standardized observation can leverage capacity . Educational Researcher , 38 , 109 - 119 . https://doi.org/10.3102/0013189X09332374 Renkl , A. , Atkinson , R. K. , Maier , U. H. , & Stanley , R. ( 2002 ). From example study to problem solving: Smooth transitions help learning . Journal of Experimental Education , 70 , 293 - 315 . https://doi.org/10.1080/00220970209599510 Rosaen , C. R. , Carlisle , J. F. , Mihocko , E. , Melnick , A. , & Johnson , J. ( 2013 ). Teachers learning from analysis of other teachers' reading lessons . Teaching and Teacher Education , 35 , 170 - 184 . https://doi.org/10.1016/j.tate. 2013 . 06 .007 Santagata , R. , & Guarino , J. ( 2011 ). Using video to teach future teachers to learn from teaching . ZDM The International Journal of Mathematics Education , 43 , 133 - 145 . https://doi.org/10.1007/s11858-010-0292-3 Scheerens , J. , & Bosker , R. J. ( 1997 ). The foundations of educational effectiveness . Oxford: Pergamon. Schoenfeld , A. H. ( 2013 ). Classroom observation in theory and practice . ZDM Mathematics Education , 45 , 607 - 621 . https://doi.org/10.1007/s11858-012-0483-1 Schoenfeld , A. H. ( 2014 ). What makes for powerful classrooms, and how can we support teachers in creating them? A story of research and practice, productively intertwined . Educational Researcher , 83 , 404 - 412 . https://doi.org/10.3102/0013189X14554450 Schwartz , D. , Chase , C. , Oppezzo , M. , & Chin , D. ( 2011 ). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer . Journal of Educational Psychology , 103 , 759 - 775 . https://doi.org/10.1037/a0025140 Seidel , T. , & Shavelson , R. ( 2007 ). Teaching effectiveness research in the past decade: the role of theory and research design in disentangling meta-analysis results . Review of Educational Research , 77 , 454 - 499 . https://doi.org/10.3102/0034654307310317 Star , J. , & Strickland , S. ( 2008 ). Learning to observe: Using video to improve preservice mathematics teachers' ability to notice . Journal of Mathematics Teacher Education , 11 , 107 - 125 . https://doi.org/10.1007/s10857-007-9063-7 St?rmer , K. , & Seidel . T. ( 2017 ). A standardized approach for measuring teachers' professional vision: The Observer Research Tool . In Schack, E., Fisher, M. , & Wilhelm , J . (Ed.), Teacher Noticing: Bridging and broadening perspectives, contexts, and frameworks (pp. 359 - 380 ). Cham: Springer. van Es , E. A. , & Sherin , M. G. ( 2002 ). Learning to notice: scaffolding new teachers' interpretations of classroom interactions . Journal of Technology and Teacher Education , 10 , 571 - 596 . Walberg , H. , & Paik , S. ( 2000 ). Effective educational practices . Educational Practices Series 3 . Published by the International Academy of Education . http://www.ibe.unesco.org. Wang , M. , Haertel , G. , & Walberg , H. ( 1993 ). What helps students learn? Educational leadership , 51 , 74 - 79 . Waxman , H. C. , Wang , M. C. , Anderson , K. A. , & Walberg , H. J. ( 1985 ). Synthesis of research on the effects of adaptive education . Educational Leadership , 43 , 26 - 29 . Wiens , P. D. , Hessberg , K. , Lo-Casale-Crouch , J. , & DeCoster , J. ( 2013 ). Using a standardized video-based assessment in a university teacher education program to examine preservice teachers' knowledge related to effective teaching . Teaching and Teacher Education , 33 , 24 - 33 . https://doi.org/10.1016/j.tate. 2013 . 01 .010 Wolff , C. E., van den Bogert, N., Jarodzka , H. , & Boshuizen , H. ( 2015 ). Keeping an eye on Learning: Differences between expert and novice teachers' representations of classroom management events . Journal of Teacher Education , 66 , 68 - 85 . https://doi.org/10.1177/0022487114549810 Zeichner , K. ( 2012 ) The turn once again toward practice-based teacher education . Journal of Teacher Education , 63 , 376 - 382 . https://doi.org/10.1177/0022487112445789


This is a preview of a remote PDF: https://ro.ecu.edu.au/cgi/viewcontent.cgi?article=3927&context=ajte

Wilfried Plöger, Matthias Krepf, Daniel Scholl, Andreas Seifert. Looking in the Heads of Experienced Teachers – Do they use the Wide Range of Principles of Effective Teaching when Analysing Lessons?, Australian Journal of Teacher Education, 2019,