The effect of written standardized feedback on the structure and quality of surgical lectures: A prospective cohort study

Research article Open Access Open Peer Review The effect of written standardized feedback on the structure and quality of surgical lectures: A prospective cohort study Jasmina Sterz1, Sebastian H. Höfer1, Bernd Bender1, 3, Maren Janko1, 3, Farzin Adili2 and Miriam Ruesseler1, 3Email author BMC Medical EducationBMC series – open, inclusive and trusted201616:292 https://doi.org/10.1186/s12909-016-0806-y ©  The Author(s). 2016 Received: 20 May 2016Accepted: 27 October 2016Published: 14 November 2016 Open Peer Review reports Abstract Background Lectures remain an important teaching method to present and structure knowledge to many students concurrently. Adequate measures are necessary to maintain the quality of the lectures. The aim of this study was to determine the impact on the lecture quality using written structured feedback and to compare the ratings of surgical lectures between students and surgical peers. Methods Prospective analysis of two consecutive surgical lecture series for undergraduate students at Goethe-University Medical School was performed before and after evaluation of the lecturers via independent written feedback from trained undergraduate students and surgeons. The 22-item feedback instrument covered three areas of performance: content, visualization, and delivery. Additional suggestions for improvement were provided from both students and surgical peers who anonymously attended the lectures. The lecturers, experienced surgeons, as well as the student and peer raters were blinded in terms of the aim and content of the study. Their response to the feedback was collected using a web-based 13-item questionnaire. The Kendall’s-W coefficient was computed to calculate inter-rater reliability (IRR). Differences between ratings before and after feedback were analyzed using Student’s t-test for dependent samples. The Kolmogorov-Smirnov-test was used for independent samples. Results A total of 22 lectures from a possible 32 given by 13 lecturers were included and analyzed by at least three surgeons and two students. There were significant improvements in overall score as well as in the details of 9 of the 13 items were found. The average inter-rater reliability was 0.71. There were no differences in the ratings as a function of the rater’s level of expertise (peers vs. students). We found that 13/23 lecturers (56.5%) answered the questionnaire, and 92% strongly agreed that the written feedback was useful. 76.9% of the lecturers revised their lecture based on the written feedback requiring on average 112.5 min (range from 20 to 300 min). Conclusions Overall, this study indicates that structured written feedback provided by trained peers and students that is subsequently discussed by the lecturers concerned is a highly effective and efficient method to improve aspects of lecturing. We anticipate that structured written feedback by trained students that is discussed by the lecturers concerned will improve lecturing. Keywords LectureFeedbackSurgeryPeer-feedbackEvaluationUndergraduate training Background Often criticized by teachers and students, lectures are still meaningful in medical education and are an efficient method of teaching [1–4]. Lectures are even more important with high student numbers because the lecturer can present information to many students at the same time and with the same learning outcome regardless of the number of students [5]. Furthermore students can learn from the lecturer’s experience and can discuss academic issues with her or him, and the lecturer can help them to prepare themselves for the faculty’s exams [1]. Medical knowledge continuously increases and is now available from everywhere via the internet. Thus, lectures can help the students to structure, assess and synchronize the available information based on the lecturers’ priorities. However, lectures can definitely be boring or demotivating if badly prepared. Thus, many authors analyze the quality criteria of successful lecturing [1, 5–10] including: Appropriate amount of data [5, 6] Clearly defined content [6] with clearly stated goals of the talk [9] Interactions with the audience [5, 9–11] Coherent and well prepared slides [5, 9] To improve one’s own teaching quality - especially regarding lecturing - a critical self-reflection of one’s own performance is necessary. Here, feedback plays an important role, because it improves knowledge and competence and helps to reflect on one’s performance [12–15]. However, non-specific, unclear, and irrelevant feedback is useless and may hinder the learning process [13]. Thus, feedback must fulfill defined quality criteria to be successful [12, 16, 17]. It must be constructive, specific and offer concrete suggestions for improvement [12, 17]. Furthermore it should be based on direct observations and be made timely [17, 18]. The easiest way to give feedback is orally. Direct talk between the person offering feedback and the receiver can be initialized via oral feedback. This feedback should be timely because important aspects might be forgotten if delayed due to the spontaneous character of oral feedback [19]. Another way of giving feedback is written feedback. It provides an enduring record and reference point that can be taken home [19]. This allows the receiver (e.g. the lecturer) to reflect on it repetitiously and to reread the personal feedback while they revise their lectures [11]. It also allows the receivers to directly compare with others [19]. Haghani et al. demonstrated that written feedback had a higher learning effect than oral feedback [20]. However, written feedback requires a time consuming preparation [19]. Feedback on teaching quality by peers has been shown to be very successful in terms of improving teaching quality. It is highly accepted by feedback providers and recipients [21–28]. Despite the advantages of peer feedback, students must realize the structure, learning objectives and aims of the lectures. The content and context must be presented more clearly to students than experts. Thus, their feedback is critical. On the other hand, student’s evaluations are influenced by many factors that cannot be influenced by the lecturer and lecture’s quality including age [29], expected grades in the relating tests [30], intrinsic motivation or general interest in the topic [31]. In fact, the lecture rating by students is often influenced by how entertaining the lecture is. Thus, a combination of student and peer feedback might be reasonable. The aim of this study is to analyze if student and expert raters are using the feedback sheet differently and to measure the impact of written, structured feedback on the quality of a lecture series in surgery for undergraduate medical students. We also want to analyze the lecturers’ response to this kind of feedback. Methods Study design This study has a prospective design and analyzed the effect of structu (...truncated)