Validating the modified System for Evaluation of Teaching Qualities: a teaching quality assessment instrument

Advances in Medical Education and Practice, Nov 2018

Validating the modified System for Evaluation of Teaching Qualities: a teaching quality assessment instrument Ahmed Al Ansari,1–3 Mona R Arekat,4 Abdel Halim Salem5 1Medical Education, Training and Education Department, Bahrain Defense Force Hospital, Riffa, Kingdom of Bahrain; 2General Surgery and Medical Education Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain; 3RCSI Bahrain, Manama, Kingdom of Bahrain; 4Internal Medicine Department, Arabian Gulf University, Manama, Kingdom of Bahrain; 5Anatomy Department, Arabian Gulf University, Manama, Kingdom of Bahrain Construct: We assessed the validity of the modified System for Evaluation of Teaching Qualities (mSETQ) in evaluating clinical teachers in Bahrain.Background: Clinical teacher assessment tools are essential for improving teaching quality. The mSETQ is a teaching quality measurement tool, and demonstrating the validity of this tool could provide a stronger evidence base for the utilization of this questionnaire for assessing medical teachers in Bahrain.Approach: This study assessed the construct validity of this questionnaire in medical schools across Bahrain using 400 medical students and 149 clinical teachers. Data were analyzed using confirmatory factor analysis (CFA). The goodness-of-fit index (GFI), comparative fit index (CFI), root mean square residual, and standardized root mean square error of approximation (RMSEA) indices were used to evaluate the model fit. The internal consistency reliability was assessed using Cronbach’s alpha.Results: The results of the CFA revealed an acceptable fit. All criteria for a good model fit were met except for the RMSEA fit index and the standardized root mean square residual (SRMR) value, which was very close to an acceptable value. Good overall reliability was found in the study (α=0.94).Conclusion: The overall findings of this study provided some evidence supporting the reliability and validity of the mSETQ instrument. Keywords: clinical teachers, student-centered learning, validity, teacher evaluation

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://www.dovepress.com/getfile.php?fileID=46664

Validating the modified System for Evaluation of Teaching Qualities: a teaching quality assessment instrument

Authors Al Ansari A, Arekat MR, Salem AH Received 22 July 2018 Accepted for publication 30 October 2018 Published 30 November 2018 Volume 2018:9 Pages 881—886 DOI https://doi.org/10.2147/AMEP.S181094 Checked for plagiarism Yes Review by Single-blind Peer reviewers approved by Dr Colin Mak Peer reviewer comments 2 Editor who approved publication: Dr Anwarul Azim Majumder Ahmed Al Ansari,1–3 Mona R Arekat,4 Abdel Halim Salem5 1Medical Education, Training and Education Department, Bahrain Defense Force Hospital, Riffa, Kingdom of Bahrain; 2General Surgery and Medical Education Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain; 3RCSI Bahrain, Manama, Kingdom of Bahrain; 4Internal Medicine Department, Arabian Gulf University, Manama, Kingdom of Bahrain; 5Anatomy Department, Arabian Gulf University, Manama, Kingdom of Bahrain Construct: We assessed the validity of the modified System for Evaluation of Teaching Qualities (mSETQ) in evaluating clinical teachers in Bahrain. Background: Clinical teacher assessment tools are essential for improving teaching quality. The mSETQ is a teaching quality measurement tool, and demonstrating the validity of this tool could provide a stronger evidence base for the utilization of this questionnaire for assessing medical teachers in Bahrain. Approach: This study assessed the construct validity of this questionnaire in medical schools across Bahrain using 400 medical students and 149 clinical teachers. Data were analyzed using confirmatory factor analysis (CFA). The goodness-of-fit index (GFI), comparative fit index (CFI), root mean square residual, and standardized root mean square error of approximation (RMSEA) indices were used to evaluate the model fit. The internal consistency reliability was assessed using Cronbach’s alpha. Results: The results of the CFA revealed an acceptable fit. All criteria for a good model fit were met except for the RMSEA fit index and the standardized root mean square residual (SRMR) value, which was very close to an acceptable value. Good overall reliability was found in the study (α=0.94). Conclusion: The overall findings of this study provided some evidence supporting the reliability and validity of the mSETQ instrument. Keywords: clinical teachers, student-centered learning, validity, teacher evaluation Introduction The quality of patient care is indirectly dependent on the quality of the training that the physicians receive. This student–teacher rapport necessitates the sharing of knowledge, attitudes, skills, experiences, influences, and interactions in the relationship in an appropriate manner.1 In the recent years of learner-centered education, teaching hospitals around the world have placed greater emphasis on evaluating their clinical teachers based on their clinical competency, teaching skills, personal qualities, involvement of teachers with the students, involvement of students in the provision of patient care, and the provision of guidance and feedback.2–4 Past literature reviews have identified many different instruments that have been used to assess clinical teachers.5–7 The results from these reviews indicate that approximately 30–35 instruments were available in the form of questionnaires that included 1–58 items. The assessments were mainly based on responses from student learners or residents. These assessments were used to provide formative feedback on the students’ teaching efficiency, resource allocation, promotions, and performance review.5–7 Considering the implications of these instruments, they should display high validity and reliability.8 There are five sources of validity evidence which have been identified by the American Psychological and Educational Research Associations as follows: content, response process, internal structure, relation to other variables, and consequences.9 These assessments are essential for continuous development and improvement in clinical teaching skills and need to be applied to a wide variety of samples at different points in the learning process.8 The System for Evaluation of Teaching Qualities (SETQ) was developed and has been used extensively in the Netherlands to assess clinical teachers.10–13 The SETQ consists of two sets of questionnaires containing the same items: one is for the supervisor’s self-evaluation and the other collects the learners’ assessments of their clinical teachers.11 Studies of the SETQ using confirmatory factor analysis (CFA) in teaching hospitals in the Netherlands have concluded that this instrument is a highly reliable and valid tool for assessing physicians’ teaching performances.14–16 This original instrument has been modified and tested for reliability and validity by Al Ansari et al15 in a teaching school in Bahrain. However, to increase the strength of the evidence regarding the psychometric properties of the instrument, it must be (...truncated)


This is a preview of a remote PDF: https://www.dovepress.com/getfile.php?fileID=46664

Ahmed Al Ansari, Mona R Arekat, Abdel Halim Salem. Validating the modified System for Evaluation of Teaching Qualities: a teaching quality assessment instrument, Advances in Medical Education and Practice, 2018, pp. 881-886, DOI: 10.2147/AMEP.S181094