Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study
October
Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study
Hamza Mohammad Abdulghani 0 1
Mohammad Irshad 1
Shafiul Haque 1
Tauseef Ahmad 0 1
Kamran Sattar 0 1
Mahmoud Salah Khalil 0 1
0 Department of Medical Education, College of Medicine, King Saud University , Riyadh , Saudi Arabia , 2 Oral Microbiology Laboratory, Department of Bioclinical Sciences, Faculty of Dentistry, Health Sciences Centre, Kuwait University, Kuwait, 3 Research and Scientific Studies Unit, College of Nursing and Allied Health Sciences, Jazan University , Jazan , Saudi Arabia
1 Editor: Emmanuel Manalo, Kyoto University , JAPAN
This study examines the long-term impact of the faculty development programs on the multiple choice question (MCQ) items' quality leading to study its effect on the students' overall competency level during their yearly academic assessment. A series of longitudinal highly constructed faculty development workshops were conducted to improve the quality of the MCQs items writing skills. A total of 2207 MCQs were constructed by 58 participants for the assessment of 882 students' cognitive competency level during the academic years 2012± 2015. The MCQs were analyzed for the difficulty index (P-value), discriminating index (DI), presence/absence of item writing flaws (IWFs), and non-functioning distractors (NFDs), Bloom's taxonomy cognitive levels, test reliability, and the rate of students' scoring. Significant improvement in the difficulty index and DI were noticed during each successive academic year. Easy and poor discriminating questions, NFDs and IWFs were decreased significantly, whereas distractor efficiency (DE) mean score and high cognitive level (K2) questions were increased substantially during the each successive academic year. Improved MCQs' quality leaded to increased competency level of the borderline students. Overall, the longitudinal faculty development workshops help in improving the quality of the MCQs items writing skills of the faculty that leads to students' high competency levels.
-
Data Availability Statement: All relevant data are
within the paper and its Supporting Information
files.
Funding: The financial support for this study was
provided by the COM, Research Centre, Deanship
of Scientific Research, KSU, Riyadh, Saudi Arabia.
The funders had no role in study design, data
collection and analysis, decision to publish, or
preparation of the manuscript.
Competing interests: The authors have declared
that no competing interests exist.
Introduction
Effective delivery of medical science education demands precise and efficient assessment
system(s) capable of examining students' essential subjective knowledge, learning attitude and
practical skills at high competency level. The assessment of high level competency changes
students' learning behavior and ingrains enthusiasm in the students to accumulate subject's
explicit and implicit information from the lecturers or other available resources [1±3]. In the
assessment system, the use of multiple choice questions (MCQs) is a very common and well
accepted method of evaluating diverse characteristics of the professional competencies of
medical science education [
4
]. Moreover, medical education agencies are adopting the use of
MCQs in the assessment system due to several advantages over the other assessment formats
[
5,6
]. Most importantly, MCQs based assessment system is being used to test the cognitive
competencies of a large number of students at-a-time with a broad range of curriculum
content along with its objectivity, higher reliability, validity and ease of scoring [
7,8
]. Earlier
studies have reported that MCQs are superior and apposite competency test for evaluating the
subject's knowledge, comprehension, and can be designed to examine applications and
analyses [
9
]. Construction of high quality MCQs can test advanced level critical thinking and
application of the knowledge by evaluating the examinee's ability to integrate, synthesize and
evaluate medical information [10±12]. Use of high quality MCQs in the assessment process
determines the deep learning approach(s) of the students towards higher Bloom's taxonomical
level of cognitive abilities such as interpretation, synthesis and application of the acquired
knowledge instead of testing the recalling of the isolated facts [
13
].
Construction of high quality MCQ items is a very challenging task for the faculty members,
especially for those who have never undergone precise and dedicated training [
14,15
].
However, MCQ items writing practice guidance has been well documented in the literature since
long to guide the faculties who wish to construct MCQ items meeting with the standard
assessment format [16±18]. Regardless of the availability of MCQ items writing guidelines, many
reports have been published stating about various deficiencies and flaws in MCQ items of
medical tests [17±20]. Deviations from MCQ items (...truncated)