The construction and evaluation of a generic work performance questionnaire for use with administrative and operational staff

Empirical Research The construction and evaluation of a generic Work Performance Questionnaire for use with administrative and operational staff Johann M Schepers Department of Human Resource Management University of Johannesburg South Africa Correspondence to: Johann M Schepers e-mail: Abstract The principal objective of the study was the construction and evaluation of a work performance questionnaire for use with administrative and operational staff. Work performance is a multidimensional construct that indicates how well a worker performs in his/her work, the degree of initiative he/she takes, the ingenuity he/she shows in the finding of solutions for problems, and the manner in which he/she uses the human resources at his/her disposal. Two questionnaires were constructed – one for staff performing managerial functions (the full scale) and one for staff in non-managerial positions (the shortened scale). The sample consisted of 278 staff at a South African university. The full scale yielded a reliability of 0,983 and the shortened scale a reliability of 0,978. The implications of the findings are discussed. Keywords: performance, halo error, citizenship, orientation, counterproductive Well-developed psychometric instruments (aptitude tests, personality tests, attitude scales, interest inventories, etc.) have been in use in the selection and training context for the best part of fifty years, but as far as the assessment of work performance is concerned there has been a paucity of suitable measuring instruments, particularly prior to 1990 (Campbell, McCloy, Oppler & Sager, 1993, pp. 36–37). SA Journal of Industrial Psychology Prior to the publication of Campbell et al.’s “theory of job performance”, job performance was simply defined as “that which is to be predicted, the dependent variable” (Schmitt & Chan, 1998, p. 71). With this restricted definition of job performance in mind the following techniques were used to assess the work performance of staff: ratings by superiors, peers and subordinates; standardised job samples in which the content of the job was simulated; direct task observation; and outcome measures (Campbell et al., 1993, pp. 53–56). Most of these procedures were developed in-house for the assessment of performance in specific positions. They could therefore not be extended for use in other positions without first doing the necessary research. Each of the approaches mentioned above suffer from their own limitations: Raters are inclined to rate others highly only if they perceive them to be like themselves. All raters have their own unit of measurement (mean and standard deviation). Some are very lenient while others are very strict. Halo errors occur when raters do not “discriminate among different facets of performance” (Ployhart, Schneider & Schmitt, 2006, p. 184). Raters must be thoroughly trained before making any ratings (Guilford, 1954; Cronbach, 1970, pp. 571–607). Performance in a simulated job negates the motivational factor that operates in real jobs. Direct task observation is closely linked to specific positions – one cannot generalise the findings to all jobs. Outcome measures are not always under the full control of a worker, particularly when teamwork is being done. His/her true performance might then not be visible (Schmitt & Chan, 1998, p. 79). To overcome the deficiencies of graphic rating scales (low reliability and poor discriminability), Behaviourally Anchored Rating Scales (BARS) were developed. Firstly, 10 SA Tydskrif vir Bedryfsielkunde the major dimensions of job performance (five to ten performance dimensions) were determined. Thereafter, each of the dimensions was anchored by five to seven behavioural statements (Ployhart et al., 2006, pp. 180–181). In practice it was found that “although BARS require much effort and time to construct, they surprisingly do not result in scales with better measurement properties” (Ployhart et al., 2006, p. 181). Schmitt and Chan (1998, pp. 91–92) critically examined the structure of BARS. They developed ten behaviourally anchored scales and applied them to 467 investigative officers. The ratings were done by their immediate supervisors. The ten behaviourally anchored scales plus an overall dimension of effectiveness were intercorrelated and subjected to a factor analysis. Two factors accounted for 65% of the common variance. Recording and Writing, Making Presentations, Gathering Information, Analysing Information, Planning and Organising and Monitoring Work/Detail had substantial loadings on the first factor, which was identified as Core Technical Proficiency. The second factor had substantial loadings on Develops Relationships, Effort and Initiative, Professional Image and Overall Effectiveness. This factor was identified as Effort (Schmidt & Chan, 1998, p. 92). The authors found that “most of the correlations are in the .40s and .50s”, and concluded that it is “evidence of halo error” (Schmidt & Chan, 1998, p. 91). In order to further improve the properties of graphic rating scales, Behavioural Observation Scales (BOS) were developed. They retained “the behavioral specificity of BARS”, but raters were required to indicate “how often each behavior occurred”. They were not required to evaluate the behaviour of the person (Ployhart et al., 2006, p. 182). Another variant of behavioural rating scales is the Mixed Standard Scales (MSS). The rater “checks which one of three statements in a set is most like the person rated” (Ployhart et al., 2006, p. 182). “Research to explore differences among rating scale formats in their ability to eliminate halo, central tendency, and leniency Vol. 34 No. 1 pp. 10 - 22 http://www.sajip.co.za Empirical Research Generic work performance questionnaire indicates that the more involved attempts (BARS, BOS, MSS) yield little improvement in measurement” (Ployhart et al., 2006, p. 186). The status of research on work performance radically changed following the publication of Campbell et al.’s (1990) model of work performance: new research concerning the content, structure and metrical properties of work performance measures emerged (Campbell et al., 1993; Viswesvaran & Ones, 2000), and several generic work performance scales were produced. Campbell et al. (1993, pp. 40–41) defined performance as “goalrelevant actions that are under the control of the individual, regardless of whether they are cognitive, motor, psychomotor, or interpersonal”. Their model consists of eight factors, viz. 300 studies and found that over 50% of the variance is shared across the different dimensions. “There is a general factor in job performance assessments” (Viswesvaren & Ones, 2000, p. 223). Furthermore, “this general factor is substantively meaningful and not just a manifestation of halo error” (Viswesvaran & Ones, 2000, p. 223). Their final conclusion is that “research to date suggests that ability and conscientiousness predict both task and contextual performance” (p. 22 (...truncated)