The team KSA test: development and validation

Abstract

People working together in teams for a common purpose has been a centerpiece of human social organization since our ancient ancestors first banded together to hunt game, raise families, and defend their communities. But despite the ubiquity throughout human history of teams and groups working together to achieve common goals, the scientific study of teams is relatively nascent. In particular, research on how to select candidates for team-based jobs has been limited. This may be, in part, because the teamwork construct space is poorly defined. To address this problem, Stevens and Campion (1994) reviewed the extant team- and group-behavior literatures and generated a taxonomy of 14 team-related knowledge, skills, and abilities (KSAs) reflecting two higher-order teamwork dimensions (interpersonal and self-management skills). Based on this taxonomy, Stevens and Campion (1999) created the Teamwork KSA Test. Despite initial optimism about the scale’s properties, in-depth evaluation demonstrated inadequacies in both construct validity and psychometric properties The research presented here addresses these problems in two separate studies. Study 1 used an online sample of 600 respondents to develop a new 15-item measure of teamwork, the Team KSA Test (15-TKT). Scale-development best practices, including item analysis and CFA, were used. Results suggested the general teamwork construct domain was best represented by two separate but related measurement models, one measuring interpersonal skills and the other measuring self-management skills. Model fit was excellent for a three-factor solution of interpersonal skills [SRMR = .04; CFI = .98; GFI = .97] and for a one-factor solution of self-management skills [SRMR = .02; CFI = .99; GFI = .98]. Internal consistency reliability estimates for the 15-TKT were excellent (α = .92). Study 2 was conducted to test the criterion-related validity of the new scale. Using convenience sampling, participants were recruited from a large West Coast government organization. Specifically, 27 team members, including five team supervisors representing five different team types, were recruited. Team members completed an online version of the 15-TKT scale. Team supervisors rated the team performance of each team member using an individual team-performance rating scale. Multiple regression analyses were conducted to predict teamwork performance from interpersonal and self-management skill scores of the new 15-TKT. Results were nonsignificant (R² = .08, F(2,19) = .473, p = n.s.). These findings offer no support for the criterion-related validity of the new 15-TKT scale. Potential reasons for the nonsignificant finding are discussed. The sample size for Study 2 was small and therefore the study likely lacked sufficient power to detect extant effects. Problems with range restriction in the dependent variable likely increased the possibility of Type II error. Overall, results from Study 1 were promising because they represent the first empirical evidence supporting measurement models related to Stevens and Campion’s (1994) teamwork KSA taxonomy. Future research conducted with adequate sample sizes and better criterion measures are needed to assess the criterion-related validity of the 15-TKT.