Final course grades in military technical schools are frequently used as the criterion in assessing the validity of a variety of cognitive, perceptual, and psychomotor ability tests. PDRI proposes an innovative approach to criterion development for use by AF researchers. During the proposed Phase I SBIR effort, we plan to begin development of two sets of criterion measures. The first set will be designed to link predictor scores directly to their training or job performance analog, thus providing evidence for construct validity using an external construct validation strategy. A second set of technical proficiency performance measures will be constructed and presented in a computerized adaptive rating format, which will help raters to provide more precise, accurate criterion performance scores. Workshops will be held with subject matter experts to review predictor constructs, target possible analog performance measures for both the training and job settings, and generate critical incidents of effective and ineffective performance. Our Phase I efforts will conclude with the development and presentation of a demonstration of the new computerized adaptive rating scale technology.
Keywords: Criterion Development Criterion Development Computerized Adaptive Rating Scales
