Assessment of structural knowledge for training evaluation in process control environments

Abstract

Objective: The objective of the present studies was to apply a novel method for structural knowledge assessment to process control to assess the potential of its measures as a training outcome. Background: Traditionally, knowledge is assessed by verbal achievement tests on the subject matter. However, traditional methods are regarded as limited in their ability to assess higher-order learning or understanding. Method: Two experiments (Experiment 1, N = 41; Experiment 2, N = 50) were conducted in which participants were given a 4-hr training session on a simulated process control task. At a later testing session, participants worked on the task for 70 min and completed knowledge tests on declarative, procedural, and structural knowledge. Structural knowledge was measured with the computer-based Association Structure Test (AST), which combines an association task and Pathfinder network on the basis of relatedness ratings. Results: In both studies, structural knowledge was significantly related to diagnostic performance, and evidence was found for internal consistency as well as convergent and predictive validity. Conclusion: Findings indicate that structural assessment with the AST shows promise as a training outcome in process control. Application: Potential applications of this research include the improvement of training design, delivery, and evaluation.