Validity, and sensitivity and specificity of ImPACT

Several studies have been completed that evaluate the validity of ImPACT. Studies include sensitivity, specificity and aspects of validity. As noted above, because, ImPACT Version 4 contains the same items and modules as ImPACT, these studies are relevant and provide validity evidence for ImPACT Version 4.

Sensitivity and specificity#

In 2006, Schatz and Pardini studied 138 high school athletes (72 concussed and 66 non-concussed) selected from a larger sample of 1,500 individuals. Non-concussed athletes self-reported no history of diagnosed concussion on their health questionnaire. All athletes from both groups self-reported no significant medical problems on their health questionnaire and all subjects from both groups successfully completed a baseline evaluation prior to participation.

A MANOVA on all ImPACT subscales and Composites found significant differences found on Verbal Memory, Visual Memory, Reaction Time, Processing Speed and Symptom scale with concussed athletes performing significantly worse on all variables than their matched controls. When considering classification accuracy, 82.4 % of concussed group were correctly identified as were 89.4% of non-concussed group.

A 2007 study (Broglio et al., 2007) found lower levels of sensitivity and specificity, but also discovered values that were superior to comparable tests, concluding that a battery of tests was ideal. ImPACT and HeadMinder Concussion Resolution Index (Erlanger, Feldman, & Kutner. (1999). were the most sensitive to concussion (79.2 and 78.6%, respectively). These tests were followed by self-reported symptoms (68.0%), the postural control evaluation (61.9%), and a brief pencil-and-paper assessment of neurocognitive function (43.5%). When the complete battery was assessed, sensitivity exceeded 90%. These findings supported previous recommendations that sports-related concussion should be approached through a multifaceted assessment with components focusing on distinct aspects of the athlete's function.

In a 2012 study, Schatz and Sandel examined the sensitivity of ImPACT in matched samples of symptomatic concussed high school athletes and matched controls who were not concussed. They reviewed the cases of 81 athletes observed to sustain a concussion and who had completed the ImPACT test within 3 days of injury. Data were compared with an independent sample of 81 athletes who completed preseason baseline cognitive assessments using ImPACT and who were matched (with concussed athletes) on the basis of sex, age, sport, concussion history, absence of attention deficit hyperactivity disorder (ADHD) and learning disability.

The authors determined that data from ImPACT yielded 91.4% sensitivity and 69.1% specificity. For asymptomatic athletes suspected of hiding their concussion, data from ImPACT yielded 94.6% sensitivity and 97.3% specificity.”

A summary of the studies described above are and additional supporting research is presented in Table 8.

Incremental validity of ImPACT compared with self-reported symptoms#

In a 2006 study, van Kampen, Lovell, Pardini, Collins, & Fu assessed the added value of neuropsychological testing (ImPACT) relative to symptom self-report in a group of 122 concussed and 72 non-concussed high school and collegiate athletes who had completed the ImPACT test battery. Both male and female athletes were included. This study found that the addition of ImPACT resulted in an increase in sensitivity of 19% relative to symptom monitoring alone. 93% of the concussed athlete group was correctly identified as being concussed in this study. In contrast, symptom data alone correctly classified only 64 percent of the sample. This study highlighted the frequent “disconnect” between self-reported symptoms and more objective neuropsychological test results. In addition, this study found that although none (0 percent) of the control group had both abnormal ImPACT neurocognitive performance and elevated symptoms relative to baseline (determined by RCI scores, relative to baseline test results). In contrast, 93 percent of the concussed group had either poorer ImPACT test scores or elevated symptoms.