ImPACT Test

Figure 1. High School Boys - Regular Education

Figure 2. University Women - Regular Education

Figure 3. Distribution of scores in concussed athletes.

According to classical test theory, obtained scores (or measures) are only estimates of "true" scores because they contain measurement error. Measurement error is closely related to test reliability. Reliability refers to the consistency or stability of test scores. Reliability can be viewed as the ability of an instrument to reflect an individual score that is minimally influenced by error. Reliability should not be considered a dichotomous concept; rather it falls on a continuum. One cannot say an instrument is reliable or unreliable, but more accurately should say it possesses a high or low degree of reliability for a specific purpose, with a specific population (Franzen, 1989, 2000)².

The internal consistency reliability of the scale was estimated using Cronbach's alpha (Cronbach, 1951). Alpha is believed to represent the lower bound for the true reliability of the scale (SPSS 9.0 Base Manual, p. 362). Alpha is influenced by the number of items on the scale, the average inter-item covariance, and the average item variance.

As seen in Table 18, internal consistency reliability ranged from .88 - .94 in the large samples of high school and college regular education students. The small sample of high school girls in special education (n = 31) had a lower reliability estimate (alpha = 0.75), but the other three larger samples of special education students had high reliability estimates (.91 - .92). The internal consistency reliability for the clinical sample of 115 concussed athletes also was high (alpha = 0.93).

The standard error of measurement (SEM) is considered an estimate of measurement error in a person's observed test score. Typically, SEMs are calculated in standard deviation units using the formula below. SEMs are calculated in three steps. First, the reliability coefficient is subtracted from one. Second, the square root of this value is obtained. Third, this square root is multiplied by the sample standard deviation.

SEMs for the different groups also are presented in Table 18. These SEMs were used to create confidence intervals. A confidence interval represents a range or band of scores, surrounding an observed score, in which the individual's "true" score is believed to fall. The 80% (.80) confidence interval is obtained by multiplying the SEM by a z-score of 1.28 and the 90% (.90) confidence interval is obtained by multiplying the SEM by a z-score of 1.64.

For college men, the 80% confidence interval for the total score is approximately +/- 4 points (i.e., 3.3) and the 90% confidence interval is approximately +/- 5 points (i.e., 4.26).

Test-retest reliability was examined in 82 concussed high school and college athletes. They completed the scale within 2 days of their concussion and again within 4 days. The test-retest reliability in this sample was .80. Notably, their mean score at time 1 was 24.6 and their mean score at time 2 was 12.0.

As seen in Figures 1 - 3, the distributions of total scores are skewed. With this degree of skew, forced-normalization of the distributions will (a) distort the true nature of the construct being measured; that is, healthy young people's total symptoms are not normally distributed in the population, and (b) result in increased interpretation error.

Therefore, the natural distribution of scores was examined and classification ranges were created that reflect proportions of normative subjects. Classification descriptors were created that reflect raw score ranges and percentile rank ranges in the natural distribution of scores. For example, in Table 19, 40.5% of high school boys obtained a total score of zero on the scale. Thus, a score of zero would be considered "Low - Normal". In contrast, only 10% scored 14 or higher, so scores between 14 and 21 are considered "High" and scores of 22 or greater are considered "Very High."

The classification ranges for high school and university students in regular education are presented in Tables 19 - 22. The ranges for those with a history of special education are presented in Tables 23 - 26. The sample of high school girls with a history of special education is very small; this table is provided for general information (Table 24). We recommend using Table 3 for all high school girls.