print this section

Appendix E: Calculating Statistical Difference in Scores

This appendix outlines the instructions to manually calculate and determine the statistical significance of differences in scores between raters and differences in scores across time. To compare scores between raters, critical values for the Conners 4™ and the Conners 4™–Short are provided. To compare scores (from the same rater) across time, Reliable Change Index (RCI) values for both T-scores and raw scores of the Conners 4™ and the Conners 4™–Short are provided.


Differences Between Raters

To compare the Conners 4 T-scores obtained from different raters, the values needed to establish statistical significance were calculated using the formula provided by Anastasi and Urbina (1997):


Difference formula


This formula takes into account the standard error of measurement (SEM) for each of the scales (see chapter 8, Reliability, for the SEM values for the Conners 4 scales).

Tables E.1a to E.1f provide the critical values needed to determine the statistical significance between a pair of T-scores at the p < .05 level of significance for the full-length Conners 4, while values are presented for the Conners 4–Short in Tables E.2a to E.2f.

To use these tables:

  1. Find the appropriate table based on the length of the form (full-length or short), the type of Normative Sample selected for scoring (Combined Gender or Gender Specific), and the age of the youth.

  2. Find the column for the relevant pair of raters (e.g., Parent to Self-Report, Teacher to Self-Report) and age group.

  3. Find the row that contains the scale that is being compared. To be significant, the absolute difference between the two raters’ standard scores on this scale must be equal to or greater than the value in the table.

For example, a clinician asked two different teachers of a 15-year-old female to rate her on the full-length Conners 4. The clinician then wanted to know if the ratings of the two teachers were significantly different from each other and wanted to use the Normative Sample Gender Specific–Females as the principal reference sample. If the Inattention/Executive Dysfunction standard score is 84 when rated by her math teacher and 70 when rated by her music teacher, this 14-point difference is compared to the value in Table E.1f (i.e., the table for the selected reference sample). This 14-point difference is statistically significant (on the Inattention/Executive Dysfunction Scale, a difference of 5 or more points is needed to be considered statistically significant). This result means that her math teacher and her music teacher provided reliably different ratings. The next step is to consider the different contexts in which the youth’s behaviors related to the Inattention/Executive Dysfunction Scale are being observed by the two raters and to explore what may have led to the different scores.


Click to expand

Table E.1a. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample–Combined Gender (Ages 6 to 11)

Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

5

5

4

6

6

5

5

5

7

6

5

5

4

Hyperactivity

7

8

6

9

8

7

8

7

9

8

7

8

6

Impulsivity

7

8

6

10

9

8

8

7

10

9

8

9

7

Emotional Dysregulation

6

8

5

9

8

6

7

5

8

7

6

7

5

Depressed Mood

8

8

8

10

10

9

9

9

11

10

9

10

8

Anxious Thoughts

9

10

8

9

9

9

9

9

9

9

9

10

9

Impairment
& Functional Outcome Scales

Schoolwork

8

9

8

10

10

8

9

7

11

10

8

9

7

Peer Interactions

9

10

7

10

10

8

9

7

11

10

8

8

7

Family Life

7

10

7

9

7

DSM
Symptom Scales

ADHD Inattentive Symptoms

7

8

6

8

8

7

7

6

9

8

6

7

6

ADHD Hyperactive/
Impulsive Symptoms

7

7

6

8

8

7

8

6

9

8

7

8

6

Total ADHD Symptoms

5

6

5

6

6

6

6

5

7

6

6

6

5

Oppositional Defiant Disorder Symptoms

6

7

5

8

8

6

7

6

8

7

6

7

5

Conduct Disorder Symptoms

6

6

7

7

7

6

5

7

6

7

5

5

6

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.1b. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample–Combined Gender (Ages 12 to 18)

Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

6

6

5

5

5

6

5

5

5

4

6

6

5

5

5

Hyperactivity

8

8

8

8

7

8

8

7

7

6

9

9

8

9

7

Impulsivity

9

9

7

8

7

8

8

7

7

7

10

9

8

9

7

Emotional Dysregulation

8

7

6

6

5

8

7

6

6

6

8

8

6

6

6

Depressed Mood

9

9

9

9

8

8

9

8

8

9

9

9

9

9

9

Anxious Thoughts

8

10

10

8

11

8

9

9

8

10

9

9

9

9

10

Impairment & Functional Outcome Scales

Schoolwork

10

9

8

8

7

10

9

8

9

7

11

9

9

10

7

Peer Interactions

11

11

8

8

8

10

11

9

8

10

11

11

8

7

9

Family Life

10

 —

7

 —

8

6

9

7

DSM
Symptom Scales

ADHD Inattentive Symptoms

8

7

6

7

6

7

7

6

7

6

8

8

6

7

6

ADHD Hyperactive/
Impulsive Symptoms

8

8

7

7

7

8

7

6

7

6

9

8

8

8

7

Total ADHD Symptoms

6

6

5

6

5

6

6

5

6

5

7

6

6

6

5

Oppositional Defiant Disorder Symptoms

7

7

6

6

5

8

7

6

7

6

8

8

6

7

6

Conduct Disorder Symptoms

7

7

6

6

6

6

6

6

5

6

6

6

6

6

6

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.1c. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Males (Ages 6 to 11)

Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

5

6

5

6

6

5

5

5

7

6

5

6

5

Hyperactivity

7

8

6

8

7

7

8

6

9

9

7

8

6

Impulsivity

7

8

7

9

9

7

8

7

9

9

8

9

7

Emotional Dysregulation

6

8

5

9

8

6

7

4

8

7

6

7

4

Depressed Mood

9

10

8

10

11

8

8

9

11

10

9

10

7

Anxious Thoughts

9

11

8

8

9

9

8

9

10

9

9

10

9

Impairment & Functional Outcome Scales

Schoolwork

8

8

8

10

9

8

8

7

10

10

8

8

8

Peer Interactions

9

10

7

10

10

8

8

7

11

10

8

9

7

Family Life

6

10

7

9

7

DSM
Symptom Scales

ADHD Inattentive Symptoms

7

8

6

8

8

6

6

6

9

8

7

7

6

ADHD Hyperactive/ Impulsive Symptoms

7

7

6

8

7

6

7

6

9

8

7

8

6

Total ADHD Symptoms

5

6

5

6

6

5

5

5

7

6

6

6

5

Oppositional Defiant Disorder Symptoms

6

7

5

8

8

6

7

6

8

7

6

7

4

Conduct Disorder Symptoms

10

13

7

8

10

9

7

10

9

9

7

8

5

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.1d. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Males (Ages 12 to 18)

Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

5

5

5

5

5

6

5

5

5

5

6

6

5

5

5

Hyperactivity

8

8

7

8

7

8

8

7

7

6

9

9

8

9

8

Impulsivity

9

9

8

8

7

8

8

7

7

6

9

8

8

9

7

Emotional Dysregulation

7

7

6

6

5

7

7

6

6

6

7

7

6

6

6

Depressed Mood

9

9

9

9

8

8

9

8

7

9

9

10

9

8

10

Anxious Thoughts

8

9

8

8

9

8

9

9

8

10

9

10

9

9

10

Impairment & Functional Outcome Scales

Schoolwork

9

9

7

8

7

10

9

8

8

7

10

10

8

9

7

Peer Interactions

10

11

7

7

8

10

11

9

7

10

11

11

8

7

9

Family Life

9

6

8

6

8

6

DSM
Symptom Scales

ADHD Inattentive Symptoms

7

7

6

6

6

7

7

6

7

6

8

8

7

7

6

ADHD Hyperactive/
Impulsive Symptoms

8

8

7

7

7

8

7

6

6

6

9

9

8

8

7

Total ADHD Symptoms

6

6

5

5

5

6

6

5

6

5

7

7

6

6

6

Oppositional Defiant Disorder Symptoms

7

7

6

6

6

7

7

6

6

6

7

7

6

6

6

Conduct Disorder Symptoms

10

10

9

9

9

10

9

10

10

9

9

8

8

9

6

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.1e. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Females (Ages 6 to 11)

Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

5

5

5

7

6

5

6

5

6

6

5

5

5

Hyperactivity

7

8

6

9

9

7

9

7

8

8

7

8

7

Impulsivity

7

8

6

10

9

7

9

7

10

9

8

8

7

Emotional Dysregulation

7

7

6

9

9

7

7

6

8

8

7

7

6

Depressed Mood

8

8

9

11

10

8

10

8

10

9

9

10

8

Anxious Thoughts

9

9

8

10

9

9

10

9

9

9

9

9

9

Impairment & Functional Outcome Scales

Schoolwork

9

10

8

11

11

9

10

7

11

10

8

9

6

Peer Interactions

8

10

7

10

9

8

10

7

10

10

7

8

7

Family Life

7

10

8

9

7

DSM
Symptom Scales

ADHD Inattentive Symptoms

7

7

6

9

9

7

8

6

8

8

6

7

6

ADHD Hyperactive/
Impulsive Symptoms

6

7

5

9

8

6

8

6

8

8

7

7

7

Total ADHD Symptoms

5

6

5

7

7

5

7

5

7

6

6

6

5

Oppositional Defiant Disorder Symptoms

6

7

5

9

8

6

7

5

8

7

6

7

5

Conduct Disorder Symptoms

8

6

9

7

8

8

6

8

7

9

9

6

11

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.1f. Conners 4 Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Females (Ages 12 to 18)

Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

6

6

5

5

5

6

6

5

5

5

6

6

5

5

5

Hyperactivity

9

9

8

8

7

8

8

7

8

6

9

8

8

8

7

Impulsivity

9

9

7

8

7

9

8

7

7

7

10

9

8

8

7

Emotional Dysregulation

8

7

6

7

6

8

7

7

7

6

8

8

6

6

6

Depressed Mood

9

9

8

9

8

8

8

8

8

9

9

9

9

10

9

Anxious Thoughts

8

10

11

9

12

8

8

9

9

9

8

8

9

9

9

Impairment & Functional Outcome Scales

Schoolwork

10

9

8

9

7

10

9

8

9

7

11

10

9

11

8

Peer Interactions

11

10

8

8

8

11

11

9

9

10

10

11

8

7

9

Family Life

10

6

9

7

10

8

DSM
Symptom Scales

ADHD Inattentive Symptoms

8

8

7

7

6

8

7

6

7

6

8

8

7

7

6

ADHD Hyperactive/
Impulsive Symptoms

9

9

7

8

6

8

8

6

7

6

9

8

7

8

6

Total ADHD Symptoms

7

6

5

6

5

6

6

5

5

5

7

7

6

6

6

Oppositional Defiant Disorder Symptoms

8

7

5

6

5

8

7

6

7

6

9

8

7

7

6

Conduct Disorder Symptoms

8

7

6

7

5

10

11

6

5

7

11

10

11

12

10

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2a. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample–Combined Gender (Ages 6 to 11)

Conners 4–Short Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

6

6

6

9

9

7

7

6

9

9

7

7

6

Hyperactivity

10

11

8

11

9

10

11

8

11

10

9

11

8

Impulsivity

9

10

9

11

11

10

11

10

11

11

10

11

9

Emotional Dysregulation

8

10

6

11

10

8

9

6

11

10

8

9

7

Impairment & Functional Outcome Scales

Schoolwork

9

9

8

11

11

8

8

8

11

10

7

8

7

Peer Interactions

8

8

7

10

9

7

8

6

10

9

7

9

6

Family Life

7

10

7

9

7

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2b. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample–Combined Gender (Ages 12 to 18)

Conners 4–Short Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

8

8

6

7

6

8

8

7

7

6

9

8

7

7

6

Hyperactivity

11

10

10

11

9

10

9

9

11

7

9

6

10

12

8

Impulsivity

12

11

10

10

9

10

10

9

9

9

12

12

11

12

10

Emotional Dysregulation

9

9

8

8

8

10

10

8

8

8

10

10

8

8

8

Impairment & Functional Outcome Scales

Schoolwork

9

9

7

7

7

9

9

7

7

6

9

10

8

7

8

Peer Interactions

11

11

8

8

8

10

11

9

8

10

12

12

9

9

9

Family Life

10

6

8

7

9

7

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2c. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Males (Ages 6 to 11)

Conners 4–Short Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

7

7

7

8

8

6

6

6

9

9

7

7

6

Hyperactivity

10

11

9

10

9

9

10

8

12

11

10

11

8

Impulsivity

10

11

9

11

10

9

10

9

11

11

10

11

9

Emotional Dysregulation

8

10

6

11

10

7

9

6

10

9

7

8

6

Impairment & Functional Outcome Scales

Schoolwork

9

9

8

10

10

8

8

8

11

10

9

9

8

Peer Interactions

8

8

8

10

10

7

8

6

10

10

7

8

6

Family Life

7

11

7

9

7

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2d. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Males (Ages 12 to 18)

Conners 4–Short Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

7

7

6

6

6

8

7

7

7

6

8

8

7

7

6

Hyperactivity

11

10

9

10

9

10

9

9

11

7

9

6

10

12

8

Impulsivity

12

11

10

10

10

9

10

9

8

9

11

11

11

12

10

Emotional Dysregulation

9

9

7

7

7

10

10

8

7

8

9

9

7

8

7

Impairment & Functional Outcome Scales

Schoolwork

9

9

6

6

6

9

9

6

6

7

9

9

8

7

8

Peer Interactions

11

10

8

8

7

9

10

8

7

10

12

12

9

8

9

Family Life

9

5

8

6

8

6

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2e. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Females (Ages 6 to 11)

Conners 4–Short Scale

6 to 7

8 to 9

10 to 11

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

7

7

6

9

9

7

7

7

9

9

7

7

6

Hyperactivity

9

11

7

12

10

11

13

9

11

10

9

10

8

Impulsivity

9

10

8

12

12

11

12

11

11

11

10

10

10

Emotional Dysregulation

9

10

7

12

11

9

10

7

11

11

9

9

8

Impairment & Functional Outcome Scales

Schoolwork

9

9

9

12

11

9

9

8

10

10

6

7

6

Peer Interactions

8

8

6

10

9

8

9

6

10

9

8

9

6

Family Life

7

10

7

9

7

Note. P = Parent; T = Teacher; S = Self-Report.


Click to expand

Table E.2f. Conners 4–Short Critical Values Denoting Statistically Significant Differences Between Raters: Normative Sample Gender Specific–Females (Ages 12 to 18)

Conners 4–Short Scale

12 to 13

14 to 15

16 to 18

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

P-S

T-S

P-T

P-P

T-T

Content Scales

Inattention/Executive Dysfunction

8

8

7

7

7

8

8

7

7

6

9

9

7

7

6

Hyperactivity

12

11

10

11

8

10

9

9

10

7

9

7

11

12

9

Impulsivity

12

11

10

10

9

10

11

9

9

9

13

12

11

12

10

Emotional Dysregulation

10

9

8

9

8

11

10

8

9

7

9

10

9

8

9

Impairment & Functional Outcome Scales

Schoolwork

9

10

8

8

8

10

9

7

8

6

10

10

8

8

8

Peer Interactions

11

11

9

9

8

11

12

9

8

10

12

11

9

10

9

Family Life

10

6

8

7

9

7

Note. P = Parent; T = Teacher; S = Self-Report.


Differences Across Time

Tables E.3 and E.4 provide the values needed to determine if scores on pairs of administrations are statistically different at the p < .05 level of significance for the Conners 4 and Conners 4–Short, respectively.

To use the tables:

  1. Select the rater type (Parent, Teacher, Self-Report) to determine which column to use.

  2. Find the row for the scale that is being compared. To be significant, the absolute difference between the two standard scores on this scale must be equal to or greater than the value in the table. A statistically significant result means that the measured change can be attributed to reliable differences between the scores, rather than random fluctuations in behavior or error in measurement. If the score has increased significantly from pre-test to post-test, then the individual’s score shows significant decline in behavior, or worsening of symptoms. If the score has decreased significantly, then the score shows significant improvement.

For example, a clinician asked a parent to rate their son using the full-length Conners 4 at Time 1 and again, a month later, at Time 2. The clinician wanted to compare the two ratings to determine whether the ratings are significantly different from each other. The parent’s rating on the full-length form resulted in a Time 1 Emotional Dysregulation T-score of 80 and a Time 2 T-score of 70. Using a significance level of p < .05, this 10-point difference between Time 1 and Time 2 is greater than the value of 7 shown in Table E.3, indicating statistically significant improvement. Conversely, the parent’s ratings on the Hyperactivity Scale (Time 1 = 74, Time 2 = 70) did not change significantly, because the 4-point difference between scores is smaller than the value of 9 shown in Table E.3. These results for Hyperactivity suggest that continued or different intervention may be needed.


Click to expand

Table E.3. Conners 4 Minimum Values Needed for Significance When Comparing Pre-test to Post-test T-scores: 95% Reliable Change Index (RCI)

Scale

Parent

Teacher

Self-Report

Content Scales

Inattention/Executive Dysfunction

7

10

16

Hyperactivity

9

9

13

Impulsivity

9

13

15

Emotional Dysregulation

7

9

13

Depressed Mood

12

10

14

Anxious Thoughts

10

12

14

Impairment & Functional Outcome Scales

Schoolwork

6

10

15

Peer Interactions

5

8

15

Family Life

9

11

DSM Symptom Scales

ADHD Inattentive Symptoms

7

11

17

ADHD Hyperactive/Impulsive Symptoms

7

9

13

Total ADHD Symptoms

6

9

14

Oppositional Defiant Disorder Symptoms

9

9

15

Conduct Disorder Symptoms

3

5

16


Click to expand

Table E.4. Conners 4–Short Minimum Values Needed for Significance When Comparing Pre-test to Post-test T-scores: 95% Reliable Change Index (RCI)

Conners 4–Short Scale

Parent

Teacher

Self-Report

Content Scales

Inattention/Executive Dysfunction

8

10

15

Hyperactivity

12

10

16

Impulsivity

12

13

15

Emotional Dysregulation

8

9

15

Impairment & Functional Outcome Scales

Schoolwork

8

11

17

Peer Interactions

7

12

20

Family Life

10

14


In most cases, the minimum values needed for significance should be used when comparing T-scores. Using the minimum values needed for significance when comparing raw scores is appropriate when a youth changed norm groups across administrations (i.e., had a birthday). Raw score critical values can be found in Tables E.5 and E.6 for the full-length Conners 4 and Conners 4–Short, respectively.


Click to expand

Table E.5. Conners 4 Minimum Values Needed for Significance When Comparing Pre-test to Post-test Raw Scores: 95% Reliable Change Index (RCI)

Scale

Parent

Teacher

Self-Report

Content Scales

Inattention/Executive Dysfunction

9

13

17

Hyperactivity

6

7

8

Impulsivity

5

7

7

Emotional Dysregulation

5

5

7

Depressed Mood

4

3

5

Anxious Thoughts

3

3

6

Impairment & Functional Outcome Scales

Schoolwork

3

4

6

Peer Interactions

2

3

5

Family Life

4

4

DSM Symptom Scales

ADHD Inattentive Symptoms

6

8

11

ADHD Hyperactive/Impulsive Symptoms

6

7

8

Total ADHD Symptoms

9

13

17

Oppositional Defiant Disorder Symptoms

6

6

8

Conduct Disorder Symptoms

3

3

7


Click to expand

Table E.6. Conners 4–Short Minimum Values Needed for Significance When Comparing Pre-test to Post-test Raw Scores: 95% Reliable Change Index (RCI)

Conners 4–Short Scale

Parent

Teacher

Self-Report

Content Scales

Inattention/Executive Dysfunction

6

7

9

Hyperactivity

4

4

6

Impulsivity

4

4

5

Emotional Dysregulation

3

0

4

Impairment & Functional Outcome Scales

Schoolwork

3

4

4

Peer Interactions

2

3

5

Family Life

3

4



<< Appendix D: DSM Symptom Scales Scoring Criteria Appendix F: Sample Descriptions >>