1. An assessment of the self-reported version of the Swedish Strengths
and Difficulties Questionnaire among children and adolescents 12-16
years old
Kenisha S. Russell Jonsson
Irina Vartanova
2. SDQ studies
• Study type 1: Examination of the psychometric properties (alpha
coefficients)
Internal consistency
Retest reliability ??
• Study type 2: Factor stucture (factor analysis & SEM)
Controversy of the five versus three structure versus bifactor
• Study type 3: Validity (ROC Analysis, mean comparison)
specificity & sensitivity
Convergent validity ??
3. Data
• Community Sample
Survey of children and young peoples mental health (Grodan) conducted in
2009, collected data from students in grade 6 and 9 (roughly between age 11-
17). In total there were 172,000 respondents.
• Service Contact Sample
During 1 mars – 30 september 2014 data a collected from 2 648 children and
young people from 27 municipalities in Sweden who visited a healthcare
center.
4. Psychometric properties (1)
Internal consistency reliability (Cronbachs Alpha) of the total difficulty scores and
subscores
SDQ scale Community Service
Contact
Widenfelt
et al. (2003)
Goodman
(2001)
Koskelainen
et al. (2000)
Total difficulties 0.63 0.56 0.70 0.80 0.71
Emotional
symptoms
0.69 0.67 0.63 0.66 0.69
Conduct problems 0.55 0.55 0.47 0.60 0.57
Hyperactivity-
inattention
0.66 0.71 0.66 0.67 0.66
Peer Problems 0.54 0.59 0.39 0.41 0.63
Prosocial 0.68 0.67 0.60 0.66 0.69
9. Validity (3) ROC Analysis
Receiver operating curves
In a ROC curve the true positive rate (Sensitivity) is plotted as a
function of the false positive rate (100-Specificity) for different cut-
off points of a parameter.
The area under the ROC curve (AUC)
a measure of how well a parameter can distinguish between two
diagnostic groups (community/service contact)
a method for reducing the entire ROC curve to a single
quantitative index of diagnostic accuracy
10. Validity (4) Caseness
True positive: cases
with condition
classified as positive
False positive: cases
without condition
classified as positiveFalse negative:
cases with condition
classified as negative
True negative: cases
without condition
classified as negative
11. Validity (5) Sensitivity–Specificity Report
Emotional Problems: Detailed report of sensitivity and specificity
Cutpoint Sensitivity Specificity
Correctly
Classified LR+ LR-
( >= 0 ) 100.00% 0.00% 0.53% 1.0000
( >= 1 ) 96.75% 16.52% 16.94% 1.1590 0.1965
( >= 2 ) 91.64% 35.90% 36.19% 1.4295 0.2330
( >= 3 ) 81.65% 54.18% 54.33% 1.7821 0.3387
( >= 4 ) 71.79% 68.67% 68.68% 2.2909 0.4109
( >= 5 ) 55.93% 79.78% 79.65% 2.7661 0.5524
( >= 6 ) 41.95% 87.91% 87.67% 3.4693 0.6604
( >= 7 ) 27.97% 93.21% 92.86% 4.1158 0.7729
( >= 8 ) 17.10% 96.42% 96.00% 4.7774 0.8597
( >= 9 ) 7.37% 98.42% 97.94% 4.6694 0.9412
( >= 10 ) 3.00% 99.38% 98.87% 4.8338 0.9761
( > 10 ) 0.00% 100.00% 99.47% 1.0000
Cutpoint:indicate the
rating used to classify
subjects with/without a
condition
Probability of correctly
classifying those with a
condition
Probability of correctly
classifying those without a
condition
The ratio of the
probability of a
negative test among
truly positive subjects
to the probability of a
negative test among
truly negative subjects
The ratio of the
probability of a positive
test among truly positive
subjects to the
probability of a positive
test among truly negative
subjects
18. Validity (12) ROC-Total difficulties
0.000.250.500.751.00
Sensitivity
0.00 0.25 0.50 0.75 1.00
1 - Specificity
Area under ROC curve = 0.7076
service contact versus community sample
Total Difficulties
19. Dilemma
• AUC low for some of the subscores -> community sample is too high
or the service contact sample is too low or vice versa.
How to solve this???
• Compare results with other countries (specifically nordic sample)
• Further analyses, restricting/more emphasis on service sample
reason for the visit
number of visit
who contacted the service center (parent/ child/teacher/other adult)
reason for contact
21. Explorative (EFA) vs Confirmative (CFA)
Factor Analysis
• In EFA, the factor structure is inferred from the obtained correlation
matrix.
• In CFA,the obtained correlation matrix is compared with a specified
theoretical model.
• The result of comparison is goodness of fit of the specified model.
Thus, we can compare different factor structures for better
understanding of the analyzed questionnaire.
22. EFA vs CFA
Correlation matrix
Factor structure
Correlation matrix
Theoretical model
compared
Model fit
23. Bifactor models – the latest suggestion of
model fit improvement
Kobor et al., 2013 Casi et al., 2015
24. Alternative models fit
Model chisq df RMSEA CFI TLI
Original 5-factor model 198,004 265 0.068 0.896 0.882
5-factor model with acquiescence style 125,592 259 0.051 0.942 0.932
Alternative 3-factor model 269,164 272 0.080 0.853 0.838
3-factor model with acquiescence style 232,670 268 0.073 0.880 0.865
Bifactor model (Kobor et al., 2013) 96,664 240 0.044 0.961 0.952
Bifactor model (Casi et al., 2015) 164,093 252 0.063 0.914 0.898
Different model fit
measures
Best fit model
Model currently
testing
25. Reference
Caci, H., Morin, A. J., & Tran, A. (2015). Investigation of a bifactor model of the Strengths and Difficulties Questionnaire. European child &
adolescent psychiatry, 24, pp 1291-1301.
Choi,B.C.K. 1998. Slopes of a receiver operating characteritic curve and the likelihood ratio for a diagnostic test.American Journal of
Epidemiology 148:1127-1132.
Di Riso, D., Salcuni, S., Chessa, D., Raudino, A., Lis, A., & Altoè, G. (2010). The Strengths and Difficulties Questionnaire (SDQ). Early evidence of
its reliability and validity in a community sample of Italian children. Personality and Individual Differences, 49(6), 570-575.
Essau, C. A., Olaya, B., Anastassiou‐Hadjicharalambous, X., Pauli, G., Gilvarry, C., Bray, D., ... & Ollendick, T. H. (2012). Psychometric properties of
the Strength and Difficulties Questionnaire from five European countries. International journal of methods in psychiatric research, 21(3), 232-
245.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: a research note. Journal of child psychology and psychiatry, 38(5), 581-586.
Goodman, R., Meltzer, H., & Bailey, V. (1998). The Strengths and Difficulties Questionnaire: A pilot study on the validity of the self-report
version. European child & adolescent psychiatry, 7(3), 125-130.
Goodman, A., Lamping, D. L., & Ploubidis, G. B. (2010). When to use broader internalising and externalising subscales instead of the
hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. Journal of
abnormal child psychology,38(8), 1179-1191.
Hanley,J.A and B.J. McNeil.1982.The meaning and the use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:9-
36.
Kóbor, A., Takács, Á., & Urbán, R. (2013). The bifactor model of the Strengths and Difficulties Questionnaire. European Journal of Psychological
Assessment, 29, pp. 299-307.
