1. The Art and Science of Test Development—Part F Psychometric/technical statistical analysis: Internal The basic structure and content of this presentation is grounded extensively on the test development procedures developed by Dr. Richard Woodcock Kevin S. McGrew, PhD. Educational Psychologist Research Director Woodcock-Muñoz Foundation

2. “ In god we trust….all others must show data” (unknown source) Test authors and publishers have standards-based responsibility to provide supporting psychometric technical information re: tests and battery Typically in the form of a series of technical chapters in manual or a separate technical manual

4. Calculate summary statistics (n, means, SDs, SEM) and reliabilities for all tests and clusters by technical age groups etc… etc…

5. Special reliability analyses required for speeded tests Traditional test-retest reliability analysis

6. Special reliability analyses for all tests More complex repeated measures reliability analysis (McArdle and Woodcock, 1989—see WJ-R Technical Manual)

7. Provide evidence based on internal structure (internal validity)

9. etc… Structural/internal validity evidence : Test and cluster inter-correlation matrices by technical age groups etc…

10. Structural/internal validity Confirmatory factor analysis by major age groups ( exploratory factor analysis if not theory-driven test blueprint)

11. Structural/internal validit y Confirmatory factor analysis by major age groups ( exploratory factor analysis if not theory-driven test blueprint) .67 .53 .40 .42 .43

14. The WJ III factor structure model provided the best fit to the data when compared to six alternative models The conclusion was the same across 5 age-differentiated samples Structural/internal validity: Confirmatory factor analysis model comparisons by major age groups

15. Internal validity evidence example: g-loadings for differentially weighted General Intellectual Ability cluster

16. Provide evidence based on internal structure: Developmental evidence?

17. Developmental evidence in the form of differential growth curves of measures

18. Provide Test Fairness Evidence

19. = White Non-White Structural/internal validity Evaluating structural invariance with Multiple Group CFA

20. = Male Female Structural/internal validity Evaluating structural invariance with Multiple Group CFA

21. = Hispanic Non-Hispanic Structural/internal validity Evaluating structural invariance with Multiple Group CFA

27. End of Part F Additional steps in test development process will be presented in subsequent modules as they are developed