Colloquium presentation for December 2014 "Rigor vs. Time: A Study of Instructional Benefits with Intelligent Tutoring Systems for students with Persistent Deficits in Mathematics" presented by Kareem Savoy
Rigor vs. Time: A Study of Instructional Benefits with Intelligent Tutoring Systems for students with Persistent Deficits in Mathematics
1. By
Kareem Savoy
RIGOR VS. TIME: A STUDY OF INSTRUCTIONAL
BENEFITS WITH INTELLIGENT TUTORING
SYSTEMS FOR STUDENTS WITH PERSISTENT
DEFICITS IN MATHEMATICS
2. • 1979 | IDENTIFIED LEARNING DISABILITIES
• 1990 | CALIFORNIA STATE UNIVERSITY OF RIVERSIDE
• 1999 | BEGAN MY TEACHING CAREER
• 2003 | NATIONAL UNIVERSITY
• 2014 | CALIFORNIA STATE UNIVERSITY, FULLERTON
RESEARCHER'S BACKGROUND
3. THE TIME NEEDED TO LEARN
(Carroll, 1963, 1989; Crawford, Carpenter, Wilson, Schmeister, & McDonald, 2012; Miller &
Mercer, 1997).
The COST TO HUMAN CAPITAL
(Acosta & Martin, 2013; Mastropieri, Scruggs, & Chung, 1998; Miller & Mercer, 1997;
Rumberger & Palardy, 2005).
PERSISTENT DEFICITS IN MATHEMATICS
ETIOLOGY
(Kovas, Haworth, Petrill, & Plomin, 2007; Mazzocco & Thompson, 2005; Miller & Mercer, 1997; Murphy et al.,
2007; Watson & Gable, 2013)
COMPUTATION, PROCEDURES, AND CONTEXT
(Fuchs, Fuchs, & Compton, 2012; Garnett, 1998; Geary, 1993; Geary, 2007, 2011a; Gersten, Chard, et al., 2009;
Gersten, Chard, et al., 2009; Gersten, Jordan, & Flojo, 2005; Mazzocco, 2005; Meyer et al., 2010; Montague & van
Garderen, 2008; Swanson & Jerman, 2006; Watson & Gable, 2013; Wilson & Swanson, 2001)
TOPIC BACKGROUND
4. INTELLIGENT TUTORING SYSTEMS (ITS)
POTENTIAL VS. PRODUCTION
TOOLS OF GOOD TEACHING
(Gersten et al., 2007; Koedinger & Corbett, 2006; Kulik, 2003)
ALEKS
INSTRUCTIONAL APPLICATION
(Canfield, 2001)
INSTRUCTIONAL APPLICATION
(National Mathematics Advisory Panel, 2008; Slavin et al., 2009)
DESIGN VS. INSTRUCTIONAL NEED
(Clark, 1963; 1989)
TOPIC BACKGROUND (CONTINUED)
9. THE PROBLEM ANALYZED HERE WAS THE LACK OF APPLICABLE
LITERATURE ON THE USE OF INTELLIGENT TUTORING SYSTEMS
(ITS) WITH STUDENTS EXHIBITING DEFICITS PERFORMANCE IN
MATHEMATICS (PDM).
Lack of understanding variables influencing instruction
Engagement Time (Carroll, 1963)
The ability to learn (Carroll, 1963)
MEASURING OUTCOME
ESTABLING PARAMETERS OF IMPLEMNTATION
MEASURING FIDELITY OF USE
(Allsopp et al., 2010; Balfanz et al., 2007; Carroll, 1963; Cheung and Slavin, 2013;
Chong and Siegel, 2008; Fuchs et al., 2008a; Fuchs and Fuchs, 2007; Fuchs, 2009;
Miller and Mercer, 1997; Murphy et al., 2007; Watson and Gable, 2013)
PROBLEM STATEMENT
10. THE PURPOSE OF THIS CROSS SECTIONAL ANALYSIS WAS TO
TEST THE CAPACITY OF CARROLL’S (1963) “MODEL OF SCHOOL
LEARNING.”
Regarding:
the time needed to learn
The use of Assessment and Learning in Knowledge Spaces (ALEKS)
To improve achievement of students with PDM.
PURPOSE OF RESEARCH
11. What effect does Engagement Time, with Assessment and
Learning in Knowledge Spaces, have on the achievement of
students with persistent deficits in mathematics (PDM)?
RESEARCH QUESTION
12. REVIEW OF THE LITERATURE
Major Sections Highlighted Works from Each Section
Characterizing Performance
• Mimicking Abilities Andersson, (2007), Anderssen(2008); Aunola, Leskinen,
Lerkkanen, & Nurmi, (2004); Brown, (2013); Bryant et al.,
2008; Fuchs et al., (2005); Fuchs, Fuchs, & Compton, (2010)
• Perceptions in Performance Cheung & Slavin,(2013); Craig et al. (2011); Li & Ma,(2010);
Min & VanLehn, (2010); Steenbergen-Hu & Cooper, (2013)
• Outcome Measures Ding and Davison (2005) ; Gersten, Chard, et al., (2009); Li &
Ma,(2010); Steenbergen-Hu and Cooper (2013)
• Curriculum in Context Fuchs, (2009); Miller & Mercer, (1997); Tamim, Bernard,
Borokhovski, Abrami, & Schmid, (2011); What Works
Clearning House (2013)
Engagement Time Crawford et al., (2012); Gersten, Chard, et al. (2009);
Steenbergen-Hu and Cooper (2013); Slavin et al. (2009)
13. Quantitative
•(Creswell, 2013; Duffy &
Chenail, 2009; Paul & Marfo,
2001) Descriptive Study
•Bickman and Rog, (2009); Creswell, (2009)
Pretest Posttest Design
•Duffy and Chenail, (2009); Paul and Marfo, (2001)
Archival Data
Reasonable Assumptions of
Causation
•(Creswell, 2013)
METHODOLOGY
Controlling for:
• Gender
• Socioeconomics
• Prior performance
• Attendance
14. Quantitative Cross-Sectional
Analysis
Archival
Data
Variable
influencing
Performance
ALEKS
RESEARCH DESIGN
MEASURE OF CENTRAL TENDENCIES
CORRELATION
REGRESSION
Y= b0 + b1X (gender) + b2X (free/reduced
lunch)+ b3X (reading achievement) + b4X
(math achievement) + b5X (attendance)
+ b6X (intervention) + εi
Y= Achievement outcome
b0 = Interception of time and achievement
B1-6= Gender, Socioeconomics, Prior
Performance, Attendance
X7 = Time engagement (in minute
increments)
εi = Error (everything else not explained by
the model
15. SETTING
Fisher Creek Unified School
District (FCUSD)
SAMPLE
138 9th Grade Students
Site A = 39
Site B = 59
Site C = 40
CRITERIA
Performing below the 25th
Percentile on CST
for 2 consecutive years
INSTRUMENT
Algebra I Assessment
VARIABLES
Independent Variables
Engagement Time
Dependent Variables
Algebra I Assessment
Current GPA
Ending Mastery on ALEKS
RESEARCH METHODS
16. RESEARCH METHODS (CONTINUED)
DATA COLLECTION
Initial Assessment
Treatment Condition
20 Weeks
300 Minutes a Week
10 Skills Mastered Per
Week
Post Assessment
DATA ANALYSIS
Measures of Central
Tendencies
Correlation
Regression
VALIDITY
Sample Size
Statistical Package for
the Social Sciences
(SPSS)
ROLE OF THE
RESEARCHER
Association
Communication
29. INTERPRETATION AND IMPLICATION
• Outcome Measures
• Fidelity to Expectation
• Engagement Time
Skills Mastered Per/Hour
• Policy
• Practice
• Theory
30. Fidelity of implementation
Protects the opportunities to learn with ALEKS
Modification of interpretation with outcome
parameters
A longitudinal analysis of growth rate, standardized
achievement, and curriculum based measure of
achievement
RECOMMENDATIONS
31. Acosta, R. A., & Mar tin, E . J. (2013). Califor n ia Ur ban Cr isis and Fiscal De cline : Tr e nds in High School
Dr opout Rate s and E conomic Implica t ions .
Allsopp, D. H., Alvar e z McHatton, P., & Cr anston-Ging ras , A. (2009). Examini ng Perceptions of Systemat ic
Inte gr ation of Instr uctiona l Te chnology in a Te ache r E ducation Pr ogr am. Teache r Education and Special
E ducatio n : The Jo urn al o f the Teache r E ducatio n Divisio n o f the Co un cil fo r E xceptio n al Children , 32(4), 337 -
350. doi: 10.1177/0888 40 64 09 34 61 44
Allsopp, D. H., Alvar e z McHatton, P., & Far me r , J. L. (2010). Te chnology, mathe mat ics ps/rti, and students
with ld: What do we know, what have we tried, and what can we do to improve outcomes now and in the
futur e ? . Learning Disabili ty Quarter ly, 33 (4), 273 -288. doi: 10.1177/0731 94 87 10 033 00 40 5
Bick man, L., & Rog, D. J. E . (2009). The Sage handbook of applied social research methods : Sage.
Bryant, D. P., Bryant, B. R., Gersten, R., Scammacca , N., & Chavez, M. M. (2008). Mathemat i cs Intervent ion
for Fir st- and Second -Grade Students With Mathemat ics Difficu l t ies : The Effects of Tier 2 Intervent ion
De live r e d as Booste r Le ssons. Remedi al and Special Education, 29 (1), 20 -32. doi:
10.1177/07419 32 50 73 09 712
Califor n i a De par tme nt of E ducation Asse ssme nt De ve lop me n t and Administ r a t io n Divis ion. (2013). Califor n i a
Standar ds Te sts Te chnical Re por t Spr ing 2012 Administ r a t i on.
Canfie ld, W. (2001). ALE KS-A web -based intell i gen t tutoring system. Mathem at ic s and Computer Education,
3 5(2), 152 -158.
Car r oll, J. B. (1963). A mode l of school le ar ning. The Teache r s College Record, 64 (8), 723 -723.
Carroll, J. B. (1989). The Carroll model a 25-year retrospecti ve and prospective view. Education al
Resea rch e r , 18(1), 26 -31. doi: 10.3102/0013 18 9X0 18 00 10 26
Che ung, A. (2013). E ffe cts of E ducational Te chnology Applicat ions on Stude nt Achie ve m e n t for
Disadvantage d Stude nts: What For ty Ye ar s of Re se ar ch Te lls Us. Cypriot Journal of Education al Sciences,
8 (1), 19 -33.
Che ung, A. C. K., & Slav in, R. E . (2013). The e ffe ctive ne ss of e ducational te chnology applicat io ns for
e nhancing mathe mat ics achie ve m e nt in K-12 classrooms: A meta -analys is. Education al Rese arc h Review , 9 ,
8 8-113. doi: 10.1016/j.edu re v. 20 13 .0 1.0 01
REFERENCES
32. Chong, S. L., & Sie ge l , L. S. (2008). Stabil i ty of computation a l de ficits in math le ar ning disabil i ty fr om
se cond thr ough fifth gr ade s. Dev Neuropsycho l , 33 (3), 300 -317. doi: 10.1080/875656 40 80 19 82 387
Clar k, R. E . (1983). Re conside r i ng Re se ar ch on Le ar ning fr om Me dia. Review of Educational Rese arc h, 53 (4),
4 4 5-459. doi: 10.3102/0034 65 430 53 00 44 45
Clar k, R. E . (1994). Me dia will ne ve r influe nce le ar ning. Education al Technology Research and Developm e nt ,
4 2(2), 21 -29. doi: 10.1007%2 F B F0 22 990 8
Ding, C. S., & Davison, M. L. (2005). A longitud ina l study of math achie ve me n t gains for initia l ly low
achie ving stude nts. Contempor ary Education al Psychology, 30 (1), 81 -95. doi:
10.1016/j.ce d psych. 20 04. 06 .00 2
E ducation, C. D. o. (2013, 08/08/). 2012 STAR Te st Re sults: Te r m and Scor e E x planations. fr om
http://star.cde.ca. go v/s ta r20 12/ he l p_sco reexp l an at i ons. aspx
Fuchs, D., Mock, D., Mor gan, P. L., & Young, C. L. (2003). Re sponsive ne ss- to- In ter v en t ion : Definit i ons,
E vide nce , and Implicat i ons for the Le ar ning Disabi l i t ie s Constr uct. Learnin g Disabilit i e s Rese arc h and
P ractice, 18(3), 157 -171. doi: 10.1111/1540 -5 82 6.0 00 72
Fuchs, L., & Fuchs, D. (1995). Acquisit ion and tr ansfe r e ffe cts of classwide peer -assisted learning strategies
in mathe mat ics for stude nts with var ying le ar ning histor ie s. School Psychology Review , 24 (4), 604 -620.
Fuchs, L. S., & Fuchs, D. (2007). A mode l for imple m e n t ing r e sponsive ne ss to inte r ve nt i on. Teachin g
E xceptio n al Children , 39(5), 14-20.
Fuchs, L. S., & Fuchs, D. (2009). Cr e ating oppor tunit i e s for inte nsive inte r ve n t ion for stude nts with le ar ning
disabil i t ies . Teachin g Exception al Children , 42 (2), 60 -62.
Ge ar y, D. C. (1993). Mathe mat ic a l disabil i t i e s: cognitiv e , ne ur opsycholog ica l , and ge ne tic compone nts.
P sycho lo gic al bulletin , 114(2), 345.
Ge ar y, D. C. (2005). Role of Cognitiv e The or y in the Study of Le ar ning Disabil i ty in Mathe mat i cs. Journal of
L earn in g Disabiliti e s , 38(4), 305-307. doi: 10.1177/0022 21 94 050 38 00 40 40 1
Ge ar y, D. C. (2007). An e voluti ona r y pe r spe ctive on le ar ning disabil i ty in mathe mat i cs. Dev Neuropsycho l ,
3 2(1), 471 -519. doi: 10.1080/8756 564 07 01 36 09 24
Ge ar y, D. C. (2011a). Cogniti ve pr e dictor s of achie ve me nt gr owth in mathe mat ics: a 5-year longitudi na l
study. Dev Psychol , 47 (6), 1539 -1552. doi: 10.1037/a0025 51 0
33. Geary, D. C. (2011b). Consequences, characterist ics, and causes of mathemat i ca l learning
disabil i t ies and persistent low achieveme nt in mathemat ics. J Dev Behav Pediat r , 32 (3), 250 -263. doi:
10.1097/DB P. 0b0 13 e31 82 09 ede f
Kulik, J. A. (2003). Effects of using instruct iona l technology in elementary and secondary schools:
What controlled evaluation studies say. Arlington: Citeseer .
Li, Q., & Ma, X. (2010). A Meta -analys is of the Effects of Computer Technology on School Students’
Mathemat ics Learning. Education al Psychology Review, 22 (3), 215-243. doi: 10.1007/s10648 - 01 0 -9 12 5 - 8
Mazzocco, M. M., & Thompson, R. E. (2005). Kindergarten Predictors of Math Learning Disabil i ty.
Learn Disabil Res Pract , 20 (3), 142 -155. doi: 10.1111/j.15 40 - 58 26 .2 005 .0 01 29.x
Murphy, M. M., Mazzocco, M. M. M., Hanich, L. B., & Early, M. C. (2007). Cognitive Characterist ics of
Children With Mathemat ics Learning Disabil i ty (MLD) Vary as a Function of the Cutoff Criteri on Used to Define
MLD. Journal of Learning Disabiliti e s , 40 (5), 458 -478. doi: 10.1177/0022 21 940 70 40 00 50 901
National Center for Education Statist ics. (2013). The nation's report card: Trends in academic
progress (Vol. 79, pp. 59 -63): The Education Digest.
National Council of Teachers of Mathemat ics . (2013). A Vision for School Mathemat ics. from
http://www.nc t m .or g/st and ards /con ten t.as px? id =3 43 4
National Mathemat i cs Advisory Panel. (2008). Foundations for Success: The Final Report of the
National Mathemat i cs Advisory Panel.
Slavi n, R. E., Lake, C., & Groff, C. (2009). Effective Programs in Middle and High School
Mathemat ics : A Best -Evid ence Synthesis. Revie w of Education al Research, 79 (2), 839 -911. doi:
10.3102/00346 54 30 83 30 968
Tienken, C. H., South Orange, N., Maher, J. A., & Consulting, M. (2008). The influence of computer -
assisted instructio n on eighth grade mathemat i cs achievemen t. RMLE Online, 32 (3).
Watson, S. R., & Gable, R. A. (2013). Unrave l in g the Complex Nature of Mathemat ics Learning
Disabil i ty: Implicat i ons for Research and Practice. Learning Disabili ty Quarter ly, 36 (3), 178 -187. doi:
10.1177/07319 48 71 24 61 489
Wilson, K. M., & Swanson, H. L. (2001). Are mathemat ics disabil i t ies due to a domain -gen era l or a
domain-spec i f ic working memory deficit? Journal of Learning Disabiliti e s , 34 (3), 237 -248.
