AERA 2013 Philip Piety, John Behrens, Roy Pea

1. Philip Piety, John Behrens, Roy Pea
American Education Research Association Annual Meeting
Monday, Apr 29 - 10:35am Parc 55 San Francisco / Divisadero Room

2. • What kind of profession will education
data sciences be?
• What are its ancestor, sister, and adjoining
disciplines?
• Which kinds of skills and dispositions are
important for preparing future practitioners
and scholars?

3. • Data exist inside a social context;
shaped by and shaping that context.

4. • Data exist inside a social context;
shaped by and shaping that context.
• Interpretation is not technical. It is
itself socially situated with
goals, predispositions/ biases, and
norms.

5. • Data exist inside a social context;
shaped by and shaping that context.
• Interpretation is not technical. It is
itself socially situated with goals,
predispositions/ biases, and norms.
• Professional communities have
developed valuable ways to reason
from imperfect evidence. We can
leverage/translate them to this new
sociotechnical terrain.

6. 1. Quantitative shifts in evidentiary artifacts (a
digital ocean) in education
2. Qualitative shifts in educational focus
3. Some contributing/relevant disciplines
4. Interpretive skills, how education data
scientists should approach data analysis?

8. Assessment
Technology
Computing
Technology
Central “Mainframe“
ComputingTabulating Technology
Cloud
Technology
Services
Traditional fixed response, short
task assessments
Analog Paper-based (Textbooks, worksheets, and manual classroom tools)
Classroom
Technology
Distributed
Integrated Assessment
Systems
Digital
Classroom
Technology
1850s 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 20101850s 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010

9. • Test scores
• Interim assessments
• In class, formative assessments
• Growth models
• Student collaboration
• Conversation records from classroom
talk and online tools
• Student work, including rich and
multimodal demonstrations of
knowledge and competency (essays,
presentations, etc.)
• Records of after-school experiences
• Records of informal learning
• Activity traces from digital media (in
school, out of school, etc.)
• Demographics
• Student-teacher relationships (TSDL)
• School improvement plans/goals
• Classifications (ex: proficiency groups)
• Video records of teaching
• Annotated/evaluated records of
teaching
• Teacher evaluations
• Individual Education Plans (IEPs) and
personalized learning maps
• Geospatial information
(mapping and trends)
• Attendance and rosters (more
important than you think!)
• FERPA/privacy blocks

10. Where to Begin?

11. Influenced by concurrent work with behrens, Mislevy, and DiCerbo for the
Learning Analytics Workgroup.

12. Structures &
Interrelationships
Influenced by concurrent work with behrens, Mislevy, and DiCerbo for the
Learning Analytics Workgroup.

13. Structures &
Interrelationships
Diachronic/Change
Processes
Influenced by concurrent work with behrens, Mislevy, and DiCerbo for the
Learning Analytics Workgroup.

14. Structures &
Interrelationships
Diachronic/Change
Processes
Variations in
Affordance
Influenced by concurrent work with behrens, Mislevy, and DiCerbo for the
Learning Analytics Workgroup.

17. 1. Reorientation of
center of control
2. Broader focus on
competencies
3. Blended/pers-
onalized learning

18. Social
Networks
&Teams
Mobile
Technology
Evidence and
Transparency
Institution Focus
Teacher Control
Institutions and Teachers

19. Social
Networks
&Teams
Mobile
Technology
Evidence and
Transparency
Institution Focus
Teacher Control
Networks and Students
Social
NetworksLearning
Networks
Learning
Communi
ties.
Expert
Sources
Open Ed.
Resources
Families
Institutions and Teachers
Related to the Education Data Movement

20. Cognitive
• Cognitive processes
and strategies
• Knowledge
• Creativity
Intrapersonal
• Intellectual openness
• Work ethic and
conscientiousness
• Positive core self-
evaluation
Interpersonal
• Teamwork and
collaboration
• Leadership
• Critical thinking
• Information literacy
• Reasoning
• Innovation
• Flexibility
• Initiative
• Appreciation for
diversity
• Metacognition
• Communication
• Collaboration
• Responsibility
• Conflict resolution

21. Cognitive
• Cognitive processes
and strategies
• Knowledge
• Creativity
Intrapersonal
• Intellectual openness
• Work ethic and
conscientiousness
• Positive core self-
evaluation
Interpersonal
• Teamwork and
collaboration
• Leadership
DigitalMediation
• Critical thinking
• Information literacy
• Reasoning
• Innovation
• Flexibility
• Initiative
• Appreciation for
diversity
• Metacognition
• Communication
• Collaboration
• Responsibility
• Conflict resolution
Artifacts

22. • Blend the best of face-to-
face/online.
• Incorporate interaction and
dynamic material coupled with
metadata and paradata to enable
feedback.
• Leverage embedded diagnostic
assessments & interactive data
visualization tools.
• “Learning algorithms” match
content/activities/ teaching
approaches with learner’s needs.
• Connect the in/out of school
learning for complete picture of
student’s development.

23. • Oriented towards new kinds of education
models while often working with data that
comes from earlier models of education.
• Not only producing evidence (data jocks), but
also change agents.
• Will be need to be innovators and draw off of
different kinds of disciplines.

24. Considering Six Adjoining Disciplines

25. 1. Growing interest from
leading universities,
foundations, USED
2. Journals, conferences, &
programs now emerging
3. What is the disciplinary
focus? What counts as
rigor and success? From
where are faculty?
Education
Data
Sciences

26. Statistical
Data
Analysis
Education
Data
Sciences
• Much of the digital ocean is
compatible with statistical
analysis.
• Exploratory data analysis (ex:
Tukey with satellite data in 70s
asked many questions that are
being asked today about “big
data”
• Already established (entrenched)
in Education power structures
• Can produce strong claims

27. Statistical
Data
Analysis
Education
Data
Sciences
Classroom/
Learning
Technology
• This area is seeing
an explosion in
media for:
• Inquiry
• Communication
• Construction
• Expression
• This is where the
data we want
most often come
from…

28. Statistical
Data
Analysis
Education
Data
Sciences
Classroom/
Learning
Technology
Learning
Sciences
• What does big data
mean for socio-
technical multimodal
learning?
• Socio-cultural and
cognitive theories
influence/informed
by data technologies
• A design science for
education practice

29. Statistical
Data
Analysis
Education
Data
Sciences
Classroom/
Learning
Technology
Learning
Sciences
Information
Sciences
• Data visualizations
and HCI
• Info. architectures
that undergird data
systems
• Codes, classifications
• Boundary objects
• In schools, media
centers evolving
with data specialists

30. Statistical
Data
Analysis
Organization
& Mgmt
Sciences
Education
Data
Sciences
Classroom/
Learning
Technology
Learning
Sciences
Information
Sciences
• Education full of
designed processes
• Blended learning
models essentially
re-structuring of org.
practices
• Inter-organizational
functions changing:
• States-districts
• Special education

31. Statistical
Data
Analysis
Organization
& Mgmt
Sciences
Education
Data
Sciences
Classroom/
Learning
Technology
Learning
Sciences
Information
Sciences
Decision
Sciences
• Established field
uses large bodies of
data to support org.
decisions
• As volume/quality of
education data
increase, more
situations where
decision sciences can
be applied emerging

32. The Seventh, Generative Discipline

39. • Broad fluency with a range of
qualitative/quantitative methods
• Ethics, privacy, and confidentiality (FERPA+)
• Technology acumen and ability to reason from
imperfect evidence

40. 1. All analytic processes are socially situated and
iterative
2. Data is a mediational tool in an iterative process
of discovery
3. Data is an imperfect lens for context and for
interactions within that context
4. Organizational/systems thinking helps expand
the reach of Education data science
5. Ethical as well as legal considerations are
important.

41. Philip Piety, John Behrens, Roy Pea
American Education Research Association Annual Meeting
Monday, Apr 29 - 10:35am Parc 55 San Francisco / Divisadero Room
Contact: ppiety@edinfoconnections.com