Keynote address to the ePIC 2013 conference, London, July 10, 2013, in which I apply principles from eportfolio research to open digital badges and learning analytics to identify future directions and points of connection
5. COMMON GOALS
Enable decisions more grounded in evidence of
learning and identity
(with the assumption that they are inextricable)
Make that evidence richer and more diverse
(with the assumption decisions will be more valid)
7. FOUR PROPOSITIONS
1. Reflection on pieces of evidence within an eportfolio
and on the eportfolio as a whole provides information for
assessment that isn't available by other means.
2. For meaningful assessment, interaction of pieces of
evidence within an eportfolio is more important than
single pieces of evidence.
3. The material practice of eportfolio composition
generates distinctive knowledge about learning.
4. Eportfolios enable meaningful comparison of
student learning without standardization.
8. Proposition Principle
Reflection Learners are key interpreters of
evidence of learning and performance
Interaction Focus on relationships rather than
individual nodes
Materiality Pay attention to embodied
consequences
Comparison Promote standards for dialogic
interpretation of evidence rather than
for evidence itself
13. RELATIONSHIPS NOT NODES:
EMERGENT PROFESSIONAL
PROFILES
Pre-defined
links to
standards or
activities
Stipulated
Individual
patterns based
on observed
success
Emergent
Group
patterns
based on
observed
success
Collective
16. EMBODIED CONSEQUENCES
Digital badges:
Focus on audiences
as was well as issuers and earners
Learning analytics:
Research use
by actual decision makers and its effects on
learners
17.
18. Our goal is to develop an understanding of validity
that begins with the questions that are being asked;
that can develop, analyze, and integrate multiple
types of evidence at different levels of scale; that is
dynamic in the sense that questions, available
evidence, and interpretations can evolve dialectically
as inquirers learn from their inquiry; that allows
attention to the antecedents and anticipated and
actual consequents … and that situates the
assessment in the broader context in which it is
used.
A validity theory can also be construed as
the representation of an
epistemology—a philosophical
stance on the nature and justification of
knowledge claims—which entails a
philosophy of science.
Moss, et.al., “Validity in
Educational Assessment”
19. STANDARDS FOR
INTERPRETIVE DIALOG
Hermeneutics and deliberative
democracy theory as alternative
epistemologies for validity
theories
Set standards for how we hold
discussions about evidence to
make decisions rather than about
what evidence to consider
Badges and learning analytics as
heuristics, mediators of meaning
making conversations
Need tools and techniques for
interpretive discussion
20. PRINCIPLES OF
DELIBERATION
Publicity
• Deliberative system
which informs and
holds accountable
Inclusiveness
• All impacted by
decisions can
participate
Reasonableness
• Economy of moral
objections
• Respect for reasonable
disagreement
Provisionality
• Openess to changing
positions and decisions
21. BETTER TOGETHER
Badges and learning analytics can enrich eportfolios through
new types and structures of evidence
algorithmically identified patterns
Eportfolios offer
authenticity for legitimacy and commitment
integration for context and meaningfulness
direct contribution to learning through compostion
Summary of 2003 story identity pathwaysbyrestructuring a set of decisions
Learning analytics = large volumes of heterogenous data relevant to understanding and predicting learning in form that decision makers in the field can interpret for themselves in as close to real time as possible
Decisions = not just, whom should I hire? But what do I care about?
Eportfolio movement needs to constructively engage!! – won’t shape what happens just by rejecting out of hand, need to apply what we know and believe to say how, not if Active debate about role of identity/learning agency Digital learning now as example from corporatized/automated version of “personalized learning” valuing more diverse evidence, seeing identity as importantnot yet a commitment to agency of learners, but learning from e-portfolios practices could helpUsing people’s identity vs. helping people act with theirs Vs. CCSSO innovation network Student voice and choice key to personalizationPersonalization vs. individualization/differentiation (Bray and McCulsky)
Will explore how the first two principles apply to the three means of representation and analysis and show how benefit from application of the fourth
Nothing stopping you from giving yourself a badge, but power is in well-definedattestation of another curating to audience as form of interpretation
Community badges Peer awarded by those who have earned the right Also for mentors
Access to data first step
Individual activity/standard: predefined – Piecemeal linking of badges to CEUs or specific standards (like a certificate) or levels within a system (we’ll be doing this for CEM, so don’t mean to trivialize) or predefined set of competencies for a profession vs. Individual profile: based on patterns in actual earning -teachers who end up being successful in this context tend to have this set of badges (with certain commonalities, from certain issuers and endorsers), and here’s how you compare to them and what it says about what learning experiences you might choose nextvs. collaborative profile: not just what they look like for individual teachers, but what they look like for an effective PLC, school, or district --This is what the open badges ecosystem is about! --Notional so far, because adoption isn’t widespread enough yet, but could be not far away for software development
Learningof the community, health of the social learning space, not just average achievement of individual learners (e.g., activity that leads to more sustained and broadbased engagement) Concepts and their relationships, not just key words (e.g., “use” key to favorable and unfavorable but not “fun”) Way to direct community managers’ attention to decide what meaning, if any, there is rather than determining See papers on ConnectedEducators.org
Professions defined by “epistemic frames”: SKIVE – skills, knowledge, identity, values, epistemology Coding chat transcripts from simulation for markers of each coaching suggestions
Add picture here
Absurdity of standarddescriptive statistics – use statistician jokeVs. Shafer – “a relational approach to psychometrics” PatriciaCross distinction between SoTL and traditional educational research - we care about specific people in this specific context and all of them (and about the character of the whole)
Badges as mediators of meaning making conversations vs. determinants LA output as heuristic for contextualized meaning making through dialog rather than automated judgmentDevelop tools for interpretative discussion, not just earning/awarding/displaying or capture/visualization/prediction Hermeneutics and deliberative democracy theory as alternative epistemologies for validity theories
Apply equally to e-portfolios and badges and analytics
authentic representation – interpretation of evidence in relationship to articulation self in relationship to shared horizons of intelligibility (right term – use Guignon quote?)Provide legitimacy, create commitment • integration across levels of evidence and spheres of life (Keegan adult challenge) Accounts for contextBoth richer by virtue of new types and structures of evidence badges and LA offer and potentially meaningful patterns algorithmic analysis can identify Capitalizes on Badges and LA to directly aid learning through composition