Open Education 2011: A case study in OER within the LMS
1. Open Education 2011:
A case study in OER within the LMS
John Rinderle @JohnRinderle
Bill Jerome
2. OER within the LMS
Why the LMS?
Interoperability standards
Our approach and findings
Looking forward
Opportunities and challenges
Discussion
3. Open Learning Initiative
Produce courses and course materials which enact
instruction and support instructors
Provide open access to these courses and materials
Develop communities of use, research and development
that enable evaluation and continuous improvement
4. Why the LMS?
Convenient for students and instructors
Single sign on
Single entry point
Mix and match
Coherent navigation
Roster management
Unified gradebook
Learning analytics
Frequently requested by students and instructors
5. Why the LMS?
Increasing access is part of our core mission
Easier to find
Metadata and search
Easier to consume
Learning management systems
To extend the reach of OER, we must bring it to where it is
needed, where it is convenient to use
6. Standards?
Not building blocks, modules, and plugins?
Too many platforms to target otherwise
Moodle, Sakai, Blackboard, Desire2Learn, Canvas, …
Changing technology, changing standards
Are they “standards” in name only?
We should not be surprised when standards work!
Confidence in mature, proven technologies
7. Which standard to choose?
OLI courses provide
User profiles
Persistent state
Multiple assignments
Scores
Student work products
Learning Dashboard
More than a simple content package!
8. IMS Learning Tools
Interoperability
Basic LTI
Tool consumer, tool provider
Trust relationship with the LMS
Fields to identify user, course, etc.
Simple, but powerful
Full LTI
Grade exchange
Multiple tool endpoints
Much more complex
9. Basic LTI
Available for most LMS systems
Easy to develop tools
Simple web request
Easy to consume tools
No programming required
Pathway to more advanced features
Extensible
Low-cost implementation
15. Challenges: Technology
Some LMS products require an extension
Requires institutional level support
We want individual instructors to be able to adopt
Some implementations are buggy
e.g. Internet Explorer on Moodle
We wrote patch for basiclti4moodle
Basic LTI has few required fields
Required, recommended, option fields
Different products supply different data
16. Challenges: Process/Policy
Security and privacy concerns
e.g. data ownership questions, security audits
Audit / control which tools instructors are using
Helping instructors get support from their institution
17. User Experience
• Why is good user experience critical to adoption?
18. User Experience
• Why is good user experience critical to adoption?
• Using complex systems is easy for some, but
much harder than one may expect for average
users, instructors and students alike
19. User Experience
• Why is good user experience critical to adoption?
• Using complex systems is easy for some, but
much harder than one may expect for average
users, instructors and students alike
• Instructors may not adopt a technology if it gives
the appearance of complicating their lives
• Setup
• Workflow
20. User Experience
• Why is good user experience critical to adoption?
• Students who struggle with interfaces experience
negative affect and their meta cognitive resources
are not spent where we’d like for learning
21. Challenges: User Experience
• Roster Management
• No record until first student access from their LMS
• Students never dropped from roster
• Teaching assistants often unsupported
22. Challenges: User Experience
• Gradebook / Learning Dashboard Tools
• Grade exchange is very limited
• Aiding instructors and students in locating no
longer centralized scores
• Access to reporting tools richer than grades
23. Challenges: User Experience
• Login
• Removed the link as it does not fully reflect user
expectation
• This proved unpopular
24. Challenges: User Experience
• Bookmarking
• Leads to unauthenticated visits that require
redirects to LMS
• Basic LTI does not support the direct back
25. Challenges: User Experience
• Desktop Support
• Who do user questions gets routed to?
• OLI questions need to reach OLI help desk
• LMS questions need to reach LMS help desk
27. Challenges: User Experience
• Where do users have accounts: OLI or LMS
• Carnegie Mellon solution: both
• Existing users solution: both
• All others: LMS-only
28. Challenges: User Experience
• Where do users have accounts: OLI or LMS
• Carnegie Mellon solution: both
• Existing users solution: both
• All others: LMS-only
• Unless you’re not using an LMS at all, then OLI only
29. Challenges: User Experience
• Where do users have accounts: OLI or LMS
• Carnegie Mellon solution: both
• Existing users solution: both
• All others: LMS-only
• Unless you’re not using an LMS at all, then OLI only
• Did you just now learn you could use your LMS but
already setup your course? We can merge your
accounts for you.
30. Challenges: User Experience
• Where do users have accounts: OLI or LMS
• Carnegie Mellon solution: both
• Existing users solution: both
• All others: LMS-only
• Unless you’re not using an LMS at all, then OLI only
• Did you just now learn you could use your LMS but
already setup your course? We can merge your
accounts for you.
• …unless you have students already registered
through OLI which might confuse them.
31. Challenges: User Experience
• Where do users have accounts: OLI or LMS
• Carnegie Mellon solution: both
• Existing users solution: both
• All others: LMS-only
• Unless you’re not using an LMS at all, then OLI only
• Did you just now learn you could use your LMS but
already setup your course? We can merge your
accounts for you.
• …unless you have students already registered
through OLI which might confuse them.
Not easy to communicate to novice users
32. Challenges: User Experience
Configuring / enabling tools in LMS is complex
Above and beyond what an instructor should have to do
Figuring out if a system is compatible is complex
Which vendor, product, version are you using?
Are the appropriate extensions installed and enabled?
We built a test and configure your LMS page
33.
34. What’s next for OLI?
Configurable Entry Points
Instructors customize LMS links to OLI
Full LTI
Report outcomes to LMS gradebook
Tool Consumer
Enable Basic LTI tools within OLI courses?
38. Learning Analytics
How do we get there?
Standards for data
Data exchange API
Platforms and tools
Policy for data exchange
LMS interoperability and learning analytics should
be automatic, not an afterthought.
40. OER App Store
Find OER from within the LMS
One click access to add to course
A basis for choice
Evaluation, context of use information
Built in communities
41. Why hasn’t this happened
[sooner]?
Publishers are now preparing offerings
– Pearson OpenClass
– Commercial and open content
42. Why hasn’t this happened
[sooner]?
Technology?
Standards?
Institutional Policy?
Licensing?
Data Ownership?
43. Why hasn’t this happened?
We need platforms which makes it easy to create,
share, find, use and evaluate OER
The user focus needs to be on the educators and
learners consuming OER
44. Discussion
How does OER make greater inroads to the LMS?
Do you agree with the app store approach?
What should an OER app store offer?
Does the app model extends or replace the content
package?
Open and “closed”, free and commercial, side by side?
Next steps to make this happen?
46. User Experience
• ISO definition : a person's perceptions and responses
that result from the use or anticipated use of a
product, system or service.
• Source: http://en.wikipedia.org/wiki/User_experience#Definitions
47. User Experience
• ISO definition : a person's perceptions and responses
that result from the use or anticipated use of a
product, system or service.
• Source: http://en.wikipedia.org/wiki/User_experience#Definitions
• We incorporate usability study into the design,
development, and evaluation of user experience
One URL to rememberSign in with your university credentialsAdd your own assignmentsWhy duplicate/recreate these features in every new OER environment?
Metadata: LRMI, Dublin Core, LOM, …Increasing access is part of our core mission. Bringing OLI to the LMS lowers another barrier to adoption: ease of use, in a familiar context.Whatever your feelings about LMS systems, the LMS is the familiar information hub of a course.A key factor in OER uptake is the ability of resources to be easily accessed, combined with other course materials, and presented in an appropriate context for learning. For many instructors, the learning management system (LMS) is the information hub of their course. To extend the reach of OER we feel it is critical for resources to be made easily accessible from within the LMS. This need is greater than providing a simple link. From the LMS, OERs should be discoverable to students and instructors who want to use them, support a single sign on interface, provide coherent navigation between LMS and OER, and seamlessly exchange key data (e.g. roster, grades, learning analytics).
Standards can now accommodate applications (tools), not just simple content packages.Differences in how vendors have implemented standards in the past.
OER is becoming increasingly sophisticated.
Basic LTI: REST based, OAuth authentication, simple browser form POST
Click 1: LMS navigation, Click 2: research consent formFall 2011: pilot (60 instructors, 1500 students)Spring 2012: general availability
Did it work as expected?Standards designed as a technology, not to serve end user needs.Current standards lean towards content publishers and institutional needs, instead of instructors and learners who consume content.
OER is no longer under the security radar (which, in a way, is good). Even though no new information is changing hands as compared to when OLI is used outside the LMS, institutions are more concerned about information security where systems connect. An advantage of Basic LTI is that there is no direct communication between the LMS and tool (e.g. OLI). The information comes from the LMS, but is exchanged through the user’s web browser. Privacy settings allow institutions to control what data is exchanged.
There is UX work to do for entry points. Multiple links could be confusing.Allow instructors to choose the assignments for which we send scores back to the LMS.
The most powerful feature of technology based instruction is the ability to embed assessment in each learning activity and use the resulting data to drive powerful feedback loops to the learner, the instructor, the course development team, and learningscience.
While today’s standards allow learning environment to report simple score outcomes to the LMS, the more robust measures of learning required to drive learning analytics remain locked up in individual tools. The result is that it is difficult to mix resources and achieve a unified view of how learning is progressing and the overall effectiveness of the learning design. We feel this is a missed opportunity. The next generation of standards and LMS systems will need to simplify the discovery and adoption of OER and facilitate finer grained data exchange.
Ease of Development:LMS interoperability and learning analytics should be automatic
LMS systems and OER need a renewed focus on user experience. Acknowledge the work of OLI, Sakai, and others.