1. Building MOOCs:
Scalable Course Development &
Delivery
Learning E-ssentials 2013
Ken Currie
Opus Learning
12th December 2013
1

2. This Talk
• Three main themes:
– Why Design is critical, but not necessary complex
– Why an Engineering approach is the only way to
build and deliver
– The implications of Delivery, Support and
Assessment
• Fourth theme?
– Implementation
– MOOC specifics
2

3. Useful Quote
• “Large scale online learning could provide
continuity for learners throughout their lives
between different education systems.”
– Association of Learning Technology Draft Green
Paper 2013.
3

4. Useless Quote
• “Enterprise technology for authoring,
publishing and distributing is still largely in an
immature state.”
– Steve Barden, Lead Consultant, LINE
Communications, Inside Learning Technologies &
Skills Magazine - November 2013
4

5. MOOCs or Programmes?
• Some of you may be here to hear about
MOOCs
– The story doesn’t really change, though there are
some delivery and operational extras
• An example: Swim Coaching
– http://ties.courseworker.com
– University of Iceland offering
– Exploitation of the output of an EU LdV Project
5

6. History of Opus
• A spinout from CAPDM.com
– Developing and delivering Masters programmes
for over 20 years
• Opus delivers highly interactive courses,
delivered via platforms such as Moodle
– With tools to manage and monitor learning with
or without tutor support.
• An SQA Approved Centre
– A wholly online College
6

7. Show of hands
• How many people here build and deliver
online distance education?
• What scale? Is it a challenge?
• Opus has 99 SQA SCQF Level 7/8 courses
– All content full
• Edinburgh Napier University’s Global Online
– 18 masters courses @ 20 credits
– All content full
• Sound online pedagogy? 7

8. Case: Heriot-Watt University
• Edinburgh Business School Online MBA
– Europe’s largest DL MBA; the world's most international
– 14,000 successful MBA graduates from 180 countries
– Twice awarded the Queen's Award for Exports
– Students from over 50% of Fortune 100 companies
– 4,400 study hours; English, Spanish, Chinese, Arabic.
– 800 different deliverables for 37 released courses in print and online.
– 20 million words of English; 8,000 multi-lingual vector diagrams, etc.
– 14 year unbroken record of revision controlled single source XML
– Has earned over £150 million
– Now on its third VLE – but not a single character has had to change to
accommodate these moves!
8

9. MOOCs vs Programmes
• Not a lot of difference?
– Opus aims for tutor-lite courses (so high quality)
– MOOCs take this to the limit
• Content full
– Socially constructed content? No.
– Opus delivers all the content a student needs
– Complete, interactive, interesting!
• But develop social learners
9

10. Theme 1: the Design Challenge
• To build, deliver and manage
– 99 HN courses for Opus
– 18 masters level courses for Edinburgh Napier
– … + others!
• To manage > 100 academics/teachers/authors
• To manage very large domains of content
• To deliver to many platforms
– Moodle, Blackboard, ePub, tablet/iPad, PDF, …
10

11. Strategic , not Tactical
• These challenges must be planned for
– A strategy
• Many (large) organisations have no such
strategy for this scale of development
– Does yours? Most universities don’t, for example
• Structure & Content Management is critical
– Communicate a consistent and understood
message to all parties
– Minimise the entropy
11

12. Three Linked Parties
• Authors, engineers, students
– Three links in a chain
• Many authors write great works
– Few write good distance learning materials
– Learning materials have to be fit for purpose
• Numerous authors
– They must all hear the same message
– They must all understand it
12

13. Talk in simple terms - components
13

14. Standard programme design
• Standardise course design within programmes
14

15. Simple message
• This structure carries a message that is:
– Familiar: all components understood
– Understood: can be communicated safely
– Scoped: easy to put a scale to components
– Quality Assured: all content must comply
structurally, in scope, academically, pedagogically
15

16. The Three Links
• Communicating the message
– Authors work to a common template
– Students consume a consistent product
• Know how to use it
• Appreciate the clarity?
• Keep retention rates up?
– Engineers, such as CAPDM, can build the product
• Cost effectively
• Consistently
16

17. SQA: the Start Point
• SQA provide ‘arrangement’ documents for all
– Frameworks (e.g. HND Business) and
– Units (e.g. Business Accounting)
• Unit specifications of:
– Outcomes
– Knowledge and/or Skills
– Evidence Requirements
– + Assessment Guidelines
• Universities have similar, perhaps less
rigorous, guidelines 17

18. Theme 2: Engineering
• These courses are complex (in a structural
sense)
– Content-full, pedagogically rich?
– Highly integrated and linked
– Delivered to multiple platforms
– Highly functional and interactive
– Rich navigation and search
• Too complex to build by hand
– The typical ‘bricolage’ approach cannot work
18

19. Standards
• All course materials are designed for their
learning potential
– Completely agnostic wrt technology
– Single source masters of all content
– Standards based: XML, MP3/4, etc.
• Structural descriptions of the assembly of this
content through …
– An XML manifest
• Focus on learning
19

20. Publishing Goals
• To go from the single source master to full
delivery
– On all chosen platforms (e.g. Moodle)
– In about 5 minutes
– All published forms (xHTML, PDF, …)
– Automatically
• Delivery to be
– Complete and fully integrated
– Fully functional
20

21. Where’s the middle bit?
• Where’s the instructional design, the delays,
the faffing about with the VLE?
– Eliminated!
– This is all in the engineering, so fully automated
– Styles, structures, etc., are all parameterised
– Consistency of interpretations and delivery
• The Authors/Module Leaders can be in control
– … in theory!
21

22. Semantic Mark-up
• Word, HTML and many other formats are
‘dumb’
– XML can embody rich metadata and semantics
– Though “File | Save As” XML is also dumb
• Our job?
– To capture all learning content in a rich mark-up
– To apply a consistent and rich semantic
interpretation to that content for delivery
22

23. Empowering Authors
• Authors write what they need
– If the mention a ‘Forum’ then we guarantee its
presence and a link to it
• It is our job to interpret their needs and to
engineer a solution
– You will never hear “we can’t do that as the
platform doesn’t offer that”
– A challenge at times
• We interpret good learning content
23

24. Flexibility
• Single source masters with an ‘engineered’
delivery provides flexibility
– Changes made in one place
– Style(s) added during publishing
– New platforms added non-disruptively
– Multiple instances easily supported
– Freed from specific technologies
• Smart, agile content
– Standards in action
24

25. Theme 3: Delivery to Assessment
• Assessment can take many forms
– Formative, e.g. through topic quizzes
– Summative, e.g. exams and assignments
– Ipsative, e.g. through comparison of self with
earlier marks
• Students build up a Reflective Log, over time
– A longitudinal measure (Digital Work Book)
– Continually improve it
– Shared with the tutor (peers potentially)
25

26. Assessments & the SQA
• Why are we trying to ‘change’ things?
• Units are stand alone
– Can be delivered in a very flexible manner
– But very clear pointers as to how assessment
should be carried out
• No longer appropriate to base an assessment
of competence on a final three hour written
examination.
26

27. HND Business: Unit by Unit
• HND Business: each student has to
– Sit 6 formal examinations
– Deliver 6 documented portfolio exercises
– Complete 22 other assessment reports
• total word count of 34,500
• An HE Diploma?
– 12 modules each with final examinations or
assignment coursework
27

28. Need: Learning Analytics
• MOOCs are gathering global analytics
– Measure everything but the individual
• Opus is looking to move towards developing
the ‘Digital Self’
– How you model yourself as an individual, and
access your data
– How you are measured at all times
– The DWB is our first step in recognising the
potential of ‘always-on’ learning
28

29. A Response: a Digital Work Book
• Longitudinal assessment
– Augments examination and assignment
• Part of our ‘Social Learner’ aim
– No faith in social learning (students creating the
learning content)
– Great belief in breeding social learners
– Many levels from ‘none’ to
• Interact with content for own ends (e.g. DWB)
• Comment locally with others (e.g. Scotsman)
• Interact and discuss with all peers (e.g. Forums)
29

30. Interacting with Content Objects
Learner
interacting with
content, leaving
traces
Student interacts with content
Makes notes, responds to
quiz, send messages, etc
Capture notesStore in MLE
Print off
record of
learning The Digital Workbook
30

31. Social Networks
• Opus is looking to build 000s of small groups
– MOOCs are looking for a group of 000s
• Our social networks are small scale but highly
linked (see Anderson 2003)
• Our goal: 5,000 students, but
– 100 centres with 50 students
– Model the individual, or a small group
31

32. How do students interact
when learning
Student
Environment
Content Tutor
Student/Student
Content/Content
Tutor/Tutor
Environment/Environment
Student/Tutor
Tutor/Environment
Student/Environment
Student/Content
Content/Environment
Tutor/Content
Based on diagram in
Anderson 2003 32

33. Big Data?
• The analytics of the individual are of more
interest to us
– Not specifically looking at group behaviour
– Individuals, in depth and across courses
• Digital Self profiles may be limited
– E.g. me to Facebook and Linked-In (no pedometer
around my waist!)
– How do we enhance profiles?
33

34. The Role of the Learning Materials
• Authors are encouraged to develop clear
‘concept maps’ for students to follow
– Learning paths must go through key ‘concept
gateways’
– Evidence of competent/critical thinking gathered
• The core is learning content that
– Is interactive
– Measures its own consumption
– Builds ‘Digital Self’ data
• How to make it more intelligent? 34

35. Student Learning Aims
• Our aims for our learners
– To see how they change over time
– To teach these learners how to learn
• Their interface
– Our content, systems and support
– Must be smart
– Back to the interpretation of the semantically rich
XML base
35

36. Analytics: Some Conclusions
• Analytics and tracking, via the DWB, etc.,
appears to
– Provide the potential of personalised learning
within mechanical presentation of study
– Permit formative assessment in the content,
driving tracking and learning progress
– Provide feedback on Tutor effectiveness and the
effectiveness of the pedagogy underpinning the
content
36

37. Theme 4: Implementation Details
• Note
– No mention of Authoring Tools or proprietary
technologies
– No real mention of e-learning
– But massive exploitation of the technology of
delivery
• Standards imply
– Technology agnosticism
– Development outside of deliver environment(s)
– Delivery into that/these environment(s) 37

38. A complete System
• Our final systems integrate
– All assignment statements, requests and
submissions
– Online tutor support and a Help Desk
– Payment clearance and immediate access to study
materials
• Our MOOCs include
– 100% automation
– Social interaction
– Auto generation of a Certificate of Completion 38

39. Back to Opus Learning
• Opus is a wholly online college
– SQA Approved
– Offering HNC/Ds and other courses
– With its own students and tutors
• Opus is also a B2B provider of
– Ready packaged courses and programmes
– A full own-brand ‘College in a Box’
– Consultancy for aspiring SQA centres
• Opus /CAPDM is a custom publisher &
developer of ODL 39

40. Strategic?
40

41. Quote Revisited: Mature?
• “Enterprise technology for authoring,
publishing and distributing is still largely in an
immature state.”
– Steve Barden, Lead Consultant, LINE
Communications, Inside Learning Technologies &
Skills Magazine - November 2013
41

42. What people think we do
42

43. What we think we do
43

44. The End
Ken Currie
Opus Learning
kwc@capdm.com
44

45. The Digital Workbook is
• A portfolio embedded in
the content
• A dashboard for learning
• Rich seam of data and
behaviours for mining
• Ongoing formative
assessment, and potential
summative assessment
• A pdf for the student
45

46. Case study HND Business
Unit by Unit Assessment
In a typical instance of the HND Business each
student has to:
– Sit 6 formal examinations
– Deliver 6 documented portfolio exercises , and
– 22 other assessments - reports with a total word
count of 34,500.
A university Dip HE might only have 12 modules
with 12 final examinations or courseworks
46

47. Managing Learning
with Technology
• At Opus we track student behaviour.
• We use 2 levels of tracking
– Moodle Based (the VLE) and
– Content Based.
47

48. Tracking through the learning content
• Opus content is massively connected to the
students workbook
• Every reflection, exercise, quiz, short response
and so on are logged within the learning
content and in the workbook
• The tutor can view, track and compare
workbooks
48

49. DWB Examples – Source Page
49

50. 50

51. 51

52. DWB Dashboard –
Managing groups of students
52

53. The Commercial Imperative
We wish to
– drive down cost of assessment
– Drive up the quality and efficiency of assessment
within the context of a well resourced learner.
We have to manage costs, and use of human
resource
53

54. Assessor/Tutor Activity Model
in Distance Learning
1. Monitor VLE activity, identify any candidate with
lack of progress and contact course leader for
action.
2. Monitor Digital Workbook (if used) and identify any
outliers in progress, contact course leader with
details.
3. Respond to direct tickets from candidate, making
use of FAQs for responses, if new response is
required, copy response to course leader.
4. Responding to assessments; remediation advice
54

55. Learning Centre Cost Model
• Based on the previous models
– Time per tutor/unit/student can be estimated
– Payment to Tutor/assessor is known
– Viability of the Distance Course can be calculated.
• We need to know these numbers and to
watch them
55

56. Potential Cost Model
Possible costs/charges per student
awarding body charges = £250;
tutoring = £1000;
tech + online content licence = £300;
agent fees = £250
Centre overheads per student = £500
Total cost per student = £2300
56

57. Potential Centre Surplus
• In UK Charge to student £3500-£4500
• In India Charge to student £1500-£2500
We cannot serve one of largest markets in the
world, with this model.
We have to be able to manage costs.
57

58. Learning from MOOCS
• Massive size of cohort
• Highly programmed week by week
• Clearly signposted weekly targets
• Using the student body as the main tutor
toolset
• Building a community of activists and lurkers
• Managing and controlling costs
• Generate revenue by conversions
58

59. Learning from MOOCS
• Do not do assessments
– Until students join full programme
• Keep the granularity of the course fairly small
– not full degrees
• Set up an honour system – make the students
police themselves.
59

60. Applying all these lessons
• Manage the cohort size
• Make sure investment reflects assessment and
tutor support provided – the added value of a
non-free course
• Integrate assessments across units; manage
the assessment overload
• Make assessments social, encourage group
help. Track this activity!
60

61. Opus Strategy
Small Private Open Courses
• Organise cohorts in groups of 10s, not groups of
10k’s or 100k’s
• Localise tutors and assessors
• Build communities including assessors
• Track activity and behaviour at keyboard level and at
cognitive level
• Record activity – perhaps award a badge for partially
completed activity
• Guide the learners towards completion, intervene as
necessary
61

62. Creating the Social Learner
• Our DWB is part of a social learner strategy
– Rich interaction spaces
– Personal interaction with the content (DWB)
– Interaction on specific issues (traversing the web)
– Social learner (Forums, Facebook, etc.)
• Manage forum and virtual space activity
62

63. The Future
• The DWB is sharable, including
– with the Tutor
– with other students, groups
• In the future it will
– support many content types
– integrate with other portfolio components
• The DWB will be our main assessment toolset
• The DWB will be our main behaviour manager
63

64. Conclusion
• Do not just set exams and give badges
– Track and Assess
– Intervene and support
– Audit and learn lessons
• Build a community for the whole qualification
• Record learning and make available for future
generations
• Continually improve learning content.
64