SlideShare une entreprise Scribd logo
1  sur  15
Télécharger pour lire hors ligne
Experimenting between
MOOCs and DOCCs
Adeline Koh @adelinekoh
Tuesday, March 18, 14
MOOCs: The Explosion
• First courses in February 2012 by
Sebastian Thrun (then of Stanford)
• Major players: Udacity, Coursera,
Harvard’s EdX
• Promises:
• scale,
• making education affordable
• global reach
• Criticisms
• Not good pedagogy, as it is
primarily content-based and
lecture-based
• Low retention rates
• New Scientist article in March
2014: Who takes MOOCs?
Educated, Employed, First
World Men. http://
www.slate.com/articles/
health_and_science/
Tuesday, March 18, 14
Anti-MOOCs: FemTechNet’s DOCC
Tuesday, March 18, 14
DOCCs: Emphasis on building community rather than content delivery:
a specifically feminist pedagogy
Tuesday, March 18, 14
Cathy Davidson’s #FutureEd Coursera MOOC turned into a DOCC
Tuesday, March 18, 14
Similar to #FemTechNet: encourages creation of community through
feedback, peer grading
Tuesday, March 18, 14
Used #FutureEd in my MA level course, Introduction to Digital
Humanities. Link to syllabus: http://tinyurl.com/kohdh
Tuesday, March 18, 14
• Student learning about the industrial model for education rather than
specialized, individual-focused education that is peer driven
• Students working on:
• Coming up with their own “Class Manifesto”
• Remixing the Syllabus
• Coming up with their own “grading contracts”
Most Valuable Exercises
Tuesday, March 18, 14
Highlights: Class Manifesto
• “AMST 5011 is an experimental graduate level course committed to identifying,
evaluating, creating and rethinking solutions to educational challenges that our
changing society faces in the twenty-first century. We aim to seize opportunities
to fully realize and harness the possibilities of twenty-first-century literacies,
which we define as the mindsets, skills, and collaborative techniques needed to
make full use of the Internet as a space of learning. We believe that the Internet
and technology are changing how individuals and communities understand
themselves and the world around them, and that this connected age offers a
tremendous opportunity to make teaching, learning, and knowledge more
accessible, more affordable, and more meaningful for everyone involved. AMST
5011’s purpose is to examine the ways in which technology influences
educational dynamics and to collaboratively propose and share new
possibilities for the Information Age so that we--scholars, teachers, and
students--can best respond collectively to the challenges this new
paradigm poses for learning.”
Tuesday, March 18, 14
Highlights: Class Manifesto
• AMST 5011 Class Goals and Practices for the Semester (Spring 2014)
• Create an evolving individual online presence aimed at advancing our digital literacy to
exemplify the possibilities of technology in higher education
• Collaborate in a constructive manner as we assign, guide, and assess the work of peers.
• Practice judicious time management in assigning tasks to others and completing our own.
• Arrive to class on time, fully prepared to participate, having completed assignments on time.
• Represent ourselves and the class both online and face-to-face as engaged scholars who
are committed to advancing the above principles.
Tuesday, March 18, 14
Highlights: Grading Contract
• Contract Grading + Peer Evaluation:  Explanation and Contract
Evaluation Method:
You determine your grade for this course by fulfilling a contract that spells out in advance
the requirements as well as the penalties for not fulfilling the terms of your contract.  
Learning together giving and receiving feedback is the core concept of our class.   
Every student will be responsible for submitting to Professor Koh anonymous peer-
grading of his classmates twice this semester, a midterm evaluation (due by March
7th)  and the final (due by April 23d). A peer grader will evaluate the work of his / her
classmates using Satisfactory / Unsatisfactory descriptive evaluation format (up to 300
words). The unsatisfactory midterm peer assessment is intended to provide the
opportunity for a student to improve his / her performance by the final evaluation. In case
of a student receiving an Unsatisfactory peer evaluation, recommendations for
improvements are to be determined by the peers collaboratively. The goal is for everyone
to produce satisfactory work, and the peers will work with each student to achieve that
goal.
Tuesday, March 18, 14
Highlights: Grading Contract
• Contract Grading:
The advantage of contract grading is that, the student, decides how much work
he / she wishes to do this semester; if the work is completed on time and
satisfactorily, the student will receive the grade for which he / she contracted.  This
means planning ahead, thinking about all of the obligations and responsibilities
this semester and also determining desired grade in the course.
Tuesday, March 18, 14
Results
• Most empowered class of students I’ve ever had the experience of teaching
• Great commitment to work that goes over and above the other classes that
I’ve taught
• Collegial classroom environment
• Amazing levels of engagement
Tuesday, March 18, 14
Challenges
• Faculty “loss of control”
• Giving up power, stepping outside of the classroom when students work
on these documents
• Faculty fear of lacking “rigor” etc. versus student empowerment, plus
emphasis on collaboration
• How to move beyond these issues?
Tuesday, March 18, 14
Finally... I leave you with a Youtube video created by my student William Albertson
as part of his work for the class. Available here and on his website, http://
wpastocktondh.com/
Tuesday, March 18, 14

Contenu connexe

Plus de Adeline Koh

The Political Power of Play
The Political Power of PlayThe Political Power of Play
The Political Power of PlayAdeline Koh
 
Postcolonial Digital Humanities: From Hashtag to Movement
Postcolonial Digital Humanities: From Hashtag to MovementPostcolonial Digital Humanities: From Hashtag to Movement
Postcolonial Digital Humanities: From Hashtag to MovementAdeline Koh
 
Apply to be a DH@Stockton Intern!
Apply to be a DH@Stockton Intern!Apply to be a DH@Stockton Intern!
Apply to be a DH@Stockton Intern!Adeline Koh
 
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014Adeline Koh
 
From Print to Digital: Reconfiguring Postcolonial Knowledge
From Print to Digital: Reconfiguring Postcolonial KnowledgeFrom Print to Digital: Reconfiguring Postcolonial Knowledge
From Print to Digital: Reconfiguring Postcolonial KnowledgeAdeline Koh
 
Postcolonial Digital Humanities: Concepts and Challenges
Postcolonial Digital Humanities: Concepts and ChallengesPostcolonial Digital Humanities: Concepts and Challenges
Postcolonial Digital Humanities: Concepts and ChallengesAdeline Koh
 
Digital Humanities for Undergraduates: Getting Started
Digital Humanities for Undergraduates: Getting StartedDigital Humanities for Undergraduates: Getting Started
Digital Humanities for Undergraduates: Getting StartedAdeline Koh
 
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...#DHThis for Social Media and the Research Cycle, Research without Borders, Co...
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...Adeline Koh
 
Koh Representing Race Opening Remarks MLA 2013
Koh Representing Race Opening Remarks MLA 2013Koh Representing Race Opening Remarks MLA 2013
Koh Representing Race Opening Remarks MLA 2013Adeline Koh
 
Navigating Archival Silence koh mla 2013
Navigating Archival Silence koh mla 2013Navigating Archival Silence koh mla 2013
Navigating Archival Silence koh mla 2013Adeline Koh
 
Race and the Digital Humanities: An Introduction
Race and the Digital Humanities: An IntroductionRace and the Digital Humanities: An Introduction
Race and the Digital Humanities: An IntroductionAdeline Koh
 

Plus de Adeline Koh (11)

The Political Power of Play
The Political Power of PlayThe Political Power of Play
The Political Power of Play
 
Postcolonial Digital Humanities: From Hashtag to Movement
Postcolonial Digital Humanities: From Hashtag to MovementPostcolonial Digital Humanities: From Hashtag to Movement
Postcolonial Digital Humanities: From Hashtag to Movement
 
Apply to be a DH@Stockton Intern!
Apply to be a DH@Stockton Intern!Apply to be a DH@Stockton Intern!
Apply to be a DH@Stockton Intern!
 
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014
DHThis a Peer Review Experiment Adeline Koh Rutgers Jan 2014
 
From Print to Digital: Reconfiguring Postcolonial Knowledge
From Print to Digital: Reconfiguring Postcolonial KnowledgeFrom Print to Digital: Reconfiguring Postcolonial Knowledge
From Print to Digital: Reconfiguring Postcolonial Knowledge
 
Postcolonial Digital Humanities: Concepts and Challenges
Postcolonial Digital Humanities: Concepts and ChallengesPostcolonial Digital Humanities: Concepts and Challenges
Postcolonial Digital Humanities: Concepts and Challenges
 
Digital Humanities for Undergraduates: Getting Started
Digital Humanities for Undergraduates: Getting StartedDigital Humanities for Undergraduates: Getting Started
Digital Humanities for Undergraduates: Getting Started
 
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...#DHThis for Social Media and the Research Cycle, Research without Borders, Co...
#DHThis for Social Media and the Research Cycle, Research without Borders, Co...
 
Koh Representing Race Opening Remarks MLA 2013
Koh Representing Race Opening Remarks MLA 2013Koh Representing Race Opening Remarks MLA 2013
Koh Representing Race Opening Remarks MLA 2013
 
Navigating Archival Silence koh mla 2013
Navigating Archival Silence koh mla 2013Navigating Archival Silence koh mla 2013
Navigating Archival Silence koh mla 2013
 
Race and the Digital Humanities: An Introduction
Race and the Digital Humanities: An IntroductionRace and the Digital Humanities: An Introduction
Race and the Digital Humanities: An Introduction
 

Dernier

Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...EduSkills OECD
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
General AI for Medical Educators April 2024
General AI for Medical Educators April 2024General AI for Medical Educators April 2024
General AI for Medical Educators April 2024Janet Corral
 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxiammrhaywood
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxVishalSingh1417
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfchloefrazer622
 
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...Sapna Thakur
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...christianmathematics
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactdawncurless
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfAyushMahapatra5
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDThiyagu K
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsTechSoup
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhikauryashika82
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingTechSoup
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfJayanti Pande
 

Dernier (20)

Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
General AI for Medical Educators April 2024
General AI for Medical Educators April 2024General AI for Medical Educators April 2024
General AI for Medical Educators April 2024
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdf
 
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impact
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdf
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SD
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 

Experimenting Between MOOCs and DOOCs

  • 1. Experimenting between MOOCs and DOCCs Adeline Koh @adelinekoh Tuesday, March 18, 14
  • 2. MOOCs: The Explosion • First courses in February 2012 by Sebastian Thrun (then of Stanford) • Major players: Udacity, Coursera, Harvard’s EdX • Promises: • scale, • making education affordable • global reach • Criticisms • Not good pedagogy, as it is primarily content-based and lecture-based • Low retention rates • New Scientist article in March 2014: Who takes MOOCs? Educated, Employed, First World Men. http:// www.slate.com/articles/ health_and_science/ Tuesday, March 18, 14
  • 4. DOCCs: Emphasis on building community rather than content delivery: a specifically feminist pedagogy Tuesday, March 18, 14
  • 5. Cathy Davidson’s #FutureEd Coursera MOOC turned into a DOCC Tuesday, March 18, 14
  • 6. Similar to #FemTechNet: encourages creation of community through feedback, peer grading Tuesday, March 18, 14
  • 7. Used #FutureEd in my MA level course, Introduction to Digital Humanities. Link to syllabus: http://tinyurl.com/kohdh Tuesday, March 18, 14
  • 8. • Student learning about the industrial model for education rather than specialized, individual-focused education that is peer driven • Students working on: • Coming up with their own “Class Manifesto” • Remixing the Syllabus • Coming up with their own “grading contracts” Most Valuable Exercises Tuesday, March 18, 14
  • 9. Highlights: Class Manifesto • “AMST 5011 is an experimental graduate level course committed to identifying, evaluating, creating and rethinking solutions to educational challenges that our changing society faces in the twenty-first century. We aim to seize opportunities to fully realize and harness the possibilities of twenty-first-century literacies, which we define as the mindsets, skills, and collaborative techniques needed to make full use of the Internet as a space of learning. We believe that the Internet and technology are changing how individuals and communities understand themselves and the world around them, and that this connected age offers a tremendous opportunity to make teaching, learning, and knowledge more accessible, more affordable, and more meaningful for everyone involved. AMST 5011’s purpose is to examine the ways in which technology influences educational dynamics and to collaboratively propose and share new possibilities for the Information Age so that we--scholars, teachers, and students--can best respond collectively to the challenges this new paradigm poses for learning.” Tuesday, March 18, 14
  • 10. Highlights: Class Manifesto • AMST 5011 Class Goals and Practices for the Semester (Spring 2014) • Create an evolving individual online presence aimed at advancing our digital literacy to exemplify the possibilities of technology in higher education • Collaborate in a constructive manner as we assign, guide, and assess the work of peers. • Practice judicious time management in assigning tasks to others and completing our own. • Arrive to class on time, fully prepared to participate, having completed assignments on time. • Represent ourselves and the class both online and face-to-face as engaged scholars who are committed to advancing the above principles. Tuesday, March 18, 14
  • 11. Highlights: Grading Contract • Contract Grading + Peer Evaluation:  Explanation and Contract Evaluation Method: You determine your grade for this course by fulfilling a contract that spells out in advance the requirements as well as the penalties for not fulfilling the terms of your contract.   Learning together giving and receiving feedback is the core concept of our class.    Every student will be responsible for submitting to Professor Koh anonymous peer- grading of his classmates twice this semester, a midterm evaluation (due by March 7th)  and the final (due by April 23d). A peer grader will evaluate the work of his / her classmates using Satisfactory / Unsatisfactory descriptive evaluation format (up to 300 words). The unsatisfactory midterm peer assessment is intended to provide the opportunity for a student to improve his / her performance by the final evaluation. In case of a student receiving an Unsatisfactory peer evaluation, recommendations for improvements are to be determined by the peers collaboratively. The goal is for everyone to produce satisfactory work, and the peers will work with each student to achieve that goal. Tuesday, March 18, 14
  • 12. Highlights: Grading Contract • Contract Grading: The advantage of contract grading is that, the student, decides how much work he / she wishes to do this semester; if the work is completed on time and satisfactorily, the student will receive the grade for which he / she contracted.  This means planning ahead, thinking about all of the obligations and responsibilities this semester and also determining desired grade in the course. Tuesday, March 18, 14
  • 13. Results • Most empowered class of students I’ve ever had the experience of teaching • Great commitment to work that goes over and above the other classes that I’ve taught • Collegial classroom environment • Amazing levels of engagement Tuesday, March 18, 14
  • 14. Challenges • Faculty “loss of control” • Giving up power, stepping outside of the classroom when students work on these documents • Faculty fear of lacking “rigor” etc. versus student empowerment, plus emphasis on collaboration • How to move beyond these issues? Tuesday, March 18, 14
  • 15. Finally... I leave you with a Youtube video created by my student William Albertson as part of his work for the class. Available here and on his website, http:// wpastocktondh.com/ Tuesday, March 18, 14