MOOC Research: What Can We Do with Big Data?
•Télécharger en tant que PPTX, PDF•
4 j'aime•2,516 vues
This document discusses research opportunities using data from massive open online courses (MOOCs). It provides an overview of MOOC participation numbers and data on the University of California, Irvine's experience offering open educational resources through online channels. Some key findings are that MOOC users tend to be well-educated, career-oriented informal learners and that data from UCI courses suggest MOOCs can help at-risk students and that online and on-campus course performance is comparable. The document concludes that addressing barriers like privacy issues is important to enable further collaborative MOOC research.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (6)
Similaire à MOOC Research: What Can We Do with Big Data?
Similaire à MOOC Research: What Can We Do with Big Data? (20)
Plus de Open Education Consortium
Plus de Open Education Consortium (20)
Dernier
Dernier (20)
MOOC Research: What Can We Do with Big Data?
- 1. MOOC RESEARCH: WHAT CAN WE DO WITH BIG DATA? GARY W. MATKIN, PH.D. DEAN, CONTINUING EDUCATION, DISTANCE LEARNING, AND SUMMER SESSIO N UNIVERSITY OF CALIFORNIA, IRVINE OPEN EDUCATION CONSORTIUM GLOBAL CONFERENCE BANFF, ALBERTA, CANADA THURSDAY, APRIL 23, 2015
- 3. What Can MOOCs Tell Us About the Future Of Education?
- 4. MOOCs AND BIG DATA: 2014 OVERVIEW • OpenCourseWare website: 2.7 million page views • YouTube: 1.5 million visits • OpenChem: 5.5 million minutes watched • MOOCs: 1.3 million participants (2013- 2015)
- 5. UCI’S EXPERIENCE WITH OER • Three channels • UCI OCW • YouTube • MOOCs (Coursera, Canvas)
- 6. UC IRVINE’S OCW • 2014 average visits per month was 27,722 • 2015 monthly average is 53,762 for January – March • Over 430,000 page views per month
- 7. UCI on YouTube As of the end of December 2014, UCI’s YouTube channel had received: • 1.5 million visits • 13 million minutes watched
- 8. UCI MOOCs 1.3 MILLION PARTICIPANTS IN COURSERA & COURSERA COURSES (TOTAL AS OF MARCH 2015) • 249,067 in Career Readiness courses • 683,146 in Undergraduate courses • UCI was one of the first Coursera partners to adopt the “always open” format for presenting course sequences.
- 9. TYPES OF MOOC RESEARCH • Use of MOOCs • Impact of MOOCs • MOOCs and Learning
- 10. UCI CAREER READINESS • Majority of students are ages 25-34 • A majority of students are taking the specialization to learn new skills and to advance in their careers • 40% of responders said they plan on earning a verified certificate • The majority of responders are from North America (34%), but a large portion are from Europe (26%) and Asia (21%) • 73% have enrolled in a MOOC • 60% hold full-time jobs, working 40+ hours a week • Highly educated: 35% Bachelor’s, 34% Master’s and 6% Ph.D., Law or Medical degree • 46% had never heard of UCI prior to enrolling in this MOOC
- 11. USE OF MOOCS: WHO USES MOOCS • MOOC users are typically • 25-64 years old • Well educated and economically advantaged • Informal learners • Pursuing a personal interest • Residents of a country other than the U.S. • Male
- 12. IMPACT OF MOOCs: GENERAL • Prompts discussion of higher education and its reaction to technology • Pushes online education forward on campuses • Become objects for government policy • Proved promise for addressing worldwide issue over access and affordability • MOOCs and academic credit • MOOCs and credentialing
- 13. THE IMPACT OF MOOCs: SPECIFIC (THE FUTURE) • Deserving audiences • Research objects • Translations and localizations
- 14. UC IRVINE’S CSET OCW OVER 30,000 PAGE VIEWS THE FRIDAY PRIOR TO THE EXAM
- 15. MOOCS AND LEARNING • Flipped classroom • Informal learning • At risk students • Universal courses • Course commons
- 16. UC IRVINE’S OPEN CHEMISTRY Over 630,000 views in 2014 Over 5.5 million minutes watched in 2014
- 17. UC IRVINE’S PRE-BIOLOGY RESEARCH • UCI counseled at-risk, entering freshmen to take a Pre-Biology MOOC offered through Coursera • Incentive to complete the course with “Distinction” to transfer into biology as a major more quickly • 450 UCI students joined 37,000 students who enrolled in the MOOC • Students who completed the MOOC did significantly better than the two control groups
- 18. UCI MATH COURSE RESEARCH • Data includes student grades and final exam scores from face-to-face, online, and MOOC versions of Pre-Calculus and similar grades and final exam scores in a subsequent Calculus course • Students in online vs. on-ground Pre-Calculus courses performed comparably on equivalent final exams • Students in online courses spend more time on task in the course than on-ground students • MOOC students who opt to use the paid ALEKS software in parallel with the course achieve and complete the course at a higher level than those who did not • There are no discernible differences in subsequent Calculus course success between students taking on-ground and online courses
- 19. BARRIERS TO MOOC RESEARCH • Personally identifiable information (and FERPA) • Ownership of data • Messy data • Intellectual property
- 20. THE FUTURE • Collaborative efforts to coordinate research • Institutional stakes in research • Replicability • Dissemination
- 21. FOR MORE INFORMATION Visit UC Irvine’s Open Education Website: http://ocw.uci.edu Or contact Gary W. Matkin at gmatkin@uci.edu Download this presentation at www.slideshare.net/garymatkin/oec2015
Notes de l'éditeur
- This presentation provides a summary of Massive Open Online Course (MOOC) research and how it’s being organized around the world. MOOCs offer research objects that have the potential to address many of the issues higher education researchers face. They present new and unique opportunities to understand how people learn across a broad spectrum of educational mediums. MOOCs cross the boundaries between formal and informal learning in an unprecedented way, with each MOOC course offering opportunities for researchers to study how people select and engage with learning resources. This presentation will identify important questions: how are these research efforts being focused? What are they trying to learn? What impact are they having? What are they revealing about higher education? It also will explore the current state of MOOC research, summarize the approaches being taken, highlight some of the results that are coming from the research, and make predictions about what we might expect in the future.
- The hype around MOOCs ebbs and flows. But insofar as MOOCs are open educational resources, they are now a permanent fixture in international higher education.
- A more relevant question is what we can project about the future of higher education from our current experience in OER and MOOCs. The reaction to and backlash from MOOCs tell us quite a bit about how higher education reacts to potentially transformative factors. Research on MOOCs is beginning to paint a picture of the broader landscape for education in the world and about how people choose to consume education as part of their daily lives.
- It is clear that MOOCs present a lot of “big data.” Here are some examples from just one university showing the massive scale of the educational opportunities offered through MOOCs and other open offerings. Mining this data can provide scale that is required to gain statistical validity allowing us to go more deeply into the learning process than ever before. Most learning studies are burdened with low numbers of students in highly unique circumstances which are largely ungeneralizable to the larger possibilities.
- UCI has considerable experience in the production and publishing of OER, through three main channels: UCI OCW, YouTube and MOOCs (Coursera & Canvas).
- UCI’s OpenCourseWare site, established in November 2006 is currently attracting over 53,000 visits per month, an increase over the average for 2014 of 27,000 visits per month. This site allows UCI to feature its faculty and courses in many ways, including the clustering of courses and learning assets into “collections” around a specific subject area, assets both generated by UCI faculty and curated by them from open material form other courses. The UCI OCW site now offers 750 courses and 1,800 video lectures.
- UCI’s YouTube offerings are attracting over 1.3 million minutes watched per month (as of March 2015), about half of which are our OpenChem courses (1.1 million views in 2014).
- UCI has been a leader in the MOOC movement with currently about 500,000 active learners, mainly in three course sequences, Virtual Teacher, Career Readiness, and Project Management. UCI was one of the first Coursera partners to adopt the “always open” format for presenting course sequences.
- So far MOOC research has concentrated on three categories of research questions. The use of MOOCs category involves an investigation about who is using MOOCs and why. A second category looks at the impact of MOOCs, on institutions, faculty roles, and as projected into the future. Third category looks at how MOOCs might improve learning and teaching.
- A very recent example of how MOOCs are used comes from a survey UCI did of its just recently launched career readiness course sequence. The survey was offered to all students who enrolled in the courses. Within 24 hours over 4,200 students had completed the survey with the results as shown on the slide. As the next slide indicates these results are typical of most such surveys.
- This slide summarizes many investigations into who are taking MOOCs.
- Most of the research on the impact of MOOCs comes from the gathering of press and policy statements from many sources. The main impact of MOOCs can be covered by the six factors listed on this slide. MOOCs have provided an entry point for a discussion, often in the form of criticisms, over the role and future of higher education. They are often mentioned as either something that will transform higher education as we know it or a fad that will quickly disappear and its underlying weaknesses as a pedagogical tool are exposed. In general this discussion is healthy, although there are some nasty edges to some discussions. As MOOCs were offered by the top ranked universities in the country and world, the acceptance of online education as a legitimate means of course offerings increased on most campuses. It is almost as if this extreme and limited form of online education distracted those who were focusing their negativity on online education to this new form so that regular online education became more tolerable. Also, MOOCs exposed faculty members to some very high quality online material and this exposure helped faculty members see the value of online education in general. MOOCs also, slowly at first, and then more rapidly captured the attention of governmental policy makers around the world who began to see that free education had the possibility of solving one of the world’s most pressing problems—how to educate an additional 1 billion people. Now many governments, including the Obama administration, through a huge department of labor grant, is mandating that government funded course development be offered under open licenses for general uses. This last possibility has been seen not only by governments but also by large patrons of higher education such as UNESCO, the World Bank, and large Foundations. These organizations also have seen the utility of using MOOCs to serve deserving, underserved, and economically disadvantaged audiences such as elementary teachers in Africa. The many efforts to attach academic credit to MOOCs and offer degrees based on MOOCs is growing rapidly. Examples range from highly respected major Universities such as Georgia Tech, with its low cost master’s degrees in computer science to the start-up University for the People which seeks to use MOOCs produced by other universities as major parts of the degrees it seeks to offer. MOOCs are also becoming aligned with the micro-credentialing movement which seeks to provide highly specific knowledge and know how to students at low cost.
- The future of MOOCs will quickly shift from general, university-level degree courses to the more specific. For instance, MOOCs will target specific audiences such as K-12 teachers in California who want to pass the qualifying exams for becoming certified science and math teachers or clinicians in Africa who need to be trained on how to keep themselves safe as they care for Ebola patients. MOOCs may also be designed to achieve specific research goals such as how students choose to advance through beginning biology. The future of MOOCs will also involve the reuse of existing material in alternative cultures through the translation and, more importantly, the localization of course material to fit the culture of the audience.
- Perhaps the first open courses directed at a specific and deserving audience was UCI’s CSET exam set of courses. It was also the first open adaptive learning course ever offered.
- The most exciting relationship between MOOCs and research is the possibilities we have for learning more about human learning, aided by the massive nature of the data we can gather. MOOCs, particularly in the “always open” mode have the ability to decant for every student essential information conveying material, most impact fully now in the form of video lecture capture delivered in “packages” that are just right for the learning objective of the student. This is sometimes expressed as the flipped classroom where MOOC lectures take the place of classroom lectures, allowing the classroom to be used for other, more personal interactions among and between students and teachers. The example of the UCI Open Chem project is of relevance here. MOOCs also help us understand informal learning—learning that takes place outside the often confining boundaries of the college course. For instance, MOOCs help us understand how students consume the material offered in MOOCs, counting how much and in what order learning objects are attended. MOOCs also can be directed at at risk students, allowing for a rich supplemental learning experience complete with self-assessments. The examples of the UCI biology and math courses are relevant. High quality MOOCs can also serve as universal courses—courses which serve as models for colleges and universities in the developing world for instance or even for peer institutions wishing to “benchmark” their courses with others. Many university courses, whether delivered by research universities or faith based small liberal arts colleges, cover the same material making comparisons relatively easy. MOOCs also can serve as the core of course commons, repositories open to others for review and use, reuse, and improvement.
- OpenChem is first and foremost to extend the benefit that we have seen since 2009 from open and free publication of individual chemistry courses to an entire curriculum. What MIT did ten years ago with its OpenCourseWare initiative was to plant the idea of making quality educational resources universally accessible. MOOCs have laudably extended this approach by providing instructional paths through individual courses at scale. What UCI hopes to do with this initiative is to present a coherent, full curriculum by a top faculty. Today, a learner can sit with us in our lecture halls and follow four years' worth of chemistry core classes and electives. That is the key innovation: making a full undergraduate education's worth of classes available for immediate incorporation in part or in full by institutions of higher education or by individual professors
- In this last category one of the most interesting research questions is to determine how learning pathways created by instructors match the pathways students naturally take in learning something. Because MOOCs are directed largely at informal learners and their platforms can trace learner behavior at very granular levels, researchers can begin to gage the gap in learning contexts between instructors, students, and diverse groups of learners. Formal and informal learning can be studied side by side. Cross national studies of learners from different countries can help us understand how MOOCs can be effective in delivering low cost education and how they must be localized. So far, no serious research has been done to understand what MOOCs might reveal in these areas. What has been done is a few studies on whether MOOCs can help students (at risk, low income, marginally qualified) do better academically in targeted academic courses. For instance, the University of California, Irvine, (UCI) counseled at-risk, entering freshman to take a “pre-biology” MOOC that was offered through Coursera. These students were given an incentive to complete the Coursera course “with distinction” to be able to transfer into biology as a major more quickly than if they did not take the course. The incentive worked; about 450 UCI students joined the 37,000 who enrolled in the MOOC. UCI found that those who competed the MOOC did significantly better than the two control groups, one that engaged in, but did not complete the Coursera MOOC, and one group that did not enroll in the MOOC.
- UCI has a rich data set from which to explore the effect of course format on student success in math courses. This data includes student grades and final exam scores from face-to-face, online, and MOOC versions of Pre-Calculus and similar grades and final exam scores in a subsequent Calculus course. The Pre-Calculus final exam from the various course formats has been produced in different ways, but the problem types and topics covered are controlled between the various course offerings. The Calculus final exam is a common exam administered across all sections of the course and is designed to be comparable between quarters. Further, all three delivery approaches to the Pre-Calculus course utilized ALEKS® learning software, which has a very rich set of learning analytics available to study. Some key preliminary findings on this data include: Students in online vs. on-ground Pre-Calculus courses performed comparably on equivalent final exams. Students in online courses spend more time on task in the course than on-ground students. MOOC students who opt to use the paid ALEKS software in parallel with the course achieve and complete the course at a higher level than those who did not. There are no discernible differences in subsequent Calculus course success between students taking on-ground and online courses.
- The advantages that MOOCs have of generating “big data” and highly valid statistical analysis has not been lost on higher education institutions. A consortium of researchers are beginning to form and the large offerors of MOOCs are encouraging the sharing of information among institutions. So, the future should see more collaborative research projects based on MOOC-generated data. One reason for this interest in collaborative research is that the data is useful to the institutions who can best seek and analyze it. Comparisons between courses and institutions can provide valuable information about learners and who is taking and who is benefiting from MOOCs and why. The value of this information spills over into regular offerings, particularly online offerings. Institutions also have the incentive to try to replicate studies so as to gain comparisons between their own offerings and those offerings of others. Finally, as a result of the factors listed here, dissemination of research results should be very broad. While higher education research can be significantly advanced through analyses of MOOCs, there are some barriers that need to be addressed. These issues include personally identifiable information (PII), ownership of data, the “messy” nature of the data, and intellectual property constraints. The following examines each of these barriers. Personally Identifiable Information (PII) Students have a federally mandated right to privacy. Any information that can be directly attributed to a student (name, email address, student I.D. number, or pattern of course interaction) must be isolated and protected. Coursera identifies three “tiers” of PII: unanonymizable data, public forum, and general course data. Unanonymizable data includes the PII elements described above and other natural products of MOOC education such as peer-graded assessments. Public forum data are student responses where the student clearly understands that his/her comments are available to the public. General course data are aggregate data where the student can’t be identified with responses or other attributes. The most effective education research has to identify responses with individual students even if only to collect group data, so PII comes into play. Because of this, most MOOC research on actual learning will probably require a human subject’s protocol to be completed with the researching institution. In more general terms, with the use of the “cloud” for storing data, university attorneys are becoming worried that data about students may be leaking to third parties every time a student opens an app or watches a recorded lecture. Ownership of Data Most MOOCs are provided through third parties such as Coursera, edX, and Udacity, rather than through university-controlled platforms. Thus the data generated from MOOCs is generally owned and controlled by the third party. Those third parties are generally willing to share the data with universities as long as their reputation is upheld and the data isn’t misused. However, the use of data for research has been less of an issue than the use of data for marketing purposes. Google, for instance, has disabled the scanning of student data for advertising purposes (19). This reflects the sensitivity that MOOC providers have about the use of data for commercial purposes even when those purposes are restricted to partner institutions. Messy Data Interpretation and focus is an inherent problem in the collection of large-scale data across a broad spectrum of learners. It is tempting to analyze the data and draw conclusions from it, even with little knowledge about how learners learn, what motivates them, and what they respond to. The spread and heterogeneity of the audience present large issues of interpretation. This problem is a growing concern among users of large amounts of data. In higher education there are few examples linking aggregate data collected over large groups with effective help for individual students. The extremely large size and great span of MOOC data is a cause of the “messiness.” Carefully created research questions are a partial answer to this issue. Intellectual Property There continues to be controversy over “who owns the course” when it is developed by a faculty member. In some institutions this issue has not been resolved. However, it is possible that a university could conduct a research study on a course without the permission of the author of the course (22). The issue of conducting research among students in a class is not usually addressed even when an institution has clearly established ownership policies or specific written agreements are required to define ownership rights and the use of the intellectual property.
- Much has been written about MOOCs and their potential disruptive impact on higher education. Some commentators welcome this impact seeing MOOCs as a way of decreasing the cost of higher education and improving learning. Others view MOOCs as disruptive to the true intent of higher education, as the commodification of education, and as a movement away from the traditional values of university education. More likely than either of these predictions is the probability that the biggest impact of MOOCs will be an increase in world knowledge about learning and the learning process. That knowledge, like most knowledge in the world, will be derived from a scientific investigation of learning through MOOCs conducted by university researchers.