Driving evidence-based practices through research ensuring students access and success
•Télécharger en tant que PPTX, PDF•
0 j'aime•202 vues
A presentation for University of Nebraska Online and System
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Driving evidence-based practices through research ensuring students access and success
Similaire à Driving evidence-based practices through research ensuring students access and success (20)
Dernier
Dernier (20)
Driving evidence-based practices through research ensuring students access and success
- 1. Dr. Tanya Joosten Co-Director and PI National Research Center for Distance Education and Technological Advancement (DETA) @UWMDETA @tjoosten slideshare.net/tjoostenDriving evidence-based practices through research ensuring students access and success
- 2. How do we ensure all students have access to a quality higher education?
- 5. F2F gold standard Flickr xenonbpics
- 7. Identify practices (instructional and institutional) that impact those outcomes
- 8. Conduct rigorous, interdisciplinary, and standardized research Flicker cc astronomyblog
- 9. Understand and determine distance education outcomes Flicker cc heycoach
- 10. Year 1 activity: Desired outcomes Access Learning Effectiveness Instructional Effectiveness Satisfaction http://uwm.edu/deta/desired-outcomes/
- 11. Goal: Develop research models Outcomes Research questions Framework of inquiry Measures Instrumentation
- 12. Year 1 activity: Host national summit
- 14. Research questions Defined What are the different design components (content, interactivity, assessments) that impact student learning? What patterns of behaviors lead to increased student learning for different populations? What support structures are critical to providing quality access to online instruction? Exploratory (Interpretive) What are the definitions of success from students’ perspective? How can we define and measure student success beyond traditional outcomes? What is the currency of student learning beyond the existing credit hours? What are the key components that promote a sustainable and an effective teaching and learning ecosystem?
- 15. Variables
- 16. Year 1 activity: Establish Framework of Inquiry
- 17. Year 2 activity: Research toolkits Shared measures – Student performance is based on numerical representation of grade converted to a 4.0 scale received in the course on assessments and as an overall grade. RQs - What are the different design components (content, interactivity, assessments) that impact student learning?
- 19. Year 2 activity: Research toolkit uwm.edu/deta/toolkits
- 20. Year 2 activity: RFP for pilot grant awards Flicker cc yourdon
- 21. Goal: Implement technology system for data
- 22. Year 2/3 goal: Produce effective use cases Flicker cc yourdon Content •Text •Images •Audio •Video Interactivity •Discussions •Groups •Feedback Assessment •Written and oral examination •Discursive •Portfolio
- 23. Year 3 goal: Technological diffusion Technology Rubric Easy 503b compliant Engagement Learning Satisfaction Cost
- 24. Year 2/3 activities: Evaluation rubric Flicker cc yourdon Learner support Organization Design Delivery Assessment Evaluation Engagement Integration Technology Appropriateness
- 26. Partners: UW-System, UW-Extension, Milwaukee Area Technical College, EDUCAUSE Year Goals Activities 1-2 Develop National Distance Education and Technology Advancements (DETA) research models for online education 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation 1-2 Implement technology systems for data collection, storage, and retrieval 1. Develop protocols for data collection 2. Implement software for the collection of data 3. Establish guidelines for data storage 4. Develop protocols for data retrieval 5. Implement hardware and software for the storage and retrieval of data 2-3 Collect data by replicating research models at the course, program, and institutional level 1. Produce research toolkits 2. Evaluate pilot grant proposal submissions 3. Engage research fellows in research projects 4. Ensure data collection addresses audiences of interest 2-3 Produce effective use cases for instructional improvement to enhance student learning through identification of success factors 1. Develop blueprints for best practices 2. Tag items for specific audiences 3. Develop rubric for course and program evaluation 2-3 Identify and disseminate success factors for competency-based distance education 1. Conduct research on CBE 2. Develop best practices blueprints for CBE 3 Develop guidelines and a rubric for the diffusion of technological advancements 1. Advance guidelines for technology diffusion 2. Develop technology evaluation rubric 1-3 Include disability in all aspects of the research goals 1. Embed disability specific questions/data fields within research instrumentation 2. Create curricular disability accessibility assessment tools 3. Identify which interventions impact learning for which type of disabilities
- 28. Research toolkit: Grant awards and RFP Flicker cc yourdon
- 29. Research toolkit: Research model
- 31. Research toolkit: Guides to research
- 32. Research toolkit: Data collection
- 33. Research toolkit: Data codebooks
- 34. An example What are the different design components (content, interactivity, assessments) that impact student learning?
- 35. An example
- 36. An example
- 37. An example
- 38. An example
- 39. An example
- 40. An example
- 41. Questions
- 42. Dr. Tanya Joosten, @tjoosten, tjoosten@uwm.edu http://www.uwm.edu/deta http://www.slideshare.edu/tjoosten
Notes de l'éditeur
- How can we promote practices on our campus that are evidence-based. This is argued any many institutions. Some have almost a divergent force when we talk about fulfilling our access mission, helping our students be successful – and pursuing research. At UWM we have lived this reality for years. Many efforts are to ensure student access and success while other efforts are furthering research though externally funded projects. However, I want to talk about how we can answer some key questions through collaborative research efforts that will help us promote student access and success in higher education, in particular distance education of which we see blended and online as prominent forms in this decade.
- At the end of the day, we want all students to be successful. They have met the criteria to be at our institutions, we hope to educate them and retain them through their degree completion. At public instititons…
- First, how do we provide access…. Our student demographic is changing across the country. Students needs are changing across the country. Students are working more hours than ever before and paying more to attend college than in previous years. I love when some of my colleagues tell me of a time when college was pretty much free. Well, that is not the case. It becomes a challenge for students to attend traditional courses. Also, students are understanding the efficiencies afforded by technology in their personal/social lives and bring those expectations to our campuses. Some of our research tells us students want mediated classes or distance education because -- Students are allowed to push time Courses from k-12 and post secondary/higher ed are based on a system of time and credit hours. Yet, as we live in a time that is much more hectic, with greater demands on our time due to changes in our society (two income households, more students, if not all, are working students, more undergraduates coming with families and jobs), we see that technology can help us become more efficient and overcome some of these barriers. Mediated learning opportunities, such as CBE, Online, and Blended, offer new pathways to degree providing students an access to an education that provides them more flexibility to manage their time and their lives. However, these alternate forms of mediated learning have been questions through the decades, which has often raised the question of quality.
- Second, how do we ensure quality? Traditionally, quality has been examined by comparing F2F to online, otherwise known as mode comparison studies Research indicates that there was little difference in student learning (Allen et al., 2004; Park & Gemino, 2001) or satisfaction satisfaction (Allen et al., 2002; Castle & McGuide, 2010; Lim, Morris, & Kupritz, 2006)… However, as Dziuban and Picciano (2015) discuss the “no significant difference phenomenon” referring to Roberts (2007) where they allude to the idea that research in online learning as “a kind of collective amnesia surrounds changes that happened over a more distant time frame. We tend to trust what we have seen for ourselves and thus dismiss events that occurred in the more distant past” (p. 13). Some researchers in disciplines newer to online learning tend to replicate the same studies with very little new to contribute to the understanding of the phenomenon. Moore and Kearsley (2011) mention “…one of the major threats to good practice as well as to good scholarship in distance education is the common failure of the newcomers to the field to understand what a depth of knowledge there is” (p. xvi). More recently, some practitioners and researchers realized to better understand online learning they need to look more at process variables and build off of the previous decades of research.
- Ensuring quality isn’t about comparing f2f and online…ensure quality is more. We know that F2F is not the gold standard either. There is good f2f and bad f2f, good and bad online…now, let’s ensure all education is good, quality education. The online medium brings many new questions to an instructor. It is no longer about just how do I put my lectures and exams online. Well, those probably should have never been the questions. The transformation becomes just as much, if not more, about the pedagogy and social processes in the classroom as it does about the technology.
- Which brings us back to the question…how do we ensure quality. When we think about how we might teach in our f2f and then consider how do we transform our course for the online environment, we have new questions. Or, we should. Designing courses and teaching online for 15 years and leading faculty development efforts for blended and online for 10 years, there are lots of questions we need to be asking ourselves to ensure quality. What documentation and evidence can students’ provide for us to assess them in the online environment? Is this rigorous? Does it provide opportunity for frequent feedback? Is the assessment plan varied, in small manageable chunks keeping student engaged? Am I helping them develop 21st century literacy skills What learning experience or activities can we develop that utilize technology that will assist them in producing this assessment? What technologies do we use for which activities? What is the best way to delivery content as part of this learning activity? What type of media due I use and when? Text, Text and Images, Video Do I mimick the f2f? How do I organize this stuff online? How do I best support my students to ensure they have a quality experience and do well in the course? What is this peer instruction and team-based learning? Why is it so important in distance education? UWM actually developed a document called the 10 questions for blended, then online, then MOOCs… These questions and opportunities provided when we think about our instruction and learning in online courses are then in return impacting our f2f courses.
- How do we design our instruction? And how do we design our institutional support mechanisms? Is there sound, rigorous research backing up these decisions? There are an array of considerations in content, assessment, interactivity (including supporting our students) that need to be contemplated in making these design and development decisions for our programs and our institutions. It isn’t about time – hours spent in a seat or even ours spent out of class or online. It is about documented learning outcomes and their careful alignment to assessments and to learning activities.
- So, the efforts of the National Research Center for Distance Education and Technological Advancements (DETA) funded by a US Dept of Ed FIPSE grant Is to conduct research across institutions that helps bring greater clarity to these key factors in course and program design to help student have a quality experience. Ensure outcomes for all students, including those with disabilities We as a community need to conduct more research. Add -- Blog post…why research is not as great as it could be…
- Before we got started, we had a few questions to think about. At the end of the grant -- What is success? What are we trying to influence/improve? What are our desired results? Something we have learned in designing online courses is that we need to know what our desired results are before we can develop assessments and activities. The same goes for research in distance education. We need to understand and determine distance education outcomes before we can develop research to ensure we are meeting those outcomes.
- Desired Outcomes Access All learners who wish to learn online can access learning in a wide array of programs and courses,1 particularly underrepresented, those with disabilities and minorities.2 An essential component in distance education is a comprehensive infrastructure for learning that provides all individuals with the resources they need when and where they are needed. The underlying principle is that infrastructure includes people, instructional resources, processes, learning resources, policies, broadband, hardware, and software. It brings state-of-the art technology into learning to enable, motivate, and inspire all students, regardless of background, languages, or disabilities, to achieve.4 Data can be collected by examining administrative and technical infrastructure, which provides access to all prospective and enrolled learners. Access quality metrics are used for information dissemination, learning resource delivery, and tutoring services.1 Other possibilities include data gathered from student information systems, from student perception surveys, or objective accessibility ratings of online courses and programs. Learning effectiveness Learning effectiveness indicates a demonstration that learning outcomes were met or exceeded standards.1 This includes areas of study with research outcomes focusing on student success in achieving learning outcomes2 and other potential indicators of achievement (success, failure, achievement gains, academic achievement, improvement).3 Moreover, learning effectiveness could also include topics of retention (of content) and retention in a course (sometimes called attrition) or program (degree completion). Typically data are gathered through direct assessment of student learning (e.g. overall grades, exam grades, or other assessments), faculty perception surveys, faculty interviews comparing learning effectiveness in delivery modes, and student focus groups or interviews measuring learning gains.1 Additionally, requests for new and better ways to measure what matters include concurrent data collection. Here, focusing on diagnosing strengths and weakness during the course of learning provides the opportunity for more immediate improved student performance. Furthermore, these technology-based assessments provide the opportunity to allow data to drive decisions on the basis of what is best for each and every student based on their unique attributes and interactivity in class.4 Other possibilities include data gathered from student information systems or from student perception surveys. Satisfaction Faculty are pleased with teaching online, citing appreciation and happiness. Students are pleased with their experiences in learning online, including interaction with instructors and peers, learning outcomes that match expectations, services, and orientation.1 Faculty and student surveys can indicate equal or growing satisfaction to traditional forms of learning. Other metrics can include repeat teaching of online courses by individual faculty and increase in percentage of faculty teaching online showing growing endorsement. Qualitative methods can include interviews, focus groups, testimonials with faculty, staff (including advisors and tutors), and/or students.1 Instructional effectiveness Instructional effectiveness indicates the quality of education meets program, institutional, and national standards.1 The focus is on what and how we teach to match what people need to know, how they learn, where and when they will learn, and who needs to learn.4 The areas of study might include instructional improvement, program effectiveness, administrator effectiveness, curriculum evaluation, educational quality, outcomes of education programs, and instructional media.3 Additionally, instructional effectiveness is not limited to instruction provided inside the classroom, but extends itself to instructional support or supplemental instruction and guidance provided through institutional services or through staff and individuals outside of the classroom. Traditionally, as in face-to-face delivered courses, student ratings of instructional effectiveness are collected. However, typically these standards in distance education and online learning are communicated in a course or program rubric (e.g., UC Chico, QM) which is administered through an objective rating of a course or program in addition to traditional methods. Recent work looks to gather this data through student perceptions of instructional effectiveness through course and program rubrics converted to student surveys. Other possibilities include objective ratings of online course and program design and instructional delivery. References: 1. Online Learning Consortium, 5 Pillars 2. U.S. Department of Education, Application for Grants 3. What Works Clearinghouse 4. National Ed Tech Plan, U.S. Department of Education
- Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
- Provide details on the summit at the ELI Annual Meeting Where, when, why, who, how…
- Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
- Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
- Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
- Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
- Final Year 1 Grant Activities: 4. Formulate measures 5. Establish research instrumentation Which were included in the research toolkit. The toolkits contain several research models or research designs to facilitate coherent cross institutional analysis including measures and instrumentation
- Diane: Now, as mentioned, the toolkits are a year 2 activity that has already taken place, to assist us in our primary year 2 goal -- Collect data by replicating research models at the course, program, and institutional level Let’s talk in a bit more detail how the contents of the toolkits are going to help make this happen…
- Diane The research toolkits produced from year 1 activities will be used by our research fellows, institutional partners, CBEN institutions Furthermore, they will provide the guides and tools necessary too facilitate rigorous cross institutional research from experimental and survey designed studies gathering data from in-class assessment, survey packets, and student information system or data warehoused student data. Note: Do not go into detail until slide 22 Now, let’s talk about the potential for you to get funding to collaborate with us….
- Diane: Request for proposals for DETA subgrant awards is available in the research toolkit in Section 1. You will find background information, grant participation requirements, proposal requirements, and proposal review criteria. The goal with the process is not to build hurdles in obtaining funding and support, but to ensure rigorous cross-institutional research and data collection and address audiences of interest Goal 2 activities 2. Evaluate pilot grant proposal submissions 3. Engage research fellows in research projects 4. Ensure data collection addresses audiences of interest
- Collect data by replicating research models at the course, program, and institutional level Discuss research toolkits produced from year 1 being used by our research fellows, institutional partners, CBEN institutions
- Produce effective use cases for instructional improvement to enhance student learning through identification of success factors Activities 1. Develop blueprints for best practices 2. Tag items for specific audiences Dissemination 1. Effective practices use cases (accessible multimedia) through DETA Community Website and ELI SEI Website
- Goal 6: Develop Guidelines and a Rubric for the Diffusion of Technological Advancements 1. Advance guidelines for technology diffusion 2. Develop technology evaluation rubric Dissemination 1. Rubric through DETA Community Website 2. Guidelines for technology diffusion through DETA Community Website
- 3. Develop rubric for course and program evaluation 2. Rubric through DETA Community Website and ELI SEI Website
- Dissemination of year 1 activities will include the research toolkits we mentioned including priority research questions, a framework of inquiry, including variables and shared measures, methodologies, reporting standards In year 2 and 3 we will use these findings to develop effective use cases
- Diane The research toolkits produced from year 1 activities will be used by our research fellows, institutional partners, CBEN institutions Furthermore, they will provide the guides and tools necessary too facilitate rigorous cross institutional research from experimental and survey designed studies gathering data from in-class assessment, survey packets, and student information system or data warehoused student data. Note: Do not go into detail until slide 22 Now, let’s talk about the potential for you to get funding to collaborate with us….
- Diane: Request for proposals for DETA subgrant awards is available in the research toolkit in Section 1. You will find background information, grant participation requirements, proposal requirements, and proposal review criteria. The goal with the process is not to build hurdles in obtaining funding and support, but to ensure rigorous cross-institutional research and data collection and address audiences of interest Goal 2 activities 2. Evaluate pilot grant proposal submissions 3. Engage research fellows in research projects 4. Ensure data collection addresses audiences of interest
- Tanya: The first goal of the grant activities is to develop research models for online learning that provide guidance in the practice of distance education research. The models were intended to facilitate the exploration of instructional practices, inform future instructional practices, serve as a model for future research practices across educational institutions, and enhance consistency in the field. In the development process, it became clear that a more general research model was needed to represent the various research models and designs that would be deployed as part of the DETA research efforts rather than several specific research models. More information is available in the toolkit or on our website.
- Diane could discuss methodological considerations – how it can help them do research, etc. She could mention WWC standards
- Tanya can discuss gathering data through SIS and Surveys Diane can mention how these will compliment experimental studies
- Tanya can discuss how codebooks will help with data merging and how these are iterative…they will be improved once partners are identified and as data acquisition and merging