SlideShare une entreprise Scribd logo
1  sur  29
A Student Measure of the Quality of
      Open Digital Textbooks
                 Or
   A Method For Getting Data-Backed
          Student Opinions

                                    TJ Bliss

     This work is licensed under a Creative Commons Attribution 3.0 Unported License.
      All credited images fall under Fair Use, are intended for transformative purposes
                           only, and retain full copyright privileges.
John Dewey


    Education is necessary to the
  individual “to prepare him for the
    future life means to give him
command of himself; it means so to
 train him that he will have the full
 and ready use of all his capacities.”
                               Image Credit: http://c250.columbia.edu/
“Serving society is only one of
            higher education’s functions, but
            it is surely among the most
            important. At a time when the
            nation has its full share of
            difficulties...the question is not
Derek Bok
            whether universities need to
            concern themselves with society’s
            problems but whether they are
            discharging this responsibility as
            well as they should.”
                               Image Credit: http://helenedison.ucsd.edu/
Image Credit: http://thethinkerblog.com/
Image Credit: http://thethinkerblog.com/
Image Credit: http://edudemic.com/
Image Credits: flatworldknowledge.com; www.etftrends.com; www.eukicks.com
Image Credits: www.flatworldknowledge.com; www.mindingthecampus.com; www.debatepolitics.com
Dilbert     By Scott Adams




          Image credit: www.dilbert.com
It’s Alive!
The Birth of a Textbook Quality Measure
   Hard data to back up student opinions about
    the quality of their (open) digital textbooks



                                   Image Credit: www.yankeeroundtable.com
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final” Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final”Questionnaire
                                       Final Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
Project Kaleidoscope

8 community colleges and 4-year schools
       California to New York
Project Kaleidoscope

         Cross-institutional faculty teams
Aggregate OER-based textbook replacements (FWK)
     11 courses, 9,000 students in 2011-2012
490 Student Survey Responses
     From your perspective, what are the
characteristics of a high quality digital textbook?
Overall, what did you think of the digital textbook
            you used in your course?
  How would you rate the quality of the digital
   textbook you used in your course? Why?
10 Student Interviews

    In-depth probing
  Clarifying questioning
      Transcriptions
70 Journal Articles

          Peer reviewed
9 main textbook evaluation criteria
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final”Questionnaire
                                       Final Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
Thematic Analysis

              7 main steps
Data prep, reading, sorting, writing, axial
     coding, more sorting, finalizing
Target Themes

Navigation, Access, Performance, Interaction, Rel
     evance, Presentation, Impact, Diversity
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final”Questionnaire
                                       Final Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
Mental Modeling

Identify construct relationships
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final”Questionnaire
                                       Final Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
Item Development
        Writing
     Expert review
       Revision
  Cognitive interviews
       Revision
     Expert review
       Revision
Pilot Questionnaire
    6 Background Items
       44 Main Items
      8 Target Themes
October 2012 Administration
student surveys
student interviews                          Qualitative Data
literature review
                                                               thematic analysis
                      item revision


   Validated Model                                              Target Themes

                                      “Final”Questionnaire
                                       Final Questionnaire

    factor analysis                                            mental modeling



  Pilot Questionnaire                                          Conceptual Model


                                      item development
Next Steps

   Model evaluation and validation (Nov. ‘12)
             Item revision (Jan. ‘13)
      “Final” Questionnaire 2.0 (Feb. ‘13)
Further evaluation, validation, revision (Apr. ‘13)
      “Final” Questionnaire 3.0 (May ‘13)

Contenu connexe

En vedette

The Economist: Securing Livelihoods Special Edition
The Economist: Securing Livelihoods Special EditionThe Economist: Securing Livelihoods Special Edition
The Economist: Securing Livelihoods Special EditionThe Rockefeller Foundation
 
Securing the livelihoods and nutritional needs of fish dependent communities
Securing the livelihoods and nutritional needs of fish dependent communitiesSecuring the livelihoods and nutritional needs of fish dependent communities
Securing the livelihoods and nutritional needs of fish dependent communitiesThe Rockefeller Foundation
 
2014 digital marketing + social media fundamentals for nonprofits
2014 digital marketing + social media fundamentals for nonprofits2014 digital marketing + social media fundamentals for nonprofits
2014 digital marketing + social media fundamentals for nonprofitsCharleston PR & Design, LLC
 
August 1977 The Eighth Annual Report Of The Council On Environmental Quality
August 1977 The Eighth Annual Report Of The Council On Environmental QualityAugust 1977 The Eighth Annual Report Of The Council On Environmental Quality
August 1977 The Eighth Annual Report Of The Council On Environmental QualityObama White House
 
Crew Documents 020700 - 020754
Crew Documents 020700 - 020754Crew Documents 020700 - 020754
Crew Documents 020700 - 020754Obama White House
 
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannesOgilvy
 
Crew, Foia, Documents 011528 - 011724
Crew, Foia, Documents 011528 - 011724Crew, Foia, Documents 011528 - 011724
Crew, Foia, Documents 011528 - 011724Obama White House
 
The Neglected Dimension of Global Security: A Framework to Counter Infectious...
The Neglected Dimension of Global Security: A Framework to Counter Infectious...The Neglected Dimension of Global Security: A Framework to Counter Infectious...
The Neglected Dimension of Global Security: A Framework to Counter Infectious...The Rockefeller Foundation
 

En vedette (14)

The Economist: Securing Livelihoods Special Edition
The Economist: Securing Livelihoods Special EditionThe Economist: Securing Livelihoods Special Edition
The Economist: Securing Livelihoods Special Edition
 
EPA DROE Email 6.19.03 (b)
EPA DROE Email 6.19.03 (b)EPA DROE Email 6.19.03 (b)
EPA DROE Email 6.19.03 (b)
 
CAR Email 6.4.02 (ss)
CAR Email 6.4.02 (ss)CAR Email 6.4.02 (ss)
CAR Email 6.4.02 (ss)
 
Securing the livelihoods and nutritional needs of fish dependent communities
Securing the livelihoods and nutritional needs of fish dependent communitiesSecuring the livelihoods and nutritional needs of fish dependent communities
Securing the livelihoods and nutritional needs of fish dependent communities
 
2014 digital marketing + social media fundamentals for nonprofits
2014 digital marketing + social media fundamentals for nonprofits2014 digital marketing + social media fundamentals for nonprofits
2014 digital marketing + social media fundamentals for nonprofits
 
August 1977 The Eighth Annual Report Of The Council On Environmental Quality
August 1977 The Eighth Annual Report Of The Council On Environmental QualityAugust 1977 The Eighth Annual Report Of The Council On Environmental Quality
August 1977 The Eighth Annual Report Of The Council On Environmental Quality
 
CEI Email 10.24.03
CEI Email 10.24.03CEI Email 10.24.03
CEI Email 10.24.03
 
Crew Documents 020700 - 020754
Crew Documents 020700 - 020754Crew Documents 020700 - 020754
Crew Documents 020700 - 020754
 
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes
5 Quick Things to Consider Before Posting #CannesLions / #OgilvyCannes
 
ACCCRN City Projects
ACCCRN City ProjectsACCCRN City Projects
ACCCRN City Projects
 
RCEC Email 6.23.03
RCEC Email 6.23.03RCEC Email 6.23.03
RCEC Email 6.23.03
 
CEI Email 10.30.03
CEI Email 10.30.03CEI Email 10.30.03
CEI Email 10.30.03
 
Crew, Foia, Documents 011528 - 011724
Crew, Foia, Documents 011528 - 011724Crew, Foia, Documents 011528 - 011724
Crew, Foia, Documents 011528 - 011724
 
The Neglected Dimension of Global Security: A Framework to Counter Infectious...
The Neglected Dimension of Global Security: A Framework to Counter Infectious...The Neglected Dimension of Global Security: A Framework to Counter Infectious...
The Neglected Dimension of Global Security: A Framework to Counter Infectious...
 

Similaire à A Student Measure of the Quality of Open Digital Textbooks

Ashbaugh dissertation defense presentation
Ashbaugh dissertation defense presentationAshbaugh dissertation defense presentation
Ashbaugh dissertation defense presentationDRMLAID
 
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...IL Group (CILIP Information Literacy Group)
 
Inquiry based learning in Psychological Research Methods
Inquiry based learning in Psychological Research MethodsInquiry based learning in Psychological Research Methods
Inquiry based learning in Psychological Research Methodscilass.slideshare
 
Blurring boundaries to spark motivation: collaborative approaches to teaching...
Blurring boundaries to spark motivation: collaborative approaches to teaching...Blurring boundaries to spark motivation: collaborative approaches to teaching...
Blurring boundaries to spark motivation: collaborative approaches to teaching...megan.fitzgibbons
 
Coding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxCoding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxwrite12
 
Coding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxCoding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxwrite31
 
Web quests in the language classroom 2014
Web quests in the language classroom 2014Web quests in the language classroom 2014
Web quests in the language classroom 2014Cyndi Cardoso
 
Dr Andrea Reid
Dr Andrea ReidDr Andrea Reid
Dr Andrea Reid campone
 
Educ 526 presentation
Educ 526 presentationEduc 526 presentation
Educ 526 presentationchervitz
 
eAssessment in practice
eAssessment in practiceeAssessment in practice
eAssessment in practiceIan Brown
 
Arc credit hour
Arc credit hourArc credit hour
Arc credit hourPresone
 
Proj A Lesson Plan
Proj A Lesson PlanProj A Lesson Plan
Proj A Lesson Planfmccutchen
 
Asynchronous curriculum creation and id models
Asynchronous curriculum creation and id modelsAsynchronous curriculum creation and id models
Asynchronous curriculum creation and id modelsekramer4
 
Action research for_librarians_carl2012
Action research for_librarians_carl2012Action research for_librarians_carl2012
Action research for_librarians_carl2012srosenblatt
 
Quality Metrics for Learning Object Metadata
Quality Metrics for Learning Object MetadataQuality Metrics for Learning Object Metadata
Quality Metrics for Learning Object MetadataXavier Ochoa
 
Action research for_librarians_carl2012
Action research for_librarians_carl2012Action research for_librarians_carl2012
Action research for_librarians_carl2012srosenblatt
 
Qualitative Methods Workshop Day 2
Qualitative Methods Workshop Day 2Qualitative Methods Workshop Day 2
Qualitative Methods Workshop Day 2Jason Rutter
 
Introduction to the 5Es Model
Introduction to the 5Es ModelIntroduction to the 5Es Model
Introduction to the 5Es ModelDavid Geelan
 

Similaire à A Student Measure of the Quality of Open Digital Textbooks (20)

Ashbaugh dissertation defense presentation
Ashbaugh dissertation defense presentationAshbaugh dissertation defense presentation
Ashbaugh dissertation defense presentation
 
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...
Weiner & Weiner - Is There a Difference Between Critical Thinking and Informa...
 
Inquiry based learning in Psychological Research Methods
Inquiry based learning in Psychological Research MethodsInquiry based learning in Psychological Research Methods
Inquiry based learning in Psychological Research Methods
 
Blurring boundaries to spark motivation: collaborative approaches to teaching...
Blurring boundaries to spark motivation: collaborative approaches to teaching...Blurring boundaries to spark motivation: collaborative approaches to teaching...
Blurring boundaries to spark motivation: collaborative approaches to teaching...
 
Coding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxCoding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docx
 
Coding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docxCoding of the Interview Theme Analysis.docx
Coding of the Interview Theme Analysis.docx
 
Web quests in the language classroom 2014
Web quests in the language classroom 2014Web quests in the language classroom 2014
Web quests in the language classroom 2014
 
Dr Andrea Reid
Dr Andrea ReidDr Andrea Reid
Dr Andrea Reid
 
Educ 526 presentation
Educ 526 presentationEduc 526 presentation
Educ 526 presentation
 
eAssessment in practice
eAssessment in practiceeAssessment in practice
eAssessment in practice
 
Arc credit hour
Arc credit hourArc credit hour
Arc credit hour
 
Presentatie ID aan UA
Presentatie ID aan UAPresentatie ID aan UA
Presentatie ID aan UA
 
Proj A Lesson Plan
Proj A Lesson PlanProj A Lesson Plan
Proj A Lesson Plan
 
Asynchronous curriculum creation and id models
Asynchronous curriculum creation and id modelsAsynchronous curriculum creation and id models
Asynchronous curriculum creation and id models
 
Action research for_librarians_carl2012
Action research for_librarians_carl2012Action research for_librarians_carl2012
Action research for_librarians_carl2012
 
LOTW
LOTWLOTW
LOTW
 
Quality Metrics for Learning Object Metadata
Quality Metrics for Learning Object MetadataQuality Metrics for Learning Object Metadata
Quality Metrics for Learning Object Metadata
 
Action research for_librarians_carl2012
Action research for_librarians_carl2012Action research for_librarians_carl2012
Action research for_librarians_carl2012
 
Qualitative Methods Workshop Day 2
Qualitative Methods Workshop Day 2Qualitative Methods Workshop Day 2
Qualitative Methods Workshop Day 2
 
Introduction to the 5Es Model
Introduction to the 5Es ModelIntroduction to the 5Es Model
Introduction to the 5Es Model
 

Plus de The William and Flora Hewlett Foundation (8)

Textbook Affordability and OER: A National Strategy
Textbook Affordability and OER: A National StrategyTextbook Affordability and OER: A National Strategy
Textbook Affordability and OER: A National Strategy
 
OER Symposium at Utah Valley University 2_9_2017
OER Symposium at Utah Valley University 2_9_2017OER Symposium at Utah Valley University 2_9_2017
OER Symposium at Utah Valley University 2_9_2017
 
An "I-Frame" for Open Educational Resources (OER)
An "I-Frame" for Open Educational Resources (OER)An "I-Frame" for Open Educational Resources (OER)
An "I-Frame" for Open Educational Resources (OER)
 
Open Educational Resources
Open Educational ResourcesOpen Educational Resources
Open Educational Resources
 
An Open Conversation about Open Educational Resources
An Open Conversation about Open Educational ResourcesAn Open Conversation about Open Educational Resources
An Open Conversation about Open Educational Resources
 
Textbook Study Strategies
Textbook Study StrategiesTextbook Study Strategies
Textbook Study Strategies
 
OER Policy and Development
OER Policy and DevelopmentOER Policy and Development
OER Policy and Development
 
Introduction to OER - Workshop
Introduction to OER - WorkshopIntroduction to OER - Workshop
Introduction to OER - Workshop
 

A Student Measure of the Quality of Open Digital Textbooks

  • 1. A Student Measure of the Quality of Open Digital Textbooks Or A Method For Getting Data-Backed Student Opinions TJ Bliss This work is licensed under a Creative Commons Attribution 3.0 Unported License. All credited images fall under Fair Use, are intended for transformative purposes only, and retain full copyright privileges.
  • 2. John Dewey Education is necessary to the individual “to prepare him for the future life means to give him command of himself; it means so to train him that he will have the full and ready use of all his capacities.” Image Credit: http://c250.columbia.edu/
  • 3. “Serving society is only one of higher education’s functions, but it is surely among the most important. At a time when the nation has its full share of difficulties...the question is not Derek Bok whether universities need to concern themselves with society’s problems but whether they are discharging this responsibility as well as they should.” Image Credit: http://helenedison.ucsd.edu/
  • 7. Image Credits: flatworldknowledge.com; www.etftrends.com; www.eukicks.com
  • 8. Image Credits: www.flatworldknowledge.com; www.mindingthecampus.com; www.debatepolitics.com
  • 9. Dilbert By Scott Adams Image credit: www.dilbert.com
  • 10. It’s Alive! The Birth of a Textbook Quality Measure Hard data to back up student opinions about the quality of their (open) digital textbooks Image Credit: www.yankeeroundtable.com
  • 11. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final” Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 12. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final”Questionnaire Final Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 13. Project Kaleidoscope 8 community colleges and 4-year schools California to New York
  • 14. Project Kaleidoscope Cross-institutional faculty teams Aggregate OER-based textbook replacements (FWK) 11 courses, 9,000 students in 2011-2012
  • 15. 490 Student Survey Responses From your perspective, what are the characteristics of a high quality digital textbook? Overall, what did you think of the digital textbook you used in your course? How would you rate the quality of the digital textbook you used in your course? Why?
  • 16. 10 Student Interviews In-depth probing Clarifying questioning Transcriptions
  • 17. 70 Journal Articles Peer reviewed 9 main textbook evaluation criteria
  • 18. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final”Questionnaire Final Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 19. Thematic Analysis 7 main steps Data prep, reading, sorting, writing, axial coding, more sorting, finalizing
  • 20. Target Themes Navigation, Access, Performance, Interaction, Rel evance, Presentation, Impact, Diversity
  • 21. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final”Questionnaire Final Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 23.
  • 24. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final”Questionnaire Final Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 25. Item Development Writing Expert review Revision Cognitive interviews Revision Expert review Revision
  • 26. Pilot Questionnaire 6 Background Items 44 Main Items 8 Target Themes October 2012 Administration
  • 27.
  • 28. student surveys student interviews Qualitative Data literature review thematic analysis item revision Validated Model Target Themes “Final”Questionnaire Final Questionnaire factor analysis mental modeling Pilot Questionnaire Conceptual Model item development
  • 29. Next Steps Model evaluation and validation (Nov. ‘12) Item revision (Jan. ‘13) “Final” Questionnaire 2.0 (Feb. ‘13) Further evaluation, validation, revision (Apr. ‘13) “Final” Questionnaire 3.0 (May ‘13)

Notes de l'éditeur

  1. Former President of Harvard University
  2. Former President of Harvard University$17.5K up from $7.5KTextbooks – avg. student spends over $900/year
  3. One of, if not the most important of motivations behind the open education movement is the desire to do a better job of discharging the great responsibilities of education by lowering costs and increasing access.
  4. The power of the open textbook:Reduces costCustomizable
  5. We want to give faculty hard data to back up their opinions about how to adapt their open digital textbooks from
  6. Version 2.0 will be openly licensed and made publicly available in early 2013.Version 3.0 will be openly licensed and made publicly available in mid 2013