SlideShare une entreprise Scribd logo
1  sur  19
Télécharger pour lire hors ligne
The Application of IRT using
    the Rasch Model in
  Constructing Cognitive
         Measures

          Carlo Magno, PhD
     De La Salle University-Manila


                                     1
Outline
•   Psychometric Theory
•   Classical test Theory (CTT)
•   Item Response Theory (IRT)
•   Approaches in IRT
•   Issues in CTT
•   Advantages of the Rasch Model
•   Assumptions of the Rasch Model
•   Uses of the Rasch Model
•   Procedure in Rasch Model
•   Workshop

                                     2
Psychometrics
• Psychometrics concerns itself with the
  science of measuring psychological
  constructs such as ability, personality,
  affect and skills.
• Research in psychology involves the
  measurement of variables in order to
  conduct further analysis.


                                             3
Branches of Psychometric
            Theory
• Classical Test Theory
• Item Response Theory




                               4
Classical Test Theory (CTT)
• Regarded as the “True Score Theory”
• Responses of examinees are due only to
  variation in ability of interest
• All other potential sources of variation existing in
  the testing materials such as external conditions
  or internal conditions of examinees are assumed
  either to be constant through rigorous
  standardization or to have an effect that is
  nonsystematic or random by nature.

                                                     5
Classical Test Theory (CTT)
                   TO = T + E
• The implication of the classical test theory
  for test takers is that test are fallible
  imprecise tools
• Error = standard error of measurement
                 Sm = S√ 1 - r
• True score = M +- Sm = 68% of the
  normal curve
                                                 6
Normal curve
            One SE




            Range of True
     mean   Scores
                            7
Focus of Analysis in CTT
• frequency of correct responses (to indicate
  question difficulty);
• frequency of responses (to examine
  distracters);
• reliability of the test and item-total correlation
  (to evaluate discrimination at the item level)




                                                       8
Item Response Theory
• Synonymous with latent trait theory, strong true
  score theory or modern mental test theory
• More applicable for tests with right and wrong
  (dichotomous) responses
• An approach to testing based on item analysis
  considering the chance of getting particular
  items right or wrong
• Each item on a test has its own item
  characteristic curve that describes the
  probability of getting each particular item right or
  wrong given the ability of the test takers (Kaplan
  & Saccuzzo, 1997)
                                                         9
Logistic Item Characteristic
               Curve
– A function of ability
  (θ) – latent trait
– Forms the boundary
  between the
  probability areas of
  answering an item
  incorrectly and
  answering the item
  correctly



                                  10
Approaches of IRT
• One dimension (Rasch)→ One parameter
  model = uses only the difficulty parameter
• Two dimension → Two parameter Model
  = difficulty and ability parameter
• Three dimension (Logistic Model)→ Three
  Parameter Model = ability, correct
  response, item discrimination


                                           11
Issues in CTT
• A score is dependent on the performance of the group
  tested (Norm referenced)
• The group on which the test has been scaled has
  outlived has usefulness across time
   – Changes in the defined population
   – Changes in educational emphasis
• There is a need to rapidly make new norms to adopt to
  the changing times
• If the characteristics of a person changes and does not
  fit s specified norm then a norm for that person needs to
  be created.
• Each collection of norms has an ability of its own =
  rubber yardstick
                                                          12
Advantages of the Rasch Model
• The calibration of test item difficulty is
  independent of the person used for the
  calibration.
• The method of test calibration does not matter
  whose responses to these items use for
  comparison
• It gives the same results regardless on who
  takes the test
• The scores a person obtain on the test can be
  used to remove the influence of their abilities
  from the estimation of their difficulty. The result
  is a sample free item calibration.

                                                        13
Rasch Model
• Rasch’s (1960) main motivation for his
  model was to eliminate references to
  populations of examinees in analyses of
  tests.
• According to him that test analysis would
  only be worthwhile if it were individual
  centered with separate parameters for the
  items and the examinees (van der Linden
  & Hambleton, 2004).

                                          14
Rasch Model
• The Rasch model is a probabilistic
  unidimensional model which asserts that:
       (1) the easier the question the more
  likely the student will respond correctly to
  it, and
       (2) the more able the student, the more
  likely he/she will pass the question
  compared to a less able student.
                                             15
Rasch Model
• The model was enhanced to assume that the
  probability that a student will correctly answer a
  question is a logistic function of the difference
  between the student's ability [θ] and the difficulty
  of the question [β] (i.e. the ability required to
  answer the question correctly), and only a
  function of that difference giving way to the
  Rasch model
• Thus, when data fit the model, the relative
  difficulties of the questions are independent of
  the relative abilities of the students, and vice
  versa (Rasch, 1977).
                                                     16
Assumptions of the Rasch Model
   According to Fisher (1974)
• (1) Unidimensionality. All items are functionally
  dependent upon only one underlying continuum.

• (2) Monotonicity. All item characteristic functions
  are strictly monotonic in the latent trait. The item
  characteristic function describes the probability
  of a predefined response as a function of the
  latent trait.
• (3) Local stochastic independence. Every person
  has a certain probability of giving a predefined
  response to each item and this probability is
  independent of the answers given to the
  preceding items.
                                                    17
Assumptions of the Rasch Model
   According to Fisher (1974)
• (4) Sufficiency of a simple sum statistic. The
  number of predefined responses is a sufficient
  statistic for the latent parameter.
• (5) Dichotomy of the items. For each item there
  are only two different responses, for example
  positive and negative. The Rasch model
  requires that an additive structure underlies the
  observed data. This additive structure applies to
  the logit of Pij, where Pij is the probability that
  subject i will give a predefined response to item
  j, being the sum of a subject scale value ui and
  an item scale value vj, i.e. In (Pij/1 - Pij) = ui + vj
                                                        18
Uses of the Rasch Model
•   Identifies items that are acceptable –
    items that are significantly different from
    0 are good items
•   Indicates whether an item is extremely
    difficult or easy




                                                  19

Contenu connexe

Tendances

Item Analysis: Classical and Beyond
Item Analysis: Classical and BeyondItem Analysis: Classical and Beyond
Item Analysis: Classical and BeyondMhairi Mcalpine
 
A visual guide to item response theory
A visual guide to item response theoryA visual guide to item response theory
A visual guide to item response theoryahmad rustam
 
Item Response Theory in Constructing Measures
Item Response Theory in Constructing MeasuresItem Response Theory in Constructing Measures
Item Response Theory in Constructing MeasuresCarlo Magno
 
Standard error of measurement
Standard error of measurementStandard error of measurement
Standard error of measurementtlcoffman
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliabilitysongoten77
 
Characteristics of a good test
Characteristics of a good testCharacteristics of a good test
Characteristics of a good testcyrilcoscos
 
Reliability
ReliabilityReliability
ReliabilityRoi Xcel
 
Reliability for testing and assessment
Reliability for testing and assessmentReliability for testing and assessment
Reliability for testing and assessmentErlwinmer Mangmang
 
IRT - Item response Theory
IRT - Item response TheoryIRT - Item response Theory
IRT - Item response TheoryAjay Dhamija
 
Test Reliability and Validity
Test Reliability and ValidityTest Reliability and Validity
Test Reliability and ValidityBrian Ebie
 
Educational testing and assessment
Educational testing and assessmentEducational testing and assessment
Educational testing and assessmentAbdul Majid
 
Validity & reliability seminar
Validity & reliability seminarValidity & reliability seminar
Validity & reliability seminarmrikara185
 
Irt 1 pl, 2pl, 3pl.pdf
Irt 1 pl, 2pl, 3pl.pdfIrt 1 pl, 2pl, 3pl.pdf
Irt 1 pl, 2pl, 3pl.pdfCarlo Magno
 
Using Item Response Theory to Improve Assessment
Using Item Response Theory to Improve AssessmentUsing Item Response Theory to Improve Assessment
Using Item Response Theory to Improve AssessmentNathan Thompson
 

Tendances (20)

Item Analysis: Classical and Beyond
Item Analysis: Classical and BeyondItem Analysis: Classical and Beyond
Item Analysis: Classical and Beyond
 
A visual guide to item response theory
A visual guide to item response theoryA visual guide to item response theory
A visual guide to item response theory
 
Classical Test Theory (CTT)- By Dr. Jai Singh
Classical Test Theory (CTT)- By Dr. Jai SinghClassical Test Theory (CTT)- By Dr. Jai Singh
Classical Test Theory (CTT)- By Dr. Jai Singh
 
Irt assessment
Irt assessmentIrt assessment
Irt assessment
 
Item Response Theory in Constructing Measures
Item Response Theory in Constructing MeasuresItem Response Theory in Constructing Measures
Item Response Theory in Constructing Measures
 
Standard error of measurement
Standard error of measurementStandard error of measurement
Standard error of measurement
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliability
 
Characteristics of a good test
Characteristics of a good testCharacteristics of a good test
Characteristics of a good test
 
IRT in Test Construction
IRT in Test Construction IRT in Test Construction
IRT in Test Construction
 
Reliability and validity
Reliability and validityReliability and validity
Reliability and validity
 
Reliability
ReliabilityReliability
Reliability
 
Reliability
ReliabilityReliability
Reliability
 
Reliability
ReliabilityReliability
Reliability
 
Reliability for testing and assessment
Reliability for testing and assessmentReliability for testing and assessment
Reliability for testing and assessment
 
IRT - Item response Theory
IRT - Item response TheoryIRT - Item response Theory
IRT - Item response Theory
 
Test Reliability and Validity
Test Reliability and ValidityTest Reliability and Validity
Test Reliability and Validity
 
Educational testing and assessment
Educational testing and assessmentEducational testing and assessment
Educational testing and assessment
 
Validity & reliability seminar
Validity & reliability seminarValidity & reliability seminar
Validity & reliability seminar
 
Irt 1 pl, 2pl, 3pl.pdf
Irt 1 pl, 2pl, 3pl.pdfIrt 1 pl, 2pl, 3pl.pdf
Irt 1 pl, 2pl, 3pl.pdf
 
Using Item Response Theory to Improve Assessment
Using Item Response Theory to Improve AssessmentUsing Item Response Theory to Improve Assessment
Using Item Response Theory to Improve Assessment
 

En vedette

Objective Standard Setting_An application of Many Facet Rasch Model
Objective Standard Setting_An application of Many Facet Rasch ModelObjective Standard Setting_An application of Many Facet Rasch Model
Objective Standard Setting_An application of Many Facet Rasch ModelSaidfudin Mas'udi
 
Applied Psych Test Design: Part C - Use of Rasch scaling technology
Applied Psych Test Design: Part C - Use of Rasch scaling technologyApplied Psych Test Design: Part C - Use of Rasch scaling technology
Applied Psych Test Design: Part C - Use of Rasch scaling technologyKevin McGrew
 
Implementing Item Response Theory
Implementing Item Response TheoryImplementing Item Response Theory
Implementing Item Response TheoryNathan Thompson
 
Item and Distracter Analysis
Item and Distracter AnalysisItem and Distracter Analysis
Item and Distracter AnalysisSue Quirante
 
Item Analysis - Discrimination and Difficulty Index
Item Analysis - Discrimination and Difficulty IndexItem Analysis - Discrimination and Difficulty Index
Item Analysis - Discrimination and Difficulty IndexMr. Ronald Quileste, PhD
 
Item analysis and validation
Item analysis and validationItem analysis and validation
Item analysis and validationKEnkenken Tan
 

En vedette (12)

Performance Assessments with MFRM
Performance Assessments with MFRMPerformance Assessments with MFRM
Performance Assessments with MFRM
 
Objective Standard Setting_An application of Many Facet Rasch Model
Objective Standard Setting_An application of Many Facet Rasch ModelObjective Standard Setting_An application of Many Facet Rasch Model
Objective Standard Setting_An application of Many Facet Rasch Model
 
Applied Psych Test Design: Part C - Use of Rasch scaling technology
Applied Psych Test Design: Part C - Use of Rasch scaling technologyApplied Psych Test Design: Part C - Use of Rasch scaling technology
Applied Psych Test Design: Part C - Use of Rasch scaling technology
 
Implementing Item Response Theory
Implementing Item Response TheoryImplementing Item Response Theory
Implementing Item Response Theory
 
T est item analysis
T est item analysisT est item analysis
T est item analysis
 
Item Analysis
Item AnalysisItem Analysis
Item Analysis
 
Item and Distracter Analysis
Item and Distracter AnalysisItem and Distracter Analysis
Item and Distracter Analysis
 
Item analysis
Item analysisItem analysis
Item analysis
 
Item analysis ppt
Item analysis pptItem analysis ppt
Item analysis ppt
 
Item Analysis - Discrimination and Difficulty Index
Item Analysis - Discrimination and Difficulty IndexItem Analysis - Discrimination and Difficulty Index
Item Analysis - Discrimination and Difficulty Index
 
Item analysis and validation
Item analysis and validationItem analysis and validation
Item analysis and validation
 
Item analysis
Item analysis Item analysis
Item analysis
 

Similaire à The application of irt using the rasch model presnetation1

2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)
2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)
2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)Yun Huang
 
Introduction to unidimensional item response model
Introduction to unidimensional item response modelIntroduction to unidimensional item response model
Introduction to unidimensional item response modelSumit Das
 
Research_Methods_Unit_II_2021.pptx
Research_Methods_Unit_II_2021.pptxResearch_Methods_Unit_II_2021.pptx
Research_Methods_Unit_II_2021.pptxSasi Kumar
 
An Adaptive Evaluation System to Test Student Caliber using Item Response Theory
An Adaptive Evaluation System to Test Student Caliber using Item Response TheoryAn Adaptive Evaluation System to Test Student Caliber using Item Response Theory
An Adaptive Evaluation System to Test Student Caliber using Item Response TheoryEditor IJMTER
 
Sampling procedure and sample I Quantitative Research
Sampling procedure and sample I Quantitative ResearchSampling procedure and sample I Quantitative Research
Sampling procedure and sample I Quantitative ResearchJimnaira Abanto
 
evolution of data mining.pptx
evolution of data mining.pptxevolution of data mining.pptx
evolution of data mining.pptxShimaaIbrahim33
 
Overview of the Possibilities of Quantitative Methods in Political Science
Overview of the Possibilities of Quantitative Methods in Political ScienceOverview of the Possibilities of Quantitative Methods in Political Science
Overview of the Possibilities of Quantitative Methods in Political Scienceenvironmentalconflicts
 
Experimental method of Educational Research.
Experimental method of Educational Research.Experimental method of Educational Research.
Experimental method of Educational Research.Neha Deo
 
Exploratory
Exploratory Exploratory
Exploratory toby2036
 
Chapter 5 class version b(1)
Chapter 5 class version b(1)Chapter 5 class version b(1)
Chapter 5 class version b(1)jbnx
 
Experimental research
Experimental researchExperimental research
Experimental researchSYIKIN MARIA
 
Experimental Design
Experimental DesignExperimental Design
Experimental DesignThiyagu K
 

Similaire à The application of irt using the rasch model presnetation1 (20)

2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)
2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)
2015EDM: A Framework for Multifaceted Evaluation of Student Models (Polygon)
 
Introduction to unidimensional item response model
Introduction to unidimensional item response modelIntroduction to unidimensional item response model
Introduction to unidimensional item response model
 
Slide Psikologi.docx
Slide Psikologi.docxSlide Psikologi.docx
Slide Psikologi.docx
 
Quantitative research design
Quantitative research designQuantitative research design
Quantitative research design
 
Methodology and IRB/URR
Methodology and IRB/URRMethodology and IRB/URR
Methodology and IRB/URR
 
Research_Methods_Unit_II_2021.pptx
Research_Methods_Unit_II_2021.pptxResearch_Methods_Unit_II_2021.pptx
Research_Methods_Unit_II_2021.pptx
 
An Adaptive Evaluation System to Test Student Caliber using Item Response Theory
An Adaptive Evaluation System to Test Student Caliber using Item Response TheoryAn Adaptive Evaluation System to Test Student Caliber using Item Response Theory
An Adaptive Evaluation System to Test Student Caliber using Item Response Theory
 
Sampling procedure and sample I Quantitative Research
Sampling procedure and sample I Quantitative ResearchSampling procedure and sample I Quantitative Research
Sampling procedure and sample I Quantitative Research
 
evolution of data mining.pptx
evolution of data mining.pptxevolution of data mining.pptx
evolution of data mining.pptx
 
Overview of the Possibilities of Quantitative Methods in Political Science
Overview of the Possibilities of Quantitative Methods in Political ScienceOverview of the Possibilities of Quantitative Methods in Political Science
Overview of the Possibilities of Quantitative Methods in Political Science
 
TCI in primary care - SEM (2006)
TCI in primary care - SEM (2006)TCI in primary care - SEM (2006)
TCI in primary care - SEM (2006)
 
Experimental method of Educational Research.
Experimental method of Educational Research.Experimental method of Educational Research.
Experimental method of Educational Research.
 
R m 99
R m 99R m 99
R m 99
 
Unit. 6.doc
Unit. 6.docUnit. 6.doc
Unit. 6.doc
 
Exploratory
Exploratory Exploratory
Exploratory
 
Chapter 5 class version b(1)
Chapter 5 class version b(1)Chapter 5 class version b(1)
Chapter 5 class version b(1)
 
Experimental research design
Experimental research designExperimental research design
Experimental research design
 
Experimental research
Experimental researchExperimental research
Experimental research
 
Experimental Design
Experimental DesignExperimental Design
Experimental Design
 
Research design
Research designResearch design
Research design
 

Plus de Carlo Magno

Assessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxAssessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxCarlo Magno
 
Social and Emotional Learning
Social and Emotional LearningSocial and Emotional Learning
Social and Emotional LearningCarlo Magno
 
Educational assessment in the 4 ir
Educational assessment in the 4 irEducational assessment in the 4 ir
Educational assessment in the 4 irCarlo Magno
 
The process of research mentoring
The process of research mentoringThe process of research mentoring
The process of research mentoringCarlo Magno
 
Quality management services sustainability training
Quality management services sustainability trainingQuality management services sustainability training
Quality management services sustainability trainingCarlo Magno
 
Managing technology integration in schools
Managing technology integration in schoolsManaging technology integration in schools
Managing technology integration in schoolsCarlo Magno
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teachingCarlo Magno
 
Empowering educators on technology integration
Empowering educators on technology integrationEmpowering educators on technology integration
Empowering educators on technology integrationCarlo Magno
 
Designing an online lesson
Designing an online lessonDesigning an online lesson
Designing an online lessonCarlo Magno
 
Curriculum integration
Curriculum integrationCurriculum integration
Curriculum integrationCarlo Magno
 
Accountability in Developing Student Learning
Accountability in Developing Student LearningAccountability in Developing Student Learning
Accountability in Developing Student LearningCarlo Magno
 
The Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementThe Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementCarlo Magno
 
Guiding your child on their career decision making
Guiding your child on their career decision makingGuiding your child on their career decision making
Guiding your child on their career decision makingCarlo Magno
 
Assessing Science Inquiry Skills
Assessing Science Inquiry SkillsAssessing Science Inquiry Skills
Assessing Science Inquiry SkillsCarlo Magno
 
Assessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumAssessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumCarlo Magno
 
Quantitative analysis in language research
Quantitative analysis in language researchQuantitative analysis in language research
Quantitative analysis in language researchCarlo Magno
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teachingCarlo Magno
 
Hallmarks of textbook
Hallmarks of textbookHallmarks of textbook
Hallmarks of textbookCarlo Magno
 
managing the learner centered-classroom
managing the learner centered-classroommanaging the learner centered-classroom
managing the learner centered-classroomCarlo Magno
 
Assessing learning objectives
Assessing learning objectivesAssessing learning objectives
Assessing learning objectivesCarlo Magno
 

Plus de Carlo Magno (20)

Assessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxAssessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptx
 
Social and Emotional Learning
Social and Emotional LearningSocial and Emotional Learning
Social and Emotional Learning
 
Educational assessment in the 4 ir
Educational assessment in the 4 irEducational assessment in the 4 ir
Educational assessment in the 4 ir
 
The process of research mentoring
The process of research mentoringThe process of research mentoring
The process of research mentoring
 
Quality management services sustainability training
Quality management services sustainability trainingQuality management services sustainability training
Quality management services sustainability training
 
Managing technology integration in schools
Managing technology integration in schoolsManaging technology integration in schools
Managing technology integration in schools
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teaching
 
Empowering educators on technology integration
Empowering educators on technology integrationEmpowering educators on technology integration
Empowering educators on technology integration
 
Designing an online lesson
Designing an online lessonDesigning an online lesson
Designing an online lesson
 
Curriculum integration
Curriculum integrationCurriculum integration
Curriculum integration
 
Accountability in Developing Student Learning
Accountability in Developing Student LearningAccountability in Developing Student Learning
Accountability in Developing Student Learning
 
The Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementThe Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School Improvement
 
Guiding your child on their career decision making
Guiding your child on their career decision makingGuiding your child on their career decision making
Guiding your child on their career decision making
 
Assessing Science Inquiry Skills
Assessing Science Inquiry SkillsAssessing Science Inquiry Skills
Assessing Science Inquiry Skills
 
Assessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumAssessment in the Social Studies Curriculum
Assessment in the Social Studies Curriculum
 
Quantitative analysis in language research
Quantitative analysis in language researchQuantitative analysis in language research
Quantitative analysis in language research
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teaching
 
Hallmarks of textbook
Hallmarks of textbookHallmarks of textbook
Hallmarks of textbook
 
managing the learner centered-classroom
managing the learner centered-classroommanaging the learner centered-classroom
managing the learner centered-classroom
 
Assessing learning objectives
Assessing learning objectivesAssessing learning objectives
Assessing learning objectives
 

The application of irt using the rasch model presnetation1

  • 1. The Application of IRT using the Rasch Model in Constructing Cognitive Measures Carlo Magno, PhD De La Salle University-Manila 1
  • 2. Outline • Psychometric Theory • Classical test Theory (CTT) • Item Response Theory (IRT) • Approaches in IRT • Issues in CTT • Advantages of the Rasch Model • Assumptions of the Rasch Model • Uses of the Rasch Model • Procedure in Rasch Model • Workshop 2
  • 3. Psychometrics • Psychometrics concerns itself with the science of measuring psychological constructs such as ability, personality, affect and skills. • Research in psychology involves the measurement of variables in order to conduct further analysis. 3
  • 4. Branches of Psychometric Theory • Classical Test Theory • Item Response Theory 4
  • 5. Classical Test Theory (CTT) • Regarded as the “True Score Theory” • Responses of examinees are due only to variation in ability of interest • All other potential sources of variation existing in the testing materials such as external conditions or internal conditions of examinees are assumed either to be constant through rigorous standardization or to have an effect that is nonsystematic or random by nature. 5
  • 6. Classical Test Theory (CTT) TO = T + E • The implication of the classical test theory for test takers is that test are fallible imprecise tools • Error = standard error of measurement Sm = S√ 1 - r • True score = M +- Sm = 68% of the normal curve 6
  • 7. Normal curve One SE Range of True mean Scores 7
  • 8. Focus of Analysis in CTT • frequency of correct responses (to indicate question difficulty); • frequency of responses (to examine distracters); • reliability of the test and item-total correlation (to evaluate discrimination at the item level) 8
  • 9. Item Response Theory • Synonymous with latent trait theory, strong true score theory or modern mental test theory • More applicable for tests with right and wrong (dichotomous) responses • An approach to testing based on item analysis considering the chance of getting particular items right or wrong • Each item on a test has its own item characteristic curve that describes the probability of getting each particular item right or wrong given the ability of the test takers (Kaplan & Saccuzzo, 1997) 9
  • 10. Logistic Item Characteristic Curve – A function of ability (θ) – latent trait – Forms the boundary between the probability areas of answering an item incorrectly and answering the item correctly 10
  • 11. Approaches of IRT • One dimension (Rasch)→ One parameter model = uses only the difficulty parameter • Two dimension → Two parameter Model = difficulty and ability parameter • Three dimension (Logistic Model)→ Three Parameter Model = ability, correct response, item discrimination 11
  • 12. Issues in CTT • A score is dependent on the performance of the group tested (Norm referenced) • The group on which the test has been scaled has outlived has usefulness across time – Changes in the defined population – Changes in educational emphasis • There is a need to rapidly make new norms to adopt to the changing times • If the characteristics of a person changes and does not fit s specified norm then a norm for that person needs to be created. • Each collection of norms has an ability of its own = rubber yardstick 12
  • 13. Advantages of the Rasch Model • The calibration of test item difficulty is independent of the person used for the calibration. • The method of test calibration does not matter whose responses to these items use for comparison • It gives the same results regardless on who takes the test • The scores a person obtain on the test can be used to remove the influence of their abilities from the estimation of their difficulty. The result is a sample free item calibration. 13
  • 14. Rasch Model • Rasch’s (1960) main motivation for his model was to eliminate references to populations of examinees in analyses of tests. • According to him that test analysis would only be worthwhile if it were individual centered with separate parameters for the items and the examinees (van der Linden & Hambleton, 2004). 14
  • 15. Rasch Model • The Rasch model is a probabilistic unidimensional model which asserts that: (1) the easier the question the more likely the student will respond correctly to it, and (2) the more able the student, the more likely he/she will pass the question compared to a less able student. 15
  • 16. Rasch Model • The model was enhanced to assume that the probability that a student will correctly answer a question is a logistic function of the difference between the student's ability [θ] and the difficulty of the question [β] (i.e. the ability required to answer the question correctly), and only a function of that difference giving way to the Rasch model • Thus, when data fit the model, the relative difficulties of the questions are independent of the relative abilities of the students, and vice versa (Rasch, 1977). 16
  • 17. Assumptions of the Rasch Model According to Fisher (1974) • (1) Unidimensionality. All items are functionally dependent upon only one underlying continuum. • (2) Monotonicity. All item characteristic functions are strictly monotonic in the latent trait. The item characteristic function describes the probability of a predefined response as a function of the latent trait. • (3) Local stochastic independence. Every person has a certain probability of giving a predefined response to each item and this probability is independent of the answers given to the preceding items. 17
  • 18. Assumptions of the Rasch Model According to Fisher (1974) • (4) Sufficiency of a simple sum statistic. The number of predefined responses is a sufficient statistic for the latent parameter. • (5) Dichotomy of the items. For each item there are only two different responses, for example positive and negative. The Rasch model requires that an additive structure underlies the observed data. This additive structure applies to the logit of Pij, where Pij is the probability that subject i will give a predefined response to item j, being the sum of a subject scale value ui and an item scale value vj, i.e. In (Pij/1 - Pij) = ui + vj 18
  • 19. Uses of the Rasch Model • Identifies items that are acceptable – items that are significantly different from 0 are good items • Indicates whether an item is extremely difficult or easy 19