SlideShare une entreprise Scribd logo
1  sur  26
WHY USABILITY TESTING WORKS FOR IT AUDIT Voraphan Manomuth Raungpaka, PhD  Kasetsart University,  THAILAND [email_address] Usana Patramontree, CPA Kasetsart University,  THAILAND [email_address] Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 Presented by  Voraphan M. Raungpaka 7 October, 2010
[object Object],[object Object],[object Object],[object Object],[object Object],OUTLINE Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],[object Object],[object Object],IN OUR ENGAGEMENT PLAN Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object]
TYPICAL IT AUDIT TECHNIQUES Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 Is used to discover weaknesses In program controls. Is used to discover  data, database inaccuracy and Incompleteness. What IT techniques should be used to discover usability problems of  usability problems of real users? user Application Program Test Data Audit Software ???? Data Data
AN AUDITOR EVER KNOW  ABOUT ANY SYSTEM USABILITY PROBLEMS OF REAL USERS?  ,[object Object],[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
WHAT WE HAVE GOT FROM PRELIMINARY   SURVEYS ? Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 management and staff were quite satisfied about the various aspects of  FMS ,  but there were also  management  and staff with negative user interface experiences So…… I came up with the idea of applying  usability testing techniques  to gain better evidence on the users’ aspect when using fund management system in specified tasks .   ( In addition to administering the test data and the generalized audit package)
WHAT IS FMS SOFTWARE ,[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
SOLUTION POSTED! Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 Is used to discover weaknesses In program controls. Is used to discover  data, database inaccuracy and Incompleteness. Is used to discover usability problems of  usability problems. Usability testing? user Application Program Test Data Audit Software Data Data
USABILITY TESTING TECHNIQUE ,[object Object],[object Object],[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],USABILITY TESTING Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],METHODOLOGY USED TO EVALUATE USABILITY PROBLEMS Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],OBSERVATION Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],THINK-ALOUD Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],POST TEST QUSTIONNAIRE Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
UTAUT THEORY Unified Theory of Acceptance and Use of Technology (UTAUT),  by Venkatesh et al (2003). Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
[object Object],[object Object],[object Object],RESULTS Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
POST-TEST QUESTIONNAIRE RESULTS Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
OBSERVATION RESULTS ,[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 ,[object Object],[object Object],[object Object],Higher Risk
OBSERVATION RESULTS Figure 2:  Results from Post Test uestionnaire Lower Risk Higher Risk Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 Inefficiency
PROS AND CONS OF USING THE USABILITY TESTING TECHNIQUES IN IT AUDIT ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010 We still need other typical computer assisted audit techniques to detect the actual risk and to correct the problem.
[object Object],[object Object],CONTRIBUTION OF THIS REPORT Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
PRACTITIONER’S TAKE AWAY ,[object Object],[object Object],[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
PRACTITIONER’S TAKE AWAY (CONT.) ,[object Object],[object Object],[object Object],Voraphan Manomuth Raungpaka,  IACIS  Conference, Las Vegas, USA,2010
THANK YOU! Question?
REFERENCES ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Contenu connexe

Tendances

A&D - Fact Finding Techniques
A&D - Fact Finding TechniquesA&D - Fact Finding Techniques
A&D - Fact Finding Techniquesvinay arora
 
System Analysis Fact Finding Methods
System Analysis Fact Finding MethodsSystem Analysis Fact Finding Methods
System Analysis Fact Finding MethodsMoshikur Rahman
 
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012Jason Hong
 
User Research Techniques by Vikram Rao, RSA
User Research Techniques by Vikram Rao, RSAUser Research Techniques by Vikram Rao, RSA
User Research Techniques by Vikram Rao, RSASTC India UX SIG
 
HCI LAB MANUAL
HCI LAB MANUAL HCI LAB MANUAL
HCI LAB MANUAL Um e Farwa
 
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth Presentation
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth PresentationRemote Proctor: Software Secure MSCHE 2013 Annual Conference Booth Presentation
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth PresentationSoftware Secure, Inc.
 
Remote Proctor: Software Secure Sloan-C 2013 Booth Presentation
Remote Proctor: Software Secure Sloan-C 2013 Booth PresentationRemote Proctor: Software Secure Sloan-C 2013 Booth Presentation
Remote Proctor: Software Secure Sloan-C 2013 Booth PresentationSoftware Secure, Inc.
 
Remote Proctor: Software Secure SACSCOC Conference 2013
Remote Proctor: Software Secure SACSCOC Conference 2013Remote Proctor: Software Secure SACSCOC Conference 2013
Remote Proctor: Software Secure SACSCOC Conference 2013Software Secure, Inc.
 
Evaluation techniques in HCI
Evaluation techniques in HCIEvaluation techniques in HCI
Evaluation techniques in HCIsawsan slii
 
ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)Shweta Gupte
 
Increasing Productivity And Reducing Errors Through Usability Analysis
Increasing Productivity And Reducing Errors Through Usability AnalysisIncreasing Productivity And Reducing Errors Through Usability Analysis
Increasing Productivity And Reducing Errors Through Usability Analysisbrighteyes
 
Fact - Finding Techniques
Fact - Finding TechniquesFact - Finding Techniques
Fact - Finding Techniquesgomzy22
 
An in depth study of mobile application testing in reference to real time sce...
An in depth study of mobile application testing in reference to real time sce...An in depth study of mobile application testing in reference to real time sce...
An in depth study of mobile application testing in reference to real time sce...Amit Aggarwal
 

Tendances (17)

Intranet Usability Testing
Intranet Usability TestingIntranet Usability Testing
Intranet Usability Testing
 
A&D - Fact Finding Techniques
A&D - Fact Finding TechniquesA&D - Fact Finding Techniques
A&D - Fact Finding Techniques
 
System Analysis Fact Finding Methods
System Analysis Fact Finding MethodsSystem Analysis Fact Finding Methods
System Analysis Fact Finding Methods
 
A Study on Online Testing, Personality profiling & Psychometrics
A Study on Online Testing, Personality profiling & PsychometricsA Study on Online Testing, Personality profiling & Psychometrics
A Study on Online Testing, Personality profiling & Psychometrics
 
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012
OTO: Online Trust Oracle for User-Centric Trust Establishment, at CCS 2012
 
User Research Techniques by Vikram Rao, RSA
User Research Techniques by Vikram Rao, RSAUser Research Techniques by Vikram Rao, RSA
User Research Techniques by Vikram Rao, RSA
 
HCI LAB MANUAL
HCI LAB MANUAL HCI LAB MANUAL
HCI LAB MANUAL
 
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth Presentation
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth PresentationRemote Proctor: Software Secure MSCHE 2013 Annual Conference Booth Presentation
Remote Proctor: Software Secure MSCHE 2013 Annual Conference Booth Presentation
 
Remote Proctor: Software Secure Sloan-C 2013 Booth Presentation
Remote Proctor: Software Secure Sloan-C 2013 Booth PresentationRemote Proctor: Software Secure Sloan-C 2013 Booth Presentation
Remote Proctor: Software Secure Sloan-C 2013 Booth Presentation
 
Remote Proctor: Software Secure SACSCOC Conference 2013
Remote Proctor: Software Secure SACSCOC Conference 2013Remote Proctor: Software Secure SACSCOC Conference 2013
Remote Proctor: Software Secure SACSCOC Conference 2013
 
Evaluation techniques in HCI
Evaluation techniques in HCIEvaluation techniques in HCI
Evaluation techniques in HCI
 
ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)
 
Increasing Productivity And Reducing Errors Through Usability Analysis
Increasing Productivity And Reducing Errors Through Usability AnalysisIncreasing Productivity And Reducing Errors Through Usability Analysis
Increasing Productivity And Reducing Errors Through Usability Analysis
 
Fact - Finding Techniques
Fact - Finding TechniquesFact - Finding Techniques
Fact - Finding Techniques
 
An in depth study of mobile application testing in reference to real time sce...
An in depth study of mobile application testing in reference to real time sce...An in depth study of mobile application testing in reference to real time sce...
An in depth study of mobile application testing in reference to real time sce...
 
ISTQB REX BLACK book
ISTQB REX BLACK bookISTQB REX BLACK book
ISTQB REX BLACK book
 
The Design and Evaluation of Beahvior Change Tech
The Design and Evaluation of Beahvior Change TechThe Design and Evaluation of Beahvior Change Tech
The Design and Evaluation of Beahvior Change Tech
 

En vedette

Accessibility awareness brown bag
Accessibility awareness brown bagAccessibility awareness brown bag
Accessibility awareness brown bagLawrenceNajjar
 
Usability awareness brown bag
Usability awareness brown bagUsability awareness brown bag
Usability awareness brown bagLawrenceNajjar
 
Accessibility, Usability and User Centred Design (introduction)
Accessibility, Usability and User Centred Design (introduction)Accessibility, Usability and User Centred Design (introduction)
Accessibility, Usability and User Centred Design (introduction)David Lamas
 
Design Simple but Powerful application
Design Simple but Powerful applicationDesign Simple but Powerful application
Design Simple but Powerful applicationJim Liang
 
Human Centered Computing (introduction)
Human Centered Computing (introduction)Human Centered Computing (introduction)
Human Centered Computing (introduction)Sónia
 

En vedette (6)

Accessibility awareness brown bag
Accessibility awareness brown bagAccessibility awareness brown bag
Accessibility awareness brown bag
 
Usability awareness brown bag
Usability awareness brown bagUsability awareness brown bag
Usability awareness brown bag
 
Accessibility, Usability and User Centred Design (introduction)
Accessibility, Usability and User Centred Design (introduction)Accessibility, Usability and User Centred Design (introduction)
Accessibility, Usability and User Centred Design (introduction)
 
Auditing Your Website for Usability Issues
Auditing Your Website for Usability IssuesAuditing Your Website for Usability Issues
Auditing Your Website for Usability Issues
 
Design Simple but Powerful application
Design Simple but Powerful applicationDesign Simple but Powerful application
Design Simple but Powerful application
 
Human Centered Computing (introduction)
Human Centered Computing (introduction)Human Centered Computing (introduction)
Human Centered Computing (introduction)
 

Similaire à Why Usability Works For It Audit Iacis 2010

Usability in product development
Usability in product developmentUsability in product development
Usability in product developmentRavi Shyam
 
Fundamentals of testing
Fundamentals of testingFundamentals of testing
Fundamentals of testingYusran5
 
National e-Learning Laboratory
National e-Learning LaboratoryNational e-Learning Laboratory
National e-Learning LaboratoryStephan Weibelzahl
 
Using Automated Testing Tools to Empower Your User Research
Using Automated Testing Tools to Empower Your User ResearchUsing Automated Testing Tools to Empower Your User Research
Using Automated Testing Tools to Empower Your User ResearchUserZoom
 
Principles of Health Informatics: Evaluating medical software
Principles of Health Informatics: Evaluating medical softwarePrinciples of Health Informatics: Evaluating medical software
Principles of Health Informatics: Evaluating medical softwareMartin Chapman
 
User-Interface Usability Evaluation
User-Interface Usability EvaluationUser-Interface Usability Evaluation
User-Interface Usability EvaluationCSCJournals
 
Key Reasons to Embrace User Acceptance Testing (UAT).pdf
Key Reasons to Embrace User Acceptance Testing (UAT).pdfKey Reasons to Embrace User Acceptance Testing (UAT).pdf
Key Reasons to Embrace User Acceptance Testing (UAT).pdfRohitBhandari66
 
User Experience Evaluation for Automation Tools: An Industrial Experience
User Experience Evaluation for Automation Tools: An Industrial ExperienceUser Experience Evaluation for Automation Tools: An Industrial Experience
User Experience Evaluation for Automation Tools: An Industrial ExperienceIJCI JOURNAL
 
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?Why Don't Software Developers Use Static Analysis Tools to Find Bugs?
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?PVS-Studio
 
John Kerher's Resume
John Kerher's ResumeJohn Kerher's Resume
John Kerher's Resumejohnkerher
 
Chapter 7 - Evaluation Tekhnique
Chapter 7 - Evaluation TekhniqueChapter 7 - Evaluation Tekhnique
Chapter 7 - Evaluation TekhniqueMuhammad Najib
 
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBus
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBusCorso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBus
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBusAlessandro Longo
 
User Zoom Webinar Monster Aug09 Vf
User Zoom Webinar Monster Aug09 VfUser Zoom Webinar Monster Aug09 Vf
User Zoom Webinar Monster Aug09 VfUserZoom
 
Essay On Reliability Of Visualization Tools
Essay On Reliability Of Visualization ToolsEssay On Reliability Of Visualization Tools
Essay On Reliability Of Visualization ToolsNatasha Barnett
 
Hpe program rating #4 human factors engineering
Hpe program rating #4 human factors engineeringHpe program rating #4 human factors engineering
Hpe program rating #4 human factors engineeringPMHaas
 
Testing desktop application police station information management system
Testing desktop application police station information management systemTesting desktop application police station information management system
Testing desktop application police station information management systemSalam Shah
 

Similaire à Why Usability Works For It Audit Iacis 2010 (20)

kritika_resume2
kritika_resume2kritika_resume2
kritika_resume2
 
Usability in product development
Usability in product developmentUsability in product development
Usability in product development
 
load_testing
load_testingload_testing
load_testing
 
Fundamentals of testing
Fundamentals of testingFundamentals of testing
Fundamentals of testing
 
National e-Learning Laboratory
National e-Learning LaboratoryNational e-Learning Laboratory
National e-Learning Laboratory
 
Using Automated Testing Tools to Empower Your User Research
Using Automated Testing Tools to Empower Your User ResearchUsing Automated Testing Tools to Empower Your User Research
Using Automated Testing Tools to Empower Your User Research
 
Principles of Health Informatics: Evaluating medical software
Principles of Health Informatics: Evaluating medical softwarePrinciples of Health Informatics: Evaluating medical software
Principles of Health Informatics: Evaluating medical software
 
User-Interface Usability Evaluation
User-Interface Usability EvaluationUser-Interface Usability Evaluation
User-Interface Usability Evaluation
 
Key Reasons to Embrace User Acceptance Testing (UAT).pdf
Key Reasons to Embrace User Acceptance Testing (UAT).pdfKey Reasons to Embrace User Acceptance Testing (UAT).pdf
Key Reasons to Embrace User Acceptance Testing (UAT).pdf
 
User Experience Evaluation for Automation Tools: An Industrial Experience
User Experience Evaluation for Automation Tools: An Industrial ExperienceUser Experience Evaluation for Automation Tools: An Industrial Experience
User Experience Evaluation for Automation Tools: An Industrial Experience
 
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?Why Don't Software Developers Use Static Analysis Tools to Find Bugs?
Why Don't Software Developers Use Static Analysis Tools to Find Bugs?
 
John Kerher's Resume
John Kerher's ResumeJohn Kerher's Resume
John Kerher's Resume
 
Chapter 7 - Evaluation Tekhnique
Chapter 7 - Evaluation TekhniqueChapter 7 - Evaluation Tekhnique
Chapter 7 - Evaluation Tekhnique
 
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBus
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBusCorso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBus
Corso Interazione Uomo Macchina e Sviluppo Applicazioni Mobile - GoBus
 
User Zoom Webinar Monster Aug09 Vf
User Zoom Webinar Monster Aug09 VfUser Zoom Webinar Monster Aug09 Vf
User Zoom Webinar Monster Aug09 Vf
 
Enterprises resource planning
Enterprises resource planningEnterprises resource planning
Enterprises resource planning
 
Essay On Reliability Of Visualization Tools
Essay On Reliability Of Visualization ToolsEssay On Reliability Of Visualization Tools
Essay On Reliability Of Visualization Tools
 
Usability Testing.docx
Usability Testing.docxUsability Testing.docx
Usability Testing.docx
 
Hpe program rating #4 human factors engineering
Hpe program rating #4 human factors engineeringHpe program rating #4 human factors engineering
Hpe program rating #4 human factors engineering
 
Testing desktop application police station information management system
Testing desktop application police station information management systemTesting desktop application police station information management system
Testing desktop application police station information management system
 

Why Usability Works For It Audit Iacis 2010

  • 1. WHY USABILITY TESTING WORKS FOR IT AUDIT Voraphan Manomuth Raungpaka, PhD Kasetsart University, THAILAND [email_address] Usana Patramontree, CPA Kasetsart University, THAILAND [email_address] Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010 Presented by Voraphan M. Raungpaka 7 October, 2010
  • 2.
  • 3.
  • 4.
  • 5. TYPICAL IT AUDIT TECHNIQUES Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010 Is used to discover weaknesses In program controls. Is used to discover data, database inaccuracy and Incompleteness. What IT techniques should be used to discover usability problems of usability problems of real users? user Application Program Test Data Audit Software ???? Data Data
  • 6.
  • 7. WHAT WE HAVE GOT FROM PRELIMINARY SURVEYS ? Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010 management and staff were quite satisfied about the various aspects of FMS , but there were also management and staff with negative user interface experiences So…… I came up with the idea of applying usability testing techniques to gain better evidence on the users’ aspect when using fund management system in specified tasks . ( In addition to administering the test data and the generalized audit package)
  • 8.
  • 9. SOLUTION POSTED! Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010 Is used to discover weaknesses In program controls. Is used to discover data, database inaccuracy and Incompleteness. Is used to discover usability problems of usability problems. Usability testing? user Application Program Test Data Audit Software Data Data
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16. UTAUT THEORY Unified Theory of Acceptance and Use of Technology (UTAUT), by Venkatesh et al (2003). Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010
  • 17.
  • 18. POST-TEST QUESTIONNAIRE RESULTS Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010
  • 19.
  • 20. OBSERVATION RESULTS Figure 2: Results from Post Test uestionnaire Lower Risk Higher Risk Voraphan Manomuth Raungpaka, IACIS Conference, Las Vegas, USA,2010 Inefficiency
  • 21.
  • 22.
  • 23.
  • 24.
  • 26.

Notes de l'éditeur

  1. This study was a traditional experimental design that involved 2 simulated websites. the high complexity site & the low complexity site. the 4 dependent variables used to evaluate Cognitive Processing were 1) Purchase task success Rate 2) Time for subjects to purchase products, 3) User satisfaction Score & 4) voluntary exiting Rate The data collection booklet was developed to collect the Dependent Variables. It includes 5 things. NEXT
  2. This study was a traditional experimental design that involved 2 simulated websites. the high complexity site & the low complexity site. the 4 dependent variables used to evaluate Cognitive Processing were 1) Purchase task success Rate 2) Time for subjects to purchase products, 3) User satisfaction Score & 4) voluntary exiting Rate The data collection booklet was developed to collect the Dependent Variables. It includes 5 things. NEXT