SlideShare a Scribd company logo
1 of 39
Ubuntu Usability Test Report Bentley University: Human Factors in Information Design HF750: Usability Testing & Assessment December 15, 2008  Dan Fitek, Don Goetz, Karen Gosciminski, Sunil Muniraju, Allison Yale
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Executive summary
Test Methodology
[object Object],[object Object],[object Object],[object Object],Ubuntu version 8.04.1 * http://www.desktoplinux.com/articles/AT5816278551.html
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Stakeholders & goals
[object Object],[object Object],[object Object],[object Object],[object Object],Test goals
Test sessions
Target participant
Test participants (n = 6)‏ All participants used computers at least once daily to perform common tasks (e.g., emailing, word processing, browsing the internet). None of the participants had experience developing computer software or using Linux-based OS.
[object Object],[object Object],[object Object],Test scenario
Directed tasks 1.  You decide that the current display setting makes on-screen information hard to see. Decrease the monitor’s display size to the 1024 x 768 setting  (time limit: 5 min) . 2.  You are trying to get online to check your email and you realize there is no Internet connection. Connect wirelessly to the Internet with the following information  (time limit: 10 min) : Network name:  campus Password:  11bb22bb33bb44bb55bb66bb77 Encryption:  open WEP, 128-bit 3.  In your inbox, there is an email from a friend telling you that the most watched video on YouTube.com is great and that you should watch it. Go to YouTube.com to view this video  (time limit: 15 min) . Participant will need to install an Adobe Flash plug-in to complete this task. (continued on next slide)‏
Directed tasks  (continued)‏ 4.  You recently took a trip home to visit family for the holidays and you took some pictures. Three of the pictures are on the desktop. Create a new folder called ‘Family’ in an appropriate place to store the pictures. Move them off of the desktop and into this folder  (time limit: 5 min) . 5.  Earlier, you downloaded a few songs to the  Music  folder. Play one of these songs  (time limit: 10 min).  Participant will need to download a codec to complete this task. 6.  Your grandmother sent you a batch of homemade cookies for your birthday and since you have terrible handwriting and your grandmother has poor vision, you want to type up a “thank you” letter in 20-point font. Type the following letter and save it onto the hard drive so that you can print it out later  (time limit: 7 min).  Dear Grandma, Thank you very much for the delicious cookies. I can’t wait to see you next  month for dinner!  Love, Pat
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Data collection
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Data analysis
Task Data
Task times The graph below shows the average task completion times of participants who performed tasks successfully without test moderator assistance.
Ease of use ratings The graph below shows the average ease of use rating participants reported after performing each task (Scale: 1-5, 1 = poor, 5 = excellent).
Task completion
Findings
Summary ,[object Object],[object Object],[object Object],[object Object],[object Object]
Strengths ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],* http://www.ubuntu.com/products/WhatIsUbuntu/desktopedition
Strengths  (continued)‏ ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],Severity rating scale
Unlocking network settings Priority:   High ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Participant quote: “Usually when things are grayed, that means there’s nothing you can do.”  Networks Settings window
Non-intuitive network icon Priority:   Medium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Participant quote: “It doesn’t seem to let me access a wireless internet connection from there.”  Left-click menu Right-click menu
Confusing 802.1x network Priority:  Medium   ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Participant quote: (after filling it out) “Looking at the screen now, it seems to be a wired network.” Connect to 802.1X protected wired network window
Misleading network option  Priority:   Medium   ,[object Object],[object Object],[object Object],[object Object],[object Object],Wireless Networks window
Unclear menu labels Priority:   Medium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Participant quote: “I’m not clearly understanding what the difference is between them.” Main menus (located in top left corner of desktop)‏ System > Preferences menu System > Administration menu
Multiple network-related labels Priority:   Medium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Figure A Figure B Figure C Figure D
Non-intuitive path for changing screen resolution Priority:   Medium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Appearance Preferences window Participant quotes:  “For Windows, normally I right-click and go for the desktop settings or something like that."  "I right-clicked on the desktop because that is the general windows shortcut to access the display properties." Desktop right-click menu System > Preferences > Appearance menu option
Lack of feedback Priority:  Low   ,[object Object],[object Object],[object Object],[object Object],Participant quote: “Hopefully it’s looking to try to connect, but I’m not really sure.” Network Settings window with progress indicator
Incorrect codec listing Priority:   Low ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Participant quote: “I’m going to try installing the first one and I’ll see what happens.” Install multimedia codecs window
Participant Feedback
[object Object],[object Object],[object Object],[object Object],[object Object],Barriers to conversion
Likelihood of supplementing Windows with Ubuntu:   Two participants would consider using Ubuntu in addition to their current Windows OS. Other participants were less enthused about Ubuntu in general and/or thought running two OS was unnecessary or overly complicated. Barriers to conversion  (continued)‏
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],What would increase interest* * Paraphrased participant comments
[object Object],[object Object],[object Object],[object Object],[object Object],Suggestions for improvement* * Paraphrased participant comments
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Function & menu identification* * Paraphrased participant comments
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Overall confidence* * Paraphrased participant comments

More Related Content

Viewers also liked

Usability Testing Fundamentals
Usability Testing FundamentalsUsability Testing Fundamentals
Usability Testing Fundamentals
debcook
 
USA TODAY iPad Application Usability Report
USA TODAY iPad Application Usability ReportUSA TODAY iPad Application Usability Report
USA TODAY iPad Application Usability Report
kathleensulli
 
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
Di Zhang
 
Usability Testing Report
Usability Testing ReportUsability Testing Report
Usability Testing Report
feifei2011
 
Ar3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research ReportAr3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research Report
Shakarah Levy
 

Viewers also liked (20)

Usability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best PracticesUsability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best Practices
 
Usability Testing Fundamentals
Usability Testing FundamentalsUsability Testing Fundamentals
Usability Testing Fundamentals
 
Usability test report for queso
Usability test report for quesoUsability test report for queso
Usability test report for queso
 
Usability Testing and QA 12 11-15
Usability Testing and QA 12 11-15Usability Testing and QA 12 11-15
Usability Testing and QA 12 11-15
 
USA TODAY iPad Application Usability Report
USA TODAY iPad Application Usability ReportUSA TODAY iPad Application Usability Report
USA TODAY iPad Application Usability Report
 
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
Web Site Usability Test - Client Report - Victorian Deaf Society (Ver 1....
 
Usability Analysis Final Report
Usability Analysis Final ReportUsability Analysis Final Report
Usability Analysis Final Report
 
Prototyping and Usability Testing your designs
Prototyping and Usability Testing your designsPrototyping and Usability Testing your designs
Prototyping and Usability Testing your designs
 
The Case for Ubuntu: Linux Operating System Performance and Usability for Use...
The Case for Ubuntu: Linux Operating System Performance and Usability for Use...The Case for Ubuntu: Linux Operating System Performance and Usability for Use...
The Case for Ubuntu: Linux Operating System Performance and Usability for Use...
 
Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010Usability Training - UDSM 06/2010
Usability Training - UDSM 06/2010
 
Website Usability - Jasmine Sante
Website Usability - Jasmine SanteWebsite Usability - Jasmine Sante
Website Usability - Jasmine Sante
 
usability_slides
usability_slidesusability_slides
usability_slides
 
El reto de generar código útil: más allá de UX y Agile
El reto de generar código útil: más allá de UX y Agile El reto de generar código útil: más allá de UX y Agile
El reto de generar código útil: más allá de UX y Agile
 
Mainstream UX
Mainstream UXMainstream UX
Mainstream UX
 
Usability Testing Report
Usability Testing ReportUsability Testing Report
Usability Testing Report
 
Evaluación de sitios web
Evaluación de sitios webEvaluación de sitios web
Evaluación de sitios web
 
Ar3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research ReportAr3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research Report
 
adsfasdf
adsfasdfadsfasdf
adsfasdf
 
Usability and Health IT
Usability and Health ITUsability and Health IT
Usability and Health IT
 
Bvh report
Bvh reportBvh report
Bvh report
 

Similar to Ubuntu Usability Test Report

ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACHICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
JamaicaCabrales
 
ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)
Shweta Gupte
 
Designing user interface module 4
Designing user interface module 4Designing user interface module 4
Designing user interface module 4
DBPMCF
 
Making the New Notes - Community, Cooperation, Concepts
Making the New Notes - Community, Cooperation, ConceptsMaking the New Notes - Community, Cooperation, Concepts
Making the New Notes - Community, Cooperation, Concepts
Maximilian Odendahl
 
SWE481 – Software Engineering Capstone 1 Page 2SWE.docx
SWE481 – Software Engineering Capstone 1     Page  2SWE.docxSWE481 – Software Engineering Capstone 1     Page  2SWE.docx
SWE481 – Software Engineering Capstone 1 Page 2SWE.docx
mattinsonjanel
 
Ui Design And Usability For Everybody
Ui Design And Usability For EverybodyUi Design And Usability For Everybody
Ui Design And Usability For Everybody
Empatika
 

Similar to Ubuntu Usability Test Report (20)

Andhra pradesh workshop user manual october 2016
Andhra pradesh workshop user manual october 2016Andhra pradesh workshop user manual october 2016
Andhra pradesh workshop user manual october 2016
 
ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACHICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
ICT-CONTACTSJDHJBQFFVEGVCGSVCHCHDCHDCVCKACH
 
8 Usability Lessons from the UPA Conference by Mark Alves
8 Usability Lessons from the UPA Conference by Mark Alves8 Usability Lessons from the UPA Conference by Mark Alves
8 Usability Lessons from the UPA Conference by Mark Alves
 
ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)ECE695DVisualAnalyticsprojectproposal (2)
ECE695DVisualAnalyticsprojectproposal (2)
 
The quality & richness of E-Education
The quality & richness of E-EducationThe quality & richness of E-Education
The quality & richness of E-Education
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
 
ISDD Testing
ISDD TestingISDD Testing
ISDD Testing
 
ASE Notes - Kuntucky.docx
ASE Notes - Kuntucky.docxASE Notes - Kuntucky.docx
ASE Notes - Kuntucky.docx
 
Interface Design
Interface DesignInterface Design
Interface Design
 
Chapter 3 - Variety of Dialogue
Chapter 3  - Variety of DialogueChapter 3  - Variety of Dialogue
Chapter 3 - Variety of Dialogue
 
Designing user interface module 4
Designing user interface module 4Designing user interface module 4
Designing user interface module 4
 
Universal usability engineering
Universal usability engineeringUniversal usability engineering
Universal usability engineering
 
Making the New Notes - Community, Cooperation, Concepts
Making the New Notes - Community, Cooperation, ConceptsMaking the New Notes - Community, Cooperation, Concepts
Making the New Notes - Community, Cooperation, Concepts
 
The case for ubuntu linux operating system performance and usabil
The case for ubuntu linux operating system performance and usabilThe case for ubuntu linux operating system performance and usabil
The case for ubuntu linux operating system performance and usabil
 
SWE481 – Software Engineering Capstone 1 Page 2SWE.docx
SWE481 – Software Engineering Capstone 1     Page  2SWE.docxSWE481 – Software Engineering Capstone 1     Page  2SWE.docx
SWE481 – Software Engineering Capstone 1 Page 2SWE.docx
 
Ui Design And Usability For Everybody
Ui Design And Usability For EverybodyUi Design And Usability For Everybody
Ui Design And Usability For Everybody
 
Barcamp Bangalore 2 - On User Experience and Usability Testing
Barcamp Bangalore 2 - On User Experience and Usability TestingBarcamp Bangalore 2 - On User Experience and Usability Testing
Barcamp Bangalore 2 - On User Experience and Usability Testing
 
ICT_CSS_SHS11_Q2_LAS1.docx
ICT_CSS_SHS11_Q2_LAS1.docxICT_CSS_SHS11_Q2_LAS1.docx
ICT_CSS_SHS11_Q2_LAS1.docx
 
Home mess systems- Prototype 2 & Evaluation
Home mess systems- Prototype 2 & EvaluationHome mess systems- Prototype 2 & Evaluation
Home mess systems- Prototype 2 & Evaluation
 
Smas Hits May 11, 2009 Sensex Down 193 Points On Profit Booking
Smas Hits May 11, 2009 Sensex Down 193 Points On Profit BookingSmas Hits May 11, 2009 Sensex Down 193 Points On Profit Booking
Smas Hits May 11, 2009 Sensex Down 193 Points On Profit Booking
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 

Ubuntu Usability Test Report

  • 1. Ubuntu Usability Test Report Bentley University: Human Factors in Information Design HF750: Usability Testing & Assessment December 15, 2008 Dan Fitek, Don Goetz, Karen Gosciminski, Sunil Muniraju, Allison Yale
  • 2.
  • 4.
  • 5.
  • 6.
  • 9. Test participants (n = 6)‏ All participants used computers at least once daily to perform common tasks (e.g., emailing, word processing, browsing the internet). None of the participants had experience developing computer software or using Linux-based OS.
  • 10.
  • 11. Directed tasks 1. You decide that the current display setting makes on-screen information hard to see. Decrease the monitor’s display size to the 1024 x 768 setting (time limit: 5 min) . 2. You are trying to get online to check your email and you realize there is no Internet connection. Connect wirelessly to the Internet with the following information (time limit: 10 min) : Network name: campus Password: 11bb22bb33bb44bb55bb66bb77 Encryption: open WEP, 128-bit 3. In your inbox, there is an email from a friend telling you that the most watched video on YouTube.com is great and that you should watch it. Go to YouTube.com to view this video (time limit: 15 min) . Participant will need to install an Adobe Flash plug-in to complete this task. (continued on next slide)‏
  • 12. Directed tasks (continued)‏ 4. You recently took a trip home to visit family for the holidays and you took some pictures. Three of the pictures are on the desktop. Create a new folder called ‘Family’ in an appropriate place to store the pictures. Move them off of the desktop and into this folder (time limit: 5 min) . 5. Earlier, you downloaded a few songs to the Music folder. Play one of these songs (time limit: 10 min). Participant will need to download a codec to complete this task. 6. Your grandmother sent you a batch of homemade cookies for your birthday and since you have terrible handwriting and your grandmother has poor vision, you want to type up a “thank you” letter in 20-point font. Type the following letter and save it onto the hard drive so that you can print it out later (time limit: 7 min). Dear Grandma, Thank you very much for the delicious cookies. I can’t wait to see you next month for dinner! Love, Pat
  • 13.
  • 14.
  • 16. Task times The graph below shows the average task completion times of participants who performed tasks successfully without test moderator assistance.
  • 17. Ease of use ratings The graph below shows the average ease of use rating participants reported after performing each task (Scale: 1-5, 1 = poor, 5 = excellent).
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 34.
  • 35. Likelihood of supplementing Windows with Ubuntu: Two participants would consider using Ubuntu in addition to their current Windows OS. Other participants were less enthused about Ubuntu in general and/or thought running two OS was unnecessary or overly complicated. Barriers to conversion (continued)‏
  • 36.
  • 37.
  • 38.
  • 39.