The document discusses remote user testing as a method of user-centered design. It summarizes that remote user testing identifies areas of confusion and frustration for users as well as where users are unable to complete tasks, allowing problems to be addressed early when still inexpensive. It provides details on conducting remote user testing sessions, including necessary software, estimated time and costs for preparation, testing and follow up, and both advantages and disadvantages compared to in-person testing.
2. Agenda Show how User-centered Design helps teams to save: Time Money Aggravation from clients Usability testing early in the design process identifies: Areas of confusion and frustration Where the user can’t complete a task While it is still cheap and easy to mitigate these problems 03/18/2009 Remote User Testing 2
3. 03/18/2009 Remote User Testing 3 Agenda User-centered Design User Testing Output (Qualitative & Quantitative) Software & Usage Time Estimates & Cost Advantages & Disadvantages
4. Introduction to User-centered Design UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process. Its measures are: Effectiveness (accuracy and completeness to achieve goals) Efficiency (resources expended in relation to the accuracy and completeness) Satisfaction (comfort and acceptability of the work system to its users) 03/18/2009 Remote User Testing 4
5. User Testing is a UCD Method 03/18/2009 Remote User Testing 5
6. Overview User testing involves: A design A user (participant) An observer (facilitator). Observe user behavior & receive user feedback. Test tasks (not functions). 03/18/2009 Remote User Testing 6 User Design Observer
7. Qualitative Output Understand mental model of the user (navigation). Validate requirements and design assumptions (focus on important functionality early). Discover non-functional design mistakes and areas of improvement (unforeseen user requirements). Evaluate effectiveness of design elements. 03/18/2009 Remote User Testing 7
8. Quantitative Output Success rates / Error rates Task time Satisfaction ratings 03/18/2009 Remote User Testing 8 Task Success Task Time
9. User Testing Software TechSmithMorae Manager: $ 1,295.- Bundle: $ 1,495.- Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements. 03/18/2009 Remote User Testing 9
10. Remote User Testing Software TechSmithUserVue $ 149.- / Month or $ 1,495. / Year Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication. Multiple Observers can passively join the Session and share their observations with the Facilitator. 03/18/2009 Remote User Testing 10
11. The Session Setting up a session. Connecting (download of software bundle required). Calling participant. Starting the session. Viewing (and recording) the participant. Removing software bundle from computers of involved parties. 03/18/2009 Remote User Testing 11
12. Time Estimates & Cost Preparation (18 hours) User screening: 8 hours Task creation: 8 hours Environment: 2 hours User testing (10 hours) 2 hours per user 5 users per round Post-test (32 hours) Test report: 16 hours Implementation: 12 hours Communication: 4 hours 03/18/2009 Remote User Testing 12 Our time: 1 Round = 60 hours 2 Rounds = 110 hrs. 3 Rounds = 160 hrs. + User comps + Time additional observers
13. Advantages of remote UT Reduced (or no) travel time and expenses. Higher exposure through easy screen sharing (managers can sneak in easily). Actual user environment, familiar and comfortable. Possibly fewer drop-outs. 03/18/2009 Remote User Testing 13
14. Disadvantages of remote UT Facilitator not physically present (degree of separation can be challenging). Can't see facial expressions or non-verbal cues. Difficult to build rapport and trust. Difficult to control environment. Possibly technical difficulties (firewall, etc.). Setup and use of software or usability lab might be challenging and requires a liaison. 03/18/2009 Remote User Testing 14
15. Summary Increased usability Higher degree of client satisfaction Continued business = Higher revenues Lesser client aggravation (“This is completely unusable”) Assist PM during requirements and design Focus and prioritize on important functionality Identify unforeseen user requirements early! Reduced costs Training and service Viewer Help-Desk calls 03/18/2009 Remote User Testing 15
Editor's Notes
Projects are initiated by USER needs!Our biggest mistake as a SW provider is to not validate our assumptions, requirements and design against our target audience = users.
Highest cost-gain ratio of all methods
Goal is to observe user behavior.Goal can be achieved by testing tasks VS. use cases
- User model NOT EQUALS Organization view Requirement: Signup required to make purchase Input Form has high drop-out rate (investigate why) Form is filled out completely but never submitted
Correlate with our measurements mentioned earlier: effectiveness, efficiency and satisfaction.1.) Your boss told you to upload a raster file [actual file will be provided] to the SPF application (http://mikeg05/usfs)2.) Create an alias for the ‘Fire Potential 2006’ layer and reclassify it using the following values [list of values].3.) A high-level executive wants to see a detailed overview of the analysis “US 2006 National” you are working on. Send an email to mgaigg@esri.com that outlines the data source and scenarios used in this analysis.4.) Create an analysis that is based on a theme called ‘Political Sensitivity’ and includes the following data layers: [3 layers given]5.) Create a new analysis that is based on the data of an already existing analysis called “Redlands Tree Mortality”6.)Create a new scenario called “Spring 2006” as part of the already existing analysis “Redlands Tree Mortality” and add weights to the layers of this new scenario.
In summary: The implementation of User-centered Design and especially User Testing early in the project management cycle will result in: