SlideShare utilise les cookies pour améliorer les fonctionnalités et les performances, et également pour vous montrer des publicités pertinentes. Si vous continuez à naviguer sur ce site, vous acceptez l’utilisation de cookies. Consultez nos Conditions d’utilisation et notre Politique de confidentialité.
SlideShare utilise les cookies pour améliorer les fonctionnalités et les performances, et également pour vous montrer des publicités pertinentes. Si vous continuez à naviguer sur ce site, vous acceptez l’utilisation de cookies. Consultez notre Politique de confidentialité et nos Conditions d’utilisation pour en savoir plus.
today’s schedule➝ review of UX evaluation plans➝ test run of scripts➝ individual walk throughs
overview➝ about UX evaluations➝ UX evaluation criteria➝ UX evaluation methods➝ tips and hints➝ for more information ✱Attribution: parts of this presentation are based on " Forum Nokia’s UX Evaluation series
why UX evaluation?Success factor ➝ UX is a critical success factor for products and services ➝ We need conﬁrmation during design and development that we are on the right track ➝ It is challenging for the software developer to keep anThe designer is objective view of the developed software not the user ➝ It is human nature to become blind of your own outcomes ➝ Thus it is important to get a second opinion of the UX, from outside of the development teamSimulate real ➝ Preferably from real users trying it in a realisticuse situations environment
usability and product quality SW Product Quality (ISO-9126) Functionality Reliability Efficiency Usability Portability Maintainability Suitability Maturity Time behavior Operability Installability Testability Accuracy Fault tolerance Resource Learnability Adaptability Stability utilization … … ... … … … Implemented Code Number of sw Function point Source lines of functional coverage bugs analysis code requirements➝ there are many metrics to measure software performance and quality
UX and product quality In this deﬁnition, UX is not a part of product quality. A SW SW Product Quality (ISO-9126) engineering point-of-view? Functionality Reliability Efficiency Usability Portability Maintainability Suitability Maturity Time behavior Operability Installability Testability Accuracy Fault tolerance Resource Learnability Adaptability Stability utilization … … ... … … …➝ product quality Expectations, motives, ➝ measuring UX is not straight- has a strong actions, interpretations, forward impact on UX … ➝ there are many intervening issues User Experience
UX evaluation criteria 1/5➝ Nielsen (1993) on usability ➝ learnability ➝ Novices’ ability to reach a reasonable level of performance rapidly UX ➝ efﬁciency ➝ Expert user’s level of performance ➝ errors utility usabi social enjoy lity value ment ➝ Number of errors users make ➝ User’s ability to recover from errors pleasure stimulation useful relia ease- efﬁci acces connec identiﬁ sibility ting cation ➝ Existence of catastrophic errors? ness bility of-use ency people ➝ memorability ➝ Casual user’s ability to remember drivers tailored issues how to use a system as such for your detailed for ➝ satisfaction product your product ➝ User’s subjective assessment of the system basic usability heuristic UX drivers derived from the UX elements
UX evaluation criteria 2/5 Melvin, 35! Jason, 25! Louise, 27! use case storyboard accessing phone from web browser • Engineer from Germany • Customer Service • Web designer, graduate • Lives with his wife, 2 kids Assistant. student on university. and a dog. • Keeps a blog for his friends • SMS addict - 50 • Both parents need to and family messages every day. sometimes travel due their Uses his device for capturing • Likes arts, especially indie jobs, and kids have lots of images to blog. movies and old films. hobbies. Most of the images • Is currently organizing • Would like better manage surprise parties for her related to snowboarding. his everyday schedules friend. • Plays guitar in a wildly with family. not-yet-so-popular band.➝ Evaluate the design against the speciﬁed UX artefacts
UX evaluation criteria 3/5➝ evaluate against benchmark ➝ products and experiences
UX evaluation criteria 4/5 Bar Slide Full hw qwerty Full sw qwerty Touch screen Touch only screen Resistive screen Capacitative screenClamshell Swivel➝ Run your tests with all the target devices and form factors
UX evaluation criteria 5/5 people places context Things time Indirect, unspoken criteria Direct, spoken criteria - originating from the context ➝ the ultimate criterion: ➝ potential users try the product in their mobile context
planning the UX evaluations 1/2 A step-by-step process would specify needs be easy to and context of deﬁne the UX create UI evaluate the UI manage. use concept designs designs In practice, design and evaluation are specify needs deﬁne the UX often in a parallel create UI designs and iterative and context of use concept relationship. evaluate concepts and designs➝ for any UX artifacts, you can think can and should it be evaluated, somehow
planning the UX evaluations 2/2My cool travel-mate concept ➝ What artefacts do we want to evaluate ➝ Sketches, visualisations, prototypes,... ➝ What issues are we evaluating? ➝ Concept, interaction, grahical design, ... ➝ Who will do the evaluation? ➝ Experts, end users,... Evaluate concepts and designs
UX evalution: expert evaluation ➝ Usability experts evaluate a system based on common heuristics (i.e. design principles) and personal expertise ➝ Common heuristics are based on knowledge gained through expertise and experience ➝ Knowledge about perceptional and cognitive processes - such as the function of memory – are utilized ➝ At least two experts take part in the evaluation to ensure reliable results ➝ At the beginning of an expert evaluation, the value of the system to its user and buyer/provider is determined ➝ Based on these values, usability criteria are deﬁned ➝ At the end, the ﬁndings documented and prioritized. Solutions proposals are made ➝ Also user tasks can be deﬁned and walkthroughs conducted as part of the evaluation
UX evaluation: focus groups ➝ a group of people (4-7) that has something in common will be invited to semi-structured group discussion session to share their views on certain topic. Lasts usually 2 hours ➝ face-to-face sessions ➝ Sometimes on-line ➝ variations of the method ➝ Exploratory ➝ Feature prioritization ➝ Competitive analysis ➝ Trend explanation
UX evaluation: usability testing➝ the test user is asked to conduct certain tasks with the user interface under design ➝ paper prototypes can be used ➝ users are asked to think aloud Special equipment can be useful to catch the Moderator actions on mobile Observer devices Tested sw Test user Test cases➝ sometimes good to test the UI with pairs of users to trigger more discussion or to test social aspects➝ usability problems are identiﬁed and prioritized based on the user data gathered during the usability tests
UX evaluation: observation ➝ the researcher does not interrupt the user but just observes the user s actions ➝ the user is followed usually shorter time than when shadowing in her/his own environment, and usually only certain activities/time period are observed ➝ good for studying users that cannot be interrupted ➝ reveals task ﬂows and possible detectableIf observation is not possible (e.g. in hospital) video problems in them and behavioral patternsrecording can be alternative. but not motivations, attitudes, values, Video can be gone through with the participant(s) concerns afterwards
dont listen to users➝ watch users attempting to perform tasks in the UI ➝ watch what people actually do ➝ be careful with what people say they do ➝ dont believe what people predict they may do in the future.
when you listenbeware:➝ users often tell you what they think you want to hear➝ often based only on recall➝ rationalized behavior
UX evaluation: make a mix ➝ an expert evaluation and a usability test can be used as complementary methods ➝ an expert evaluation require less time and preparation than a usability test ➝ expert evaluations are recommended before usability tests in order to support designing the test tasks ➝ a satisfaction questionnaire can be combined with a usability test
UX evaluation: tips and hints ➝ the UX evaluation criteria should be in-line with the deﬁned UX drivers, targets and requirements ➝ make sure to share your UX evaluation criteria with the designers before they start their work ➝ do evaluations with real users ➝ expert evaluations can provide quick useful ﬁndings ➝ try to have a end user mindset ➝ ideally your evaluation methods should scale to weekly (or designevaluate even daily) evaluations ➝ design & evaluation iteration loops do not have to be iterate big monolithic steps ➝ UX evaluation(s) should be an integral part of software development process, just like software testing