Purchase Path User Testing Report

These materials (and the information contained therein) are confidential and proprietary in nature to Dick’s Sporting Goods, and they are intended solely for the participant(s) receiving them. You are permitted to review and use these materials for the limited
and express business purpose provided, and you are prohibited from copying, transmitting, sharing or distributing these materials for any purpose. Additionally, you must protect these confidential materials in accordance with the confidentially provisions set
forth in the contractual agreements in effect between you and Dick’s Sporting Goods.
PROJECT EAGLE
SOFT LAUNCH USABILITY STUDY
PURCHASE PATH - DESKTOP
ZackPontious&TimBroadwater
12/21/2016
CONTENTS
Executive Summary 2
Methodology 3
Test Participants 4
Issue Classification 5
Guest User Checkout 6
Registered User Checkout 12
My Account 18
Supplementary Data 24

PURCHASE PATH SOFT LAUNCH USABILITY TESTING | 12.21.2016
eCommerce User Experience | Page 2
EXECUTIVE SUMMARY
This report will examine findings related to key features in the Purchase Path desktop
experience. Please note that the website stability was in development, which isn’t
ideal, and contributed to some scripts being altered.
The following pages will outline what features were assessed, primary factors influencing success or failure, and
provide detailed usability findings. Additionally, the report will describe observations of user behavior that could
be indicative of pain points within the experience.
The key features assessed in this study were identified by the Purchase Path Pod during Q3/Q4 of 2016.
Functional pass/fail criteria is in parity to the scripts utilized by testing teams in Project Eagle UAT.
Usability pass/fail and issue classification will be assigned based on the frequency and potential impact of the
issue in question.
KEY FEATURES
• Delivery Methods Page
• Overall Cart & Checkout Layout for Phone
• Guest User Checkout & Leaving Checkout Flow
• Registered User Checkout, Saved Addresses, & Cards
• ScoreCard Widget and Modal in Checkout
• Shipping Eligibility & Restrictions
• BOPIS
• Promo Rewards
• My Account
• Save for Later
• Inventory Calls on Cart & Checkout

METHODOLOGY
TEST SCRIPTS
Guest User Test Script
http://bit.ly/2hUerEn
Registered User Test Script
http://bit.ly/2hUj3up
My Account Test Script
http://bit.ly/2hIiXaU
Users were tested via moderated and unmoderated task-based studies. During the moderated sessions, users
sat with a moderator and observer from the UX Team, who facilitated the tasks and noted issues/observations.
Unmoderated testing was taken by the user alone, utilizing a testing interface via the UserTesting platform.
All sessions were recorded (audio and screen capture) via UserTesting and TechSmith Camtasia.

TEST PARTICIPANTS
UnmoderatedSession1
SHOPPING BEHAVIOR
100% of Users Shopped Both In-Store
and Online at DICK’S Sporting Goods
Desktop Operating System
08% Macintosh
92% Windows
Online Shopping Device Preference
35% Use Desktop / Laptop
65% Use Desktop / Laptop / Tablet / Smartphone
Overlap Percentages of Most Used
Shopping Websites
91% Amazon
21% Target
17% Kohl’s
13% Best Buy
13% Walmart
SEX
BROWSER
0
1
2
3
4
5
6
7
8
45 - 5435 - 4425 - 3418 - 24
0
1
2
3
4
5
6
7
8
N/AIEFirefoxSafariChrome
AGE
0
3
6
9
12
15
No ResponseMaleFemale
0
1
2
3
4
5
6
7
8
45 - 5435 - 4425 - 3418 - 24
0
1
2
3
4
5
6
7
8
0
2
4
6
8
10
> 53 - 52 - 3< 1N/A
0
3
6
9
12
15
0
1
2
3
4
5
6
7
8
45 - 5435 - 4425 - 3418 - 24
0
1
2
3
4
5
6
7
8
0
2
4
6
8
10
> 53 - 52 - 3< 1N/A
SHOP ONLINE PER MONTH
0
3
6
9
12
15
0
1
2
3
4
5
6
7
8
45 - 5435 - 4425 - 3418 - 24
0
1
2
3
4
5
6
7
8
0
2
4
6
8
10
> 53 - 52 - 3< 1N/A

ISSUE CLASSIFICATION
All identified issues will be assigned a severity and a frequency. This criteria helps
with prioritizing corrective actions to remedy reported usability issues.
SEVERITY
High : User has no obvious way to work around
Medium: User has to actively try to overcome or work around
Low: User may pause or stop for a moment
FREQUENCY
A Few: Approximately 5-15% of users
Some: Approximately 16-50% of users
Most: Approximately 51-80% of users
Almost All: Approximately 81-90% of users
All: 100% of users
Based on a rating system implemented by the Baymard Institute: baymard.com
FREQUENCY
100%80%60%
LOWMEDHIGH
40%20%0%
SEVERITY

SUPPLEMENTARY DATAGUEST USER CHECKOUT

GUEST USER CHECKOUT UAT FEATURE VALIDATIONS
A summary of tasks as they relate to key features within the experience. The success rate is the number of total users who completed the task accurately. Observed issues is a high-level
summary of themes impacting usability of these features.
eCommerce User Experience | Page 7*See “Explanation of Terms” for more detail on these themes
FEATURE TASK USER SUCCESS RATE OBSERVED ISSUES*
Promo Rewards Show how you would view more details about the promotion. 8/8 Findability, Interaction
Add a warranty to the tent. 8/8 Comprehension, Content
Shopping Cart
Show how you would pick-up the tent at a local Dick’s Sporting
Goods store near you.
8/8 Comprehension, Content, Function, Performance
Click on the ‘Checkout’ button, and checkout as a Guest. 8/8 Interaction, Performance
Billing Address Fill out the billing address page. 8/8 Function
Fill out the shipping address page 8/8 --
Show how you would view more details about the different types
of shipping.
8/8 Comprehension, Findability, Interaction
Delivery Methods
Show how you would change your shipping methods to deliver as
quick as possible.
8/8 Findability, Performance
Use the following credit card information to checkout. 8/8 Interaction
Show how you would add the following promo code to your order. 8/8 Findability, Function, Performance
ScoreCard Modal
Dick’s Sporting Goods wants to know if you want to apply your
Scorecard to this order. Please choose an option.
4/8 Function
Review Page Review your order, then click the “Submit Order” button. 6/8 Performance

GUEST USER CHECKOUT USABILITY ISSUES BY FEATURE
FEATURE ISSUE SEVERITY FREQUENCY
Loading & Performance
Environment is slow to update delivery date
Adding BOPIS store took a very long time
Delivery page took a very long time to load
Applying a promo took a very long time
Submitting an order takes a very long time
Low
Low
Low
Low
High
All
All
All
All
All
General Experience
When user edits the address first name isn’t updating
User receives we are experiencing technical difficulties error
Flash sale modal with x in graphic keeps popping up
No images showing in cart
High
Medium
Medium
Low
All
A Few
Most
Almost All
Promotions
Promo submission displayed error then accepted
Language changes under promo code to “Lorem Ipsum” when applying promo
Promo code is invalid, then accepted
Promo code is already used, then accepted
Low
Medium
Low
Low
All
All
Most
Most
Shipping Methods
Shipping details needs to be completed/has placeholder text
Threshold or room of choice detail information is not present
Medium
Medium
All
All
ScoreCard
Scorecard detection and modal didn’t trigger
“Lorem ipsum” text appears above promo
0 out to left of promo appears in order summary
Medium
Medium
Low
Most
All
Some
BOPIS
BOPIS produces error with various zip codes
BOPIS modal shows no black overlay in background
Medium
Low
Some
Some

GUEST USER CHECKOUT OBSERVATIONS
Users expected that typing ‘PA’ or ‘Pen’ in the state drop-down would populate Pennsylvania. Users had a difficult time
seeing when items would be shipped, and didn’t notice the delivery dates. Users also didn’t notice instructional text
below input fields, and in general users thought the form fields were clunky and cumbersome.
Users expressed frustration with the performance of drop-down form fields being cut off, and would prefer to click
on the detail text as links rather than the ‘i’ icons. Users also expect visual feedback for hover and click state of all
buttons, and to be scrolled to anchored errors.
ENHANCEMENTS / QUICK WINS
• Zip code-first address entry to populate city/state
• State drop-down that takes ‘OH’ and ‘Ohio’ text entry
• Move expected delivery dates next to shipping options
• Make input instructions stand out more
• Drop-downs populate above and below the browser window
• Make shipping, promotion, and question detail text also a link
• Add button hover and click states
• Add functionality to scroll/anchor to errors after user clicks on call to action

GUEST USER CHECKOUT FEEDBACK
I think that I would like to use this online
checkout frequently.
I found the entire checkout experience to
be unnecessarily complex.
I thought that the online checkout was a
pleasant experience.
I think that I would need help to be able
to use the online checkout.
I found the various steps of the checkout
experience to be intuitive.

GUEST USER CHECKOUT RECOMMENDATIONS
DESCRIPTION RATIONALE
Explore different solutions for bundling/displaying orders that have multiple shipping options. Users find the presentation of shipping orders confusing, and counter intuitive.
Conduct a label audit or taxonomy study for shipping terminology.
Standard, expedited, threshold, and other terminology doesn’t meet user expectations. We have a
study on this previously ran for the optimization team that could prove useful.
Perform content strategy and testing for specific shipping and promotion details for relevancy,
user expectations, and comprehension.
Users want relevant information, and not have to read through walls of text, a glossary, or codex
of information.
Prototype and test tooltip functionality on desktop for details as opposed to modals.
Users want information quickly, and not have to interact with pop-up or modals, when their
expectation is that this can be done in-page.
Conduct an optimization test for BOPIS that only populates a user interface with results that have
BOPIS buttons.
‘Limited Stock’ doesn’t meet user expectations, and if they are going to use BOPIS some may only
want to see relevant store locations.
Prototype and test user being able to change and edit information in the shopping cart, without
having to go back to the beginning of the checkout process.
User feel like they should be able to change information dynamically inside of a linear checkout
process, and not have to restart the process if a change is made.
Prototyping and test a more intuitive way for users to make accounts in the process of guest
checkout.
80% of checkouts are from guest users, and users will make an account if they have a positive
experience, and the account is easy to make.
User test incentive for guest users making accounts in the guest checkout, like 20% off this/next
purchase.
80% of checkouts are from guest users, and users will make an account if they have incentive to
do so.

SUPPLEMENTARY DATAREGISTERED USER CHECKOUT

REGISTERED USER CHECKOUT UAT FEATURE VALIDATIONS
Promo Rewards Show how you would view more details about the promotion. 7/7 Findability, Interaction
Shopping Cart Show how you would add a warranty to the tent. 7/7 Comprehension, Content
Show how you would login with your Dick’s Sporting Goods
account and start to checkout.
7/7 Findability
Show how you would continue through the checkout process. 7/7 --
Delivery Methods
Show how you would view more details about the different types
of shipping.
8/8 Comprehension, Findability, Interaction
Show how you would change your shipping methods to deliver as
quick as possible.
7/7 Comprehension, Findability, Performance
Show how you would add the following credit card to your
account
7/7 Function, Interaction
Payment
Show how you would update your expiration date to your saved
card to any future date of what it is currently.
7/7 --
Show how you would add the following promo code to your order. 7/7 Findability, Function, Performance
ScoreCard
Has your Scorecard been applied to this order? How do you
know? Answer, and click the “Continue” button.
4/7 Comprehension, Content, Visual
Review Page Review your order, then click the “Submit Order” button. 5/7 Performance

REGISTERED USER CHECKOUT USABILITY ISSUES BY FEATURE
Loading & Performance
Environment is slow to update delivery date
Adding BOPIS store took a very long time
Delivery page took a very long time to load
Applying a promo took a very long time
Submitting an order takes a very long time
Low
Low
Low
Low
High
All
All
All
All
All
General Experience
When user edits the address first name isn’t updating
User receives we are experiencing technical difficulties error
Flash sale modal with x in graphic keeps popping up, and is confusing for viewers
No images showing in cart
User returns to cart page when hitting submit order button
High
Medium
Medium
Low
Medium
All
A Few
Most
Almost All
Some
Payment
Entered visa card doesn’t save when user clicks to edit other card
Update button credit card isn’t very visible
User isn’t seeing an immediate button click feedback
Promotion Modal doesn’t load background transparency
Medium
Low
Low
Low
All
All
All
Some
Shipping Methods
Shipping details needs to be completed
Threshold or room of choice detail information is not present
Shipping costs remain with free shipping.
Medium
Medium
High
All
All
Some
ScoreCard
Scorecard detection and modal didn’t trigger
Lorem ipsum text appears above promo
0 out to left of promo appears in order summary
User thinks ScoreCard hasn’t been applied because it’s grayed-out
Medium
Medium
Low
Medium
Most
All
Some
Most
Checkout
Two credit cards are showing up in checkout.
Fee Gift Modal not working properly
Low
Medium
A Few
All

REGISTERED USER CHECKOUT OBSERVATIONS
Users expressed a desire for more obvious placement of delivery estimates as being next to the radial buttons. Users
are also confused by the form field entry methods, and why ScoreCards aren’t being automatically applied when they
are logged-in to their account.
Users exhibited a tendency to clicking on promotion and shipping detail text as opposed to the ‘i’ icon, as well as a
desire to read specific information by doing so.
• Utilize a more obvious placement of delivery info in line with radio buttons
• Auto caps for form fields that aren’t case sensitive
• Revise ScoreCard to apply all of the time for logged-in users
• Implement clicking on promo text to open promo details modal
• Don’t clear login username in form fields so user can see their mistakes
• Incorporate specific information per the promotion, and not generic information in modal

REGISTERED USER CHECKOUT FEEDBACK
I think that I would like to use this online
checkout frequently.
I found the entire checkout experience to
be unnecessarily complex.
I thought that the online checkout was a
pleasant experience.
to use the online checkout.
I found the various steps of the checkout
experience to be intuitive.

REGISTERED USER CHECKOUT RECOMMENDATIONS
Test users to establish more clarity around shipping cost, promos, credits
Users seem to lose information that isn’t in the account summary, so testing
perception and visibility of users for promos, shipping cost, and credits could yield
valuable information.
Test visibility of promo code input in and out of accordion on the payment page
Users wonder why these are in accordions, and why they aren’t inside order summary,
or more visible.
Conduct label audit or test taxonomy against user expectations for extension plan,
warranty, or contract terminology.
Extension plan, warranty, or contract terminology doesn’t meet user expectations.
Test user expectations and understanding of check-boxes for billing and shipping on
the same page, as opposed to other display methodologies
Users want information quickly, and not have to interact with pop-up or modals, when
their expectation is that this can be done in-page.
Conduct label audit or test taxonomy for threshold and room of choice verbiage
Standard, expedited, threshold, and other terminology doesn’t meet user
expectations. The UX Team have a study on this previously ran for the optimization
team that could prove useful.
Test user expectations and prototype solutions for products being broken out by
shipping methods, versus display alternatives for bundling
Users find the presentation of shipping orders confusing, and counterintuitive.

MY ACCOUNT UAT FEATURE VALIDATIONS
Save For Later
Show how you would save these shoes to purchase at
a later time.
5/7 Findability, Function, Interaction, Performance
Find a shirt that you like on the Dick’s Sporting Good’s
website. Show how you would save this shirt to
purchase at a later time.
7/7 Findability, Performance
My Account
Now, show how you would view all the items that you
saved for later.
3/7 Comprehension, Content, Findability, Navigation
Saved Products
Go ahead and click on the ‘Shop Now’ button next to
an item that you saved.
7/7 Comprehension, Content, Navigation
Click the ‘Back’ button on the browser, and show how
you would remove one of your saved items.
4/7 Function
Subscription
Show where you would go to sign-up or unsubscribe
from Dick’s Sporting Goods emails?
5/7 Comprehension, Content, Navigation
Manage Login
Show where you would go if you wanted to change
your email or password.
7/7 Comprehension, Content, Navigation, Performance
Credit Card Show where you would add or edit a credit card. 7/7 --
Address Book Show where you would add or edit an address. 7/7 --

MY ACCOUNT USABILITY ISSUES BY FEATURE
My Account Utility Navigation
Account drop-down not working
Users not seeing My Account
High
Medium
Most
Most
Saved Products
Remove link doesn’t remove items
Remove link causes an error
No error when user saved item without attributions
selected
No scrolling when user saved item without options
Confirmation modal didn’t show after user login
Saved items weren’t in saved products
Medium
Low
Medium
Medium
Medium
High
Some
Some
All
All
Almost All
Some
Performance / Load Times Webpage load times were extremely long Medium All
Subscription Subscription page was broken on-load, user had to reload Medium A Few

MY ACCOUNT OBSERVATIONS
Users find the My Account experience to be too broken out, sparse, and cumbersome. Users feel they have to go too
many places to get pieces of information, rather than it being combined in relevant locations.
Users expect to be able to see saved products in cart, and don’t understand the semi-list/semi-saved product
functionality for that reason. Users also expect the button text in Saved Products to be ‘See Item’ or ‘Add To Cart’, and
don’t understand why it takes them back to the PDP.
Finally, users don’t understand the majority of the left-hand navigation terminology in My Account, and generally want
more robust email subscription options like weekly, based on deals, or holidays.
• Merge ScoreCard points and rewards onto the same page
• Make saved products viewable in cart
• Change button text from ‘Shop Now’ to ‘See Item’, or ‘Add To Cart’ and then add product to cart
• Explore ways to make email opt-in to be more visible and not counter-intuitive
• Allow users to subscribe to emails weekly, based on deals, or holidays, or other preferences
• Explore ways to make ‘Save for Later’ on PDP and in cart more dominant
• Remove the ticker from My Account to reduce accidental clicking of registered users

MY ACCOUNT FEEDBACK
I think that I would like to use the online
account frequently.
I found the entire account management
experience to be unnecessarily complex
I thought that using the online account
was a pleasant experience.
to use the online account.
I found the various parts of the account
management experience to be intuitive.

MY ACCOUNT RECOMMENDATIONS
Prototype and conduct user tests for saved items being visible in cart, and their
placement in the cart
A decent percentage of users expect saved products to be accessible on return
through their cart.
Test functionality, expectations, verbiage, and usability of ‘Lists’ or ‘Wish Lists’ versus
‘Saved Products’
Users are confused between the verbiage and functionality of lists, saved products,
and the quasi-methodology employed on the DSG website.
Prototype and incorporate more recognizable user utility navigations that show a
person icon, and let them have quick links to their needs
Users feel that the My Account utility navigation isn’t located, recognizable, or
visually similar with their expectations of other utility navigation accounts.
Label audit/taxonomy study for subscriptions, preferences, manage login, and
account summary
Users find the My Account experience to be too broken out, sparse , and cumbersome.
The terminology doesn’t meet user expectations, and should be audited.
Prototype and conduct user tests for merging various my account functionality and
sections
Users feel they have to go too many places to get pieces of information, rather than it
being combined in relevant locations.

SUPPLEMENTARY DATASUPPLEMENTARY DATA

TITLE HYPERLINK
UserTesting Raw Data http://bit.ly/2hhEng4
User Feedback Data Charts http://bit.ly/2herzp9
Guest User Moderated UserTesting Videos
https://www.usertesting.com/v/a827cca6-9738-4616-a560-a4d4373a9790?back=%23study_2011042#/notes
https://www.usertesting.com/v/ec9e6d13-55df-49ab-9102-5c554cb647c3?back=%23study_2011042#/notes
https://www.usertesting.com/v/c72190d9-4814-403f-81b4-13ee7a2d5245?back=%23study_2011042#/notes
https://www.usertesting.com/v/82aeea47-6a7f-45c4-b79f-a49404bab8f5?back=%23study_2011042#/notes
Guest User Unmoderated UserTesting Videos
https://www.usertesting.com/v/8f2e7a46-4848-4b6c-824d-13de511b374a?back=%23study_2011039#/notes
https://www.usertesting.com/v/0760c84d-e332-4e86-853f-1243f294f52a?back=%23study_2011039#/notes
https://www.usertesting.com/v/15ca9ea6-1d38-4616-8a25-2d24718d7c8b?back=%23study_2011039#/notes
Registered User Moderated UserTesting Videos
https://www.usertesting.com/v/91a1e302-06e8-43a8-b23d-c22d0c637ce4?back=%23study_2011049#/notes
https://www.usertesting.com/v/ce54cc64-b7ca-4113-adad-f7058c901f34?back=%23study_2011049#/notes
https://www.usertesting.com/v/2b14a49a-4e90-40ce-9f0e-e23bf7ea5dc6?back=%23study_2011049#/notes
https://www.usertesting.com/v/6d936386-0610-4181-8a38-48ba3612b169?back=%23study_2011049#/notes
Registered User Unmoderated UserTesting Videos
https://www.usertesting.com/v/598e093e-9f73-48a3-9f0b-05712119f8e2?back=%23study_2011047#/notes
https://www.usertesting.com/v/74a09e02-2f48-4bf0-8f4c-acd9c4643f09?back=%23study_2011047#/notes
https://www.usertesting.com/v/8b4db064-a8f8-42fa-9291-9dc30a4fb0c0?back=%23study_2011047#/notes
https://www.usertesting.com/v/c2c8db33-1adf-4a1f-9163-23556f695d88?back=%23study_2011047#/notes
My Account Moderated UserTesting Videos
https://www.usertesting.com/v/c5f1e4d4-7753-4410-a826-f0b41ff1f2e5?back=%23study_2026473#/notes
https://www.usertesting.com/v/c9543169-672f-44f9-8b73-63126266c1e7?back=%23study_2026473#/notes
https://www.usertesting.com/v/0da13f8f-9a35-411b-b2e1-bb380792767f?back=%23study_2026473#/notes
https://www.usertesting.com/v/1202edd2-c7d2-48c5-9d30-e818b4130892?back=%23study_2026473#/notes
My Account Unmoderated UserTesting Videos
https://www.usertesting.com/v/f4064334-bd06-4570-8b73-d3a074606f98?back=%23study_2011019#/notes
https://www.usertesting.com/v/778493b0-39ad-43ad-8aa2-045782e1a774?back=%23study_2011019#/notes
https://www.usertesting.com/v/73ca5fa3-1e70-4ef0-8d3c-a2fe0291030e?back=%23study_2011019#/notes
EXTERNAL DATA

TERM ISSUES ENCOMPASSED
Comprehension Understanding, interpretation, wording, tone
Content Interpretation or understanding of graphic/text elements
Findability Noticing elements
Function Unexpected or broken behavior, including bugs
Interaction Triggering actions, noticing elements, understanding behavior resulting from clicks or gestures
Legibility Visual contrast lending to comfortable reading
Navigation Ability to find pages, go back and forward, taxonomy
Performance Site speed, page load
Site-Specific Promotions, product availability, loyalty, pricing
Visual Insufficient contrast for non-text elements, hierarchy, proportion, consistency
EXPLANATION OF TERMS

Purchase Path User Testing Report

Recommandé

Recommandé

Contenu connexe

Similaire à Purchase Path User Testing Report

Similaire à Purchase Path User Testing Report (20)

Plus de Tim Broadwater

Plus de Tim Broadwater (20)

Dernier

Dernier (8)

Purchase Path User Testing Report