A summary report of user research done onsite at the 2014 Imagine conference in Las Vegas

1. Magento Redesign / Cover Confidential Modified Tue Jun 03 2014 1
CREATED: LAST MODIFIED:
AUTHOR:
MagentoConfidential
Commerce Experience Research Labs
Summary Report • Research Project • Imagine 2014 Conference
JIRA: MTD-815
05.20.2014 Tue Jun 03 2014
Coburn Hawk

2. Magento Redesign / Original Whiteboard Concept Confidential Modified Tue Jun 03 2014 2
Original Whiteboard Concept
1. The original concept for Imagine 2014 emerged from
a conversation between Keith Howard, Jonathan
Atkins, and Coburn Hawk.
2. The experience at the show would be a
combination of two common* research methods:
The Graffiti Wall and
the Love Note / Breakup Letter
3. The purpose of the project was to engage with
Magento merchants, developers, System Integrators,
and Partners.
We would get real time feedback on the Magento
products while also building a pipeline of customers
who would be willing to participate in more formal
research projects in the future.
C.E.R.L.
Commerce Experience Research Labs
1
2
3
* from the book Universal Methods of Design by Bella
Martin & Bruce Hanington. (p. 96-97 and p. 114-115)

3. Magento Redesign / What's in a name? Confidential Modified Tue Jun 03 2014 3
What's in a name?
Quite a bit actually. The name was chosen very
purposefully to let our people know this was a serious
effort while still having fun. (important to engagement at
a trade show)
C.E.R.L.
Commerce Experience Research Labs
The word Commerce was used rather than
eCommerce to speak clearly that commerce is an
omnichannel, connected experience.
1. The brand team was consulted to put together a
logo treatment we could use without breaking their
guidelines. The design team had an approved logo
treatment in 3 days.
2. The new logo and name were put on lab coats to
create a fun way to approach customers at the show.
1
2

4. Magento Redesign / The Resulting Installation Confidential Modified Tue Jun 03 2014 4
The Resulting Installation
1. The concept was simple: A participant could either
choose to write a Love Note or a We Need To Talk…
communication.
2. In the process of collecting real time feedback, many
personal connections were made between the Magento
community and the User Experience designers that are
both improving the current product and design then
future releases, starting with Magento 2.0
3. It is noted that this is qualitative research and not
meant to be statistically significant.
More than anything, this was meant to establish a
series of relationships with which we could build a solid
research practice moving forward.
1
2
3

5. Magento Redesign / The Basics Confidential Modified Tue Jun 03 2014 5
The Base Numbers
What I Love About… Notes : 33
We Need to Talk… Notes : 76
Business Cards submitted : 60
Total Points of Engagement : 169
The Dear Magento... board was initially manned by
2 researchers in and was rolled out into public areas
of the Hard Rock Hotel starting on May 13th, 2014 at
approximately 11:30am.
After trying one location for about 20 minutes,
we moved the installation. The final location
made it easier to engage with participants.
At the final location, we increased to 5 researchers
and we all stayed very busy talking to participants
for about 3 - 4 hours.
We left the board unattended overnight with plenty
of supplies so participants could leave notes. By
morning, the boards was full of feedback.

6. Magento Redesign / High Level Categories • 33 notes Confidential Modified Tue Jun 03 2014 6
High Level Categories • 33 notes
Community
Open Source
Show me the Money
Training
Ease of Use
Support
Other
Random Graffiti
Encouragement
After a basic card sort of the submissions, here were some categories that seemed to emerge.
6 Notes
5 Notes
3
3
3
2
4
4
3

7. Magento Redesign / High Level Categories • 76 notes Confidential Modified Tue Jun 03 2014 7
High Level Categories • 76 notes
After a basic card sort of the submissions, here were some categories that seemed to emerge.
Extensions & Connect
Speed
Product Creation
Developer Feature Requests
International Markets
Admin Related
Product Related
Price Concerns
Updates/Patches/Install
Merchant Feature Requests
Integrations
Segmentation
Support
Analytics
Partners
Site Search
CMS
Other
8 Notes
6 Notes
6
6
6
5
4
4
4
4
3
3
3
3
2
2
2
5

8. Magento Redesign / High Level Findings : Negative Notes Confidential Modified Tue Jun 03 2014 8
Negative communications in regards to
the load times, speed, and overall
performance of Magento received 6 total
notes.
International Markets had a score of
6 total notes. These were requests for
Magento expansion mostly in Asia.
Extensions received the most "We need to talk…"
communications with 8 total notes.
High Level Findings : Negative Notes
It was no surprise to the UX team that
Product Creation scored 6 total notes
with comments about the process being
difficult.
Developer Feature Requests had a score
of 6 total notes. These were requests that
were both very technical and very specific.

9. Magento Redesign / High Level Findings : Positive Notes Confidential Modified Tue Jun 03 2014 9
Community (the social construct, not the product) received the most "What I love…"
communications with 6 total notes. Our customers seem to like every aspect of the Magento
community… it beat out "Money" in things they love.
A category that emerged was Ease of Use.
Even with a modest 3 total notes, the participants
who left accolades in this categories were
passionate about how easy they found Magento.
High Level Findings : Positive Notes
In a close second with 5 total notes
was Open Source and our continued
commitment to it.

10. Magento Redesign / Details Confidential Modified Tue Jun 03 2014 10
MagentoConfidential
Detailed Results
Photos & transcript of what participants wrote
Revision History
06.03.2014 Updates based on peer review. Some feedback moved to more appropriate categories. Visual layout changed for better read-ability
All data entry of the cards complete. First cut of the High level findings created for peer review.05.22.2014
PAGE #
Document Created - Presentation of Draft to the team in 5.21.2014 UX Milestone 4 Deliverable Demo
NOTES
05.20.2014
MODIFIED
All

11. Magento Redesign / Details: We need to talk... Confidential Modified Tue Jun 03 2014 11
MagentoConfidential
Detailed Responses • Total : 76
Extensions & Connect
Speed
Product Creation
Developer Feature Requests
International Markets
Admin Related
Product Related
Price Concerns
Updates/Patches/Install
Merchant Feature Requests
Integrations
Segmentation
Support
Analytics
Partners
Site Search
CMS
Other
8 Notes
6 Notes
6
6
6
5
4
4
4
4
3
3
3
3
2
2
2
5

12. Magento Redesign / Extensions & Magento Connect Confidential Modified Tue Jun 03 2014 12
Extensions & Connect
8 Notes
Some low quality extensions
can break the whole site
Indexing is a nightmare in CC
on large product databases
Sub Par extensions and how to
report them
+1 +1
More qualified extensions!!
+1
Include unit test files when
submitting new extensions!
+1
What will happen to extension-
generated data when we make
the move to M2?? Does it stay
or go away?
Search on Connect
Rewards / Royalties for successful
extensions, including promoting
good 3rd party accomplishment
+1
All the +1's were added by other
participants who agreed
CE?
On the individual extension page, we'd like to be
able to:
• Have more freedom to customize the look of
pour detail page - current HTML is limiting
(i.e. fonts, colors, etc)
• FAQ page that we can post instead of waiting
for users to post questions
• When uploading screenshots, the ability to
rearrange the order of the pictures
by: rachel.hiraldo@oct8ne.com

13. Magento Redesign / Speed & Product Creation Confidential Modified Tue Jun 03 2014 13
Speed
6 Notes
Have Speed issues!!!
I wish you were faster.
15 second page load?
Load times for shopping carts
with 50 plus items - Front end
Performance needs to improve.
Some sites lost $
very true!
Index
This note was added by
another participant
Product Creation
6 Notes
PRODUCT CREATION
Make it Easier!
Easier product linking
I hate the way we link products
today but tomorrow… we can be
friends
(can we link products easier?)
Easy import / export of product feed
- product setup (New products)
* Edit product by store view
Export orders to CSV in UI
Add free product to purchase
on checkout
Copy Categories + tree

14. Magento Redesign / Developer Feature Requests & International Confidential Modified Tue Jun 03 2014 14
Developer Feature Requests
6 Notes
FIX ENTERPRISE
TARGET RULE
PLEASE : )
Why use Compass versus
LESS-based CSS framework
which is used in Magento 2 ???
ORMs should plug into multiple
databases. support postgres,
mongo and others!
SASS?
Coding Standards Please!
Scoring?
Fix that data model.
International Markets
6 Notes
Come to Asia
Don't forget this
part of the world
Cover the asia market
We are Growing!
You definitely need to be in
India! Don't increase prices…
PLEASE!
Unfortunately the translation for CE
1.9 / EE 1.14 does not really work
- a lot of wrong translations
- community module not able to
replace them
• We need payment gateway in Latin America
• You can take examples of Latin American
business
• we need Spanish language support
That we are the unique company from
Brazil present on the event
WEBJUMP
Sass is a scripting language
that is interpreted into
Cascading Style Sheets

15. Magento Redesign / Product Related & Admin Related Confidential Modified Tue Jun 03 2014 15
Admin Related
5 Notes
The layout of the list filter in
admin needs some love
Expand dashboards + Reports
make them configurable
There's no mobile responsive
version of Admin!!
+1
Centralize configurations
*Can I Make Magento my product master?
- - > Use a PIM
Improve search in admin
This note was added by
another participant
Product Related
4 Notes
Intermediate product between
CE and EE
I want to see your product
experiences more tangibly and
visibly shown in Keynote
Being able to change the order
of the flow
Gift Lists
Specifically the checkout
flow, I spoke to them

16. Magento Redesign / Price Concerns & Updates/Patches/Install Confidential Modified Tue Jun 03 2014 16
Price Concerns
4 Notes
Scared of random price
increases!
EE costs have risen 50%
over 4 years. Feels like an
annual shake down!
Enterprise should be free'r
Metered billing of Enterprise
Edition for cloud/AWS
scaling
I disagree. We kinda like our
price points now
Updates/Patches/Install
4 Notes
Consistent patches + upgrades
(bugs + Patches)
Detailed notes for changes
(Updates + Patches)
We Can't Upgrade!
1.11.1.0
Do you know who your
customers are?
ask for an email during install
This note was added by
another participant

17. Magento Redesign / Feature Requests & Integrations Confidential Modified Tue Jun 03 2014 17
Merchant Feature Requests
4 Notes
We need to apply multiple
coupons + promos on one
order
We need better resource
management on front end,
especially w/ responsive
theming.
Not enough docs on
abandon cart
Help finding things.
P.O.S. systems, etc.
Integrations
3 Notes
Overwrites for shipping
(look at Mageworks)
Integration capability
w/ salesforce + other services
(CRM, ERP)
Doesn't play nice with NetSuite!
A NetSuite rep saw this and
they resolved it right there
on the board
They wanted better
documentation specifically
for abandon cart

18. Magento Redesign / Segmentation & Support Confidential Modified Tue Jun 03 2014 18
Segmentation
3 Notes
Improve customer
segmentation nuturing via
custom navigation
+1
It's difficult to create
nuanced demographic
segmentation
…about
Referral Segmentation
Support
3 Notes
EE Support - The solution isn't
always root access to our
servers
We need to access advance
ebay support to overcome
account issues using plug-in to
sell on ebay
Be more supporty to your EE
customers

19. Magento Redesign / Analytics, Partners, Site Search Confidential Modified Tue Jun 03 2014 19
Analytics
3 Notes
Expose Customer ID
further up conversion funnel
(i.e. pre-Login)
UniversalAnalytics tie in.
Analytics require
Coburn-size databases!
Checkout is difficult to do A/
B testing on.
Partners
2 Notes
Conversation with potential
partners need a better rhythm
and speed
Make your partner program
EASIER
Extensions Easier
Site Search
2 Notes
Site Search needs some
"Improvement"
+1
Site search needs very serious
improvement! Search results
are simply unacceptable !!!
I think this was a reference
to extra large researcher,
Coburn Hawk

20. Magento Redesign / CMS & Other Confidential Modified Tue Jun 03 2014 20
CMS
2 Notes
We have tons of CMS blocks.
Please list them
alphabetically
CMS Hierarchy
Training
1 Note
We need more opportunities
for Magento training
Other
5 Notes
about making hosting easier.
Less Expensive +
More Performance
Bring the humanity of the brand
success stories to life more
You keep me up at night
X0X0!
Auto-correct changes your
name to "Magneto"
Waiting for Magento 2.0

21. Magento Redesign / Details: What I love about you... Confidential Modified Tue Jun 03 2014 21
MagentoConfidential
Detailed Responses • Total : 33
Community
Open Source
Show me the Money
Training
Ease of Use
Support
Other
Random Graffiti
Encouragement
6 Notes
5 Notes
3
3
3
2
4
4
3

22. Magento Redesign / Community & Open Source Confidential Modified Tue Jun 03 2014 22
Community
6 Notes
Community
150 developers directly involved
in Magento!
I love the opportunity you
provide to your Merchants and
Solution Partners.
• It has transform us to a new level
of business
• we love you!
Bringing small biz awesome &
powerful tools
Supporting the small agencies
Open Source
5 Notes
Open Source is AWESOME!!
Open Source means you're
proud of your code!
Commitment to CE, knowing it
doesn't directly create revenue
for Magento.
The range of applications and the
types of companies it works for
+ In complexity lies elegance
+ open source
+ Fall-backs

23. Magento Redesign / Money, Training, Ease of Use, Support Confidential Modified Tue Jun 03 2014 23
Show me the Money
3 Notes
You Make Me
MONEY
1/3 of the cost of every other
provider
Total cost of ownership
Training
3 Notes
Magento U : )
I ♡ Ben Marks
+1 +1 +1 +1 +1
User friendly. easy to teach
other people.
Ease of Use
3 Notes
You make it easy -
Sh*t you are easy! !
We got up and running in 2 months!
End to End in 6 months!
Love the scope of options available
out of the box!
Support
2 Notes
I love Magento Enterprise support!
Life & sho savers ♡
Is how quickly you respond to us & how
helpful you ares to new extensions
entering the marketplace
All the +1's were added in the
middle of the night… Ben Fans
She Savers? Show Savers?

24. Magento Redesign / Other, Encouragement, Random Confidential Modified Tue Jun 03 2014 24
Other
4 Notes
Your Next Version.
(Get Here Soon, Plz!)
So many useful Extensions!
1.14 works! New Template!
Great employees & partners
Encouragement
3 Notes
IMAGINE!
You can do it!
Orange is my favorite color!
Random Graffiti
4 Notes
Not transcribed. Mostly self promotion &
scribbles you get by leaving a message
board unattended at a trade show.

25. Magento Redesign / The Researchers Confidential Modified Tue Jun 03 2014 25
The Researchers
Jonathan Atkins, Keith Howard, Coburn Hawk,
Daniel Mennega, John Goff

26. Magento Redesign / What we could do differently... Confidential Modified Tue Jun 03 2014 26
What we could do differently...
1. It might be more effective to position the "Love
Notes" in a different way. If the sentence started:
"Dear Magento, please don't change…"
That positioning might yield more actionable data.
2. Some simple lecturns would make it easier for
participants to engage, especially when they are at a
trade show with the hands full.
3. Whenever possible, having options for the location
is critical. Some locations just don't inspire the kind of
engagement that is necessary to make this type of
interactive experience work.
4. We wanted to also conduct sit down interviews with
participants who were willing to spend a few minutes
talking. This was scrapped due to the inadequate
lighting for good video and the unexpected fast pace of
the experience.
5. If we want to have video interviews in the future, we
probably need one person dedicated to just that as well
as some portable lights.
6. Although we did have a twitter account set up,
having a digital display of the notes linked to a hashtag
would be a nice upgrade to the experience in the future.
1
2 3
4
5
@MagentoCERL
#dearmagento 6