3. Today’s Presenters
Chris Musto,
GM, Keynote Competitive
Research and Financial Services
Keynote Systems
Ken Harker
Senior Consultant
Keynote Systems
5. To learn what works in online banking, Keynote asked online bankers
as they used online banking
SITES EVALUATED: SAMPLE:
Bank of America 7 evaluations; 1519 panelists total
Chase ~200 per site; US online adults 18+
Citibank Current online banking customer of site being
evaluated
SunTrust All panelists successfully logged in to account
U.S. Bank
Wachovia TIMING:
Wells Fargo
Wave 7 evaluation period: May 2010
Previous evaluation period: May 2009
Next evaluation: May 2011
7. To the user, the evaluation was a natural experience
The user’s view
Keynote asks user
to perform a task
Keynote asks
questions about
the user’s
experience
performing that
task
Keynote asks the
user’s impressions
of the bank
To us, it was an opportunity to find out how some banks get better
business outcomes from online banking
8. We asked users to perform six tasks
1. Log In
2. Check Balances and Transactions
3. Update Personal Information
4. Schedule Payment or Explore Online Bill Payment
5. Use Customer Support
6. Explore the bank’s other products
12. Customer Support Rankings
Top metrics in this driver
SunTrust and Bank of America among top sites • Options to resolve
satisfaction
• Agree in‐context help
provided
• Agree site cares
• Online help not well
organized
• Difficult to locate FAQs and
other online help
• Difficult to search online
help
• Site doesn't provide
specific tips/ instructions as I
complete different tasks
15. Live Chat gets noticed: Bank of America
Multiple contact/ help options
• “There are different options to get customer
service such as telephone, email or chat.”
• “…seem to be there for you 24/7. If you need
to talk to someone, you can chat, e‐mail, or
call the 1800 number.”
• “I like the fact that there are different
channels of support whether it is live or
web/phone. I do like having immediate
support.”
• “I like that when I click help it pops up help for
the section of the site I am on, ex if I am in bill
pay I am given help for bill pay.”
Live chat helpful for immediate help
• “Like very much online live chat. Very very
helpful.”
• “I used the live chat for a problem and it was
resolved right away.”
• “…I also really like that there is an online chat.
That makes it really easy when I am already
on the computer and I need help solving an
issue.”
17. Bill Pay Satisfaction Rankings
All metrics
• Schedule payment
satisfaction (Q58)
• Schedule payment
difficulty (Q56)
• Inadequate online banking
feature: Pay bills online
(Q117)
• Reason to Stop Online Bill
Pay: Difficult to use (Q61)
• Reason to Stop Online Bill
Pay: Not enough bill pay
features/options (Q61)
• Inadequate online banking
feature: Receive bills online
(i.e., e‐bills) (Q117)
• Reason to Stop Online Bill
Pay: Fees are too high (Q61)
19. Bill Pay Satisfaction Best Practices: U.S. Bank
Quick and easy to use
• “Quick, clear, easy.”
• “It's very easy to use, the process is fast.”
• “It's convenient and very easy to use.”
• “I like the simplicity of it. Very clean.”
Ease of looking up/ adding a new bill
• “It can pay a person by looking up his phone
number.”
• “Setting up new payee started with asking for
phone number, to simplify process.”
• “It's so easy to look up a person if you are
paying them a bill. finds all of their
information for you.”
• “Easy to figure out how to add bills, and lots
of good instructions.”
Helpful ‘Payment Assistant’ window
• “When you click on a bill to pay, a box comes
up with information on the last 3 payments
that were made to the account.”
20. Bill Pay Satisfaction Best Practices: SunTrust
Quick and easy to make payments, service
is free
• “It's easy to use, & fast.”
• “The ease of use and it's free!”
• “The online bill pay procedure is very straight
forward and simple. And it is free.”
• “It is fast and easy to arrange the payment
online and hassle‐free.”
• “The service is very easy to use and
convenient.”
Helpful ‘Payment Assistant’ window
• “My payment history is right there so that I
can review previous payments and time when
they were made.”
• “I like being able to see what my payment
history to each biller is…”
Ease of adding new payee
• “I like how easy it is to add a bill and to make
a payment.”
• “It is easy to use and I can pay any bill or
anyone.”
24. Webster: The little bank whose customers could… do it more easliy
What do I need to know and what
can I do?
Unlike customers at most large
banks, Webster customers don’t
have to click past the secure site
landing page to see if a recent
transaction has been processed.
The site also provides an on‐page
transfer interface.
The link directly to the current
statement is an easily ignored but
important feature, with most banks
requiring a one click to pick an
account and a second click to view
its statements.
29. Banking Prospects
How do banking sites compare in technical quality?
Site performance and site availability are a good start
Keynote uses a more holistic approach to assessing site quality
Consider a common path (transaction) through the site
In this case, start at a bank home page, step through the process of
finding information on checking accounts, and begin the application
process
Knowing how your site ranks in a competitive study is important,
but knowing why you rank where you do is more important
33. Study Period
For one‐time studies, Keynote uses at least three weeks of data
• Important to measure weekdays and weekends
• Multiple weeks reduce the impact of any holidays or unusual news events
• For banking studies, Keynote does not consider weekends to be “peak
period”, but as more and more customers interact with financial
institutions online, this might need to change
• Long‐term continuous measurement is the best way to manage site
performance
35. Action Areas – Profiling Page Performance by Element
Many things can impact site
performance, including issues
with application calls, CDNs, and
third‐party elements
You cannot rely upon
implementing best practices to
solve all problems, you need to
measure
37. Action Areas – Client‐Side Processing 150 ms of client‐side
processing delay
Although banking sites are not pushing
the AJAX, Javascript, and Flash
model as much as other industries,
we are seeing it more and more
You need to understand
the impact of client‐ 200 ms of client‐
side processing time side processing
delay
This page has two periods of client‐side
processing, early in the page
download, that impact the end user
perception of the page
38. Action Areas – Third‐Party Elements
Managing the performance and reliability of third‐party calls (ad tags,
analytics tags, marketing tags, etc.) is a challenge
Delays from third‐party elements can block critical page content from loading
The delay is very damaging to end user perception of the site
sales.liveperson.net
call takes an
extraordinarily long
time
Critical page content
from the banking site
is blocked from
loading
41. Common Challenges to Overcome
1. Increasing demand for your services on mobile
2. Need for consistent QoS across devices and networks
3. Dependency on 3rd Parties to deliver mobile services
4. Lack of ability enforce SLA due to insufficient “evidence”
5. Service downtime and poor performance can affect revenue and brand
reputation
6. Quick release cycles impacting QoS
7. Need for performance trend analysis
54. Resources
To View This or Other Events On-Demand Please Visit:
http://www.netseminar.com
For more information please visit:
http://www.keynote.com/company/events/events.html