SlideShare une entreprise Scribd logo
1  sur  44
Copyright © 2010 Business Over Broadway
Validation of COMPANY Customer
Satisfaction Survey
Business growth through customer insight
Business Over Broadway
Copyright © 2010 Business Over Broadway
• Executive Summary
• Components of Customer Feedback Programs
• Uses of Customer Feedback Data
• Company Surveys
• Definitions of Reliability and Validity
• Evidence of Reliability
• Evidence of Validity
• Summary
• Conclusions
• Recommendations
Overview
Copyright © 2010 Business Over Broadway
• The company can feel confident using their customer
survey results when making business decisions
• Loyalty questions can be used to make reliable, useful
decisions to grow business through new and existing
customers
• Build customer-centricity in COMPANY by using loyalty
indices in:
– executive dashboards, strategy/governance, decision making,
incentive compensation, benchmark against industry standards
• Proposed Recommendations
– Change Customer Loyalty Measurement
– Change Technical Support Measurement
– Adopt best practices in Strategy/Governance, Business
Integration
Executive Summary
Copyright © 2010 Business Over Broadway
• Adoption of best practices in each area ensures program success1
Components of Customer Feedback
Programs
1 In Hayes, B.E. (2009). Beyond the Ultimate Question: A Systematic Approach to Improve Customer Loyalty. Quality Press.
Milwaukee, WI. Loyalty Leading companies adopt specific business practices compared to Loyalty Lagging companies
Copyright © 2010 Business Over Broadway
Employee, Partner and Customer Loyalty
Drive Business Results
Based on the book, The Service Profit Chain: How leading Companies Link Profit and Growth to Loyalty, Satisfaction and Value
(Heskett, Sasser & Schlesinger, 1997)
Employee Survey
Customer Survey
Partner Survey
Business Strategy
Business Programs
Company
PerformanceInternal
System
Employee
Satisfaction
Employee
Productivity
Perceived
Value
Customer
Satisfaction
Revenue
Profits
Employee
Loyalty
Customer
Loyalty
Partner
Advocacy
Purchasing
Retention Market
Share
Copyright © 2010 Business Over Broadway
• Relationship Satisfaction Surveys
– Focused on overall relationship with the customer
– Delivery is not time-dependent (done at company discretion)
– Includes Business Attribute questions and Loyalty-based questions
• Typically have ratings of Satisfaction
• Transactional Satisfaction Surveys
– Focused on specific customer transaction
– Delivery is time-dependent on transaction occurrence
– Includes Business Attribute questions and Loyalty-based questions
• Typically have ratings of Satisfaction
• Targeted Surveys
– Focused on drilling into specific areas for deeper insight
– Delivery is on an as-needed basis
– Compliment the Relationship and Transactional surveys
• Three survey types can be used in any given Account
Company Surveys
Copyright © 2010 Business Over Broadway
• Customer surveys should result in reliable and valid
scores of customers’ attitudes
– Good measurement processes lead to good business decisions
• Reliability reflects the degree to which scores are
free from random error
– Reliability deals with precision/consistency
– Goal is to have a measurement system that
delivers reliable results
– Reliability varies from 0 (no reliability) to 1
(perfect reliability)
• Validity reflects the degree to which
scores are measuring what they
are designed to measure
– Validity deals with the meaning of the scores from the measurement system
– Goal is to have a measurement system that delivers valid results
Reliability and Validity
ValidNot Valid
ReliableNotReliable
Copyright © 2010 Business Over Broadway
• Four general forms of reliability
1. Inter-rater reliability1
– Degree of agreement between two or more
raters who are rating same attribute
2. Test-retest reliability1
– Degree of agreement between same
measure/rater over two time periods
3. Parallel-forms reliability
– Degree of agreement between two
measures (A vs. B) that measure same thing
4. Internal consistency reliability1
– Degree of consistency of ratings across
items within a measure
Reliability
=
Rater 1 Rater 2
=
Rater n Rater n
Attribute
1 Determination of reliability depends on number of questions in survey and types of surveys. Typically,
parallel-forms reliability cannot be studied as only one measure of customer satisfaction is available.
Time 1 Time 2
=
Rater n
Survey
Vers. A
Attribute
2
Attribute
m=
Survey
Vers. B=
Attribute
1
Rater n
Copyright © 2010 Business Over Broadway
• Three approaches at establishing validity
1. Content-related approach
– Examining the questions of the instrument to determine if they are a representative
sample of the universe of all possible questions
2. Criterion-related approach
– Correlating scores from the instrument with some other criteria
• Based on some theory of the customer satisfaction construct
• Customer satisfaction scores should be correlated to other criteria (e.g., other
measures of Account quality)
3. Construct-related approach
– Developing a nomological (lawful) network for the customer satisfaction scores
– Framework/Theory/Model for understanding and using customer satisfaction scores
should be supported by data
Validity
Copyright © 2010 Business Over Broadway
Evidence of Reliability
Copyright © 2010 Business Over Broadway
• Internal consistency determines extent to which
Contacts give similar ratings to items within a scale
– Indexed by three indices
1. Item-total correlations - correlation between rating of single item
to aggregate of remaining items in the survey
2. Split-half correlation – correlation between two halves of the
survey
3. Cronbach’s alpha – formula to calculate internal consistency
• Addresses the quality of the items in a scale
– If reliability is high, ratings do not tend to vary across items
– Expect some degree of internal consistency
• Do not expect perfect reliability – Perfect reliability would indicate
no variability in ratings and no need to ask multiple questions
Internal Consistency Reliability
Copyright © 2010 Business Over Broadway
Surveys
Contact 1
Contact 2
Contact n
Business
Attribute 1
Business
Attribute 2
Reliability Analysis
Internal Consistency Reliability
• Item-Total Correlation is the correlation of
specific attribute with aggregate of
remaining attributes
• Split-half reliability is the correlation
between average of the two halves of the
survey
• Cronbach’s alpha () estimate used to
determine internal consistency of ratings
(satisfaction) within a survey
• Calculated for any type of survey
that contains multiple questions
• Reliability varies from 0 (no internal
consistency) to 1 (perfect internal
consistency)
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
.
.
.
=
Rater n
Attribute
2
Attribute
m=Attribute
1
Data Model for Internal Consistency
Reliability Analysis
Copyright © 2010 Business Over Broadway
Index (Measure) Definition Survey Questions
Customer Loyalty Index (4
questions)
Extent to which customers feel positively
toward COMPANY
Overall Satisfaction, Quality, Value, Recommend
Perceived Value of External
Information1 (7 questions)
Extent to which customers value external
sources of information regarding security
products
Security vendor websites, Whitepapers, Industry
analysts, Blogs / online communities, Security
publications, Peers, Webcasts / podcasts
Perceived Importance of
Doc/Mgmt/Reporting (3 questions)
Stated importance of Documentation,
Management, Reporting
Reporting, Management / Administration,
Documentation
Perceived Importance of Ease of
Use (3 questions)
Stated importance of user experience Ease of installation, End user experience, Ease of
upgrade
Quality of Security Product (8
questions)
Rating of quality of security product Security effectiveness, Reporting, Management /
Administration,
Documentation, Ease of installation, End user
experience, Ease of upgrade, Price
Quality of Technical Support (3
questions)
Rating of quality of technical support Technical support, Telephone, Email
Indices (Summated Scales)*
1 Items only appear in 2009 survey.
*Indices were created based on factor analyses of the survey questions; factor-analytic results show that specific survey items can be combined
together into one index. Survey questions that loaded highly on the same factor were combined into a single scale by averaging the responses
across items.
Factor analyses of survey items exposed several indices, each
measuring a general content area (e.g. loyalty, tech support).
Copyright © 2010 Business Over Broadway
Internal Consistency Reliability by
Decision Influence
Decision-Making Role
INDEX
Entire
Sample
Final
Decision Influencer Purchasing
2009 Indices
1 Customer Loyalty Index* .89 .89 .89 .88
2 Perceived Value of External
Information
.85 .85 .83 .87
3 Perceived Importance of
Doc/Mgmt/Reporting
.80 .78 .79 .78
4 Perceived Importance of Ease of Use .79 .76 .80 .80
5 Quality of Security Products .91 .92 .91 .92
6 Quality of Tech Support .91 .91 .90 .91
2010 Indices
1 Customer Loyalty Index .82 .81 .83 .81
2 Perceived Importance of
Doc/Mgmt/Reporting
.70 .69 .66 .78
3 Perceived Importance of Ease of Use .66 .68 .64 .58
4 Perceived Quality of Security Products .89 .89 .88 .91
5 Perceived Quality of Tech Support .92 .93 .92 .95
Internal Consistency Reliability1
Major Points
• Factor analyses of survey
items exposed several indices,
each measuring a general
content area (e.g. loyalty, tech
support).
• Definition of index is based on
the aggregated items
• Internal consistency
reliabilities are acceptable and
similar across all decision-
making categories.
• Each index would be useful for
executive dashboards since
each provides reliable, useful
information.
1 No significance testing possible for Cronbach’s alpha.
*See previous slide for description of each index and the survey items that represent each index.
Copyright © 2010 Business Over Broadway
• Inter-contact reliability determines extent to which
Contacts within an Account give similar customer
satisfaction survey ratings
– Indexed by a correlation between different Contacts within each Account across all
Accounts
• Inter-contact reliability addresses the consistency
of the implementation of the customer satisfaction
survey system across Contacts
– Expect some degree of inter-contact reliability because Contacts are rating the same
Account
• Do not want perfect reliability – Perfect reliability would indicate no utility of
surveying different management contacts within an organization
• Calculate inter-contact reliability for each survey
type across all business attributes
Inter-Rater Reliability
Copyright © 2010 Business Over Broadway
Data Model for Inter-Rater Reliability
Analysis
Type of Contact
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Inter-Contact Reliability
• Correlation of satisfaction ratings between
different Contacts within Accounts
• Calculated for each type of customer survey
• Reliability could vary from 0 (no agreement
between Contacts) to 1.0 (complete
agreement between Contacts)
.
.
.
...
=
Rater 2
Attribute
Rater 1
Copyright © 2010 Business Over Broadway
Inter-Contact Reliability (Decision-
makers vs. Influencers)
INDICES1
Decision
maker/
Influencer
1 Customer Loyalty Index* .19
2 Security Effectiveness -.12
3 Reporting .05
4 Management / Administration .15
5 Documentation -.04
6 Ease of Installation .16
7 End User Experience .02
8 Ease of Upgrade .22
9 Technical Support .05
10 Price .22
Major Points
• There is low agreement
between Contacts from different
decision-making standpoint.
• Contacts from different
decision-making standpoints
(e.g., Decision maker vs.
Influencer) show low agreement
in their ratings of COMPANY
• The degree of disagreement
across different survey
respondents within an Account
could reflect:
•Poor communication
within an Account
•Different expectations of
different functions
•Different usage of
COMPANY products
1 Survey data are from 2010 Survey; N = 47 to 50. No statistically significant correlations.
*Customer Loyalty Index is composite of following four questions: Overall Satisfaction, Quality, Value
and Recommend. Functional job function includes Finance,
Not enough sample to examine “Purchasing” role.
Copyright © 2010 Business Over Broadway
• Test-retest reliability determines extent to which
Contacts give similar ratings over time
– Indexed by a correlation between ratings across two time periods
– Higher correlation indicates that Contacts provide similar ratings over time
• Addresses the consistency of ratings over time
– If reliability is high, relative ordering of ratings do not tend to vary over time across
different Accounts
– Expect good degree of test-retest reliability because Contacts are rating the same
Account over time
• Contacts that are satisfied/dissatisfied at time 1 should also be
satisfied/dissatisfied at time 2
• Contacts with high/low value ratings at time 1 should also have high/low value
ratings at time 2
• Calculate test-retest reliability for satisfaction and
value ratings
Test-Retest Reliability
Copyright © 2010 Business Over Broadway
Data Model for Test-Retest Reliability
Analysis
Time 1
Survey
Time 2
Survey
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Test-Retest Reliability
• Correlation of satisfaction ratings between
same Contacts across two time periods
• Correlation of satisfaction ratings (averaged
over Contacts) between same Accounts
across two time periods
• Calculated only for Surveys that are
repeated
• Reliability varies from 0 (no
agreement between time 1 and time
2) to 1.0 (complete agreement
between time 1 and time 2)
.
.
.
.
.
.
.. .
=
Contact n Contact n
Time 1 Time 2
Copyright © 2010 Business Over Broadway
Test-Retest Reliability
INDICES/MEASURES
1 Customer Loyalty Index .69
2 Overall Satisfaction .32
3 Quality .58
4 Value .52
5 Recommend .70
6 Security Effectiveness .43
7 Reporting .53
8 Management / Administration .54
9 Documentation .44
10 Ease of Installation .50
11 End User Experience .49
12 Ease of Upgrade .45
13 Technical Support .51
14 Price .54
Test-Retest
Reliability1
1 Survey data are from 2009 (Time 1) and 2010 (Time 2). N = 472 to 478. All correlations significant at p < .01 level. CLI is a composite score
consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey
questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
=
Contact n Contact n
Time 1 Time 2
Major Points
• Test-retest reliability is high
• Customers tend to respond
in a similar way over two
time periods.
• Those who are satisfied
(dissatisfied) at time 1 tend
to be satisfied (dissatisfied)
at time 2.
Copyright © 2010 Business Over Broadway
Evidence of Validity
Copyright © 2010 Business Over Broadway
• Content validity is established when items in the
survey are a representative sample of some defined
universe of items
• Universe of survey items represents all items that
define the important elements of a company’s
performance
• Does the customer survey comprehensively assess
customer requirements?
• Must show “link” between set of items (survey) and
the universe of items
– Link determined by Subject Matter Experts (SMEs)
Content-related Approach
Copyright © 2010 Business Over Broadway
• Disloyal customers provide
comments regarding
improvements1
• Comments/issues range
from product to service issues
• All issues/reasons
covered by content of
customer survey
• Issues aligned with
results of driver analysis
Content-related Approach
1 Customer comments from those who rated Recommend question low.
Copyright © 2010 Business Over Broadway
• Criterion-related validity relies on statistical
analysis rather than judgment
• Established when scores on the survey are related
to some other variable (criterion) already known
to measure similar attributes
• Key is developing/selecting right criteria to use in
validation study
– Measures of Account Quality – by Account Team
– Objective Assessment Scores
Criterion-related Approach
Copyright © 2010 Business Over Broadway
• Construct-related validity addresses what is being
measured by examining relationships between
constructs (e.g., customer satisfaction, customer
loyalty, other measures of Account quality)
• Concerned with theory and the proposed relationships
between constructs based on that theory
• Understanding the theoretical relationship of
constructs to other constructs
– To what should customer satisfaction be (convergent validity)
and not be related (discriminant validity)?
• Need to examine patterns of relationships using the
survey and other instruments
Construct-related Approach
Copyright © 2010 Business Over Broadway
2009 Mean SD N 1 2 3 4 5 6
1. Customer Loyalty Index 4.84 0.94 2941 0.89
2. Quality of Security Product 4.59 0.86 2917 0.77 0.91
3. Quality of Technical Support 4.81 1.09 2880 0.68 0.69 0.91
4. Perceived Value of External Information 4.34 0.97 2787 0.31 0.37 0.29 0.85
5. Perceived Importance of Doc/Mgmt/Reporting 4.77 0.87 2914 0.29 0.43 0.31 0.34 0.80
6. Perceived Importance of Ease of Use 5.05 0.87 2919 0.33 0.48 0.33 0.28 0.54 0.79
Correlations Among Indices
 Indices are correlated logically with each other
 Higher ratings on key indices (quality, value) correspond to higher
customer loyalty ratings compared to other indices (perceived
importance)
2010 Mean SD N 1 2 3 4 5
1. Customer Loyalty Index 5.54 1.27 3449 0.82
2. Quality of Security Product 4.72 0.75 3417 0.78 0.89
3. Quality of Technical Support 4.95 1.04 3387 0.67 0.65 0.92
4. Perceived Importance of Doc/Mgmt/Reporting 5.01 0.66 3434 0.11 0.21 0.12 0.70
5. Perceived Importance of Ease of Use 5.17 0.69 3437 0.10 0.23 0.12 0.41 0.66
1 Cronbach’s alpha (internal consistency estimates) are located in the diagonal.
Copyright © 2010 Business Over Broadway
• Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2009)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 4.84 2941
Understands Needs 4.57 2926 0.73
Addresses Needs 4.52 2924 0.78
Understands Better Than Competitors 4.30 2884 0.69
Security Effectiveness 5.18 2911 0.70
Reporting 4.40 2875 0.54
Management / Administration 4.52 2866 0.63
Documentation 4.23 2871 0.55
Ease of Installation 4.61 2884 0.58
End User Experience 4.66 2880 0.66
Ease of Upgrade 4.77 2856 0.61
Technical Support 4.80 2860 0.67
Price 4.35 2877 0.55
Compare COMPANY to Alternatives 4.06 2645 0.69
Tech Support via Telephone 4.96 1815 0.61
Tech Support via Email 4.79 1792 0.62
Tech Support via Web Knowledgebase 4.39 2102 0.49
1 No significance testing possible for Cronbach’s alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
Copyright © 2010 Business Over Broadway
• Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2010)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 5.54 3449
Understands Needs 4.32 3414 0.44
Addresses Needs 4.31 3408 0.48
Understands Better Than Competitors 4.37 3337 0.52
Security Effectiveness 5.21 3407 0.65
Reporting 4.58 3371 0.54
Management / Administration 4.67 3387 0.67
Documentation 4.48 3372 0.54
Ease of Installation 4.72 3382 0.56
End User Experience 4.73 3371 0.66
Ease of Upgrade 4.78 3356 0.59
Technical Support 4.91 3360 0.66
Price 4.62 3348 0.47
Compare COMPANY to Alternatives 4.01 3020 0.77
Tech Support via Telephone 5.05 2408 0.63
Tech Support via Email 4.91 2337 0.64
Tech Support via Web Knowledgebase 4.47 2585 0.55
1 No significance testing possible for Cronbach’s alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
Copyright © 2010 Business Over Broadway
• Use previous validity approaches (content and
criterion) to support a theoretical model of customer
satisfaction
• Various theoretical models
state that customer loyalty
is impacted by satisfaction with
product and satisfaction with
service
• Need to examine the pattern of
relationships among these
constructs
• Pattern of correlations should support this model
Construct-related Approach
Service
Satisfaction
Product
Satisfaction
Customer
Loyalty
Basic Theory of
Customer Satisfaction
Copyright © 2010 Business Over Broadway
• Test the model by examining
correlations among three
variables:
– Product Quality (Quality of Security Eff.)
– Tech Support Quality
– Customer Loyalty (CLI)
• Path analysis1 applied to
observed correlations
– Isolates degree of impact a given
variable has on other variables
• Analysis supports theory that
both product quality and tech
tech support quality impact
customer loyalty independently
Construct-related Approach
CORRELATIONS
CLI
Tech
Qual
Product Quality .70 .53
Customer Loyalty (CLI) - .68
1 For detailed description of path analysis methodology, please see: Pedhazur, E. J. (1982). Multiple
regression in behavioral research: Explanation and prediction (2nd Ed.) Holt, Rinehart and Winston.
New York, NY.
Tech Support
Product
Effectiveness
Customer
Loyalty
Path Analysis
.11
.54
.51
Copyright © 2010 Business Over Broadway
• Support Level and Customer Loyalty
• Three levels of Support (Standard, Premium, Platinum)
• Lower customer loyalty reported by customers with
Platinum support
– Platinum support customers could have higher expectations than
Standard support customers.
Support Level
Copyright © 2010 Business Over Broadway
• Difference found across regions1
– APAC has highest levels of customer loyalty and quality ratings
– Southern Europe as lowest levels of customer loyalty and quality
ratings
Segmentation By Region
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
Segmentation By Industry
Difference found across industries1
HED has the lowest levels of customer loyalty
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
Segmentation By License Tier
Difference found across license tiers1
License tiers 3 and 4 have highest levels of customer loyalty and quality ratings
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
• Internal consistency reliability
– Contacts tend to give similar ratings across the different items of the same
scale
• Non-perfect reliability indicates that different items still provide different
information about customers’ attitudes
• Reasons for extremely high reliability
– High degree of redundancy of items
– Rater fatigue due to survey length so respondents rate all questions similarly
• Inter-contact reliability
– Contacts within Accounts show little agreement in ratings
– Different types of Contacts provide different vantage point when rating the
company
• Poor communication within Account across levels and/job functions
• Different Contacts have differing expectations
• Contacts are treated differently by Account team depending on their
level/function
Summary of Reliability of Survey Ratings
Copyright © 2010 Business Over Broadway
• Content-related approach
– Questions in customer survey are representative sample
of universe of questions
• Items comprehensively cover customer requirements
throughout all phases of the customer lifecycle
• Dissatisfied customers’ open-ended comments are addressed
by questions in the survey
• Criterion-related approach
– No external criteria are available at this time
• Construct-related approach
– Pattern of relationships of customer satisfaction data
supports theory of customer satisfaction
• Customer satisfaction is related to customer loyalty
• Product quality impacts loyalty through perceived value
Summary of Validity
Copyright © 2010 Business Over Broadway
• Self-assessment survey identified strengths and areas
needing improvement1
• Overall, low to moderate adoption rates of best practices
across all components of customer feedback programs
– Customer feedback not as important as financial metrics,
customer feedback not integrated with CRM system, problem
resolution not integrated into CRM system
• Strengths seen in specific areas of COMPANY’S customer
feedback program:
– Top executive is champion of the program, multiple methods of
data collection, Web surveys, various measures of customer
loyalty, customer research presented externally, existing
customer data used for segmentation
Customer Feedback Programs Best
Practices Survey
1 Survey identifies extent to which company adopts best practices in their customer feedback program. EMPLOYEE JANE DOE
completed the online survey (http://www.businessoverbroadway.com/cfpbpi.htm) for COMPANY on June 10, 2010. See survey for
questions.
Copyright © 2010 Business Over Broadway
• The company’s survey process produces reliable,
valid indices of customers’ attitudes
• Evidence of reliability
– All indices are highly reliable – items within indices
can be combined to create summary scores
• Evidence of validity
– Survey questions comprehensively assess important
areas to the customers
– Each index measures unique construct apart from the
other indices
– Relationships among indices supports customer
satisfaction models
Conclusions
Copyright © 2010 Business Over Broadway
• The company can feel confident using customer
survey results when making business decisions
– Identifying drivers of customer loyalty
– Using Indexes to manage customer relationships
• Understand why different customers within the
same Account hold different attitudes toward
COMPANY
– Do they communicate with each other (through
Company-sanctioned User groups or individually)? They
could get together and talk about the benefits of
COMPANY. Could COMPANY assist in this process?
Conclusions
Copyright © 2010 Business Over Broadway
• Changes in Customer Loyalty Measurement
– Put loyalty question at start of survey (to assess top of mind
responses) with common rating scale (recommend 0 to 10)
scale)
– Add questions to measure other components of customer
loyalty
• Purchasing Loyalty: Example: How likely are you to purchase
additional/different types of products from COMPANY?
• Retention Loyalty: Example: How likely are you to switch to a different
security provider within the next 12 months?
• Benefits
– Make decisions to grow business through new and existing
customers
– Loyalty indices can be used to build customer-centricity
• Executive dashboards, guide strategy/governance, resource
allocation, incentive compensation, benchmark against industry
standards
Recommendations on Method
Copyright © 2010 Business Over Broadway
• Changes in Technical Support Measurement
– Move all technical support questions into one section
• Telephone, Email, and technical support
– Consider adding other technical support questions to
broaden technical support coverage
• Knowledge, professionalism, care
• Benefits
– Easier for customers to complete survey
– Provide more specific feedback to technical support
personnel
• Consider ways to increase response rates
– Email invitation content
Recommendations on Method
Copyright © 2010 Business Over Broadway
• Integrate customer feedback into company’s
strategy/governance1
– Company’s strategic vision, mission and goals; executives’
objectives and incentive compensation, front-line employees’
objectives and incentive compensation
• Integrate customer feedback into daily business
processes1
– CRM system; company/executive dashboards; communicate
results regularly to the entire company
• Conduct in-depth research using customer survey results1
– Linkage analysis between customer feedback and operational
metrics, financial metrics, employee metrics.
Recommendations on Strategy,
Integration and Research
1 Based on Customer Feedback Programs Self-Assessment Survey Results.
Copyright © 2010 Business Over Broadway
• Consider other “financial” metrics to track and link to
customer survey
– Consult with CFO to consider other financial metrics that are
tracked and consider using those to identify which aspects of
customer satisfaction/loyalty relate to financial growth of
COMPANY
• e.g. Renewals
– Track financial metrics over time in conjunction with the
customer survey to conduct longitudinal studies
• Study impact of attitudes at time 1 on financial metrics at time 2 and
beyond
• Linkage study between COMPANY’S transactional survey
and relationship survey
– To what extent do transactions (e.g. in call center) impact long-
term customer loyalty
Recommendations for Future Research
Copyright © 2010 Business Over Broadway
For More Information
Business growth through customer insight
Business Over Broadway
Bob E. Hayes, Ph.D.
Email: bob@businessoverbroadway.com
Web: www.businessoverbroadway.com
Blog: www.businessoverbroadway.blogspot.com
Twitter: www.twitter.com/bobehayes

Contenu connexe

Tendances

12-designing-channel-systems
 12-designing-channel-systems 12-designing-channel-systems
12-designing-channel-systemsManilyn Angel
 
Brand Tracking - Taking The Pulse On Your Brand
Brand Tracking - Taking The Pulse On Your BrandBrand Tracking - Taking The Pulse On Your Brand
Brand Tracking - Taking The Pulse On Your BrandMichaela Mora
 
Clow imc8 inppt_01
Clow imc8 inppt_01Clow imc8 inppt_01
Clow imc8 inppt_01Tuf02771
 
The t Test for Two Independent Samples
The t Test for Two Independent SamplesThe t Test for Two Independent Samples
The t Test for Two Independent Samplesjasondroesch
 
Market Based Management, business presentations
Market Based Management, business presentationsMarket Based Management, business presentations
Market Based Management, business presentationshttp://www.drawpack.com
 
Chapter 9 Creating Brand Equity
Chapter 9 Creating Brand EquityChapter 9 Creating Brand Equity
Chapter 9 Creating Brand Equitydona_sia
 
Product in theory and practice
Product in theory and practiceProduct in theory and practice
Product in theory and practiceRavi Chandegara
 
Value chain Analysis including Value Creation
Value chain Analysis including Value CreationValue chain Analysis including Value Creation
Value chain Analysis including Value CreationCMA Himanshu Bahl
 
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)PERSUASION (Psych 201 - Chapter 8 - Spring 2014)
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)Melanie Tannenbaum
 
Integrated Marketing communications
Integrated Marketing communicationsIntegrated Marketing communications
Integrated Marketing communicationsTom Chapman
 
Marketing research ch 5_malhotra
Marketing research ch 5_malhotraMarketing research ch 5_malhotra
Marketing research ch 5_malhotraJamil Ahmed AKASH
 
Chapter 15 Marketing Research Malhotra
Chapter 15 Marketing Research MalhotraChapter 15 Marketing Research Malhotra
Chapter 15 Marketing Research MalhotraAADITYA TANTIA
 
Frequency Distributions
Frequency DistributionsFrequency Distributions
Frequency Distributionsjasondroesch
 
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...TanveerHossainRayvee
 
Chapter 2 strategic marketing
Chapter 2   strategic marketingChapter 2   strategic marketing
Chapter 2 strategic marketingEyya Ahmed
 
"A basic guide to SPSS"
"A basic guide to SPSS""A basic guide to SPSS"
"A basic guide to SPSS"Bashir7576
 

Tendances (20)

12-designing-channel-systems
 12-designing-channel-systems 12-designing-channel-systems
12-designing-channel-systems
 
Brand Tracking - Taking The Pulse On Your Brand
Brand Tracking - Taking The Pulse On Your BrandBrand Tracking - Taking The Pulse On Your Brand
Brand Tracking - Taking The Pulse On Your Brand
 
Wal Mart Green Marketing By Monika
Wal Mart Green Marketing By MonikaWal Mart Green Marketing By Monika
Wal Mart Green Marketing By Monika
 
Clow imc8 inppt_01
Clow imc8 inppt_01Clow imc8 inppt_01
Clow imc8 inppt_01
 
Brand extension
Brand extensionBrand extension
Brand extension
 
The t Test for Two Independent Samples
The t Test for Two Independent SamplesThe t Test for Two Independent Samples
The t Test for Two Independent Samples
 
Market Based Management, business presentations
Market Based Management, business presentationsMarket Based Management, business presentations
Market Based Management, business presentations
 
Chapter 9 Creating Brand Equity
Chapter 9 Creating Brand EquityChapter 9 Creating Brand Equity
Chapter 9 Creating Brand Equity
 
Product in theory and practice
Product in theory and practiceProduct in theory and practice
Product in theory and practice
 
Value chain Analysis including Value Creation
Value chain Analysis including Value CreationValue chain Analysis including Value Creation
Value chain Analysis including Value Creation
 
Normality
NormalityNormality
Normality
 
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)PERSUASION (Psych 201 - Chapter 8 - Spring 2014)
PERSUASION (Psych 201 - Chapter 8 - Spring 2014)
 
Integrated Marketing communications
Integrated Marketing communicationsIntegrated Marketing communications
Integrated Marketing communications
 
Marketing research ch 5_malhotra
Marketing research ch 5_malhotraMarketing research ch 5_malhotra
Marketing research ch 5_malhotra
 
Chapter 15 Marketing Research Malhotra
Chapter 15 Marketing Research MalhotraChapter 15 Marketing Research Malhotra
Chapter 15 Marketing Research Malhotra
 
Frequency Distributions
Frequency DistributionsFrequency Distributions
Frequency Distributions
 
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...
Secondary Brand Association - Leveraging Secondary Brand Associations to Buil...
 
Product management
Product managementProduct management
Product management
 
Chapter 2 strategic marketing
Chapter 2   strategic marketingChapter 2   strategic marketing
Chapter 2 strategic marketing
 
"A basic guide to SPSS"
"A basic guide to SPSS""A basic guide to SPSS"
"A basic guide to SPSS"
 

En vedette

En vedette (8)

Value proposition &amp; usp
Value proposition &amp; uspValue proposition &amp; usp
Value proposition &amp; usp
 
Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011
 
updated CV Yacoub
updated CV Yacoubupdated CV Yacoub
updated CV Yacoub
 
Customer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTDCustomer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTD
 
Textbook Campaign
Textbook CampaignTextbook Campaign
Textbook Campaign
 
Verizon Presentation
Verizon PresentationVerizon Presentation
Verizon Presentation
 
Columbus Metropolitan Library Survey
Columbus Metropolitan Library SurveyColumbus Metropolitan Library Survey
Columbus Metropolitan Library Survey
 
Usp Marketing
Usp MarketingUsp Marketing
Usp Marketing
 

Similaire à Validation of Customer Survey

Developing a Customer Centric Research Program
Developing a Customer Centric Research ProgramDeveloping a Customer Centric Research Program
Developing a Customer Centric Research ProgramBusiness Over Broadway
 
Customer Service Excellence
Customer Service ExcellenceCustomer Service Excellence
Customer Service ExcellenceKathryn Larkin
 
A Robust approach to analysis of Customer Satisfaction Survey feedback
A  Robust approach to analysis of Customer Satisfaction Survey feedbackA  Robust approach to analysis of Customer Satisfaction Survey feedback
A Robust approach to analysis of Customer Satisfaction Survey feedbackSantanoo Medhi
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for StartupsVishal Kumar
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for StartupsTCELab LLC
 
Customer Satisfaction in Shared Services
Customer Satisfaction in Shared ServicesCustomer Satisfaction in Shared Services
Customer Satisfaction in Shared ServicesScottMadden, Inc.
 
Quality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin ShekapureQuality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin ShekapureNitin Shekapure
 
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01TCELab LLC
 
A Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation ProcessA Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation ProcessTracy Drey
 
Improving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingImproving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingNuwan Ireshinie
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellencekushshah
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementTCELab LLC
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementVishal Kumar
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...TCELab LLC
 

Similaire à Validation of Customer Survey (20)

Developing a Customer Centric Research Program
Developing a Customer Centric Research ProgramDeveloping a Customer Centric Research Program
Developing a Customer Centric Research Program
 
Customer Service Excellence
Customer Service ExcellenceCustomer Service Excellence
Customer Service Excellence
 
A Robust approach to analysis of Customer Satisfaction Survey feedback
A  Robust approach to analysis of Customer Satisfaction Survey feedbackA  Robust approach to analysis of Customer Satisfaction Survey feedback
A Robust approach to analysis of Customer Satisfaction Survey feedback
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Big Data and Customer Experience
Big Data and Customer ExperienceBig Data and Customer Experience
Big Data and Customer Experience
 
Customer Satisfaction in Shared Services
Customer Satisfaction in Shared ServicesCustomer Satisfaction in Shared Services
Customer Satisfaction in Shared Services
 
Quality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin ShekapureQuality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin Shekapure
 
dasat
dasatdasat
dasat
 
Building a Customer Feedback Program
Building a Customer Feedback ProgramBuilding a Customer Feedback Program
Building a Customer Feedback Program
 
Quality control
Quality controlQuality control
Quality control
 
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
 
A Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation ProcessA Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation Process
 
Improving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingImproving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service Marketing
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellence
 
Service quality
Service qualityService quality
Service quality
 
Customer Experience Management
Customer Experience ManagementCustomer Experience Management
Customer Experience Management
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
 

Plus de Business Over Broadway

What MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysWhat MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysBusiness Over Broadway
 
In a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment IndexIn a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment IndexBusiness Over Broadway
 
Big Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience ProfessionalsBig Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience ProfessionalsBusiness Over Broadway
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Business Over Broadway
 
Customer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample ReportCustomer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample ReportBusiness Over Broadway
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for StartupsBusiness Over Broadway
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBusiness Over Broadway
 
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship SurveyAsking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship SurveyBusiness Over Broadway
 
Linkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback ProgramsLinkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback ProgramsBusiness Over Broadway
 
Competitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer LoyaltyCompetitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer LoyaltyBusiness Over Broadway
 
Managing Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachManaging Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachBusiness Over Broadway
 

Plus de Business Over Broadway (14)

What MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysWhat MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and Surveys
 
Investigating data scientists
Investigating data scientistsInvestigating data scientists
Investigating data scientists
 
In a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment IndexIn a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment Index
 
The Hidden Bias in Customer Metrics
The Hidden Bias in Customer MetricsThe Hidden Bias in Customer Metrics
The Hidden Bias in Customer Metrics
 
Big Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience ProfessionalsBig Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience Professionals
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
 
Customer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample ReportCustomer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample Report
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship SurveyAsking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
 
Linkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback ProgramsLinkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback Programs
 
Competitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer LoyaltyCompetitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer Loyalty
 
Managing Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachManaging Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro Approach
 
RAPID Loyalty Measurement
RAPID Loyalty MeasurementRAPID Loyalty Measurement
RAPID Loyalty Measurement
 

Dernier

0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdfRenandantas16
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageMatteo Carbone
 
Sales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessSales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessAggregage
 
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature Set
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature SetCreating Low-Code Loan Applications using the Trisotech Mortgage Feature Set
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature SetDenis Gagné
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Dipal Arora
 
Understanding the Pakistan Budgeting Process: Basics and Key Insights
Understanding the Pakistan Budgeting Process: Basics and Key InsightsUnderstanding the Pakistan Budgeting Process: Basics and Key Insights
Understanding the Pakistan Budgeting Process: Basics and Key Insightsseri bangash
 
Regression analysis: Simple Linear Regression Multiple Linear Regression
Regression analysis:  Simple Linear Regression Multiple Linear RegressionRegression analysis:  Simple Linear Regression Multiple Linear Regression
Regression analysis: Simple Linear Regression Multiple Linear RegressionRavindra Nath Shukla
 
Best Basmati Rice Manufacturers in India
Best Basmati Rice Manufacturers in IndiaBest Basmati Rice Manufacturers in India
Best Basmati Rice Manufacturers in IndiaShree Krishna Exports
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMANIlamathiKannappan
 
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130  Available With RoomVIP Kolkata Call Girl Howrah 👉 8250192130  Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Roomdivyansh0kumar0
 
Boost the utilization of your HCL environment by reevaluating use cases and f...
Boost the utilization of your HCL environment by reevaluating use cases and f...Boost the utilization of your HCL environment by reevaluating use cases and f...
Boost the utilization of your HCL environment by reevaluating use cases and f...Roland Driesen
 
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...Lviv Startup Club
 
Unlocking the Secrets of Affiliate Marketing.pdf
Unlocking the Secrets of Affiliate Marketing.pdfUnlocking the Secrets of Affiliate Marketing.pdf
Unlocking the Secrets of Affiliate Marketing.pdfOnline Income Engine
 
7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...Paul Menig
 
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurVIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurSuhani Kapoor
 
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
Keppel Ltd. 1Q 2024 Business Update  Presentation SlidesKeppel Ltd. 1Q 2024 Business Update  Presentation Slides
Keppel Ltd. 1Q 2024 Business Update Presentation SlidesKeppelCorporation
 
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...lizamodels9
 
Monte Carlo simulation : Simulation using MCSM
Monte Carlo simulation : Simulation using MCSMMonte Carlo simulation : Simulation using MCSM
Monte Carlo simulation : Simulation using MCSMRavindra Nath Shukla
 
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
Tech Startup Growth Hacking 101  - Basics on Growth MarketingTech Startup Growth Hacking 101  - Basics on Growth Marketing
Tech Startup Growth Hacking 101 - Basics on Growth MarketingShawn Pang
 
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsCash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsApsara Of India
 

Dernier (20)

0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usage
 
Sales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessSales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for Success
 
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature Set
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature SetCreating Low-Code Loan Applications using the Trisotech Mortgage Feature Set
Creating Low-Code Loan Applications using the Trisotech Mortgage Feature Set
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
 
Understanding the Pakistan Budgeting Process: Basics and Key Insights
Understanding the Pakistan Budgeting Process: Basics and Key InsightsUnderstanding the Pakistan Budgeting Process: Basics and Key Insights
Understanding the Pakistan Budgeting Process: Basics and Key Insights
 
Regression analysis: Simple Linear Regression Multiple Linear Regression
Regression analysis:  Simple Linear Regression Multiple Linear RegressionRegression analysis:  Simple Linear Regression Multiple Linear Regression
Regression analysis: Simple Linear Regression Multiple Linear Regression
 
Best Basmati Rice Manufacturers in India
Best Basmati Rice Manufacturers in IndiaBest Basmati Rice Manufacturers in India
Best Basmati Rice Manufacturers in India
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMAN
 
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130  Available With RoomVIP Kolkata Call Girl Howrah 👉 8250192130  Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
 
Boost the utilization of your HCL environment by reevaluating use cases and f...
Boost the utilization of your HCL environment by reevaluating use cases and f...Boost the utilization of your HCL environment by reevaluating use cases and f...
Boost the utilization of your HCL environment by reevaluating use cases and f...
 
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...
Yaroslav Rozhankivskyy: Три складові і три передумови максимальної продуктивн...
 
Unlocking the Secrets of Affiliate Marketing.pdf
Unlocking the Secrets of Affiliate Marketing.pdfUnlocking the Secrets of Affiliate Marketing.pdf
Unlocking the Secrets of Affiliate Marketing.pdf
 
7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...
 
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurVIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
 
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
Keppel Ltd. 1Q 2024 Business Update  Presentation SlidesKeppel Ltd. 1Q 2024 Business Update  Presentation Slides
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
 
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
 
Monte Carlo simulation : Simulation using MCSM
Monte Carlo simulation : Simulation using MCSMMonte Carlo simulation : Simulation using MCSM
Monte Carlo simulation : Simulation using MCSM
 
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
Tech Startup Growth Hacking 101  - Basics on Growth MarketingTech Startup Growth Hacking 101  - Basics on Growth Marketing
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
 
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsCash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
 

Validation of Customer Survey

  • 1. Copyright © 2010 Business Over Broadway Validation of COMPANY Customer Satisfaction Survey Business growth through customer insight Business Over Broadway
  • 2. Copyright © 2010 Business Over Broadway • Executive Summary • Components of Customer Feedback Programs • Uses of Customer Feedback Data • Company Surveys • Definitions of Reliability and Validity • Evidence of Reliability • Evidence of Validity • Summary • Conclusions • Recommendations Overview
  • 3. Copyright © 2010 Business Over Broadway • The company can feel confident using their customer survey results when making business decisions • Loyalty questions can be used to make reliable, useful decisions to grow business through new and existing customers • Build customer-centricity in COMPANY by using loyalty indices in: – executive dashboards, strategy/governance, decision making, incentive compensation, benchmark against industry standards • Proposed Recommendations – Change Customer Loyalty Measurement – Change Technical Support Measurement – Adopt best practices in Strategy/Governance, Business Integration Executive Summary
  • 4. Copyright © 2010 Business Over Broadway • Adoption of best practices in each area ensures program success1 Components of Customer Feedback Programs 1 In Hayes, B.E. (2009). Beyond the Ultimate Question: A Systematic Approach to Improve Customer Loyalty. Quality Press. Milwaukee, WI. Loyalty Leading companies adopt specific business practices compared to Loyalty Lagging companies
  • 5. Copyright © 2010 Business Over Broadway Employee, Partner and Customer Loyalty Drive Business Results Based on the book, The Service Profit Chain: How leading Companies Link Profit and Growth to Loyalty, Satisfaction and Value (Heskett, Sasser & Schlesinger, 1997) Employee Survey Customer Survey Partner Survey Business Strategy Business Programs Company PerformanceInternal System Employee Satisfaction Employee Productivity Perceived Value Customer Satisfaction Revenue Profits Employee Loyalty Customer Loyalty Partner Advocacy Purchasing Retention Market Share
  • 6. Copyright © 2010 Business Over Broadway • Relationship Satisfaction Surveys – Focused on overall relationship with the customer – Delivery is not time-dependent (done at company discretion) – Includes Business Attribute questions and Loyalty-based questions • Typically have ratings of Satisfaction • Transactional Satisfaction Surveys – Focused on specific customer transaction – Delivery is time-dependent on transaction occurrence – Includes Business Attribute questions and Loyalty-based questions • Typically have ratings of Satisfaction • Targeted Surveys – Focused on drilling into specific areas for deeper insight – Delivery is on an as-needed basis – Compliment the Relationship and Transactional surveys • Three survey types can be used in any given Account Company Surveys
  • 7. Copyright © 2010 Business Over Broadway • Customer surveys should result in reliable and valid scores of customers’ attitudes – Good measurement processes lead to good business decisions • Reliability reflects the degree to which scores are free from random error – Reliability deals with precision/consistency – Goal is to have a measurement system that delivers reliable results – Reliability varies from 0 (no reliability) to 1 (perfect reliability) • Validity reflects the degree to which scores are measuring what they are designed to measure – Validity deals with the meaning of the scores from the measurement system – Goal is to have a measurement system that delivers valid results Reliability and Validity ValidNot Valid ReliableNotReliable
  • 8. Copyright © 2010 Business Over Broadway • Four general forms of reliability 1. Inter-rater reliability1 – Degree of agreement between two or more raters who are rating same attribute 2. Test-retest reliability1 – Degree of agreement between same measure/rater over two time periods 3. Parallel-forms reliability – Degree of agreement between two measures (A vs. B) that measure same thing 4. Internal consistency reliability1 – Degree of consistency of ratings across items within a measure Reliability = Rater 1 Rater 2 = Rater n Rater n Attribute 1 Determination of reliability depends on number of questions in survey and types of surveys. Typically, parallel-forms reliability cannot be studied as only one measure of customer satisfaction is available. Time 1 Time 2 = Rater n Survey Vers. A Attribute 2 Attribute m= Survey Vers. B= Attribute 1 Rater n
  • 9. Copyright © 2010 Business Over Broadway • Three approaches at establishing validity 1. Content-related approach – Examining the questions of the instrument to determine if they are a representative sample of the universe of all possible questions 2. Criterion-related approach – Correlating scores from the instrument with some other criteria • Based on some theory of the customer satisfaction construct • Customer satisfaction scores should be correlated to other criteria (e.g., other measures of Account quality) 3. Construct-related approach – Developing a nomological (lawful) network for the customer satisfaction scores – Framework/Theory/Model for understanding and using customer satisfaction scores should be supported by data Validity
  • 10. Copyright © 2010 Business Over Broadway Evidence of Reliability
  • 11. Copyright © 2010 Business Over Broadway • Internal consistency determines extent to which Contacts give similar ratings to items within a scale – Indexed by three indices 1. Item-total correlations - correlation between rating of single item to aggregate of remaining items in the survey 2. Split-half correlation – correlation between two halves of the survey 3. Cronbach’s alpha – formula to calculate internal consistency • Addresses the quality of the items in a scale – If reliability is high, ratings do not tend to vary across items – Expect some degree of internal consistency • Do not expect perfect reliability – Perfect reliability would indicate no variability in ratings and no need to ask multiple questions Internal Consistency Reliability
  • 12. Copyright © 2010 Business Over Broadway Surveys Contact 1 Contact 2 Contact n Business Attribute 1 Business Attribute 2 Reliability Analysis Internal Consistency Reliability • Item-Total Correlation is the correlation of specific attribute with aggregate of remaining attributes • Split-half reliability is the correlation between average of the two halves of the survey • Cronbach’s alpha () estimate used to determine internal consistency of ratings (satisfaction) within a survey • Calculated for any type of survey that contains multiple questions • Reliability varies from 0 (no internal consistency) to 1 (perfect internal consistency) Business Attribute m Business Attribute 1 Business Attribute 2 Business Attribute m Business Attribute 1 Business Attribute 2 Business Attribute m . . . = Rater n Attribute 2 Attribute m=Attribute 1 Data Model for Internal Consistency Reliability Analysis
  • 13. Copyright © 2010 Business Over Broadway Index (Measure) Definition Survey Questions Customer Loyalty Index (4 questions) Extent to which customers feel positively toward COMPANY Overall Satisfaction, Quality, Value, Recommend Perceived Value of External Information1 (7 questions) Extent to which customers value external sources of information regarding security products Security vendor websites, Whitepapers, Industry analysts, Blogs / online communities, Security publications, Peers, Webcasts / podcasts Perceived Importance of Doc/Mgmt/Reporting (3 questions) Stated importance of Documentation, Management, Reporting Reporting, Management / Administration, Documentation Perceived Importance of Ease of Use (3 questions) Stated importance of user experience Ease of installation, End user experience, Ease of upgrade Quality of Security Product (8 questions) Rating of quality of security product Security effectiveness, Reporting, Management / Administration, Documentation, Ease of installation, End user experience, Ease of upgrade, Price Quality of Technical Support (3 questions) Rating of quality of technical support Technical support, Telephone, Email Indices (Summated Scales)* 1 Items only appear in 2009 survey. *Indices were created based on factor analyses of the survey questions; factor-analytic results show that specific survey items can be combined together into one index. Survey questions that loaded highly on the same factor were combined into a single scale by averaging the responses across items. Factor analyses of survey items exposed several indices, each measuring a general content area (e.g. loyalty, tech support).
  • 14. Copyright © 2010 Business Over Broadway Internal Consistency Reliability by Decision Influence Decision-Making Role INDEX Entire Sample Final Decision Influencer Purchasing 2009 Indices 1 Customer Loyalty Index* .89 .89 .89 .88 2 Perceived Value of External Information .85 .85 .83 .87 3 Perceived Importance of Doc/Mgmt/Reporting .80 .78 .79 .78 4 Perceived Importance of Ease of Use .79 .76 .80 .80 5 Quality of Security Products .91 .92 .91 .92 6 Quality of Tech Support .91 .91 .90 .91 2010 Indices 1 Customer Loyalty Index .82 .81 .83 .81 2 Perceived Importance of Doc/Mgmt/Reporting .70 .69 .66 .78 3 Perceived Importance of Ease of Use .66 .68 .64 .58 4 Perceived Quality of Security Products .89 .89 .88 .91 5 Perceived Quality of Tech Support .92 .93 .92 .95 Internal Consistency Reliability1 Major Points • Factor analyses of survey items exposed several indices, each measuring a general content area (e.g. loyalty, tech support). • Definition of index is based on the aggregated items • Internal consistency reliabilities are acceptable and similar across all decision- making categories. • Each index would be useful for executive dashboards since each provides reliable, useful information. 1 No significance testing possible for Cronbach’s alpha. *See previous slide for description of each index and the survey items that represent each index.
  • 15. Copyright © 2010 Business Over Broadway • Inter-contact reliability determines extent to which Contacts within an Account give similar customer satisfaction survey ratings – Indexed by a correlation between different Contacts within each Account across all Accounts • Inter-contact reliability addresses the consistency of the implementation of the customer satisfaction survey system across Contacts – Expect some degree of inter-contact reliability because Contacts are rating the same Account • Do not want perfect reliability – Perfect reliability would indicate no utility of surveying different management contacts within an organization • Calculate inter-contact reliability for each survey type across all business attributes Inter-Rater Reliability
  • 16. Copyright © 2010 Business Over Broadway Data Model for Inter-Rater Reliability Analysis Type of Contact Account 1 Account 2 Account 3 Account m Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Reliability Analysis Inter-Contact Reliability • Correlation of satisfaction ratings between different Contacts within Accounts • Calculated for each type of customer survey • Reliability could vary from 0 (no agreement between Contacts) to 1.0 (complete agreement between Contacts) . . . ... = Rater 2 Attribute Rater 1
  • 17. Copyright © 2010 Business Over Broadway Inter-Contact Reliability (Decision- makers vs. Influencers) INDICES1 Decision maker/ Influencer 1 Customer Loyalty Index* .19 2 Security Effectiveness -.12 3 Reporting .05 4 Management / Administration .15 5 Documentation -.04 6 Ease of Installation .16 7 End User Experience .02 8 Ease of Upgrade .22 9 Technical Support .05 10 Price .22 Major Points • There is low agreement between Contacts from different decision-making standpoint. • Contacts from different decision-making standpoints (e.g., Decision maker vs. Influencer) show low agreement in their ratings of COMPANY • The degree of disagreement across different survey respondents within an Account could reflect: •Poor communication within an Account •Different expectations of different functions •Different usage of COMPANY products 1 Survey data are from 2010 Survey; N = 47 to 50. No statistically significant correlations. *Customer Loyalty Index is composite of following four questions: Overall Satisfaction, Quality, Value and Recommend. Functional job function includes Finance, Not enough sample to examine “Purchasing” role.
  • 18. Copyright © 2010 Business Over Broadway • Test-retest reliability determines extent to which Contacts give similar ratings over time – Indexed by a correlation between ratings across two time periods – Higher correlation indicates that Contacts provide similar ratings over time • Addresses the consistency of ratings over time – If reliability is high, relative ordering of ratings do not tend to vary over time across different Accounts – Expect good degree of test-retest reliability because Contacts are rating the same Account over time • Contacts that are satisfied/dissatisfied at time 1 should also be satisfied/dissatisfied at time 2 • Contacts with high/low value ratings at time 1 should also have high/low value ratings at time 2 • Calculate test-retest reliability for satisfaction and value ratings Test-Retest Reliability
  • 19. Copyright © 2010 Business Over Broadway Data Model for Test-Retest Reliability Analysis Time 1 Survey Time 2 Survey Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Account 1 Account 2 Account 3 Account m Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Reliability Analysis Test-Retest Reliability • Correlation of satisfaction ratings between same Contacts across two time periods • Correlation of satisfaction ratings (averaged over Contacts) between same Accounts across two time periods • Calculated only for Surveys that are repeated • Reliability varies from 0 (no agreement between time 1 and time 2) to 1.0 (complete agreement between time 1 and time 2) . . . . . . .. . = Contact n Contact n Time 1 Time 2
  • 20. Copyright © 2010 Business Over Broadway Test-Retest Reliability INDICES/MEASURES 1 Customer Loyalty Index .69 2 Overall Satisfaction .32 3 Quality .58 4 Value .52 5 Recommend .70 6 Security Effectiveness .43 7 Reporting .53 8 Management / Administration .54 9 Documentation .44 10 Ease of Installation .50 11 End User Experience .49 12 Ease of Upgrade .45 13 Technical Support .51 14 Price .54 Test-Retest Reliability1 1 Survey data are from 2009 (Time 1) and 2010 (Time 2). N = 472 to 478. All correlations significant at p < .01 level. CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct. = Contact n Contact n Time 1 Time 2 Major Points • Test-retest reliability is high • Customers tend to respond in a similar way over two time periods. • Those who are satisfied (dissatisfied) at time 1 tend to be satisfied (dissatisfied) at time 2.
  • 21. Copyright © 2010 Business Over Broadway Evidence of Validity
  • 22. Copyright © 2010 Business Over Broadway • Content validity is established when items in the survey are a representative sample of some defined universe of items • Universe of survey items represents all items that define the important elements of a company’s performance • Does the customer survey comprehensively assess customer requirements? • Must show “link” between set of items (survey) and the universe of items – Link determined by Subject Matter Experts (SMEs) Content-related Approach
  • 23. Copyright © 2010 Business Over Broadway • Disloyal customers provide comments regarding improvements1 • Comments/issues range from product to service issues • All issues/reasons covered by content of customer survey • Issues aligned with results of driver analysis Content-related Approach 1 Customer comments from those who rated Recommend question low.
  • 24. Copyright © 2010 Business Over Broadway • Criterion-related validity relies on statistical analysis rather than judgment • Established when scores on the survey are related to some other variable (criterion) already known to measure similar attributes • Key is developing/selecting right criteria to use in validation study – Measures of Account Quality – by Account Team – Objective Assessment Scores Criterion-related Approach
  • 25. Copyright © 2010 Business Over Broadway • Construct-related validity addresses what is being measured by examining relationships between constructs (e.g., customer satisfaction, customer loyalty, other measures of Account quality) • Concerned with theory and the proposed relationships between constructs based on that theory • Understanding the theoretical relationship of constructs to other constructs – To what should customer satisfaction be (convergent validity) and not be related (discriminant validity)? • Need to examine patterns of relationships using the survey and other instruments Construct-related Approach
  • 26. Copyright © 2010 Business Over Broadway 2009 Mean SD N 1 2 3 4 5 6 1. Customer Loyalty Index 4.84 0.94 2941 0.89 2. Quality of Security Product 4.59 0.86 2917 0.77 0.91 3. Quality of Technical Support 4.81 1.09 2880 0.68 0.69 0.91 4. Perceived Value of External Information 4.34 0.97 2787 0.31 0.37 0.29 0.85 5. Perceived Importance of Doc/Mgmt/Reporting 4.77 0.87 2914 0.29 0.43 0.31 0.34 0.80 6. Perceived Importance of Ease of Use 5.05 0.87 2919 0.33 0.48 0.33 0.28 0.54 0.79 Correlations Among Indices  Indices are correlated logically with each other  Higher ratings on key indices (quality, value) correspond to higher customer loyalty ratings compared to other indices (perceived importance) 2010 Mean SD N 1 2 3 4 5 1. Customer Loyalty Index 5.54 1.27 3449 0.82 2. Quality of Security Product 4.72 0.75 3417 0.78 0.89 3. Quality of Technical Support 4.95 1.04 3387 0.67 0.65 0.92 4. Perceived Importance of Doc/Mgmt/Reporting 5.01 0.66 3434 0.11 0.21 0.12 0.70 5. Perceived Importance of Ease of Use 5.17 0.69 3437 0.10 0.23 0.12 0.41 0.66 1 Cronbach’s alpha (internal consistency estimates) are located in the diagonal.
  • 27. Copyright © 2010 Business Over Broadway • Driver analysis shows that customers who are more satisfied also report higher levels of customer loyalty Construct-related Approach (2009) Mean Rating N Impact on Loyalty Customer Loyalty Index 4.84 2941 Understands Needs 4.57 2926 0.73 Addresses Needs 4.52 2924 0.78 Understands Better Than Competitors 4.30 2884 0.69 Security Effectiveness 5.18 2911 0.70 Reporting 4.40 2875 0.54 Management / Administration 4.52 2866 0.63 Documentation 4.23 2871 0.55 Ease of Installation 4.61 2884 0.58 End User Experience 4.66 2880 0.66 Ease of Upgrade 4.77 2856 0.61 Technical Support 4.80 2860 0.67 Price 4.35 2877 0.55 Compare COMPANY to Alternatives 4.06 2645 0.69 Tech Support via Telephone 4.96 1815 0.61 Tech Support via Email 4.79 1792 0.62 Tech Support via Web Knowledgebase 4.39 2102 0.49 1 No significance testing possible for Cronbach’s alpha. *CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
  • 28. Copyright © 2010 Business Over Broadway • Driver analysis shows that customers who are more satisfied also report higher levels of customer loyalty Construct-related Approach (2010) Mean Rating N Impact on Loyalty Customer Loyalty Index 5.54 3449 Understands Needs 4.32 3414 0.44 Addresses Needs 4.31 3408 0.48 Understands Better Than Competitors 4.37 3337 0.52 Security Effectiveness 5.21 3407 0.65 Reporting 4.58 3371 0.54 Management / Administration 4.67 3387 0.67 Documentation 4.48 3372 0.54 Ease of Installation 4.72 3382 0.56 End User Experience 4.73 3371 0.66 Ease of Upgrade 4.78 3356 0.59 Technical Support 4.91 3360 0.66 Price 4.62 3348 0.47 Compare COMPANY to Alternatives 4.01 3020 0.77 Tech Support via Telephone 5.05 2408 0.63 Tech Support via Email 4.91 2337 0.64 Tech Support via Web Knowledgebase 4.47 2585 0.55 1 No significance testing possible for Cronbach’s alpha. *CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
  • 29. Copyright © 2010 Business Over Broadway • Use previous validity approaches (content and criterion) to support a theoretical model of customer satisfaction • Various theoretical models state that customer loyalty is impacted by satisfaction with product and satisfaction with service • Need to examine the pattern of relationships among these constructs • Pattern of correlations should support this model Construct-related Approach Service Satisfaction Product Satisfaction Customer Loyalty Basic Theory of Customer Satisfaction
  • 30. Copyright © 2010 Business Over Broadway • Test the model by examining correlations among three variables: – Product Quality (Quality of Security Eff.) – Tech Support Quality – Customer Loyalty (CLI) • Path analysis1 applied to observed correlations – Isolates degree of impact a given variable has on other variables • Analysis supports theory that both product quality and tech tech support quality impact customer loyalty independently Construct-related Approach CORRELATIONS CLI Tech Qual Product Quality .70 .53 Customer Loyalty (CLI) - .68 1 For detailed description of path analysis methodology, please see: Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction (2nd Ed.) Holt, Rinehart and Winston. New York, NY. Tech Support Product Effectiveness Customer Loyalty Path Analysis .11 .54 .51
  • 31. Copyright © 2010 Business Over Broadway • Support Level and Customer Loyalty • Three levels of Support (Standard, Premium, Platinum) • Lower customer loyalty reported by customers with Platinum support – Platinum support customers could have higher expectations than Standard support customers. Support Level
  • 32. Copyright © 2010 Business Over Broadway • Difference found across regions1 – APAC has highest levels of customer loyalty and quality ratings – Southern Europe as lowest levels of customer loyalty and quality ratings Segmentation By Region 1 Based on survey results from 2010.
  • 33. Copyright © 2010 Business Over Broadway Segmentation By Industry Difference found across industries1 HED has the lowest levels of customer loyalty 1 Based on survey results from 2010.
  • 34. Copyright © 2010 Business Over Broadway Segmentation By License Tier Difference found across license tiers1 License tiers 3 and 4 have highest levels of customer loyalty and quality ratings 1 Based on survey results from 2010.
  • 35. Copyright © 2010 Business Over Broadway • Internal consistency reliability – Contacts tend to give similar ratings across the different items of the same scale • Non-perfect reliability indicates that different items still provide different information about customers’ attitudes • Reasons for extremely high reliability – High degree of redundancy of items – Rater fatigue due to survey length so respondents rate all questions similarly • Inter-contact reliability – Contacts within Accounts show little agreement in ratings – Different types of Contacts provide different vantage point when rating the company • Poor communication within Account across levels and/job functions • Different Contacts have differing expectations • Contacts are treated differently by Account team depending on their level/function Summary of Reliability of Survey Ratings
  • 36. Copyright © 2010 Business Over Broadway • Content-related approach – Questions in customer survey are representative sample of universe of questions • Items comprehensively cover customer requirements throughout all phases of the customer lifecycle • Dissatisfied customers’ open-ended comments are addressed by questions in the survey • Criterion-related approach – No external criteria are available at this time • Construct-related approach – Pattern of relationships of customer satisfaction data supports theory of customer satisfaction • Customer satisfaction is related to customer loyalty • Product quality impacts loyalty through perceived value Summary of Validity
  • 37. Copyright © 2010 Business Over Broadway • Self-assessment survey identified strengths and areas needing improvement1 • Overall, low to moderate adoption rates of best practices across all components of customer feedback programs – Customer feedback not as important as financial metrics, customer feedback not integrated with CRM system, problem resolution not integrated into CRM system • Strengths seen in specific areas of COMPANY’S customer feedback program: – Top executive is champion of the program, multiple methods of data collection, Web surveys, various measures of customer loyalty, customer research presented externally, existing customer data used for segmentation Customer Feedback Programs Best Practices Survey 1 Survey identifies extent to which company adopts best practices in their customer feedback program. EMPLOYEE JANE DOE completed the online survey (http://www.businessoverbroadway.com/cfpbpi.htm) for COMPANY on June 10, 2010. See survey for questions.
  • 38. Copyright © 2010 Business Over Broadway • The company’s survey process produces reliable, valid indices of customers’ attitudes • Evidence of reliability – All indices are highly reliable – items within indices can be combined to create summary scores • Evidence of validity – Survey questions comprehensively assess important areas to the customers – Each index measures unique construct apart from the other indices – Relationships among indices supports customer satisfaction models Conclusions
  • 39. Copyright © 2010 Business Over Broadway • The company can feel confident using customer survey results when making business decisions – Identifying drivers of customer loyalty – Using Indexes to manage customer relationships • Understand why different customers within the same Account hold different attitudes toward COMPANY – Do they communicate with each other (through Company-sanctioned User groups or individually)? They could get together and talk about the benefits of COMPANY. Could COMPANY assist in this process? Conclusions
  • 40. Copyright © 2010 Business Over Broadway • Changes in Customer Loyalty Measurement – Put loyalty question at start of survey (to assess top of mind responses) with common rating scale (recommend 0 to 10) scale) – Add questions to measure other components of customer loyalty • Purchasing Loyalty: Example: How likely are you to purchase additional/different types of products from COMPANY? • Retention Loyalty: Example: How likely are you to switch to a different security provider within the next 12 months? • Benefits – Make decisions to grow business through new and existing customers – Loyalty indices can be used to build customer-centricity • Executive dashboards, guide strategy/governance, resource allocation, incentive compensation, benchmark against industry standards Recommendations on Method
  • 41. Copyright © 2010 Business Over Broadway • Changes in Technical Support Measurement – Move all technical support questions into one section • Telephone, Email, and technical support – Consider adding other technical support questions to broaden technical support coverage • Knowledge, professionalism, care • Benefits – Easier for customers to complete survey – Provide more specific feedback to technical support personnel • Consider ways to increase response rates – Email invitation content Recommendations on Method
  • 42. Copyright © 2010 Business Over Broadway • Integrate customer feedback into company’s strategy/governance1 – Company’s strategic vision, mission and goals; executives’ objectives and incentive compensation, front-line employees’ objectives and incentive compensation • Integrate customer feedback into daily business processes1 – CRM system; company/executive dashboards; communicate results regularly to the entire company • Conduct in-depth research using customer survey results1 – Linkage analysis between customer feedback and operational metrics, financial metrics, employee metrics. Recommendations on Strategy, Integration and Research 1 Based on Customer Feedback Programs Self-Assessment Survey Results.
  • 43. Copyright © 2010 Business Over Broadway • Consider other “financial” metrics to track and link to customer survey – Consult with CFO to consider other financial metrics that are tracked and consider using those to identify which aspects of customer satisfaction/loyalty relate to financial growth of COMPANY • e.g. Renewals – Track financial metrics over time in conjunction with the customer survey to conduct longitudinal studies • Study impact of attitudes at time 1 on financial metrics at time 2 and beyond • Linkage study between COMPANY’S transactional survey and relationship survey – To what extent do transactions (e.g. in call center) impact long- term customer loyalty Recommendations for Future Research
  • 44. Copyright © 2010 Business Over Broadway For More Information Business growth through customer insight Business Over Broadway Bob E. Hayes, Ph.D. Email: bob@businessoverbroadway.com Web: www.businessoverbroadway.com Blog: www.businessoverbroadway.blogspot.com Twitter: www.twitter.com/bobehayes