Government agencies continue to spend significant amounts on primary data collection to explore customer or client satisfaction at either a transactional or broader service experience level. This data is then rightly used to identify insights into how the experience can either be improved or undertaken in a more efficient manner. We believe the end of primary data collection for customer satisfaction is well within sight. Many Government agencies themselves – particularly those that deliver large scale services to the Australian community – are already drowning in data. Yet there remains a view that further primary data collection is needed to measure & understand the customer experience. Recent advancements in AI technology will revolutionise customer experience tracking for Government agencies in the very near future. Advancements in sentiment analysis technology will see the vast amounts of passive data already held by Government – in both voice and written form – used to track customer satisfaction in close to real time. And not just using a sample of interactions or transactions – across every single interaction point. This will have significant time and cost advantages to Government, as well as ending the survey impost currently placed on time poor service recipients. This presentation forms part of of the online Festival of NewMR webinar series. Presented by James Wunsch from Faster Horses.

1. Is this the death of customer
satisfaction surveys for Government?
James Wunsch
Faster Horses
The Festival of NewMR, March 2020

2. Sponsors
Gold
Silver
The Festival of NewMR, March 2020

3. WHAT WOULD
THIS GUY
KNOW
ANYWAY?
20 years working in market & social
research – almost exclusively for
Federal Government clients
Started career as a research buyer for
Australian Federal Government 1998,
before becoming a research
practitioner in 2005
Have managed literally hundreds of
qualitative & qualitative research &
evaluation projects over this time…
the vast majority of which have relied
on new primary data collection &
analysis

4. LET’S LOOK AT SOME SERVICE INTERACTION
VOLUMES…
+ DHS’s call centre operation is
the largest in size and most
complex in the southern
hemisphere.
+ In 2017–18, it handled more
than 48 million calls.
+ There were approximately
355,000 logons to myGov every
day, an increase of 34% from the
2016–17 financial year.
+ Received more than 11 million
non-digital, inbound customer
service interactions
+ Increased take up of myTax (up
9% on 2017) with over 3.5
million lodgements received
from self-preparers
+ Between 1 July 2013 and 30
June 2018, Service NSW
assisted more than 43.8
million customers.
+ 66.7 per cent of our transactions
were handled digitally.
ACCORDING TO THE
DHS 2017/18 ANNUAL REPORT
ACCORDING TO THE
ATO 2017/18 ANNUAL REPORT
ACCORDING TO THE
SERVICE NSW 2017/18 ANNUAL
REPORT

5. In competitive
markets, CX
increasingly seen
as critical to
sustainable
advantage
WHY IS MEASURING CLIENT
EXPERIENCE (CX) &
CUSTOMER SATISFACTION
IMPORTANT?“Customer experience (CX) is at the very top of everyone’s
priority list — from the CEO to the CMO to the CIO. The
emergence of new channels, new customer behaviours, new
design breakthroughs, new customer expectations and new
sources of customer data offer new opportunities to improve the
CX and deliver sustainable competitive advantage.” – GARTNER
“Data from Forrester’s Customer Experience Index (CX Index™)
shows that improving CX improves profitability; in fact, the revenue
growth of CX leaders is 5.1 times that of laggards. Great CX impacts
both top-line (higher retention, cross-sell and upsell, new customers)
and bottom-line growth (lower cost to serve, lower customer
acquisition and employee acquisition costs).” – FORRESTER

6. IMPROVING SERVICE
DELIVERY A KEY
GOVERNMENT PRIORITY
"I want to see some congestion-busting
not on our roads... but when it comes to
bureaucratic bottlenecks and regulatory
bottlenecks so Australians can get access
to those services in a more timely and
efficient way…there are services there
they rely on and we want them to be able
to access those services as easily as they
can and in as an informed way as they
possibly can.”– PM SCOTT MORISON
PM Scott Morison when announcing the creation of ‘Services Australia’
in May 2019, to be modelled on the success of Service NSW

7. HOW DO THESE AGENCIES MEASURE CX &
CSAT?
+ Random selection of current
users who have recently
interacted with the department
+ Continuously throughout the
year
+ Approximately 26,469 surveys
completed through phone
interviews and 1,525 surveys
completed through interactive
voice response
+ Cost – around AUD$1.5 million
per annum
+ Quarterly phone survey among
those with a recent interaction
+ In 2016/17 survey involved
telephone interviews with 3,956
respondents.
+ Cost – around AUD$200k
A further approx. AUD$200k on
customer satisfaction related
external research
Several CX & CSAT related studies
run in-house using primary data
collection through Qualtrics platform
+ Online survey of over 15,000
consumers and 3,500 businesses
across 6 jurisdictions in 2018
+ Asks about government service
experiences over previous 12
months
Supplemented by quarterly pulse
surveys of n=1000 consumers and
n=100 businesses
Cost – not disclosed as manged
internally
TRANSACTIONAL SURVEY CORPORATE PERCEPTIONS SURVEY
ANNUAL CUSTOMER SATISFACTION
MEASUREMENT SURVEY

8. IS THAT AN
ELEPHANT
IN THE
ROOM?
+ You bet, and it’s called non-response bias!
+ Survey response rates – including for CX and CSAT
studies - are now commonly so low they are not even
reported…survey response rates of less than 10% is
increasingly common
+ At this level, we can’t be even remotely sure our
sample of customers is genuinely representative of the
broader customer service experience!

9. IN SUMMARY…
While customer satisfaction with government service delivery has
traditionally been measured through additional primary data
collection among service users (e.g. post contact surveys), there
are several key challenges with this:
+ People are increasingly time poor and unwilling to participate in follow-up surveys – even
where their feedback could potentially lead to an improved service experience in future
+ Primary data collection is still relatively expensive – not to mention the nominal (and
unpaid) time cost being born by respondents when undertaking surveys
+ Primary data collection can only ever touch a very small proportion of customers, which
undermines the reliability of being able to extrapolate these findings to your broader
client population
+ Non-response bias is a very real (but commonly downplayed) issue
SO….IS THERE A BETTER WAY???

10. Every government agency understands the impact data
can have on the success or failure of their organisation.
Yet almost no one has been able to harness insights from
one of their most important sources of data:
Their own customer conversations
Until now…

11. MILLIONS OF
CONVERSATIONS…
…INTO ACTIONABLE
INSIGHTS
Experience
Performance
Compliance
100%
Call Coverage
…AUTOMATICALLY
ANALYSED & DECIPHERED…
THE FUTURE IS HERE, NOW
Natural Language Recognition and AI can uncover insights in your customer
conversations by listening, deciphering, and highlighting courses of action…
from every single customer service interaction, across all channels!
100%
INTERACTION
COVERAGE
SEMANTIC ENGINE
Translate
Display
Ingest
Interpret

12. CRITICAL QUESTIONS AND
CONCERNS ADDRESSED
Are we complying
with regulation?
Are we delivering an
excellent customer
experience?
Is our supervisory
structure
efficient & effective?
Are our training
programs efficient
& effective?
Which customers
are likely to
recontact?
Where are
the opportunities
for increased
efficiencies?
Analysis of unstructured voice & text data at scale, helping
agencies address critical questions and concerns

13. THIS SOUNDS TOO
GOOD TO BE TRUE…
+ Sceptical that NLR & AI - using existing interaction data - can
be as accurate as real feedback from customers?
+ We had run a traditional NPS tracking study for an energy
provider for the past three years.
+ We wanted to see how the actual NPS scores compared
with the output of an automated sentiment analysis
(using NLR & AI) of verbatim comments given to the
follow up question “Why is that?”
+ These text inputs from the survey could be readily swapped out
for actual call centre recording transcripts

14. AUTO-NPS & CSAT VIA SENTIMENT
ANALYSIS VS. PRIMARY DATA
COLLECTIONThe outcome….
AUTO-CODED SENTIMENT
MORE ACCURATE REFLECTION OF
TRUE CUSTOMER FEELING BECAUSE
IT CONSIDERS MEANING & CONTEXT
NPS
NPS NEARLY ALWAYS UNDERESTIMATES TRUE
CUSTOMER SENTIMENT
WHY?
‘8’ COUNTS AS ‘NEUTRAL’
‘5/6’ ARE OFTEN NEUTRAL RATHER THAN
NEGATIVE
R=0.5

15. SAMPLE INFORMATION FLOW
DATA SOURCES
(OMNI CHANNEL)
SOCIAL
MEDIA
CUSTOMER
CALLS
CRM
EMAIL/
CHAT
AUDIO ANALYTICS
AND
STANDARDISATION
(VOICE)
TRANSCRIPTION
(VOICE)
MATCH CUSTOMER
METADATA
(CRM)
SURVEYS
WEB/
MOBILE
IOT
EXTRACT LOAD
SPEECH
ANALYTICS
ENGINE
TRANSCRIPTION
ENGINE
INTERPRET
API
DISPLAY
CHURN
SENTIMENT
INTERACTION
REASON
REPEAT
DISSATIS-
FACTION
OTHERS
(CUSTOMISABLE)
KEY CONTACT INSIGHTS

16. High-risk
calls flagged
Performance
recognition
Trend analysis
IDENTIFY HIGH RISK INTERACTIONS
Transcribes, analyses, scores, and then highlights interactions perceived
“high risk” by the organisation (e.g. likely to complain, compliance risk)

17. PERFORMANCE
+ Unlock insights from every customer conversation,
enabled through 100% call coverage
+ Enhance operational excellence (automate QA
scoring, strengthen call to action process)
+ Continuous, unbiased compliance process violation
and conduct monitoring
+ Use AI to identify signals/patterns/themes at scale
+ Enhance current or create new products as a result of
greater customer insight
+ Instantaneous QA scalability
+ Use contact centres for value-added growth rather than
cost centre
+ Open data architecture; helping other areas of your
business grow
DELIVERING VALUE BY HELPING
BUSINESSES EXCEL IN KEY STRATEGIC
AREAS BUSINESS GROWTH
SERVICE
+ Rapid remediation of customer issues and
improved overall service delivery
+ Better understand and anticipate customer needs
+ Reinvent customer journey
+ Training tailored to each agent’s development needs
+ Deliver new and improved ways of working through
adoption and use of AI-driven tools
+ AI allows for improved human capital deployment
WORKFORCE OPTIMISATION

18. SO ARE CUSTOMER
SATISFACTION SURVEYS
DEAD FOR
GOVERNMENT?NOT YET…
+ Many established governance and customer service tracking surveys
that won’t/can’t disappear overnight
BUT…
+ The cost, speed and coverage advantages offered by this technology
will likely make much of this survey work redundant in the very near
future
+ We believe primary data collection is very much in its ‘twilight years’ -
Government agencies need to invest in new approaches that better
leverage the vast tracks of data already produced through each and
every client interaction
+ We are partnering with a number of Government agencies in trialling
these alternate approaches to tracking the customer service
experience

19. Q & A
James Wunsch
Faster Horses
Sue York
NewMR
The Festival of NewMR, March 2020

20. Sponsors
Gold
Silver
The Festival of NewMR, March 2020