On Monday April 11 2016, we have demonstrated how we applied Process Mining to improve the customer experience in a crucial customer-facing process at ING Belgium. In this case we compare traditional (six sigma-style) approaches with more modern techniques to help ING Belgium serve its clients better. We will illustrate the benefits, milestones, requirements and potential pitfalls we encountered. Presentation held on the INFORMS conference on Business Analytics and Operations Research in Orlando (USA).
Process mining - a case by ING Belgium and Python Predictions
1. Process Mining – the road to a superior
customer experience
A case by ING Belgium and Python Predictions
Pieter Van Bouwel - Python Predictions
Wouter Buckinx - Python Predictions
3. ING Belgium
3
Universal direct bank
2.4 mio active retail
customers
730 branches and an
expanded digital
network
Extremely digital
Extremely personal
4. ING Belgium & analytics
4
Human
Resources
Operations
Marketing
Risk
Analytical
Maturity
5. 5
SUBMIT ACCEPT OFFER SUCCES
• Complete documents
• Signatures
• Follow-up incomplete
submissions
• Complete request
• Assess application
• Seek additional
information
• Investigate for fraud
• Prepare a competitive
offer
• Application approval
• Signatures
• Complete file
A fictive example to get us started
Mortgage Loan: The Process
Operations
7. Operational Excellence
7
Focus on Customer Journey
End 2 End measurement: from initial request until the client needs are satisfied
TODAYYESTERDAY
Measure subprocesses Measure E2E process capability
Monitor and control
Root cause analysis
Operations
Define promises to customer
8. Process mining
8
“Process Mining helps organizations
to discover, monitor and improve
real processes by extracting
knowledge from event logs”
(Wil van der Aalst)
11. 11
What are the symptoms?
Define
Improve
How to cure
our patient ?
How sick is our patient?
MeasureControl
Is our patient
healthier?
More info: cfr. Lean Six Sigma DMAIC circle
What explains
the symptoms?
Analyse
12. What are the symptoms?
12
It’s not clear what ING
expects from me
It takes too much time before
everything is up and running
Who is actually taking care of
my request?
Our customers do not provide us
with all the information we need
Our customers do not sign the
documents correctly
We acknowledge that there’s a
problem but lack the total picture
There’s a clear need to discover and measure the
end-2-end process as experienced by our customers
Point of view Customer Point of view ING
13. How sick is our patient?
13
PAC
EB
Marketing
data
platform
1 base table
holding information
on the process (steps,
times), customers
and resources
Process discoveryData preparation
and consolidation
LAMS
CCCV
14. How sick is our patient?
14
26%
23%
16%
11%
7%
5%
3% 2% 2% 1%
5%
5 10 15 20 25 30 35 40 45 50 > 50
Weekdays
Online Branch
Starting point1 Starting point2 Starting point3
Back Office1
Back Office2 Endpoint2
Endpoint3
Endpoint4
(1) (2) (3) (1)
(4)
(3)
(2)
Median weekdays
5 8
1
1 hr
2
5
1
5
Simplified process
The median End-2-
End throughput time
is 11 weekdays, but
there’s a high
variation
25%
% Files Closed
15. What explains the symptoms?
15
49%
17%
11%
7%
4% 3% 2% 2% 1% 1%
4%
5 10 15 20 25 30 35 40 45 50 > 50
First part: Process Start
There’s a lot of
variation, however
we lack the data
to further
investigate
Weekdays
% Files
16. Last part: Process End
What explains the symptoms?
16
90%
6% 2% 1% 0,4%
5 10 15 20 > 20
Weekdays
The process end
is predictable, no
use to further
analyse this part
% Files
17. 83%
14%
2% 1% 0,6%
4%
53%
20%
12% 14%
0 5 10 15 > 15
Weekdays
% First Time Right Files
% Non First Time Right Files
Middle part: Back office
What explains the symptoms?
17
The quality of
incoming files has
a high impact on
the back office
throughput time
18. What explains the symptoms?
18
2,2
6,4
9,7
14,5
17,6
0
2
4
6
8
10
12
14
16
18
20
0
(62,4%)
1
(28,2%)
2
(7,1%)
3
(1,5%)
4
(0,8%)
# Request Info Iterations
(% of files)
Each request info iteration adds
on average 4 days to the total
Back Office 1 throughput time
If a file requires
no extra info, it
takes on average
2,2 days from
branch until the
end of Back
Office 1 process
Average
Throughput Time
Back Office 1
(Weekdays)
19. 19
What explains the symptoms?
9,5% Non Second Time Right
15,7%
16,0%
7,8%
0% 5% 10% 15% 20%
Channel = Webtool
Branch Experience < 25%
Language = French
The incoming quality
can be explained by
different parameters
20. 20
What explains the symptoms?
9,5%
8,6% 15,7%
24% 11%
Channel
Paper (87%) Webtool (13%)
Sender Type
Branch / Other (8,4%)Client (4,5%)
The incoming quality
can be explained by a
combination of
parameters
21. How to cure our patient?
21
Symptoms Medication
Our customers do not provide
us with all the information we
need
Our customers do not sign
the documents correctly
It takes too much time before
everything is up and running
Who is actually taking care
of my request?
It’s not clear what ING expects
from me
Welcome team that will provide
additional support
Set KPI on quality of the files and
not on number of files
The order of products X and Y are
no longer done in the branches,
but in one back-office team
One team is the owner of the
customer request from the start
until the very end
This team is staffed by native
sales people that speak the
“language” of our customers
22. Is our patient healthier?
22
Direct feedback will be gathered via
Customer Effort Score measurements
Point of view customer Point of view ING
Set new smart Key Performance
Indicators on End-2-End Throughput
Time, Incoming and Outgoing quality
Monitoring continuously to be able
to react immediately
24. Effect = Quality of the solution X Acceptance of the solution
Criteria for Success
24
Involvement of process experts and the different players in the
End-2-End process
Ability to access and link the different data sources
A pragmatic, but critical data scientist with business affection to do the data
preparation , discover the End-2-End process and find the key root causes.
An Operational Excellence project manager with analytical skills that knows
the business, challenge the status quo and drives for high impact solutions
25. Add aditional data elements to further complete the picture
Link process data to customer feedback data
Monitor and adjust when necessary
Plans for the future
25
On this particular case
Run process mining exercises on other key customer facing processes
A new exercise is in the pipeline, comparable to the one presented today
Build up the data, tools, skills and a data driven mindset within the organisation
In general
26. 26
Focus on Customer Journey
End 2 End measurement: from initial request until the client needs are satisfied
TODAY TOMORROW
Measure E2E process capability Forecast E2E process capability
Pro-active reach promise
transparency to customer
Workload balancing
Plans for the future
More than ever there’s much to do about:
BIG data
Advanced analytics
Creating a superior customer experience
We are not going to use these terms in our presentation.
What you may expect is a practical case by which we will show how we applied mature datamining techniques
and more recent developed process mining algoritmes to improve a key customer facing process at ING.
We will guide you through our case by telling a medical story.
And that brings our back to the case of our talk, and let’s do this by means of an example: The closing of a mortgage loan.
To close a mortgage loan, there are different steps that ING and the customer need to take. A smooth procees can look like this:
-A request need to be submitted by the customer.
-ING needs to review and accept the request
-Ing needs to make an interesting offer to the client
-Finally, if everybody agrees, the request becomes a contract.
However, typically every step in this process, is the responsibility of another person, department, part in the organization.
But, a lot of elements can go wrong. Internally, as well as in interaction with the customer.
Initial application submission:
─ A_SUBMITTED / A_PARTLYSUBMITTED
Application pre-accepted but requires additional information:
─ A_PREACCEPTED
Application accepted and pending screen for completeness:
─ A_ACCEPTED
Application finalized after passing screen for completeness:
─ A_FINALIZED
End state of successful (approved) applications:
─ A_APPROVED / A_REGISTERED / A_ACTIVATED
End states of unsuccessful applications:
─ A_CANCELLED
─ A_DECLINED
Refers to states of an offer communicated to the customer:
Applicant selected to receive offer:
─ O_SELECTED
Offer prepared and transmitted to applicant:
─ O_PREPARED / O_SENT
Offer response received from applicant:
─ O_SENT BACK
End state of successful offer:
─ O_ACCEPTED
End states of unsuccessful offers:
─ O_CANCELLED
─ O_DECLINED
Refers to states of work items that occur during the approval process.
These events capture most of the manual effort exerted by Bank’s
resources during the application approval process. The events describe
efforts during various stages of the application process.
Following up on incomplete initial submissions:
─ W_Afhandelen leads
Completing pre-accepted applications:
─ W_Completeren aanvraag
Follow up after transmitting offers to qualified applicants:
─ W_Nabellen offertes
Assessing the application:
─ W_Valideren aanvraag
Seeking additional information during assessment phase:
─ W_Nabellen incomplete dossiers
Investigating suspect fraud cases:
─ W_Beoordelen fraude
Modifying approved contracts:
─ W_Wijzigen contractgegevens
Initial application submission:
─ A_SUBMITTED / A_PARTLYSUBMITTED
Application pre-accepted but requires additional information:
─ A_PREACCEPTED
Application accepted and pending screen for completeness:
─ A_ACCEPTED
Application finalized after passing screen for completeness:
─ A_FINALIZED
End state of successful (approved) applications:
─ A_APPROVED / A_REGISTERED / A_ACTIVATED
End states of unsuccessful applications:
─ A_CANCELLED
─ A_DECLINED
Refers to states of an offer communicated to the customer:
Applicant selected to receive offer:
─ O_SELECTED
Offer prepared and transmitted to applicant:
─ O_PREPARED / O_SENT
Offer response received from applicant:
─ O_SENT BACK
End state of successful offer:
─ O_ACCEPTED
End states of unsuccessful offers:
─ O_CANCELLED
─ O_DECLINED
Refers to states of work items that occur during the approval process.
These events capture most of the manual effort exerted by Bank’s
resources during the application approval process. The events describe
efforts during various stages of the application process.
Following up on incomplete initial submissions:
─ W_Afhandelen leads
Completing pre-accepted applications:
─ W_Completeren aanvraag
Follow up after transmitting offers to qualified applicants:
─ W_Nabellen offertes
Assessing the application:
─ W_Valideren aanvraag
Seeking additional information during assessment phase:
─ W_Nabellen incomplete dossiers
Investigating suspect fraud cases:
─ W_Beoordelen fraude
Modifying approved contracts:
─ W_Wijzigen contractgegevens
-In other words: the subprocesses are managed independently, and monitored independetly.
What in the end, in most cases is not the most optimal way of managing the process.
-The business question for ING was; how can we improve this way of working?
-How can we focus on the entire customer journey to ensure a smooth experience from the very start until the very end of the process.
-the answer is:
discover the process like it really happens, like the customer experiences our process, and measure what your really deliver.
If the process capability is unsufficient, search for the root causes of te waste, and improve the process.
- We wensen veel meer de customer journey te meten, dit betekent het trajet vanaf de initiële aanvraag tot wanneer de klant zijn behoeften werden gerealiseerd. Customer journeys zijn vaak een combinatie van verschillende subprocessen die vandaag nog meestal elk afzonderlijk worden gemeten.
- Door het proces te ontdekken zoals de klant ze beleefd (dank zijn process discovery technieken) kunnen we meten welke service we vandaag aan onze klanten leveren. We spreken van de huidige process capability. Bvb. in 95% van de gevallen is de rekening fully operational in 3 werkdagen.
o Als de process capability goed is dan is het een kwestie van dit elke week te gaan meten om de controle te behouden, maar hoeven we geen kosten te maken om verder verbeteringen toe te passen.
o Als de process capability onvoldoende is dan moeten we op zoek gaan naar oorzaken van ‘waste’ (root cause analysis – process mining) om de performance te verbeteren tot een niveau die aanvaardbaar is voor de klant, of die de klant een WOW experience heeft als we beslissen op die customer journey uit te blinken.
- Als we weten wat we kunnen leveren, dan kunnen we ook promises gaan definiëren naar de klant bvb. We hebben uw aanvraag goed ontvangen, uw rekening zal volledig operationeel zijn in 3 werkdagen (maar dan een beetje beter verwoord natuurlijk ;))
- Bij wat ik hier boven heb beschreven kijken we nog steeds naar het verleden (what has happened), veel sterker is natuurlijk om te meten what is happening right now or even better what is going to happen (forecasting). Dus in plaats van op het einde van de week te zien dat we in 10% van de gevallen onze customer promise hebben gemist, wensen we elke dag op te lijsten welke tickets tegen hun promise gaan aanlopen om proactief te handelen zodat de promise wordt gehaald. Vanaf 2017 wensen we dan de mogelijkheden van forecasting te onderzoeken. Als we kunnen voorspellen hoeveel klantaanvragen we in een bepaalde week/dag zullen krijgen dan kunnen we daar ook onze workforce op aanpassen en eventueel mensen van 1 team tijdelijk verschuiven naar een andere team. We spreken hierbij van workload balancing.
- We wensen veel meer de customer journey te meten, dit betekent het trajet vanaf de initiële aanvraag tot wanneer de klant zijn behoeften werden gerealiseerd. Customer journeys zijn vaak een combinatie van verschillende subprocessen die vandaag nog meestal elk afzonderlijk worden gemeten.
- Door het proces te ontdekken zoals de klant ze beleefd (dank zijn process discovery technieken) kunnen we meten welke service we vandaag aan onze klanten leveren. We spreken van de huidige process capability. Bvb. in 95% van de gevallen is de rekening fully operational in 3 werkdagen.
o Als de process capability goed is dan is het een kwestie van dit elke week te gaan meten om de controle te behouden, maar hoeven we geen kosten te maken om verder verbeteringen toe te passen.
o Als de process capability onvoldoende is dan moeten we op zoek gaan naar oorzaken van ‘waste’ (root cause analysis – process mining) om de performance te verbeteren tot een niveau die aanvaardbaar is voor de klant, of die de klant een WOW experience heeft als we beslissen op die customer journey uit te blinken.
- Als we weten wat we kunnen leveren, dan kunnen we ook promises gaan definiëren naar de klant bvb. We hebben uw aanvraag goed ontvangen, uw rekening zal volledig operationeel zijn in 3 werkdagen (maar dan een beetje beter verwoord natuurlijk ;))
- Bij wat ik hier boven heb beschreven kijken we nog steeds naar het verleden (what has happened), veel sterker is natuurlijk om te meten what is happening right now or even better what is going to happen (forecasting). Dus in plaats van op het einde van de week te zien dat we in 10% van de gevallen onze customer promise hebben gemist, wensen we elke dag op te lijsten welke tickets tegen hun promise gaan aanlopen om proactief te handelen zodat de promise wordt gehaald. Vanaf 2017 wensen we dan de mogelijkheden van forecasting te onderzoeken. Als we kunnen voorspellen hoeveel klantaanvragen we in een bepaalde week/dag zullen krijgen dan kunnen we daar ook onze workforce op aanpassen en eventueel mensen van 1 team tijdelijk verschuiven naar een andere team. We spreken hierbij van workload balancing.
Let me ask you some simple questions
Suppose you feel miserable and you go the see a doctor in a hospital.
Q1) What’s the first actions the doctor will take? Define
Q2) What will she do next? Measure (fever, taking blood pressure, listen to your heart beat)
Q3) If she knows what the symptoms are and how bad it’s, …
what’s the next step she will make? (Apply for blood analysis, send you to the radiology service to take an X-ray)
Q4) OK, she knows what’s the problem, how big the problem is and what the root causes are…
How will she proceed?
- Q5) What’s the final step?
We have just described the DMAIC approach applied in every lean six sigma project to improve process performances.
We will use this 5 step approach to explain how we tackled the process we analysed
Symptoms expressed by your customers/sales people versus Symptoms experienced by ING.
Not signing the documents correctly leads to having to send the document back … delay in the process.
Customer journey = processtep A + processtep B + processtep C
Break down the silo’s
Confidentiality.
Unfortunately, We can not share which process we analysed, but can share all the steps that we have taken
and the results.
Pieter Van Bouwel has ran all the analysis and will present you these results.
Docter will start with some basic test to measure how sick the patient is
@ING we collect process information from different departments involved in the process Consolidation in process mining table
Discovery of process model ~ body of the patient
Process model is translated in simplified version: process start (online/branch) – middle part (back office) – process end (back office or different branch scenario’s)
We added timing of different activities and between activities ~ measurements: is the process sick? 1st time ING has complete E2E view on the process
Median throughput time of 11 weekdays which was OK for the process, although there’s a high variation
For 25% of the cases it takes at least 20 weekdays to complete the request
Conclusion: We’d like to understand the root causes that explain the difference in variation
The doctor starts with a scan of the head, however as we see on this slide, the scan is rather blurry…
~There’s a lot of variation in the time spent before the file is received by BO1, however we cannot assign it to the behaviour of our customers or the ING front desks
~We lack data on customer interactions (email/phone calls) to further analyse this part of the process
A substantial part of the End-2-End throughput time is spent in this part of the process
Example?
The scan of the end part is very clear, and shows this is a standard process
Most branches start their activity within the 24 hrs,
The process throughput time is very predictable from the moment the branch activity until the endpoint
No use to further analyse this part
Root Cause Analysis
This part of the process was most interesting to analyse, because
1. The data was available
2. This part is not a standard process
3. There was a lot of variation
Main finding is shown on this slide
FTR vs NFTR: The quality of incoming files has a high impact on the throughput time at BMM side. The BMM part takes at least 5 weekdays in 96% of the non first time right tickets compared to 17% for the first time right tickets
Root Cause Analysis
The doctor discovers one of the symptoms that is making the patient sick, and does some further analysis
Relationship TPT and number of request info iterations = incoming quality of the file linear: for each request info iteration, the process takes 4 additional days to complete.
Interesting here: 62% FTR = which means more than 1/3 NFTR is document/template too complex, not clear?
NSTR: 9,5% = 1/10 files has more than 1 iteration link with KPI: speed of process instead of quality of process…
ING knew that incominig quality was an important factor, but for the first time we were able to quantify the impact on the E2E TPT
We had some very interesting findigs, but could not explain why the quality of some files was much lower than for others? further analysis: univariate
Language has positive effect Back Office = located in Brussels and has more French speaking employees French files are handled more easy
Branch experience: maybe the request should be handled by experienced persons / branches?
Webtool: maybe the tool is too difficult, or online is not the right channel for these kind of request?
Multivariate: Combining parameters explained even more the difference in incoming quality.
Interesting subsegments discuss together with business experts
Now that we know the root causes it’s time to come up with solutions to tacke the issues in the process.
Medication
Confidential: Provide additional support risk of error becomes smaller and so the incoming quality increases
Centralise orders in back office: This seems a very logial thing to do, but you should know that originaly a lot of processteps were triggered in the branches
3) Ensure our customers understand what we need
The last phase in the DMAIC circle is the control phase:
Check if your implemented actions have the desired effects
Where to adjust if necessary
How will we do the checks:
By asking our customer’s feedback CES measurements
(New process went live 2 weeks ago, too early for results)
2) By continuously monitoring the E2E process performance
We had similar reports before, but only for the subprocess parts
New KPI’s: steer on incoming and outgoing quality. Make linkt to most important root cause of long throughput times
The last phase in the DMAIC circle is the control phase:
Check if your implemented actions have the desired effects
Where to adjust if necessary
How will we do the checks:
By asking our customer’s feedback CES measurements
(New process went live 2 weeks ago, too early for results)
2) By continuously monitoring the E2E process performance
We had similar reports before, but only for the subprocess parts
New KPI’s: steer on incoming and outgoing quality. Make linkt to most important root cause of long throughput times
Effect = quality of the solution times the acceptance of the solution.
If the doctor made a correct diagnosis, but the patient does not take the medicins he will not become better.
Involvement of process experts and the different partners in the E2E process.
They can help you in finding the data (patient express the symptoms and agrees to go through some examinations) and interpret the results (patient confirms and recognizes what the doctor discovers)
Apply to link the data. Link process steps A, B, C by one unique key
In general
What we shared are the results of a POC.
I had the change to present these results to our COO who got convinced of the added value and asked me to develop analytics within his organisation.
- We wensen veel meer de customer journey te meten, dit betekent het trajet vanaf de initiële aanvraag tot wanneer de klant zijn behoeften werden gerealiseerd. Customer journeys zijn vaak een combinatie van verschillende subprocessen die vandaag nog meestal elk afzonderlijk worden gemeten.
- Door het proces te ontdekken zoals de klant ze beleefd (dank zijn process discovery technieken) kunnen we meten welke service we vandaag aan onze klanten leveren. We spreken van de huidige process capability. Bvb. in 95% van de gevallen is de rekening fully operational in 3 werkdagen.
o Als de process capability goed is dan is het een kwestie van dit elke week te gaan meten om de controle te behouden, maar hoeven we geen kosten te maken om verder verbeteringen toe te passen.
o Als de process capability onvoldoende is dan moeten we op zoek gaan naar oorzaken van ‘waste’ (root cause analysis – process mining) om de performance te verbeteren tot een niveau die aanvaardbaar is voor de klant, of die de klant een WOW experience heeft als we beslissen op die customer journey uit te blinken.
- Als we weten wat we kunnen leveren, dan kunnen we ook promises gaan definiëren naar de klant bvb. We hebben uw aanvraag goed ontvangen, uw rekening zal volledig operationeel zijn in 3 werkdagen (maar dan een beetje beter verwoord natuurlijk ;))
- Bij wat ik hier boven heb beschreven kijken we nog steeds naar het verleden (what has happened), veel sterker is natuurlijk om te meten what is happening right now or even better what is going to happen (forecasting). Dus in plaats van op het einde van de week te zien dat we in 10% van de gevallen onze customer promise hebben gemist, wensen we elke dag op te lijsten welke tickets tegen hun promise gaan aanlopen om proactief te handelen zodat de promise wordt gehaald. Vanaf 2017 wensen we dan de mogelijkheden van forecasting te onderzoeken. Als we kunnen voorspellen hoeveel klantaanvragen we in een bepaalde week/dag zullen krijgen dan kunnen we daar ook onze workforce op aanpassen en eventueel mensen van 1 team tijdelijk verschuiven naar een andere team. We spreken hierbij van workload balancing.
Need for a good methodology
Specific tools
Collect data
Merge subprocesses
Analyze process
Different skills compared to predictive analytics projects
Collect data
Understand processes
So that was it, I hoped you liked our case and that you will not remember ING as a hospital full of sick patients,
But as a company that recognizes that some of their E2E process do run not as they should and that is determined to fix them !
Are there any questions?