SlideShare une entreprise Scribd logo
1  sur  18
© 2016. Capital One Bank - All Rights Reserved.
Loan Decisioning Transformation
Project IDEAL
(Intelligent Decisioning Engine for Auto Lending)
Fred Crable, Director, Software Engineering
Fredrick.Crable@CapitalOne.com
2© 2016. Capital One Bank - All Rights Reserved.
About Capital One
• A leading diversified bank
– One of the largest Credit Card companies
– Top 10 bank in US
– Nation’s largest digital bank
– 3rd largest independent auto loan provider in US
– Largest consumer and commercial banking institution
headquartered in the Washington, DC region
• Major operations in 15 US cities, Canada,
UK
• More than 70 million customer accounts
• Business & Leadership
– Ranked #127 on the Fortune 500
– Ranked #128 on Forbes
“Most Reputable Companies” list
– Listed on The Financial Times “Global 500” (#278)
– Ranked Training magazine's “Top 125” companies for
employer-sponsored workforce training and
development
2
3Confidential
Loan Decisioning
4Confidential
Pull Credit
• Equifax
• Transunion
• Experian
Lexis Nexis
*Analytic
Models
• Customer
FICO, Debt
• Collateral
Loan to Value
• Back/Front-
End
• Pricing Tiers
*Policy Engine
• Max $
Financed
• Tiers
• Limits
• Fees
• Exceptions
Pick Best
• Cash Down
• Sales Price
• Warranty
• APR
IDEAL - Fast Data Use Case
*Run 1000’s in Parallel and pick the best
5© 2016. Capital One Bank - All Rights Reserved.
Big Data Use Case 1 – Analyze and Adjust
• Customer
• CollateralApplication
• Credit Report
• Driver
Variables
Credit
• In/Out of
Policy
• Offers
• Scores
Model /
Policy
• Accept / Reject
• APR, Finance
Amount...
Outcome
• Adjust
• AvoidAnalyze
Refit/Test
6© 2016. Capital One Bank - All Rights Reserved.
Big Data Use Case #2 – Inventory Search & Scoring
•Inventory
•Book ValueSearch
•Model Scoring
•Policy Checks
Model /
Policy
•In/Out of
Policy
•APR, Finance
Amount...
Outcome
7© 2016. Capital One Bank - All Rights Reserved.
Our Implementation over AKKA
• A system that loads a set of services and uses these services in order specified
– Operations can run in parallel or in sequence ( if they have dependencies )
• The sequence of operations and order of operations are configuration driven
• The initial operation is evaluated and the result set tested to continue
• System takes input from Queue or API and kicks-off the sequence
• Operation chains can be expressed as data condition tests or model/mathematical
operations (expressed as configuration yaml or json)
8© 2016. Capital One Bank - All Rights Reserved.
Open Source Scoring Platform with Basic Decisioning Workflow
• Runs 10s of thousands of models per second on
a single server (4 cores).
• This will scale out with more cores or servers. As
they are added this number increases.
• We can add additional actor systems for other
asynchronous activities (e.g. Bureaus or external
data sources)
9© 2016. Capital One Bank - All Rights Reserved.
AKKA basics
• AKKA helps implement parallel computing easier. Through Actor model.
• AKKA helps to implement distributed parallel computing easier.
• Develop code as actor and message handling – (Loosely a java thread).
10Confidential
Using an Actor (Java)
public class Calculator implements Receiver {
LoggingAdapter log = Logging.getLogger(TypedActor.context().system(), this);
public void onReceive(Object msg, ActorRef actor) {
log.info("Received Message Q> {}", msg);
}
}
//Get access to the ActorRef
ActorRef calActor =
TypedActor.get(_system).getActorRefFor(calculator);
//pass message
calActor.tell("Hi'there");
11© 2016. Capital One Bank - All Rights Reserved.
Actor Flow Development – JSON Defined Workflow
API
Initiates / Blocks
Parallel
Operation
Group 1 Element 1
Parallel
Operation
Group 1 Element 2
Parallel
Operation
Group 1 Element 3
Serial
Operation
Group 2 Element 1
Serial
Operation
Group 2 Element 2
Message
Handler
Supervisor
{
"actorSystemName": ”SimpleSystem",
"context": {
"type": "Define",
"supervision": "Escalate",
"router": "RandomPool",
"lowerCount": 1,
"upperCount": 1,
"childrenMode": "Sequential”
},
"messageHandler": [
{
"name": "MessageHandler1",
"desc": "MessageHandler1",
"clazz": ”Class1",
"group": [
{
"name": "g1",
"desc": "g1",
"order": 1,
"context": {
"type": "Inherit",
"childrenMode": ”Parallel"
},
"element": [
{
"id": 1,
"name": "group1-element1",
"desc": ”first actor in sequence",
"order": 1,
"clazz": [
”MessageInput"
],
"bean": ”Class1Bean"
}
interface Class1Bean <MessageInput, MessageOutput>
extends ServiceInterface {
…
public MessageOutput execute(MessageInput M1)
12© 2016. Capital One Bank - All Rights Reserved.
Actor Workflow Supervisor Pattern
13© 2016. Capital One Bank - All Rights Reserved.
Supervisor Actor Patterns
14© 2016. Capital One Bank - All Rights Reserved.
Actor Workflow Mechanics – Operation and Service Definition
"messageHandler": [
{
"name": "ApplicationScoringService",
"desc": "ApplicationScoringService",
"clazz": "com.capitalone.auto.lo.business.Application",
"context": {
"group": [
{
"name": "ideal",
"desc": "ideal credit pull and decisioning workflow",
"order": 1,
"context": {
"childrenMode": "Sequential",
"carryoverRequest": "true"
},
"element": [
{
"name": "dataService",
"desc": ”get credit data and alternate data",
"clazz": [ "CurrentMessage" ],
"bean": ”DataService"
},
on receive
Handler
Bean
Output
Rules
Define service
Bean and
relative order of
execution
15© 2016. Capital One Bank - All Rights Reserved.
Actor Workflow Mechanics – Conditional Rules
"conditions": [
{
"id": 1,
"name": "Exception",
"reference": "any executionElementId",
"watch": {
"values": ["throws java.lang.Exception”]
}
},
{
"id": 2,
"name": "PrescoreResponse",
"reference": "executionElementId:1.1.1",
"watch": {
"values":
["com.capitalone.auto.lo.response.PreScoreResponse[getAppDecision notEquals A]"]
}
},
on receive
Handler
Bean
Output
Rules
Define service
Bean and
relative order of
execution
16© 2016. Capital One Bank - All Rights Reserved.
Actor Workflow Mechanics – Conditional Flow
"consequences": [
{
"id": 1,
"order": 1,
"name": "Exception",
"action": {
"whenMatched": [ "Exception”],
"thenGoto": "executionElementId:1.3.1"
}
},
{
"id": 2,
"order": 2,
"name": "PreScoreDecline",
"action": {
"whenMatched": [”ModelScoringResponse"],
"thenGoto": "executionElementId:1.2.1"
}
},
on receive
Handler
Bean
Output
Rules
Define service
Bean and
relative order of
execution
17© 2016. Capital One Bank - All Rights Reserved.
Decisioning Workflow
18© 2016. Capital One Bank - All Rights Reserved.
Best Practices
• Plan out your logging strategy with MDC (thread local variable) and use
tools like Splunk
• Identify actors like a person, with state and identity
• Limit granularity of fan-out or scatter-gather operations
• Make sure all actors have the data they need to do their jobs w/o calling
external resources
• Make sure actor operations are idempotent
• Take checkpoints and use AKKA persistence where applicable

Contenu connexe

Tendances

Application Management Service Offerings
Application Management Service OfferingsApplication Management Service Offerings
Application Management Service OfferingsGss America
 
Archer Daniels Midland: The Journey from 100% Paper to 100% Digital
Archer Daniels Midland: The Journey from 100% Paper to 100% DigitalArcher Daniels Midland: The Journey from 100% Paper to 100% Digital
Archer Daniels Midland: The Journey from 100% Paper to 100% DigitalTradeshift
 
Aula Sobre Navegação para Rally
Aula Sobre Navegação para RallyAula Sobre Navegação para Rally
Aula Sobre Navegação para RallyGustavo Monasterio
 
Digital Payments
Digital PaymentsDigital Payments
Digital Paymentsrohancool
 
The Journey to Digital Transformation with Touch Bank
The Journey to Digital Transformation with Touch BankThe Journey to Digital Transformation with Touch Bank
The Journey to Digital Transformation with Touch BankBackbase
 
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and Suppliers
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and SuppliersSupply Chain Finance: Win-Win Working Capital Benefits for Buyers and Suppliers
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and SuppliersSAP Ariba
 
E-commerce strategies for B2B business in India TYBMS
E-commerce strategies for B2B business in India TYBMS E-commerce strategies for B2B business in India TYBMS
E-commerce strategies for B2B business in India TYBMS NaheedaFatimaKhan
 
01 introduction-to-digital-finance
01 introduction-to-digital-finance01 introduction-to-digital-finance
01 introduction-to-digital-financeinnov-acts-ltd
 
Debit and credit card
Debit and credit cardDebit and credit card
Debit and credit card17791
 
New Practices and Opportunities in Supply Chain Finance
New Practices and Opportunities in Supply Chain FinanceNew Practices and Opportunities in Supply Chain Finance
New Practices and Opportunities in Supply Chain FinanceMisys
 
Digital Financial Services for Financial Inclusion
Digital Financial Services for Financial InclusionDigital Financial Services for Financial Inclusion
Digital Financial Services for Financial InclusionJohn Owens
 
e-wallet , The future of Cards and Money
e-wallet , The future of Cards and Moneye-wallet , The future of Cards and Money
e-wallet , The future of Cards and MoneyVikram Dahiya
 
Pay and Get Paid: How To Integrate Stripe Into Your App
Pay and Get Paid: How To Integrate Stripe Into Your AppPay and Get Paid: How To Integrate Stripe Into Your App
Pay and Get Paid: How To Integrate Stripe Into Your AppFlatiron School
 
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...mBank - the most design-driven digital bank in the world - NetFinance, Miami ...
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...Nordea
 
Adyen presentation
Adyen presentationAdyen presentation
Adyen presentationMax Dodson
 

Tendances (20)

Application Management Service Offerings
Application Management Service OfferingsApplication Management Service Offerings
Application Management Service Offerings
 
Archer Daniels Midland: The Journey from 100% Paper to 100% Digital
Archer Daniels Midland: The Journey from 100% Paper to 100% DigitalArcher Daniels Midland: The Journey from 100% Paper to 100% Digital
Archer Daniels Midland: The Journey from 100% Paper to 100% Digital
 
Aula Sobre Navegação para Rally
Aula Sobre Navegação para RallyAula Sobre Navegação para Rally
Aula Sobre Navegação para Rally
 
Digital bank
Digital bankDigital bank
Digital bank
 
Digital banking
Digital bankingDigital banking
Digital banking
 
Payment Gateway
Payment GatewayPayment Gateway
Payment Gateway
 
EMV Overview
EMV OverviewEMV Overview
EMV Overview
 
Digital Payments
Digital PaymentsDigital Payments
Digital Payments
 
The Journey to Digital Transformation with Touch Bank
The Journey to Digital Transformation with Touch BankThe Journey to Digital Transformation with Touch Bank
The Journey to Digital Transformation with Touch Bank
 
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and Suppliers
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and SuppliersSupply Chain Finance: Win-Win Working Capital Benefits for Buyers and Suppliers
Supply Chain Finance: Win-Win Working Capital Benefits for Buyers and Suppliers
 
E-commerce strategies for B2B business in India TYBMS
E-commerce strategies for B2B business in India TYBMS E-commerce strategies for B2B business in India TYBMS
E-commerce strategies for B2B business in India TYBMS
 
01 introduction-to-digital-finance
01 introduction-to-digital-finance01 introduction-to-digital-finance
01 introduction-to-digital-finance
 
Debit and credit card
Debit and credit cardDebit and credit card
Debit and credit card
 
New Practices and Opportunities in Supply Chain Finance
New Practices and Opportunities in Supply Chain FinanceNew Practices and Opportunities in Supply Chain Finance
New Practices and Opportunities in Supply Chain Finance
 
Mi fin financial product suite
Mi fin financial product suiteMi fin financial product suite
Mi fin financial product suite
 
Digital Financial Services for Financial Inclusion
Digital Financial Services for Financial InclusionDigital Financial Services for Financial Inclusion
Digital Financial Services for Financial Inclusion
 
e-wallet , The future of Cards and Money
e-wallet , The future of Cards and Moneye-wallet , The future of Cards and Money
e-wallet , The future of Cards and Money
 
Pay and Get Paid: How To Integrate Stripe Into Your App
Pay and Get Paid: How To Integrate Stripe Into Your AppPay and Get Paid: How To Integrate Stripe Into Your App
Pay and Get Paid: How To Integrate Stripe Into Your App
 
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...mBank - the most design-driven digital bank in the world - NetFinance, Miami ...
mBank - the most design-driven digital bank in the world - NetFinance, Miami ...
 
Adyen presentation
Adyen presentationAdyen presentation
Adyen presentation
 

En vedette

A New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouseA New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouseDataWorks Summit/Hadoop Summit
 
Building and managing complex dependencies pipeline using Apache Oozie
Building and managing complex dependencies pipeline using Apache OozieBuilding and managing complex dependencies pipeline using Apache Oozie
Building and managing complex dependencies pipeline using Apache OozieDataWorks Summit/Hadoop Summit
 
Solving Performance Problems on Hadoop
Solving Performance Problems on HadoopSolving Performance Problems on Hadoop
Solving Performance Problems on HadoopTyler Mitchell
 
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)Spark Summit
 
The Transformation Underway in FinTech Lending
The Transformation Underway in FinTech LendingThe Transformation Underway in FinTech Lending
The Transformation Underway in FinTech LendingDushyant Shahrawat, CFA
 
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay Listings
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay ListingsScalable Machine Learning Pipeline For Meta Data Discovery From eBay Listings
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay ListingsSpark Summit
 
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...DataWorks Summit/Hadoop Summit
 
Apache hadoop bigdata-in-banking
Apache hadoop bigdata-in-bankingApache hadoop bigdata-in-banking
Apache hadoop bigdata-in-bankingm_hepburn
 
Build a predictive analytics model on a terabyte of data within hours
Build a predictive analytics model on a terabyte of data within hoursBuild a predictive analytics model on a terabyte of data within hours
Build a predictive analytics model on a terabyte of data within hoursDataWorks Summit
 
LLAP: long-lived execution in Hive
LLAP: long-lived execution in HiveLLAP: long-lived execution in Hive
LLAP: long-lived execution in HiveDataWorks Summit
 
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...Spark Summit
 

En vedette (20)

Spark Uber Development Kit
Spark Uber Development KitSpark Uber Development Kit
Spark Uber Development Kit
 
A New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouseA New "Sparkitecture" for modernizing your data warehouse
A New "Sparkitecture" for modernizing your data warehouse
 
High-Scale Entity Resolution in Hadoop
High-Scale Entity Resolution in HadoopHigh-Scale Entity Resolution in Hadoop
High-Scale Entity Resolution in Hadoop
 
Building and managing complex dependencies pipeline using Apache Oozie
Building and managing complex dependencies pipeline using Apache OozieBuilding and managing complex dependencies pipeline using Apache Oozie
Building and managing complex dependencies pipeline using Apache Oozie
 
Solving Performance Problems on Hadoop
Solving Performance Problems on HadoopSolving Performance Problems on Hadoop
Solving Performance Problems on Hadoop
 
Apache Hive 2.0: SQL, Speed, Scale
Apache Hive 2.0: SQL, Speed, ScaleApache Hive 2.0: SQL, Speed, Scale
Apache Hive 2.0: SQL, Speed, Scale
 
H20: A platform for big math
H20: A platform for big math H20: A platform for big math
H20: A platform for big math
 
Beyond TCO
Beyond TCOBeyond TCO
Beyond TCO
 
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)
Deconstructiong Recommendations on Spark-(Ilya Ganelin, Capital One)
 
Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks
 
The Transformation Underway in FinTech Lending
The Transformation Underway in FinTech LendingThe Transformation Underway in FinTech Lending
The Transformation Underway in FinTech Lending
 
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay Listings
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay ListingsScalable Machine Learning Pipeline For Meta Data Discovery From eBay Listings
Scalable Machine Learning Pipeline For Meta Data Discovery From eBay Listings
 
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
 
Active Learning for Fraud Prevention
Active Learning for Fraud PreventionActive Learning for Fraud Prevention
Active Learning for Fraud Prevention
 
SQL and Search with Spark in your browser
SQL and Search with Spark in your browserSQL and Search with Spark in your browser
SQL and Search with Spark in your browser
 
Apache hadoop bigdata-in-banking
Apache hadoop bigdata-in-bankingApache hadoop bigdata-in-banking
Apache hadoop bigdata-in-banking
 
Build a predictive analytics model on a terabyte of data within hours
Build a predictive analytics model on a terabyte of data within hoursBuild a predictive analytics model on a terabyte of data within hours
Build a predictive analytics model on a terabyte of data within hours
 
LLAP: Sub-Second Analytical Queries in Hive
LLAP: Sub-Second Analytical Queries in HiveLLAP: Sub-Second Analytical Queries in Hive
LLAP: Sub-Second Analytical Queries in Hive
 
LLAP: long-lived execution in Hive
LLAP: long-lived execution in HiveLLAP: long-lived execution in Hive
LLAP: long-lived execution in Hive
 
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...
Appraiser: How Airbnb Generates Complex Models in Spark for Demand Prediction...
 

Similaire à Loan Decisioning Transformation

Similaire à Loan Decisioning Transformation (20)

Shanish_SQL_PLSQL_Profile
Shanish_SQL_PLSQL_ProfileShanish_SQL_PLSQL_Profile
Shanish_SQL_PLSQL_Profile
 
IOOF IT System Modernisation
IOOF IT System ModernisationIOOF IT System Modernisation
IOOF IT System Modernisation
 
Resume-Manish_Agrahari_IBM_BPM
Resume-Manish_Agrahari_IBM_BPMResume-Manish_Agrahari_IBM_BPM
Resume-Manish_Agrahari_IBM_BPM
 
amitabh.g
amitabh.gamitabh.g
amitabh.g
 
Updated resume
Updated resumeUpdated resume
Updated resume
 
Shriraam-performance test engineer 5.4 years
Shriraam-performance test engineer 5.4 yearsShriraam-performance test engineer 5.4 years
Shriraam-performance test engineer 5.4 years
 
Updated resume
Updated resumeUpdated resume
Updated resume
 
Ojas it services
Ojas it servicesOjas it services
Ojas it services
 
AbdulRasoolShaik_Pega_LSA
AbdulRasoolShaik_Pega_LSAAbdulRasoolShaik_Pega_LSA
AbdulRasoolShaik_Pega_LSA
 
Rani_Shinde_Resume
Rani_Shinde_ResumeRani_Shinde_Resume
Rani_Shinde_Resume
 
Business Applications of Predictive Modeling at Scale - KDD 2016 Tutorial
Business Applications of Predictive Modeling at Scale - KDD 2016 TutorialBusiness Applications of Predictive Modeling at Scale - KDD 2016 Tutorial
Business Applications of Predictive Modeling at Scale - KDD 2016 Tutorial
 
Umashankar_K_CV_Updated
Umashankar_K_CV_UpdatedUmashankar_K_CV_Updated
Umashankar_K_CV_Updated
 
IBM Rational HATS Overview 2013
IBM Rational HATS Overview 2013IBM Rational HATS Overview 2013
IBM Rational HATS Overview 2013
 
mayank_unix_sql_Jboss
mayank_unix_sql_Jbossmayank_unix_sql_Jboss
mayank_unix_sql_Jboss
 
Resume - Santosh B
Resume - Santosh BResume - Santosh B
Resume - Santosh B
 
Recruiting for Drupal #Hiring
Recruiting for Drupal #HiringRecruiting for Drupal #Hiring
Recruiting for Drupal #Hiring
 
Are API Services Taking Over All the Interesting Data Science Problems?
Are API Services Taking Over All the Interesting Data Science Problems?Are API Services Taking Over All the Interesting Data Science Problems?
Are API Services Taking Over All the Interesting Data Science Problems?
 
IbrahimAYussif_Resume
IbrahimAYussif_ResumeIbrahimAYussif_Resume
IbrahimAYussif_Resume
 
ashishtripathi
ashishtripathiashishtripathi
ashishtripathi
 
Chand_chanu
Chand_chanuChand_chanu
Chand_chanu
 

Plus de DataWorks Summit/Hadoop Summit

Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerUnleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerDataWorks Summit/Hadoop Summit
 
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformEnabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformDataWorks Summit/Hadoop Summit
 
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDouble Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDataWorks Summit/Hadoop Summit
 
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...DataWorks Summit/Hadoop Summit
 
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...DataWorks Summit/Hadoop Summit
 
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLMool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLDataWorks Summit/Hadoop Summit
 
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)DataWorks Summit/Hadoop Summit
 
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...DataWorks Summit/Hadoop Summit
 

Plus de DataWorks Summit/Hadoop Summit (20)

Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in ProductionRunning Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
 
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache ZeppelinState of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
 
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerUnleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
 
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformEnabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
 
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and ZeppelinRevolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
 
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDouble Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
 
Hadoop Crash Course
Hadoop Crash CourseHadoop Crash Course
Hadoop Crash Course
 
Data Science Crash Course
Data Science Crash CourseData Science Crash Course
Data Science Crash Course
 
Apache Spark Crash Course
Apache Spark Crash CourseApache Spark Crash Course
Apache Spark Crash Course
 
Dataflow with Apache NiFi
Dataflow with Apache NiFiDataflow with Apache NiFi
Dataflow with Apache NiFi
 
Schema Registry - Set you Data Free
Schema Registry - Set you Data FreeSchema Registry - Set you Data Free
Schema Registry - Set you Data Free
 
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
 
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
 
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLMool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
 
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
 
HBase in Practice
HBase in Practice HBase in Practice
HBase in Practice
 
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
 
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS HadoopBreaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
 
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
 
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
 

Dernier

Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 

Dernier (20)

Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 

Loan Decisioning Transformation

  • 1. © 2016. Capital One Bank - All Rights Reserved. Loan Decisioning Transformation Project IDEAL (Intelligent Decisioning Engine for Auto Lending) Fred Crable, Director, Software Engineering Fredrick.Crable@CapitalOne.com
  • 2. 2© 2016. Capital One Bank - All Rights Reserved. About Capital One • A leading diversified bank – One of the largest Credit Card companies – Top 10 bank in US – Nation’s largest digital bank – 3rd largest independent auto loan provider in US – Largest consumer and commercial banking institution headquartered in the Washington, DC region • Major operations in 15 US cities, Canada, UK • More than 70 million customer accounts • Business & Leadership – Ranked #127 on the Fortune 500 – Ranked #128 on Forbes “Most Reputable Companies” list – Listed on The Financial Times “Global 500” (#278) – Ranked Training magazine's “Top 125” companies for employer-sponsored workforce training and development 2
  • 4. 4Confidential Pull Credit • Equifax • Transunion • Experian Lexis Nexis *Analytic Models • Customer FICO, Debt • Collateral Loan to Value • Back/Front- End • Pricing Tiers *Policy Engine • Max $ Financed • Tiers • Limits • Fees • Exceptions Pick Best • Cash Down • Sales Price • Warranty • APR IDEAL - Fast Data Use Case *Run 1000’s in Parallel and pick the best
  • 5. 5© 2016. Capital One Bank - All Rights Reserved. Big Data Use Case 1 – Analyze and Adjust • Customer • CollateralApplication • Credit Report • Driver Variables Credit • In/Out of Policy • Offers • Scores Model / Policy • Accept / Reject • APR, Finance Amount... Outcome • Adjust • AvoidAnalyze Refit/Test
  • 6. 6© 2016. Capital One Bank - All Rights Reserved. Big Data Use Case #2 – Inventory Search & Scoring •Inventory •Book ValueSearch •Model Scoring •Policy Checks Model / Policy •In/Out of Policy •APR, Finance Amount... Outcome
  • 7. 7© 2016. Capital One Bank - All Rights Reserved. Our Implementation over AKKA • A system that loads a set of services and uses these services in order specified – Operations can run in parallel or in sequence ( if they have dependencies ) • The sequence of operations and order of operations are configuration driven • The initial operation is evaluated and the result set tested to continue • System takes input from Queue or API and kicks-off the sequence • Operation chains can be expressed as data condition tests or model/mathematical operations (expressed as configuration yaml or json)
  • 8. 8© 2016. Capital One Bank - All Rights Reserved. Open Source Scoring Platform with Basic Decisioning Workflow • Runs 10s of thousands of models per second on a single server (4 cores). • This will scale out with more cores or servers. As they are added this number increases. • We can add additional actor systems for other asynchronous activities (e.g. Bureaus or external data sources)
  • 9. 9© 2016. Capital One Bank - All Rights Reserved. AKKA basics • AKKA helps implement parallel computing easier. Through Actor model. • AKKA helps to implement distributed parallel computing easier. • Develop code as actor and message handling – (Loosely a java thread).
  • 10. 10Confidential Using an Actor (Java) public class Calculator implements Receiver { LoggingAdapter log = Logging.getLogger(TypedActor.context().system(), this); public void onReceive(Object msg, ActorRef actor) { log.info("Received Message Q> {}", msg); } } //Get access to the ActorRef ActorRef calActor = TypedActor.get(_system).getActorRefFor(calculator); //pass message calActor.tell("Hi'there");
  • 11. 11© 2016. Capital One Bank - All Rights Reserved. Actor Flow Development – JSON Defined Workflow API Initiates / Blocks Parallel Operation Group 1 Element 1 Parallel Operation Group 1 Element 2 Parallel Operation Group 1 Element 3 Serial Operation Group 2 Element 1 Serial Operation Group 2 Element 2 Message Handler Supervisor { "actorSystemName": ”SimpleSystem", "context": { "type": "Define", "supervision": "Escalate", "router": "RandomPool", "lowerCount": 1, "upperCount": 1, "childrenMode": "Sequential” }, "messageHandler": [ { "name": "MessageHandler1", "desc": "MessageHandler1", "clazz": ”Class1", "group": [ { "name": "g1", "desc": "g1", "order": 1, "context": { "type": "Inherit", "childrenMode": ”Parallel" }, "element": [ { "id": 1, "name": "group1-element1", "desc": ”first actor in sequence", "order": 1, "clazz": [ ”MessageInput" ], "bean": ”Class1Bean" } interface Class1Bean <MessageInput, MessageOutput> extends ServiceInterface { … public MessageOutput execute(MessageInput M1)
  • 12. 12© 2016. Capital One Bank - All Rights Reserved. Actor Workflow Supervisor Pattern
  • 13. 13© 2016. Capital One Bank - All Rights Reserved. Supervisor Actor Patterns
  • 14. 14© 2016. Capital One Bank - All Rights Reserved. Actor Workflow Mechanics – Operation and Service Definition "messageHandler": [ { "name": "ApplicationScoringService", "desc": "ApplicationScoringService", "clazz": "com.capitalone.auto.lo.business.Application", "context": { "group": [ { "name": "ideal", "desc": "ideal credit pull and decisioning workflow", "order": 1, "context": { "childrenMode": "Sequential", "carryoverRequest": "true" }, "element": [ { "name": "dataService", "desc": ”get credit data and alternate data", "clazz": [ "CurrentMessage" ], "bean": ”DataService" }, on receive Handler Bean Output Rules Define service Bean and relative order of execution
  • 15. 15© 2016. Capital One Bank - All Rights Reserved. Actor Workflow Mechanics – Conditional Rules "conditions": [ { "id": 1, "name": "Exception", "reference": "any executionElementId", "watch": { "values": ["throws java.lang.Exception”] } }, { "id": 2, "name": "PrescoreResponse", "reference": "executionElementId:1.1.1", "watch": { "values": ["com.capitalone.auto.lo.response.PreScoreResponse[getAppDecision notEquals A]"] } }, on receive Handler Bean Output Rules Define service Bean and relative order of execution
  • 16. 16© 2016. Capital One Bank - All Rights Reserved. Actor Workflow Mechanics – Conditional Flow "consequences": [ { "id": 1, "order": 1, "name": "Exception", "action": { "whenMatched": [ "Exception”], "thenGoto": "executionElementId:1.3.1" } }, { "id": 2, "order": 2, "name": "PreScoreDecline", "action": { "whenMatched": [”ModelScoringResponse"], "thenGoto": "executionElementId:1.2.1" } }, on receive Handler Bean Output Rules Define service Bean and relative order of execution
  • 17. 17© 2016. Capital One Bank - All Rights Reserved. Decisioning Workflow
  • 18. 18© 2016. Capital One Bank - All Rights Reserved. Best Practices • Plan out your logging strategy with MDC (thread local variable) and use tools like Splunk • Identify actors like a person, with state and identity • Limit granularity of fan-out or scatter-gather operations • Make sure all actors have the data they need to do their jobs w/o calling external resources • Make sure actor operations are idempotent • Take checkpoints and use AKKA persistence where applicable