Agency aims to extend machine learning to problems where the four pillars of effective machine learning are not fully present. It provides a formal technique for linking data science to business objectives and decisions. Agency decomposes complex dynamical systems until each component either satisfies the four pillars or can be described using known relationships, in order to determine which actions will increase the probability of achieving objectives.

1. Agency
Mark Zangari
CEO Quantellia

2. Agency
Agency’s aims:
1. Extend using machine learning into a new domain
of problems.
2. The beginnings of a formal technique for linking
data science to business levers.
3. Instead of answering “Given input data A, what
output data B does my model predict?”, Agency
answers “If I do A to complex dynamical system S,
with the intention of achieving objective B, what is
the probability that S’s outputs are closer to B?”

3. Agency
The four pillars for effective machine learning:
1. Availability of training data
2. System stability over characterstic times
(a) training data period (past)
(b) prediction period (future)
3. Availability of input data at the time the prediction
is to be made.
4. Sufficient signal-to-noise in data for pattern
recognition to be possible.

4. Agency
The four pillars for effective machine learning:
What can we do if one or more of the
four pillars are not present?

5. Agency
The four pillars for effective machine learning:
What can we do if one or more of the
four pillars are not present?
Example from finance:
Commercial lending vs. consumer lending.

6. Agency
Consumer Credit:
1. Ample, accessible data
2. Large, consistent, classifiable population
3. Relevant variables easily measured
4. Strong correlations between input variables and credit risk.
Consumer
Credit
Transactions
Data
Warehouse
Rating System Credit Score

7. Bill Fair & Earl Isaac
Contacted 58 of the
nation’s top lending
institutions in 1958
offering to show them
how using data would
help them make better
credit decisions...
Only one responded.
Image composed from stock images and portraits of Fair and
http://www.fico.com/en/about-us#our_history

8. Agency
Consumer Credit:
1. Ample, accessible data
2. Large, consistent, classifiable population
3. Relevant variables easily measured
4. Strong correlations between input variables and credit risk.
Commercial Credit for Small/Medium Businesses:
1. Little data available, hard to obtain
2. Each business different, many are very new with little history.
3. Complex inter-entity relationships affect credit risk.
4. Many correlations , hard to isolate those that are good predictors
of credit risk.

9. Agency
Fully automated credit rating is rarely used for scoring
Small/Medium Business Loans.
The process commonly used is a good example of
Agency in action…

10. Agency
Type of
Business
Loan Amount Probability of
Default
Typical Model Decomposition for Commercial Loan Decision
Grant this
Loan?
Profitable
Loan Book
Desired ObjectiveBusiness Lever
External
Inputs

11. Agency
Agency Principle One:
Learning and other analytics are designed to
discriminate actions that will increase the probability of
the objectives being met, from actions that do not.
So…
What is an “objective?”

12. Agency
Objective:
Map every element in the set of outcomes to a measure
of the favorability of that outcome occurring, relative to
the other outcomes.
Probability of default
DesiredLoanFrequency

13. Agency
Objective:
Map every element in the set of outcomes to a measure
of the favorability of that outcome occurring, relative to
the other outcomes.
Probability of default
DesiredLoanFrequency

14. Agency
Type of
Business
Loan Amount Probability of
Default
Typical Model Decomposition for Commercial Loan Decision
Grant this
Loan?
Profitable
Loan Book
Desired ObjectiveBusiness Levers
External
Inputs
Measure Measure Learn
Interest Rate

15. Agency
Agency Principle Two:
If a system does not ideally support machine learning
because it does not satisfy the four pillars, decompose it
using a dynamic system model until each link between
each node either:
(a) Satisfies the four pillars, or
(b) Can be described using known domain-specific
relationships (formulas, rules, approximations, etc.)

16. Agency
Financial
Spreading
Values
Market &
Industry Data
Management
Team Bios &
Experience
Relationships
to other
Entities
Key Ratios
Aggregate
Statistics
Scoring
Probability of
Default
Known mathematical relationships
Machine Learning
(intangibles)
Rating
Machine Learning
Transform Tables
Typical Model Decomposition for Business Credit Rating
Feature Engineering

17. Agency
Management
Team
Experience
Agency gives us insight into decisions via the model levers.
E.g. What interest rate do we charge and how does this
affect our success?
Interest Rate 1
Management
Team
Experience
Interest Rate 2
Interest Rate 1 Interest Rate 2

18. Agency
Summary of characteristics where Agency becomes
highly relevant and useful:
• Sparse data
• Complex emergent dynamics, including feedback
loops, phase changes, and other non-linear effects.
• Provides a way of including intangibles with few
measureables in the model
• Can utilize relationships that the training data does
not make apparent.

19. Agency
A (brief) mathematical representation of Agency:
We draw analogy from the information measure:
The “Agency” A of a lever L, which maps some part of
the output space to favorable outcomes, f and some
other part to unfavorable outcomes, with measure M(f)
and M(u) is given by:

20. Agency
Agency:
• Helps business users connect real-world decisions
with support from data.
• Provides a architecture for using machine learning in
situations that are not typically well suited to ML
solutions.
• Allows ML results to be extrapolated beyond
information contained in the data, by integrating
knowledge of system dynamics.