SlideShare une entreprise Scribd logo
1  sur  82
Machine Learning
with Azure & Databricks
Machine
Learning 101
Databricks
Overview &
Demos
Azure
Machine
Learning
Overview &
Demos
Machine
Learning 101
9:00-9:45 9:45-10:45
(with a break)
10:45-11:30
An introduction to Machine Learning and its uses in business
Machine Learning 101
Why should anyone care about Machine Learning?
What is Machine Learning?
How does Machine Learning work?
Ok, but how does it really work?
How can an organization use Machine Learning?
Agenda
The concepts in Machine Learning are not new.
https://www.quantinsti.com/blog/machine-learning-basics
another human.
four
Though the concepts have been around, Machine Learning has just started
getting buzz in recent years because the barriers to entry are much lower.
Flood of data and decreasing costs of storage
Increasing computational power
Increased attention from researchers
Growth of open source technologies
Support from industries
Analyzing
Images
Understanding
Language
Forming &
Executing Strategy
Personalized
Recommendations
Autonomous
Decisions
Predicting
Asset Values
Machine Learning has tons of useful applications you already encounter or
hear about every day.
Sales/Marketing
• Price Optimization
• Inventory Forecasting
• Customer Segmentation
• Cross Sell / Upsell /
Recommendation Engines
• Customer Churn Predictions
• Customer Lifetime Value
Finance
• Asset Pricing
• Risk Analysis
• Fraud detection
• Market Forecasting
• Anti Money Laundering
Operations
• Inventory Forecasting
• Robotics
• Automated Workflows
• Predictive Maintenance
• Schedule Optimization
• IoT Production Line
Monitoring
Service
• Single View of Customer
• Customer Service analysis
• Chat Bots / Digital Assistants
• Social Media Analysis
• Lead Scoring
Machine Learning doesn’t just have to be the realm of high tech.
There are practical ways to incorporate it across the business.
Machine Learning lends several benefits to enterprise decision support.
Why should anyone care about Machine Learning?
What is Machine Learning?
How does Machine Learning work?
Ok, but how does it really work?
How can an organization use Machine Learning?
Agenda
Machine Learning is a discipline that supports the data science process.
It is a technique, and its value is in the outputs it drives.
Discipline Process Decision Actions
Data Science
A broad process for generating insights that may
involve data ingestion from one or many sources
(including external data, streaming data, or big
data), data processing and cleansing, model
generation using either statistical or machine
learning approaches, model selection, model
deployment and maintenance, and visualization
of data.
Advanced Analytics
Apply data science to predictive (what
will happen?) or prescriptive (what
should we do?) business use cases.
Artificial Intelligence /
Cognitive Computing
Apply data science to approximate
human intuition and decision making
(e.g. strategy, creativity, planning) or
human sensory function s (e.g.
computer vision, natural language
understanding, etc.)
Statistics
A branch of math for generating descriptions
or inferences about a population, often based
on samples of the population. Inferences may
take the form of “models,” which are
equations that approximate the data’s
inherent relationships.
Machine Learning
Combines computer science with math
concepts to generate models by rapidly
iterating on large datasets.
Other Analytics Disciplines
High Performance Computing, Data
Engineering, Visualization, etc.
Automation /
Robotics /
Intelligent Devices
Strategy / Operations
Advanced Analytics can enable predictive and prescriptive uses of data.
Traditional
analytics focus on
understanding and
explaining the data
that has been
collected.
Advanced Analytics
focus on generating
new data in the
form of predictions
or decisions, and
going the extra step
to automate
decision-making
when possible.
Simply put, machine learning is the science of making best guesses by
iterative trial and error.
101010101010101010101010101010101010
010101010101010101010101010101010101
Why should anyone care about Machine Learning?
What is Machine Learning?
How does Machine Learning work?
Ok, but how does it really work?
How can an organization use Machine Learning?
Agenda
Machine Learning works by using “algorithms” to generate “models”.
A model is a repeatable, data-driven approach to making a best guess.
It does this by formalizing mathematical relationships between data in the form of either:
– Rules (e.g. predict applicants will default on a loan if Credit Score < 700 and Debt to Income Ratio > 30%)
– Or an equation (e.g. predict Home Price = 100*Square Footage + 2*Average Income in the Area)
Note that this is different from other types of models, like operating models or data models
Statistical Model Data ModelOperating Model
People
Process Technology
Data
Guide
Support
Enable
What’s a model?
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $2MM
Program / Model
This month sales =
(prior month +
2 months prior +
3 months prior)
/ 3
Answer
This month’s sales = $3MM?
In the past we’ve told computers how to use data to answer our questions.
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
But we’ve found that if we give the machine historic facts, we can let it
find the right program / model to plug in for future answers.
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $2MM
Program / Model
This month’s sales =
1/8 * Prior month +
1/3 * 2 months prior +
1/4 * 3 months prior
What’s a model?
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $1MM
Once we have our machine-defined program, we can use it
with new data to make better predictions.
Answer
Last month’s sales: $2MM
Data
Prior month sales: $4MM
2 months prior: $3MM
3 months prior: $2MM
Program / Model
This month’s sales =
1/8 * Prior month +
1/3 * 2 months prior +
1/4 * 3 months prior
New Data
Prior month sales: $8MM
2 months prior: $6MM
3 months prior: $8MM
Answer
This month’s sales = $5MM
What’s a model?
What is an algorithm?
The word algorithm gets used a lot, but it isn’t always defined.
A defined set of steps for solving a problem
Often involves repeating steps
In Machine Learning, it may or may not have an ending condition. Some common ending conditions are:
– The problem is solved to our satisfaction
• For example – stop when the last 4 iterations have been 95% accurate or better
– The problem hasn’t been solved but we don’t seem to be getting any closer to solving it
• For example – stop if the last 10 iterations have not seen any improvement in accuracy
– The process has run for a long time
• For example – stop after the program has run for 12 hours, regardless of whether progress is still being made
Collect the data and randomly create initial decision rules.
Design a method for measurably evaluating how good or bad your hypothesis is.
Update your hypothesis in a way that marginally improves the performance of your decision rules.
Continue this process until either you are satisfied with the results, or your hypothesis can’t improve
anymore with the data available.
What is an algorithm?
Create a
hypothesis
Evaluate the
hypothesis
Adjust the
hypothesis
Repeat until
convergence
Almost all machine learning algorithms follow the same general pattern.
Why should anyone care about Machine Learning?
What is Machine Learning?
How does Machine Learning work?
Ok, but how does it really work?
How can an organization use Machine Learning?
Agenda
Supervised Learning Unsupervised Learning
We know the “right answers” for some of the scenarios.
– We may have history we can look back on
– We may be hoping to replicate human decision making
There aren’t necessarily “right answers,” we just
want to get a better understanding of our data.
There are two main families of algorithms to choose from.
Image credit: Gowthamy Vaseekaran via Medium.com, available at https://medium.com/@gowthamy/machine-learning-supervised-learning-vs-unsupervised-learning-f1658e12a780
Predict our profits next quarter.
Identify the number written on a check.
Predict a user’s rating for a given product.
Group our customers into segments.
Find the most important variables in a dataset.
Identify credit card transactions that are out of the ordinary.
Now let’s walk through two of the most popular machine learning approaches
and discuss how the algorithms are applied.
Classification Clustering
Everyone will repay their loan.
Create a
hypothesis
20 outstanding loans
Use classification when you want to guess a non-numeric value, like a yes/no answer. We
will take a decision tree approach.
Calculate accuracy as the % of predictions that are correct based on your current set of rules.
Evaluate the
hypothesis
20 outstanding loans
12 repaid, 8 defaulted
Accuracy = 12/20 = 60%
Use classification when you want to guess a non-numeric value, like a yes/no answer. We
will take a decision tree approach.
Income > 60kIncome < 60k
Find the next branch by looking for the data split that would have the biggest impact on the purity of
each node. There are several ways to do this mathematically (Gini Index, Information Gain, Chi-
Square).
Adjust the
hypothesis
20 outstanding loans20 outstanding loans
Credit Score > 700Credit Score < 700
20 outstanding loans
DTI > 40%DTI < 40%
70%50%
60% weighted
71%53%
59% weighted
80%73%
75% weighted
Use classification when you want to guess a non-numeric value, like a yes/no answer. We
will take a decision tree approach.
Repeat the process for each of your new “leaf” nodes. Stop when you reach an acceptable level of
accuracy, or when your accuracy begins getting worse with independent data.
Repeat until
convergence
20 outstanding loans
DTI > 40%DTI < 40%
Credit Score > 700Credit Score < 700Income > $60kIncome < $60k
100%50% 100%100%
80% weighted
Use classification when you want to guess a non-numeric value, like a yes/no answer. We
will take a decision tree approach.
Classification is used for lots of problems that copy human intuition.
Think about how you classify information to identify these images!
But with more advanced
approaches like convolutional
neural networks these
pictures can definitely be
classified by a machine.
These use cases are obviously
more complex than our
simple decision tree.
Now let’s walk through two of the most popular machine learning approaches
and discuss how the algorithms are applied.
Classification Clustering
Imagine Marketing
has asked you to split
these customers into
3 groups.
How would you do it?
Use clustering when there’s no “correct” classification, but you still want to assign
individuals to groups. This algorithm is called k-means clustering.
I can segment my customers by assigning them to 3 groups. We’ll set down 3 random “anchors” and
assign each customer to its closest anchor.
Create a
hypothesis
Use clustering when there’s no “correct” classification, but you still want to assign
individuals to groups. This algorithm is called k-means clustering.
Move the anchors to the center of each cluster. Count how many anchors are actually closer to one of
the other anchors.
Evaluate the
hypothesis
Use clustering when there’s no “correct” classification, but you still want to assign
individuals to groups. This algorithm is called k-means clustering.
Re-assign each customer to the group corresponding to the center they’re closest to.
Adjust the
hypothesis
Use clustering when there’s no “correct” classification, but you still want to assign
individuals to groups. This algorithm is called k-means clustering.
Repeat until
convergence
Move the anchors again. Continue re-assigning customers and moving the anchors until the anchors
stop moving.
Use clustering when there’s no “correct” classification, but you still want to assign
individuals to groups. This algorithm is called k-means clustering.
This is just the tip of the iceberg.
There are several algorithms available for various types of problems.
Why should anyone care about Machine Learning?
What is Machine Learning?
How does Machine Learning work?
Ok, but how does it really work?
How can an organization use Machine Learning?
Agenda
Engaging with Machine Learning
Image inspired by Microsoft
Delivering analytics with Machine Learning requires
alignment across people, process, technology, and data.
The sources of data can for machine learning can be quite broad. People
Process Technology
Data
Data
Warehouses
•Curated & Governed data
•Big data
•Cloud or on-prem
Data Lakes
•Unstructured & Semi-
structured data
•Streaming data
•Partially curated
Externally
Procured
Data
•May be purchased from 3rd
party providers
•May be scraped from the
web
•May require designing
research experiments
Data science teams typically have the programming and data integration skills to use data from
anywhere it can be found.
Data scientists combine broad skills to integrate
data, build models, and drive business value.
People
Process Technology
Data
Scoring,
Performance
monitoring, etc.
Business
Understanding
Start
Modeling
Feature
Engineering
Model
Training
Model
Evaluation
Deployment
Intelligent
Applications
Web
Services
Model
Store
Data Acquisition
& Understanding
Data Source
Pipeline
Environment
Wrangling,
Exploration &
Cleaning
Transform, Binning
Temporal, Text, Image
Feature Selection
Algorithms, Ensemble
Parameter Tuning
Retraining
Model Management
Cross Validation
Model Reporting
A/B Testing
On-Premises vs Cloud
Database vs Files
Streaming vs Batch
Low vs High Frequency
On-premises vs Cloud
Database vs Data Lake vs…
Small vs Medium vs Big Data
Structured vs Unstructured
Data Validation and Cleanup
Visualization
Customer
Acceptance
End
Let’s look at the Microsoft Team Data Science Process to see how models are
built.
People
Process Technology
Data
Traditional Analytics
Store and access data. Filter and aggregate it. Visualize it.
Show it to the business
so they can take action.
Machine Learning
Filter and aggregate it.
1
𝑁
𝑛=1
𝑁
𝑥
Create a model. Generate new data
(predictions, etc.).
The new data can be stored
with the rest of the data for
use in analytics.
Or it can be visualized
directly to gain insights.
Or it can automate
decisions or actions,
allowing better processes
to run faster and 24/7.
The outputs of the data science process can be used in traditional analytics,
analyzed directly, or fed into automated decision-making.
People
Process Technology
Data
We’ll spend the rest of the workshop talking about the tools that enable all this
to happen.
+
Develop models faster with automated machine learning
Use any Python environment and ML frameworks
Manage models across the cloud and the edge.
Prepare data clean data at massive scale
Enable collaboration between data scientists and data engineers
Access machine learning optimized clusters
Azure Machine Learning
Python-based machine learning service
Azure Databricks
Apache Spark-based big-data service
People
Process Technology
Data
Machine
Learning 101
Databricks
Overview &
Demos
Azure
Machine
Learning
Overview &
Demos
Databricks
Overview &
Demos
What is Databricks?
Why scale out vs. scale up?
What is Spark?
Why Databricks?
Agenda
What is Databricks, in a nutshell?
is a unified platform powered by Apache Spark,
capable of abstracting complex cluster management to
scale out your data processing and machine learning
workloads, with intelligent optimizations to dynamically
reallocate workers given computational demands.
Scaling Out with Distributed Processing vs. Scaling Up
Option A
A-G
H-N
O-T
U-Z
Imagine this… I need to find every entry in the phone book with my first name. I’d like to hire
someone to read through the entire phone book and pick them out.
Option B
Scaling Out with Distributed Processing vs. Scaling Up
Option A
A-D
E-I
J-M
N-R
Imagine this… I need to find every entry in the phone book with my first name. I’d like to hire
someone to read through the entire phone book and pick them out.
S-V
W-Z
Option B
Imagine this… I need to extract every numeric column in my dataset and normalize the values in each.
I need to perform a grid search of hyperparameters to improve the accuracy of my classification model.
I need to train an algorithm to make correct classifications based on several features.
Scaling Out with Distributed Processing vs. Scaling Up
Option A
Option B
These common pieces of machine learning pipelines may sound simple, but in working with big data, tasks
like these can add hours, days, or weeks to your timeline, or be too cost inefficient to complete at all.
More flexible
More easily scalable
With Databricks and Spark,
easy to spin up and manage
What is Spark?
2010
Started at
UC Berkeley
2013
Databricks
started &
donated to ASF 2014
Spark 1.0 and
additions to Spark
Core (SQL, ML,
GraphX)
2015
DataFrames/Datasets
Tungsten
ML Pipelines
Apache
Spark
2.0
2016
Apache Spark 3.0
released, Adaptive
Query Execution,
new Pandas
function APIs
2020
Continued feature
development to further
support distributed ML
2018
Easier
Smarter
Faster
is an open source framework enabling distributed
cluster computing for large scale data processing.
The Spark architecture works to scale processing out
across compute resources with a managing driver
node assigning processing tasks to worker nodes.
Spark was founded with the singular goal to
“democratize” the “super power” of big data by
offering high-level APIs and a unified engine to
complete processing at all steps of the data pipeline.
Since then, thousands of contributors have developed Spark projects that improve
the accessibility and versatility of the Spark framework and distributed processing.
. . .
Scaling Out with Databricks
is a unified platform powered by Apache Spark,
capable of abstracting complex cluster management to
scale out your data processing and machine learning
workloads, with intelligent optimizations to dynamically
reallocate workers given computational demands.
Databricks brings scaling out to your workloads in a way that’s easy to spin up,
familiar to work with, and integrates with tools you already use every day.
In an
accessible
setting
Multiple languages in Databricks Notebooks (Python, R, Scala, SQL)
Databricks Connect: connect external tools with Databricks (IDEs, RStudio, Jupyter…)
Work on a single node and utilize the most common ML frameworks
Familiar Options & Distributed Frameworks on Databricks
Distributed
machine
learning
Spark MLlib for distributed models
Migrate Single Node to distributed with just a few lines of code changes
Distributed hyperparameter search (Hyperopt, Gridsearch)
PandasUDF to distribute models over subsets of data or hyperparameters
Koalas: Pandas DataFrame API on Spark
Deep Learning distributed training (HorovodRunner)
Enhanced Accessibility on Azure Databricks
Not an Azure Marketplace or
a 3rd party hosted service
PAAS: Platform as a Service
Azure
Databricks is a
first party
service on
Azure.
Azure Storage Services: Directly
access data in Azure Blob Storage
and Azure Data Lake Store
Azure Active Directory: For user
authentication, eliminate the
need to maintain two separate
sets of users in Databricks and
Azure.
Azure
Databricks is
integrated
seamlessly with
Azure services.
CCGPractices
LET’S TAKE A BREAK! Return at 10:10 AM EST
Strategy and Governance
• Data Governance Solution
• Data Privacy Solution
• Strategy Roadmap Solution
Services
• Health Assessments
• Roadmaps
• Data Governance
• Data Privacy
• Master Data Management with
Profisee
• Metadata Management
Analytics and Insights
• Customer Intelligence Solution
• Visualization & Reporting Solutions
Services
• Dashboards & Visualizations
• Operational Reporting
• Data Exploration
• Customer Insights
• Marketing Analytics
• Power Platform
• D365 Customer Insights
AI and Data Science
• Machine Learning Solution
• Model As A Service
Services
• Predictive Analytics
• Prescriptive Analytics
• Azure Cognitive Services
• Natural Language Processing
• Computer Vision / Image
• ML Ops
• Data Mining
• Data Science Enablement
• Data Science Roadmap
• Data Science Center of Excellence
Data and Infrastructure
• Platform Modernization Solution
• Cloud Migration and Management
Services
• DR/BC
• Security
• Azure Governance
• Data Warehousing
• Data Integration
• Data Architecture
• PowerApps
• Synapse DW
MLflow is an open source platform for managing the end-to-end machine learning
lifecycle. MLflow offers an integrated experience for tracking and securing machine
learning model training runs and running machine learning projects.
What is MLflow?
Tracking
• Record and query
experiments: code,
data, configuration,
results
Projects
• Package data science
code in a format to
reproduce runs on
any platform
Models
• Deploy machine
learning models in
diverse serving
environments
Registry
• Store, annotate,
discover, and manage
models in a central
repository
Serving
• Host ML models as
REST endpoints that
are updated
automatically
MLflow’s Five Key Components
Back to Databricks!
Machine
Learning 101
Databricks
Overview &
Demos
Azure
Machine
Learning
Overview &
Demos
Azure
Machine
Learning
Overview &
Demos
Train and evaluate model
Azure Machine Learning offers a suite of tools for managing the Machine
Learning lifecycle.
Organize model assets
A
B
C
Deploy and manage model
Train and evaluate model
Azure Machine Learning offers a suite of tools for managing the Machine
Learning lifecycle.
Organize model assets
A
B
C
Deploy and manage model
Automated ML provides an easy way to quickly iterate through
multiple models. Train Organize
A
B
C
Deploy
How much is this car worth?
Automated ML provides an easy way to quickly iterate through
multiple models. Train Organize
A
B
C
Deploy
Mileage
Condition
Car brand
Year of make
Regulations
…
Parameter 1
Parameter 2
Parameter 3
Parameter 4
…
Gradient Boosted
Nearest Neighbors
SGD
Bayesian Regression
LGBM
…
Mileage Gradient Boosted Criterion
Loss
Min Samples Split
Min Samples Leaf
XYZ Model
Which algorithm? Which parameters?Which features?
Car brand
Year of make
Automated ML provides an easy way to quickly iterate through
multiple models. Train Organize
A
B
C
Deploy
Which algorithm? Which parameters?Which features?
Mileage
Condition
Car brand
Year of make
Regulations
…
Gradient Boosted
Nearest Neighbors
SGD
Bayesian Regression
LGBM
…
Nearest Neighbors
Criterion
Loss
Min Samples Split
Min Samples Leaf
XYZ Model
Iterate
Gradient Boosted N Neighbors
Weights
Metric
P
ZYX
Mileage
Car brand
Year of make
Car brand
Year of make
Condition
Automated ML provides an easy way to quickly iterate through
multiple models. Train Organize
A
B
C
Deploy
Which algorithm? Which parameters?Which features?
Iterate
Automated ML provides an easy way to quickly iterate through
multiple models. Train Organize
A
B
C
Deploy
Enter data
Define goals
Apply constraints
Input Intelligently test multiple models in parallel
Optimized model
For those who prefer a no-code experience, there’s a drag-n-drop
interface in Azure Machine Learning Designer. Train Organize
A
B
C
Deploy
Azure Machine Learning allows you to take advantage of cloud
compute through local tools or Azure Notebooks. Train Organize
A
B
C
Deploy
For image data, you can also train custom object detection models
with the intuitive Labeling interface. Train Organize
A
B
C
Deploy
Train and evaluate model
Azure Machine Learning offers a suite of tools for managing the Machine
Learning lifecycle.
Organize model assets
A
B
C
Deploy and manage model
Experiments allow you to capture training metrics to run side-by-side
comparisons and easily select the best model. Train Organize
A
B
C
Deploy
Pipelines can organize multiple data preparation and modeling steps
into a single resource. Train Organize
A
B
C
Deploy
Explain machine learning models to support business users and
compliance processes. Train Organize
A
B
C
Deploy
And apply fairness assessments when needed.
Train Organize
A
B
C
Deploy
Train and evaluate model
Azure Machine Learning offers a suite of tools for managing the Machine
Learning lifecycle.
Organize model assets
A
B
C
Deploy and manage model
Apply version control in a centralized model registry.
Train Organize
A
B
C
Deploy
Models can be deployed to containers and shipped to the edge or
accessed via Rest APIs. Train Organize
A
B
C
Deploy
• Identify and promote your best models
• Capture model telemetry
• Retrain models with APIs
• Deploy models anywhere
• Scale out to containers
• Infuse intelligence into the IoT edge
• Build and deploy models in minutes
• Iterate quickly on serverless infrastructure
• Easily change environments
Proactively manage
model performance
Deploy models
closer to your data
Bring models
to life quickly
Train and evaluate models
Model MGMT, experimentation,
and run history
Azure
ML service
Containers
AKS ACI
IoT edge
Docker
Azure
ML service
Monitor data drift over time to know when your model may require
re-training. Train Organize
A
B
C
Deploy
Now let’s see some of the
Azure Machine Learning tools in action.
Wrapping Up
Machine
Learning 101
Databricks
Overview &
Demos
Azure
Machine
Learning
Overview &
Demos
We can work with your business to deliver custom predictive and prescriptive
analytics across the lifecycle.
Machine Learning Strategy
• Develop a backlog of
predictive and prescriptive
use cases
• Refine and prioritize use
cases by value
• Develop a predictive
roadmap
Model Development /
Data Mining
• Aggregate data from across
internal and external data
sources
• Perform correlation
analyses, develop models,
and find new relationships
in your data
Model Maintenance
• Monitor and maintain
statistical models to sustain
predictive power
• Develop a model telemetry
dashboard
• Test model design changes
to improve predictive power
Model Governance & Operating Model
• Assess existing Data Science & Artificial Intelligence maturity
• Develop standards and processes to help guide data science output
• Build a Data Science Center of Excellence
Model Deployment / MLOps
• Customize and deploy pre-
existing models from Azure
Cognitive Services
• Deploy custom model as an
API or batch job, or support
deployment in existing
systems
RapidInsight Prototype Offering
Model as a Service Subscription OfferingElastic AI Research & Development
MLOps POC
Managed Services
Accelerators
CCG is a full-service cloud analytics provider.
Strategy and Governance
• Data Governance Solution
• Data Privacy Solution
• Strategy Roadmap Solution
Services
• Health Assessments
• Roadmaps
• Data Governance
• Data Privacy
• Master Data Management with
Profisee
• Metadata Management
Analytics and Insights
• Customer Intelligence Solution
• Visualization & Reporting Solutions
Services
• Dashboards & Visualizations
• Operational Reporting
• Data Exploration
• Customer Insights
• Marketing Analytics
• Power BI
• D365 Customer Insights
AI and Data Science
• Machine Learning Solution
• Model As A Service
Services
• Prescriptive Analytics
• Azure Cognitive Services
• Natural Language Processing
• Computer Vision / Image
• ML Ops
• Data Mining
• Data Science Enablement
• Data Science Roadmap
Data and Infrastructure
• Platform Modernization Solution
• Cloud Migration and Management
Services
• DR/BC
• Security
• Azure Governance
• Data Warehousing
• Data Integration
• Data Architecture
• PowerApps
• Synapse DW
THANK YOU
www.ccganalytics.com | (813) 968-3238

Contenu connexe

Tendances

The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
Databricks
 

Tendances (20)

Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
Master Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and GovernanceMaster Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and Governance
 
Databricks Overview for MLOps
Databricks Overview for MLOpsDatabricks Overview for MLOps
Databricks Overview for MLOps
 
Introducing Databricks Delta
Introducing Databricks DeltaIntroducing Databricks Delta
Introducing Databricks Delta
 
Data Mesh
Data MeshData Mesh
Data Mesh
 
Data Quality Best Practices
Data Quality Best PracticesData Quality Best Practices
Data Quality Best Practices
 
The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...
 
Introduction To IPaaS: Drivers, Requirements And Use Cases
Introduction To IPaaS: Drivers, Requirements And Use CasesIntroduction To IPaaS: Drivers, Requirements And Use Cases
Introduction To IPaaS: Drivers, Requirements And Use Cases
 
Getting Started with Delta Lake on Databricks
Getting Started with Delta Lake on DatabricksGetting Started with Delta Lake on Databricks
Getting Started with Delta Lake on Databricks
 
Role of Data in Digital Transformation
Role of Data in Digital TransformationRole of Data in Digital Transformation
Role of Data in Digital Transformation
 
Azure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar PresentationAzure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar Presentation
 
Benefits of Knowledge Graphs and AI For Enterprise PLM Platforms
Benefits of Knowledge Graphs and AI For Enterprise PLM PlatformsBenefits of Knowledge Graphs and AI For Enterprise PLM Platforms
Benefits of Knowledge Graphs and AI For Enterprise PLM Platforms
 
MLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at ScaleMLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at Scale
 
What is MLOps
What is MLOpsWhat is MLOps
What is MLOps
 
Big data Analytics
Big data AnalyticsBig data Analytics
Big data Analytics
 
Mdm: why, when, how
Mdm: why, when, howMdm: why, when, how
Mdm: why, when, how
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
Creating an Enterprise AI Strategy
Creating an Enterprise AI StrategyCreating an Enterprise AI Strategy
Creating an Enterprise AI Strategy
 
Data Quality Management: Cleaner Data, Better Reporting
Data Quality Management: Cleaner Data, Better ReportingData Quality Management: Cleaner Data, Better Reporting
Data Quality Management: Cleaner Data, Better Reporting
 

Similaire à Machine Learning with Azure and Databricks Virtual Workshop

Similaire à Machine Learning with Azure and Databricks Virtual Workshop (20)

Ml in a day v 1.1
Ml in a day v 1.1Ml in a day v 1.1
Ml in a day v 1.1
 
Introduction to Machine Learning with Azure & Databricks
Introduction to Machine Learning with Azure & DatabricksIntroduction to Machine Learning with Azure & Databricks
Introduction to Machine Learning with Azure & Databricks
 
Machine learning101 v1.2
Machine learning101 v1.2Machine learning101 v1.2
Machine learning101 v1.2
 
Ml in a Day Workshop 5/1
Ml in a Day Workshop 5/1Ml in a Day Workshop 5/1
Ml in a Day Workshop 5/1
 
Lesson 1 - Overview of Machine Learning and Data Analysis.pptx
Lesson 1 - Overview of Machine Learning and Data Analysis.pptxLesson 1 - Overview of Machine Learning and Data Analysis.pptx
Lesson 1 - Overview of Machine Learning and Data Analysis.pptx
 
Analytics
AnalyticsAnalytics
Analytics
 
Better Living Through Analytics - Strategies for Data Decisions
Better Living Through Analytics - Strategies for Data DecisionsBetter Living Through Analytics - Strategies for Data Decisions
Better Living Through Analytics - Strategies for Data Decisions
 
Machine learning
Machine learningMachine learning
Machine learning
 
MB2208A- Business Analytics- unit-4.pptx
MB2208A- Business Analytics- unit-4.pptxMB2208A- Business Analytics- unit-4.pptx
MB2208A- Business Analytics- unit-4.pptx
 
Data science tutorial
Data science tutorialData science tutorial
Data science tutorial
 
Fantastic Problems and Where to Find Them: Daryl Weir
Fantastic Problems and Where to Find Them: Daryl WeirFantastic Problems and Where to Find Them: Daryl Weir
Fantastic Problems and Where to Find Them: Daryl Weir
 
Data Analytics & Visualization (Introduction)
Data Analytics & Visualization (Introduction)Data Analytics & Visualization (Introduction)
Data Analytics & Visualization (Introduction)
 
Business Analytics.pptx
Business Analytics.pptxBusiness Analytics.pptx
Business Analytics.pptx
 
Machine Learning for Business - Eight Best Practices for Getting Started
Machine Learning for Business - Eight Best Practices for Getting StartedMachine Learning for Business - Eight Best Practices for Getting Started
Machine Learning for Business - Eight Best Practices for Getting Started
 
Machine learning at b.e.s.t. summer university
Machine learning  at b.e.s.t. summer universityMachine learning  at b.e.s.t. summer university
Machine learning at b.e.s.t. summer university
 
Data Analysis - Making Big Data Work
Data Analysis - Making Big Data WorkData Analysis - Making Big Data Work
Data Analysis - Making Big Data Work
 
Intro_to_business_analytics_1707852756.pdf
Intro_to_business_analytics_1707852756.pdfIntro_to_business_analytics_1707852756.pdf
Intro_to_business_analytics_1707852756.pdf
 
Chapter 1 Introduction to Business Analytics.pdf
Chapter 1 Introduction to Business Analytics.pdfChapter 1 Introduction to Business Analytics.pdf
Chapter 1 Introduction to Business Analytics.pdf
 
Evans_Analytics2e_ppt_01.pdf
Evans_Analytics2e_ppt_01.pdfEvans_Analytics2e_ppt_01.pdf
Evans_Analytics2e_ppt_01.pdf
 
Practical Machine Learning
Practical Machine LearningPractical Machine Learning
Practical Machine Learning
 

Plus de CCG

Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
CCG
 

Plus de CCG (20)

Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Data Governance Workshop
Data Governance WorkshopData Governance Workshop
Data Governance Workshop
 
How to Monetize Your Data Assets and Gain a Competitive Advantage
How to Monetize Your Data Assets and Gain a Competitive AdvantageHow to Monetize Your Data Assets and Gain a Competitive Advantage
How to Monetize Your Data Assets and Gain a Competitive Advantage
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
How to Create a Data Analytics Roadmap
How to Create a Data Analytics RoadmapHow to Create a Data Analytics Roadmap
How to Create a Data Analytics Roadmap
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Power BI Advanced Data Modeling Virtual Workshop
Power BI Advanced Data Modeling Virtual WorkshopPower BI Advanced Data Modeling Virtual Workshop
Power BI Advanced Data Modeling Virtual Workshop
 
Artificial Intelligence Executive Brief
Artificial Intelligence Executive BriefArtificial Intelligence Executive Brief
Artificial Intelligence Executive Brief
 
Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
 
Virtual Governance in a Time of Crisis Workshop
Virtual Governance in a Time of Crisis WorkshopVirtual Governance in a Time of Crisis Workshop
Virtual Governance in a Time of Crisis Workshop
 
Advance Data Visualization and Storytelling Virtual Workshop
Advance Data Visualization and Storytelling Virtual WorkshopAdvance Data Visualization and Storytelling Virtual Workshop
Advance Data Visualization and Storytelling Virtual Workshop
 
Azure Fundamentals Part 3
Azure Fundamentals Part 3Azure Fundamentals Part 3
Azure Fundamentals Part 3
 
Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
 
Azure Fundamentals Part 2
Azure Fundamentals Part 2Azure Fundamentals Part 2
Azure Fundamentals Part 2
 
Shape Your Data into a Data Model with M
Shape Your Data into a Data Model with MShape Your Data into a Data Model with M
Shape Your Data into a Data Model with M
 
Azure Fundamentals Part 1
Azure Fundamentals Part 1Azure Fundamentals Part 1
Azure Fundamentals Part 1
 
Introduction to Microsoft Power BI
Introduction to Microsoft Power BIIntroduction to Microsoft Power BI
Introduction to Microsoft Power BI
 
Data Governance and MDM | Profisse, Microsoft, and CCG
Data Governance and MDM | Profisse, Microsoft, and CCGData Governance and MDM | Profisse, Microsoft, and CCG
Data Governance and MDM | Profisse, Microsoft, and CCG
 
Enable Better Decision Making with Power BI Visualizations & Modern Data Estate
Enable Better Decision Making with Power BI Visualizations & Modern Data EstateEnable Better Decision Making with Power BI Visualizations & Modern Data Estate
Enable Better Decision Making with Power BI Visualizations & Modern Data Estate
 

Dernier

+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
nirzagarg
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
HyderabadDolls
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
nirzagarg
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
ahmedjiabur940
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
ranjankumarbehera14
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
chadhar227
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
gajnagarg
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
gajnagarg
 

Dernier (20)

+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
 
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
 
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
Gomti Nagar & best call girls in Lucknow | 9548273370 Independent Escorts & D...
Gomti Nagar & best call girls in Lucknow | 9548273370 Independent Escorts & D...Gomti Nagar & best call girls in Lucknow | 9548273370 Independent Escorts & D...
Gomti Nagar & best call girls in Lucknow | 9548273370 Independent Escorts & D...
 
7. Epi of Chronic respiratory diseases.ppt
7. Epi of Chronic respiratory diseases.ppt7. Epi of Chronic respiratory diseases.ppt
7. Epi of Chronic respiratory diseases.ppt
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
 
20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 

Machine Learning with Azure and Databricks Virtual Workshop

  • 2. Machine Learning 101 Databricks Overview & Demos Azure Machine Learning Overview & Demos Machine Learning 101 9:00-9:45 9:45-10:45 (with a break) 10:45-11:30
  • 3. An introduction to Machine Learning and its uses in business Machine Learning 101
  • 4. Why should anyone care about Machine Learning? What is Machine Learning? How does Machine Learning work? Ok, but how does it really work? How can an organization use Machine Learning? Agenda
  • 5. The concepts in Machine Learning are not new. https://www.quantinsti.com/blog/machine-learning-basics another human. four
  • 6. Though the concepts have been around, Machine Learning has just started getting buzz in recent years because the barriers to entry are much lower. Flood of data and decreasing costs of storage Increasing computational power Increased attention from researchers Growth of open source technologies Support from industries
  • 7. Analyzing Images Understanding Language Forming & Executing Strategy Personalized Recommendations Autonomous Decisions Predicting Asset Values Machine Learning has tons of useful applications you already encounter or hear about every day.
  • 8. Sales/Marketing • Price Optimization • Inventory Forecasting • Customer Segmentation • Cross Sell / Upsell / Recommendation Engines • Customer Churn Predictions • Customer Lifetime Value Finance • Asset Pricing • Risk Analysis • Fraud detection • Market Forecasting • Anti Money Laundering Operations • Inventory Forecasting • Robotics • Automated Workflows • Predictive Maintenance • Schedule Optimization • IoT Production Line Monitoring Service • Single View of Customer • Customer Service analysis • Chat Bots / Digital Assistants • Social Media Analysis • Lead Scoring Machine Learning doesn’t just have to be the realm of high tech. There are practical ways to incorporate it across the business.
  • 9. Machine Learning lends several benefits to enterprise decision support.
  • 10. Why should anyone care about Machine Learning? What is Machine Learning? How does Machine Learning work? Ok, but how does it really work? How can an organization use Machine Learning? Agenda
  • 11. Machine Learning is a discipline that supports the data science process. It is a technique, and its value is in the outputs it drives. Discipline Process Decision Actions Data Science A broad process for generating insights that may involve data ingestion from one or many sources (including external data, streaming data, or big data), data processing and cleansing, model generation using either statistical or machine learning approaches, model selection, model deployment and maintenance, and visualization of data. Advanced Analytics Apply data science to predictive (what will happen?) or prescriptive (what should we do?) business use cases. Artificial Intelligence / Cognitive Computing Apply data science to approximate human intuition and decision making (e.g. strategy, creativity, planning) or human sensory function s (e.g. computer vision, natural language understanding, etc.) Statistics A branch of math for generating descriptions or inferences about a population, often based on samples of the population. Inferences may take the form of “models,” which are equations that approximate the data’s inherent relationships. Machine Learning Combines computer science with math concepts to generate models by rapidly iterating on large datasets. Other Analytics Disciplines High Performance Computing, Data Engineering, Visualization, etc. Automation / Robotics / Intelligent Devices Strategy / Operations
  • 12. Advanced Analytics can enable predictive and prescriptive uses of data. Traditional analytics focus on understanding and explaining the data that has been collected. Advanced Analytics focus on generating new data in the form of predictions or decisions, and going the extra step to automate decision-making when possible.
  • 13. Simply put, machine learning is the science of making best guesses by iterative trial and error. 101010101010101010101010101010101010 010101010101010101010101010101010101
  • 14. Why should anyone care about Machine Learning? What is Machine Learning? How does Machine Learning work? Ok, but how does it really work? How can an organization use Machine Learning? Agenda
  • 15. Machine Learning works by using “algorithms” to generate “models”. A model is a repeatable, data-driven approach to making a best guess. It does this by formalizing mathematical relationships between data in the form of either: – Rules (e.g. predict applicants will default on a loan if Credit Score < 700 and Debt to Income Ratio > 30%) – Or an equation (e.g. predict Home Price = 100*Square Footage + 2*Average Income in the Area) Note that this is different from other types of models, like operating models or data models Statistical Model Data ModelOperating Model People Process Technology Data Guide Support Enable
  • 16. What’s a model? Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $2MM Program / Model This month sales = (prior month + 2 months prior + 3 months prior) / 3 Answer This month’s sales = $3MM? In the past we’ve told computers how to use data to answer our questions.
  • 17. Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM But we’ve found that if we give the machine historic facts, we can let it find the right program / model to plug in for future answers. Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $2MM Program / Model This month’s sales = 1/8 * Prior month + 1/3 * 2 months prior + 1/4 * 3 months prior What’s a model?
  • 18. Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $1MM Once we have our machine-defined program, we can use it with new data to make better predictions. Answer Last month’s sales: $2MM Data Prior month sales: $4MM 2 months prior: $3MM 3 months prior: $2MM Program / Model This month’s sales = 1/8 * Prior month + 1/3 * 2 months prior + 1/4 * 3 months prior New Data Prior month sales: $8MM 2 months prior: $6MM 3 months prior: $8MM Answer This month’s sales = $5MM What’s a model?
  • 19. What is an algorithm? The word algorithm gets used a lot, but it isn’t always defined. A defined set of steps for solving a problem Often involves repeating steps In Machine Learning, it may or may not have an ending condition. Some common ending conditions are: – The problem is solved to our satisfaction • For example – stop when the last 4 iterations have been 95% accurate or better – The problem hasn’t been solved but we don’t seem to be getting any closer to solving it • For example – stop if the last 10 iterations have not seen any improvement in accuracy – The process has run for a long time • For example – stop after the program has run for 12 hours, regardless of whether progress is still being made
  • 20. Collect the data and randomly create initial decision rules. Design a method for measurably evaluating how good or bad your hypothesis is. Update your hypothesis in a way that marginally improves the performance of your decision rules. Continue this process until either you are satisfied with the results, or your hypothesis can’t improve anymore with the data available. What is an algorithm? Create a hypothesis Evaluate the hypothesis Adjust the hypothesis Repeat until convergence Almost all machine learning algorithms follow the same general pattern.
  • 21. Why should anyone care about Machine Learning? What is Machine Learning? How does Machine Learning work? Ok, but how does it really work? How can an organization use Machine Learning? Agenda
  • 22. Supervised Learning Unsupervised Learning We know the “right answers” for some of the scenarios. – We may have history we can look back on – We may be hoping to replicate human decision making There aren’t necessarily “right answers,” we just want to get a better understanding of our data. There are two main families of algorithms to choose from. Image credit: Gowthamy Vaseekaran via Medium.com, available at https://medium.com/@gowthamy/machine-learning-supervised-learning-vs-unsupervised-learning-f1658e12a780 Predict our profits next quarter. Identify the number written on a check. Predict a user’s rating for a given product. Group our customers into segments. Find the most important variables in a dataset. Identify credit card transactions that are out of the ordinary.
  • 23. Now let’s walk through two of the most popular machine learning approaches and discuss how the algorithms are applied. Classification Clustering
  • 24. Everyone will repay their loan. Create a hypothesis 20 outstanding loans Use classification when you want to guess a non-numeric value, like a yes/no answer. We will take a decision tree approach.
  • 25. Calculate accuracy as the % of predictions that are correct based on your current set of rules. Evaluate the hypothesis 20 outstanding loans 12 repaid, 8 defaulted Accuracy = 12/20 = 60% Use classification when you want to guess a non-numeric value, like a yes/no answer. We will take a decision tree approach.
  • 26. Income > 60kIncome < 60k Find the next branch by looking for the data split that would have the biggest impact on the purity of each node. There are several ways to do this mathematically (Gini Index, Information Gain, Chi- Square). Adjust the hypothesis 20 outstanding loans20 outstanding loans Credit Score > 700Credit Score < 700 20 outstanding loans DTI > 40%DTI < 40% 70%50% 60% weighted 71%53% 59% weighted 80%73% 75% weighted Use classification when you want to guess a non-numeric value, like a yes/no answer. We will take a decision tree approach.
  • 27. Repeat the process for each of your new “leaf” nodes. Stop when you reach an acceptable level of accuracy, or when your accuracy begins getting worse with independent data. Repeat until convergence 20 outstanding loans DTI > 40%DTI < 40% Credit Score > 700Credit Score < 700Income > $60kIncome < $60k 100%50% 100%100% 80% weighted Use classification when you want to guess a non-numeric value, like a yes/no answer. We will take a decision tree approach.
  • 28. Classification is used for lots of problems that copy human intuition. Think about how you classify information to identify these images! But with more advanced approaches like convolutional neural networks these pictures can definitely be classified by a machine. These use cases are obviously more complex than our simple decision tree.
  • 29. Now let’s walk through two of the most popular machine learning approaches and discuss how the algorithms are applied. Classification Clustering
  • 30. Imagine Marketing has asked you to split these customers into 3 groups. How would you do it? Use clustering when there’s no “correct” classification, but you still want to assign individuals to groups. This algorithm is called k-means clustering.
  • 31. I can segment my customers by assigning them to 3 groups. We’ll set down 3 random “anchors” and assign each customer to its closest anchor. Create a hypothesis Use clustering when there’s no “correct” classification, but you still want to assign individuals to groups. This algorithm is called k-means clustering.
  • 32. Move the anchors to the center of each cluster. Count how many anchors are actually closer to one of the other anchors. Evaluate the hypothesis Use clustering when there’s no “correct” classification, but you still want to assign individuals to groups. This algorithm is called k-means clustering.
  • 33. Re-assign each customer to the group corresponding to the center they’re closest to. Adjust the hypothesis Use clustering when there’s no “correct” classification, but you still want to assign individuals to groups. This algorithm is called k-means clustering.
  • 34. Repeat until convergence Move the anchors again. Continue re-assigning customers and moving the anchors until the anchors stop moving. Use clustering when there’s no “correct” classification, but you still want to assign individuals to groups. This algorithm is called k-means clustering.
  • 35. This is just the tip of the iceberg. There are several algorithms available for various types of problems.
  • 36. Why should anyone care about Machine Learning? What is Machine Learning? How does Machine Learning work? Ok, but how does it really work? How can an organization use Machine Learning? Agenda
  • 37. Engaging with Machine Learning Image inspired by Microsoft Delivering analytics with Machine Learning requires alignment across people, process, technology, and data.
  • 38. The sources of data can for machine learning can be quite broad. People Process Technology Data Data Warehouses •Curated & Governed data •Big data •Cloud or on-prem Data Lakes •Unstructured & Semi- structured data •Streaming data •Partially curated Externally Procured Data •May be purchased from 3rd party providers •May be scraped from the web •May require designing research experiments Data science teams typically have the programming and data integration skills to use data from anywhere it can be found.
  • 39. Data scientists combine broad skills to integrate data, build models, and drive business value. People Process Technology Data
  • 40. Scoring, Performance monitoring, etc. Business Understanding Start Modeling Feature Engineering Model Training Model Evaluation Deployment Intelligent Applications Web Services Model Store Data Acquisition & Understanding Data Source Pipeline Environment Wrangling, Exploration & Cleaning Transform, Binning Temporal, Text, Image Feature Selection Algorithms, Ensemble Parameter Tuning Retraining Model Management Cross Validation Model Reporting A/B Testing On-Premises vs Cloud Database vs Files Streaming vs Batch Low vs High Frequency On-premises vs Cloud Database vs Data Lake vs… Small vs Medium vs Big Data Structured vs Unstructured Data Validation and Cleanup Visualization Customer Acceptance End Let’s look at the Microsoft Team Data Science Process to see how models are built. People Process Technology Data
  • 41. Traditional Analytics Store and access data. Filter and aggregate it. Visualize it. Show it to the business so they can take action. Machine Learning Filter and aggregate it. 1 𝑁 𝑛=1 𝑁 𝑥 Create a model. Generate new data (predictions, etc.). The new data can be stored with the rest of the data for use in analytics. Or it can be visualized directly to gain insights. Or it can automate decisions or actions, allowing better processes to run faster and 24/7. The outputs of the data science process can be used in traditional analytics, analyzed directly, or fed into automated decision-making. People Process Technology Data
  • 42. We’ll spend the rest of the workshop talking about the tools that enable all this to happen. + Develop models faster with automated machine learning Use any Python environment and ML frameworks Manage models across the cloud and the edge. Prepare data clean data at massive scale Enable collaboration between data scientists and data engineers Access machine learning optimized clusters Azure Machine Learning Python-based machine learning service Azure Databricks Apache Spark-based big-data service People Process Technology Data
  • 44. What is Databricks? Why scale out vs. scale up? What is Spark? Why Databricks? Agenda
  • 45. What is Databricks, in a nutshell? is a unified platform powered by Apache Spark, capable of abstracting complex cluster management to scale out your data processing and machine learning workloads, with intelligent optimizations to dynamically reallocate workers given computational demands.
  • 46. Scaling Out with Distributed Processing vs. Scaling Up Option A A-G H-N O-T U-Z Imagine this… I need to find every entry in the phone book with my first name. I’d like to hire someone to read through the entire phone book and pick them out. Option B
  • 47. Scaling Out with Distributed Processing vs. Scaling Up Option A A-D E-I J-M N-R Imagine this… I need to find every entry in the phone book with my first name. I’d like to hire someone to read through the entire phone book and pick them out. S-V W-Z Option B
  • 48. Imagine this… I need to extract every numeric column in my dataset and normalize the values in each. I need to perform a grid search of hyperparameters to improve the accuracy of my classification model. I need to train an algorithm to make correct classifications based on several features. Scaling Out with Distributed Processing vs. Scaling Up Option A Option B These common pieces of machine learning pipelines may sound simple, but in working with big data, tasks like these can add hours, days, or weeks to your timeline, or be too cost inefficient to complete at all. More flexible More easily scalable With Databricks and Spark, easy to spin up and manage
  • 49. What is Spark? 2010 Started at UC Berkeley 2013 Databricks started & donated to ASF 2014 Spark 1.0 and additions to Spark Core (SQL, ML, GraphX) 2015 DataFrames/Datasets Tungsten ML Pipelines Apache Spark 2.0 2016 Apache Spark 3.0 released, Adaptive Query Execution, new Pandas function APIs 2020 Continued feature development to further support distributed ML 2018 Easier Smarter Faster is an open source framework enabling distributed cluster computing for large scale data processing. The Spark architecture works to scale processing out across compute resources with a managing driver node assigning processing tasks to worker nodes. Spark was founded with the singular goal to “democratize” the “super power” of big data by offering high-level APIs and a unified engine to complete processing at all steps of the data pipeline. Since then, thousands of contributors have developed Spark projects that improve the accessibility and versatility of the Spark framework and distributed processing. . . .
  • 50. Scaling Out with Databricks is a unified platform powered by Apache Spark, capable of abstracting complex cluster management to scale out your data processing and machine learning workloads, with intelligent optimizations to dynamically reallocate workers given computational demands. Databricks brings scaling out to your workloads in a way that’s easy to spin up, familiar to work with, and integrates with tools you already use every day.
  • 51. In an accessible setting Multiple languages in Databricks Notebooks (Python, R, Scala, SQL) Databricks Connect: connect external tools with Databricks (IDEs, RStudio, Jupyter…) Work on a single node and utilize the most common ML frameworks Familiar Options & Distributed Frameworks on Databricks Distributed machine learning Spark MLlib for distributed models Migrate Single Node to distributed with just a few lines of code changes Distributed hyperparameter search (Hyperopt, Gridsearch) PandasUDF to distribute models over subsets of data or hyperparameters Koalas: Pandas DataFrame API on Spark Deep Learning distributed training (HorovodRunner)
  • 52. Enhanced Accessibility on Azure Databricks Not an Azure Marketplace or a 3rd party hosted service PAAS: Platform as a Service Azure Databricks is a first party service on Azure. Azure Storage Services: Directly access data in Azure Blob Storage and Azure Data Lake Store Azure Active Directory: For user authentication, eliminate the need to maintain two separate sets of users in Databricks and Azure. Azure Databricks is integrated seamlessly with Azure services.
  • 53. CCGPractices LET’S TAKE A BREAK! Return at 10:10 AM EST Strategy and Governance • Data Governance Solution • Data Privacy Solution • Strategy Roadmap Solution Services • Health Assessments • Roadmaps • Data Governance • Data Privacy • Master Data Management with Profisee • Metadata Management Analytics and Insights • Customer Intelligence Solution • Visualization & Reporting Solutions Services • Dashboards & Visualizations • Operational Reporting • Data Exploration • Customer Insights • Marketing Analytics • Power Platform • D365 Customer Insights AI and Data Science • Machine Learning Solution • Model As A Service Services • Predictive Analytics • Prescriptive Analytics • Azure Cognitive Services • Natural Language Processing • Computer Vision / Image • ML Ops • Data Mining • Data Science Enablement • Data Science Roadmap • Data Science Center of Excellence Data and Infrastructure • Platform Modernization Solution • Cloud Migration and Management Services • DR/BC • Security • Azure Governance • Data Warehousing • Data Integration • Data Architecture • PowerApps • Synapse DW
  • 54. MLflow is an open source platform for managing the end-to-end machine learning lifecycle. MLflow offers an integrated experience for tracking and securing machine learning model training runs and running machine learning projects. What is MLflow?
  • 55. Tracking • Record and query experiments: code, data, configuration, results Projects • Package data science code in a format to reproduce runs on any platform Models • Deploy machine learning models in diverse serving environments Registry • Store, annotate, discover, and manage models in a central repository Serving • Host ML models as REST endpoints that are updated automatically MLflow’s Five Key Components
  • 58. Train and evaluate model Azure Machine Learning offers a suite of tools for managing the Machine Learning lifecycle. Organize model assets A B C Deploy and manage model
  • 59. Train and evaluate model Azure Machine Learning offers a suite of tools for managing the Machine Learning lifecycle. Organize model assets A B C Deploy and manage model
  • 60. Automated ML provides an easy way to quickly iterate through multiple models. Train Organize A B C Deploy How much is this car worth?
  • 61. Automated ML provides an easy way to quickly iterate through multiple models. Train Organize A B C Deploy Mileage Condition Car brand Year of make Regulations … Parameter 1 Parameter 2 Parameter 3 Parameter 4 … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Mileage Gradient Boosted Criterion Loss Min Samples Split Min Samples Leaf XYZ Model Which algorithm? Which parameters?Which features? Car brand Year of make
  • 62. Automated ML provides an easy way to quickly iterate through multiple models. Train Organize A B C Deploy Which algorithm? Which parameters?Which features? Mileage Condition Car brand Year of make Regulations … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Nearest Neighbors Criterion Loss Min Samples Split Min Samples Leaf XYZ Model Iterate Gradient Boosted N Neighbors Weights Metric P ZYX Mileage Car brand Year of make Car brand Year of make Condition
  • 63. Automated ML provides an easy way to quickly iterate through multiple models. Train Organize A B C Deploy Which algorithm? Which parameters?Which features? Iterate
  • 64. Automated ML provides an easy way to quickly iterate through multiple models. Train Organize A B C Deploy Enter data Define goals Apply constraints Input Intelligently test multiple models in parallel Optimized model
  • 65. For those who prefer a no-code experience, there’s a drag-n-drop interface in Azure Machine Learning Designer. Train Organize A B C Deploy
  • 66. Azure Machine Learning allows you to take advantage of cloud compute through local tools or Azure Notebooks. Train Organize A B C Deploy
  • 67. For image data, you can also train custom object detection models with the intuitive Labeling interface. Train Organize A B C Deploy
  • 68. Train and evaluate model Azure Machine Learning offers a suite of tools for managing the Machine Learning lifecycle. Organize model assets A B C Deploy and manage model
  • 69. Experiments allow you to capture training metrics to run side-by-side comparisons and easily select the best model. Train Organize A B C Deploy
  • 70. Pipelines can organize multiple data preparation and modeling steps into a single resource. Train Organize A B C Deploy
  • 71. Explain machine learning models to support business users and compliance processes. Train Organize A B C Deploy
  • 72. And apply fairness assessments when needed. Train Organize A B C Deploy
  • 73. Train and evaluate model Azure Machine Learning offers a suite of tools for managing the Machine Learning lifecycle. Organize model assets A B C Deploy and manage model
  • 74. Apply version control in a centralized model registry. Train Organize A B C Deploy
  • 75. Models can be deployed to containers and shipped to the edge or accessed via Rest APIs. Train Organize A B C Deploy • Identify and promote your best models • Capture model telemetry • Retrain models with APIs • Deploy models anywhere • Scale out to containers • Infuse intelligence into the IoT edge • Build and deploy models in minutes • Iterate quickly on serverless infrastructure • Easily change environments Proactively manage model performance Deploy models closer to your data Bring models to life quickly Train and evaluate models Model MGMT, experimentation, and run history Azure ML service Containers AKS ACI IoT edge Docker Azure ML service
  • 76. Monitor data drift over time to know when your model may require re-training. Train Organize A B C Deploy
  • 77. Now let’s see some of the Azure Machine Learning tools in action.
  • 80. We can work with your business to deliver custom predictive and prescriptive analytics across the lifecycle. Machine Learning Strategy • Develop a backlog of predictive and prescriptive use cases • Refine and prioritize use cases by value • Develop a predictive roadmap Model Development / Data Mining • Aggregate data from across internal and external data sources • Perform correlation analyses, develop models, and find new relationships in your data Model Maintenance • Monitor and maintain statistical models to sustain predictive power • Develop a model telemetry dashboard • Test model design changes to improve predictive power Model Governance & Operating Model • Assess existing Data Science & Artificial Intelligence maturity • Develop standards and processes to help guide data science output • Build a Data Science Center of Excellence Model Deployment / MLOps • Customize and deploy pre- existing models from Azure Cognitive Services • Deploy custom model as an API or batch job, or support deployment in existing systems RapidInsight Prototype Offering Model as a Service Subscription OfferingElastic AI Research & Development MLOps POC Managed Services Accelerators
  • 81. CCG is a full-service cloud analytics provider. Strategy and Governance • Data Governance Solution • Data Privacy Solution • Strategy Roadmap Solution Services • Health Assessments • Roadmaps • Data Governance • Data Privacy • Master Data Management with Profisee • Metadata Management Analytics and Insights • Customer Intelligence Solution • Visualization & Reporting Solutions Services • Dashboards & Visualizations • Operational Reporting • Data Exploration • Customer Insights • Marketing Analytics • Power BI • D365 Customer Insights AI and Data Science • Machine Learning Solution • Model As A Service Services • Prescriptive Analytics • Azure Cognitive Services • Natural Language Processing • Computer Vision / Image • ML Ops • Data Mining • Data Science Enablement • Data Science Roadmap Data and Infrastructure • Platform Modernization Solution • Cloud Migration and Management Services • DR/BC • Security • Azure Governance • Data Warehousing • Data Integration • Data Architecture • PowerApps • Synapse DW