SlideShare une entreprise Scribd logo
1  sur  23
Télécharger pour lire hors ligne
Plan
• Francis why
• Personal ML lib evolution & Qmining
• My bias
• Reminder: Data maturity model
• Reminder: ML place
• Tools:
– Flayers philosopgy
– Mlboost
– Digipy (flayers+mlboost+opencv+swing)
– Mlboost for clustering
– Mlboost for Confusion Matrix
– Outliers visualization (semi supervised)
– Session visualization
– Notebook integration integration, pandasm theanets, sklearn, pytrade)
KISS & efficient startup bias
Francis Evolution in ML
• 2001-2003: Bengio lab: Plearn -> flayers (compete with torch & plearn)
• Industry (2005-…): Mlboost (numpy, sklearn, scipy, maplotlib) -> boost ML
project: extreme prototyping, preprocessing & feature extraction
• QMining -> Mlboost repackaging, Big data-Mining/ML infrastructure in aws
(95% infra/5% ML)
• Nuance -> Mlboots++ (clustering, advances preprocessing)
My python bias
Flayers options
flayers
MLboost
• MLboost: Machine Learning boost library
in Python. MLboost main goal is to
speedup any Machine Learning projects
by simplifying data preprocessing, features
selection and data visualisation. Design by
Machine Learning practitioners to let them
do ML...;)
Digipy (mlboost & flayers)
• http://fraka6.blogspot.com/2009/07/digipy-
011-hand-digit-real-time-demo-is.html
Mlboost visualisation
• http://fraka6.blogspot.com/2013/04/simplif
ying-clustering-visualization.html
Mlboost summary
• Numpy & scipy
• Sklearn (Machine learning)
• Pandas (timeseries & stocks access)
• Matplotlib (visualization)
• Argparse (options)
• Improvement/simplification
– Dimention Reduction
– Semi supervised visualization
– Session preprocessing & stats
QMining Techno layter
Data Maturity
Reminder: ML place
• ML place
QUESTIONS
francis@qmining.com
hum...

Contenu connexe

Similaire à ML_tools&libs-part1.pptx

Apache Spark's MLlib's Past Trajectory and new Directions
Apache Spark's MLlib's Past Trajectory and new DirectionsApache Spark's MLlib's Past Trajectory and new Directions
Apache Spark's MLlib's Past Trajectory and new DirectionsDatabricks
 
Apache Spark MLlib's Past Trajectory and New Directions with Joseph Bradley
Apache Spark MLlib's Past Trajectory and New Directions with Joseph BradleyApache Spark MLlib's Past Trajectory and New Directions with Joseph Bradley
Apache Spark MLlib's Past Trajectory and New Directions with Joseph BradleyDatabricks
 
3 python packages
3 python packages3 python packages
3 python packagesFEG
 
Building the Next Generation Teaching and Learning Environment
Building the Next Generation Teaching and Learning EnvironmentBuilding the Next Generation Teaching and Learning Environment
Building the Next Generation Teaching and Learning EnvironmentCharles Severance
 
AI and Spark - IBM Community AI Day
AI and Spark - IBM Community AI DayAI and Spark - IBM Community AI Day
AI and Spark - IBM Community AI DayNick Pentreath
 
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
 MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ... MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...Databricks
 
Data Engineering with Solr and Spark
Data Engineering with Solr and SparkData Engineering with Solr and Spark
Data Engineering with Solr and SparkLucidworks
 
Kubernetes data science and machine learning
Kubernetes data science and machine learningKubernetes data science and machine learning
Kubernetes data science and machine learningKublr
 
CaffeOnSpark: Deep Learning On Spark Cluster
CaffeOnSpark: Deep Learning On Spark ClusterCaffeOnSpark: Deep Learning On Spark Cluster
CaffeOnSpark: Deep Learning On Spark ClusterJen Aman
 
State of the Art Logging. Kibana4Solr is Here!
State of the Art Logging. Kibana4Solr is Here! State of the Art Logging. Kibana4Solr is Here!
State of the Art Logging. Kibana4Solr is Here! lucenerevolution
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...Jose Quesada (hiring)
 
Tuning ML Models: Scaling, Workflows, and Architecture
Tuning ML Models: Scaling, Workflows, and ArchitectureTuning ML Models: Scaling, Workflows, and Architecture
Tuning ML Models: Scaling, Workflows, and ArchitectureDatabricks
 
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's Data
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's DataFrom Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's Data
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's DataDatabricks
 
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI PipelinesTimothy Spann
 
Scalable Deep Learning on AWS with Apache MXNet
Scalable Deep Learning on AWS with Apache MXNetScalable Deep Learning on AWS with Apache MXNet
Scalable Deep Learning on AWS with Apache MXNetJulien SIMON
 
Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"Fwdays
 

Similaire à ML_tools&libs-part1.pptx (20)

Apache Spark's MLlib's Past Trajectory and new Directions
Apache Spark's MLlib's Past Trajectory and new DirectionsApache Spark's MLlib's Past Trajectory and new Directions
Apache Spark's MLlib's Past Trajectory and new Directions
 
Apache Spark MLlib's Past Trajectory and New Directions with Joseph Bradley
Apache Spark MLlib's Past Trajectory and New Directions with Joseph BradleyApache Spark MLlib's Past Trajectory and New Directions with Joseph Bradley
Apache Spark MLlib's Past Trajectory and New Directions with Joseph Bradley
 
3 python packages
3 python packages3 python packages
3 python packages
 
Building the Next Generation Teaching and Learning Environment
Building the Next Generation Teaching and Learning EnvironmentBuilding the Next Generation Teaching and Learning Environment
Building the Next Generation Teaching and Learning Environment
 
AI and Spark - IBM Community AI Day
AI and Spark - IBM Community AI DayAI and Spark - IBM Community AI Day
AI and Spark - IBM Community AI Day
 
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
 MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ... MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
MLflow: Infrastructure for a Complete Machine Learning Life Cycle with Mani ...
 
Python ml
Python mlPython ml
Python ml
 
Data Engineering with Solr and Spark
Data Engineering with Solr and SparkData Engineering with Solr and Spark
Data Engineering with Solr and Spark
 
Kubernetes data science and machine learning
Kubernetes data science and machine learningKubernetes data science and machine learning
Kubernetes data science and machine learning
 
CaffeOnSpark: Deep Learning On Spark Cluster
CaffeOnSpark: Deep Learning On Spark ClusterCaffeOnSpark: Deep Learning On Spark Cluster
CaffeOnSpark: Deep Learning On Spark Cluster
 
State of the Art Logging. Kibana4Solr is Here!
State of the Art Logging. Kibana4Solr is Here! State of the Art Logging. Kibana4Solr is Here!
State of the Art Logging. Kibana4Solr is Here!
 
PyData Boston 2013
PyData Boston 2013PyData Boston 2013
PyData Boston 2013
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
 
Tuning ML Models: Scaling, Workflows, and Architecture
Tuning ML Models: Scaling, Workflows, and ArchitectureTuning ML Models: Scaling, Workflows, and Architecture
Tuning ML Models: Scaling, Workflows, and Architecture
 
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's Data
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's DataFrom Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's Data
From Pandas to Koalas: Reducing Time-To-Insight for Virgin Hyperloop's Data
 
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
 
Textpy
TextpyTextpy
Textpy
 
Scalable Deep Learning on AWS with Apache MXNet
Scalable Deep Learning on AWS with Apache MXNetScalable Deep Learning on AWS with Apache MXNet
Scalable Deep Learning on AWS with Apache MXNet
 
MLOps with Kubeflow
MLOps with Kubeflow MLOps with Kubeflow
MLOps with Kubeflow
 
Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"
 

Plus de Francis Piéraut

4th industrial revolution fuel by combining big data and deeplearning a qui...
4th industrial revolution fuel by combining big data and deeplearning   a qui...4th industrial revolution fuel by combining big data and deeplearning   a qui...
4th industrial revolution fuel by combining big data and deeplearning a qui...Francis Piéraut
 
Startups ultime experience
Startups ultime experienceStartups ultime experience
Startups ultime experienceFrancis Piéraut
 
The ultimate trick to learn faster
The ultimate trick  to learn fasterThe ultimate trick  to learn faster
The ultimate trick to learn fasterFrancis Piéraut
 
Big data barrier of entry (flash)
Big data barrier of entry (flash) Big data barrier of entry (flash)
Big data barrier of entry (flash) Francis Piéraut
 
Big data: Just another barrier of entry
Big data: Just another barrier of entryBig data: Just another barrier of entry
Big data: Just another barrier of entryFrancis Piéraut
 
The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.Francis Piéraut
 
Appengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsAppengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsFrancis Piéraut
 
No BI without Machine Learning
No BI without Machine LearningNo BI without Machine Learning
No BI without Machine LearningFrancis Piéraut
 

Plus de Francis Piéraut (10)

4th industrial revolution fuel by combining big data and deeplearning a qui...
4th industrial revolution fuel by combining big data and deeplearning   a qui...4th industrial revolution fuel by combining big data and deeplearning   a qui...
4th industrial revolution fuel by combining big data and deeplearning a qui...
 
Startups ultime experience
Startups ultime experienceStartups ultime experience
Startups ultime experience
 
The ultimate trick to learn faster
The ultimate trick  to learn fasterThe ultimate trick  to learn faster
The ultimate trick to learn faster
 
Big data barrier of entry (flash)
Big data barrier of entry (flash) Big data barrier of entry (flash)
Big data barrier of entry (flash)
 
Big data trap
Big data trapBig data trap
Big data trap
 
Big data: Just another barrier of entry
Big data: Just another barrier of entryBig data: Just another barrier of entry
Big data: Just another barrier of entry
 
The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.
 
Appengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsAppengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startups
 
No BI without Machine Learning
No BI without Machine LearningNo BI without Machine Learning
No BI without Machine Learning
 
Java Empowered by Jython
Java Empowered by JythonJava Empowered by Jython
Java Empowered by Jython
 

ML_tools&libs-part1.pptx