Cortana Analytics, de nouveaux patterns pour vos plateformes de données
•
1 j'aime•586 vues
--- Session donnée au Global Azure Bootcamp Paris 2016 --- Co-speaker : Fabien Adato Le monde de la donnée est en pleine mutation. Le Data devient Big, le SQL devient NoSQL, la BI devient Analytics, le Data Mining devient Data Science… Faux buzz ou vrais paradigmes, quoiqu'il en soit, de nouvelles architectures apparaissent pour traiter les challenges d'aujourd'hui. Microsoft propose ses solutions au travers de Cortana Analytics, veritable mashup des briques Data du Cloud Azure : Azure Data Catalog, Azure Stream Analytics, Azure Data Lake, Azure Data Factory, Azure ML, HDInsight, etc. Que font ces briques ? Lesquelles choisir ? Comment s'interconnectent-elles ? Quelle architecture implémenter ? Ce sont ces questions que nous aborderons lors de cette session.
Recommandé
Contenu connexe
Tendances
Tendances (19)
En vedette
En vedette (16)
Similaire à Cortana Analytics, de nouveaux patterns pour vos plateformes de données
Similaire à Cortana Analytics, de nouveaux patterns pour vos plateformes de données (20)
Plus de Jean-Pierre Riehl
Plus de Jean-Pierre Riehl (12)
Dernier
Dernier (11)
Cortana Analytics, de nouveaux patterns pour vos plateformes de données
- 1. http://GUSS.Pro @GUSS_FRANCE Cortana Analytics, de nouveaux patterns pour vos plateformes de données Jean-Pierre Riehl / Fabien Adato
- 2. http://GUSS.Pro @GUSS_FRANCE Rejoignez la communauté Webcasts, Conférences, Afterworks La communauté Data Microsoft @GUSS_FRANCE /GUSS /GUSS.FR Save the Date Samedi 25 juin 2016 Campus SUPINFO Tour Montparnasse, Paris
- 3. http://GUSS.Pro @GUSS_FRANCE Préconférence Cortana Analytics, une plate-forme de données moderne Vendredi 24 juin 2016 En préambule du SQLSaturday Paris Jean-Pierre Riehl – FR MVP DataPlatform depuis 2008 @djeepy1 Fabien Adato – FR Consultant Data Science @fabienAD
- 4. http://GUSS.Pro @GUSS_FRANCE Qui sommes nous ? Jean-Pierre Riehl Practice Manager Data & BI MVP SQL Server Chapter Leader– GUSS @djeepy1 http://blog.djeepy1.net Fabien Adato Consultant Data Science @fabienAD http://fadata-blog.com
- 5. http://GUSS.Pro @GUSS_FRANCE Get Model Analyze ? Store
- 6. http://GUSS.Pro @GUSS_FRANCE Nouvelles questions Nouveaux challenges Nouveaux outils Self-Service BI Agilité Big Data IoT
- 9. http://GUSS.Pro @GUSS_FRANCE Cortana Analytics Suite Transform data into intelligent action Business apps Custom apps Sensors and devices People Automated Systems Data Collection ToolsData Collection Tools
- 10. http://GUSS.Pro @GUSS_FRANCE Boite à outils « Cloud » HDInsight Azure Data Lake Azure SQL DWH HADOOP Azure DocumentDB Azure SQL IaaS Power BI Azure Data Catalog Azure Machine Learning Azure Data Factory Azure Stream Analytics
- 11. http://GUSS.Pro @GUSS_FRANCE HDInsight Azure Data Lake HADOOP Azure Machine Learning Azure Data Factory Azure Stream Analytics Boite à outils « Cloud »… …Une sélection
- 12. http://GUSS.Pro @GUSS_FRANCE HDInsight et l’écosystème HADOOP
- 13. http://GUSS.Pro @GUSS_FRANCE HDInsight : le monde Big DataHDInsight Azure Blob HDFS Sqoop Oozie Azure UX SDK Azure WebHcat/ Templeton RDP Yarn Hive Pig HCatalog AmbariMap/Reduce StormHBase HDInsight HADOOP
- 15. http://GUSS.Pro @GUSS_FRANCE Analytics Storage HDInsight (“managed clusters”) Azure Data Lake Analytics Azure Data Lake Storage Azure Data Lake U-SQL
- 16. http://GUSS.Pro @GUSS_FRANCE A hyper scale repository for big data analytic workloads Introducing Azure Data Lake • Hadoop File System compatible with HDFS™ • Integrated with HDInsight, Revolution R, Hortonworks, Cloudera • Based on YARN • Petabyte-sized files • No size limits to data in single account • Massive throughput to increase performance • AAD based access control • Data management Devices
- 18. http://GUSS.Pro @GUSS_FRANCE Azure Machine Learning
- 19. http://GUSS.Pro @GUSS_FRANCE Azure Machine Learning • Data Mining 2.0 – De nombreux modèles de classification, régression et clustering disponibles : Réseau de neurones, arbre de décision, Support vector machine… • Données en entrée – HDInsight (Hive) – Azure SQL – Azure Tables – Données locales (fichiers) Azure Machine Learning
- 20. http://GUSS.Pro @GUSS_FRANCE Azure Machine Learning • Support de R et Python • Requêtable par Web Service – Azure ML API • ML Studio Principe de la paillasse de laboratoire – Démarrage immédiat Azure Machine Learning
- 21. http://GUSS.Pro @GUSS_FRANCE AZURE MACHINE LEARNING
- 22. http://GUSS.Pro @GUSS_FRANCE Azure Data Factory
- 23. http://GUSS.Pro @GUSS_FRANCE Azure Data Factory Data Supply Chain • Ingest • Transform • Publish • Monitor Azure Data Factory Data Pipelines
- 24. http://GUSS.Pro @GUSS_FRANCE Azure Data Factory • Sources / Destinations – Données OnPrem (SQL Server) – Azure Blob / Azure Tables – SQL Azure • Transformation – Map/Reduce – Hive/Pig – Custom Activities : C# Azure Data Factory
- 25. http://GUSS.Pro @GUSS_FRANCE AZURE DATA FACTORY
- 26. http://GUSS.Pro @GUSS_FRANCE Azure Stream Analytics
- 27. http://GUSS.Pro @GUSS_FRANCE Azure Stream Analytics • Permet d’analyser des flux temps réel – Millions d’événements / seconde – 365 jours de rétention (ou 20To) • Basé sur Event Hub • Syntaxe SQL • Export vers SQL Azure/Blob/Power BI/ADL… Azure Stream Analytics SELECT Category, COUNT(*) FROM Input TIMESTAMP BY EntryTime GROUP BY Category, SlidingWindow(minute, 5)
- 28. http://GUSS.Pro @GUSS_FRANCE AZURE STREAM ANALYTICS
- 30. http://GUSS.Pro @GUSS_FRANCE Rise of Analytics...
- 31. http://GUSS.Pro @GUSS_FRANCE Skynet Paul le poulpe VS
- 32. http://GUSS.Pro @GUSS_FRANCE Intelligence Gallery https://gallery.cortanaintelligence.com/ Cognitive Services (ex: Project Oxford) https://www.microsoft.com/cognitive-services/ http://how-old.net/ - https://www.captionbot.ai/ Cortana Intelligence Services https://www.microsoft.com/en-us/server-cloud/cortana-intelligence-suite/ Deep Learning Framework http://blogs.microsoft.com/next/2016/01/25/microsoft-releases-cntk... Microsoft investit dans l’intelligence artificielle
- 34. http://GUSS.Pro @GUSS_FRANCE Cortana Analytics Suite Transform data into intelligent action Business apps Custom apps Sensors and devices People Automated Systems Data Collection ToolsData Collection Tools
- 35. http://GUSS.Pro @GUSS_FRANCE Un nouveau métier : Data Architect
Notes de l'éditeur
- Et tout cela dans un monde où tout va toujours de + en + vite time to market
- Cortana Analytics Suite delivers an end-to-end platform with integrated and comprehensive set of tools and services to help you build intelligent applications that let you easily take advantage of Advanced Analytics. First Cortana Analytics Suite provides services to bring data in, so that you can analyze it. It provides information management capabilities like Azure Data Factory so that you can pull data from any source (relational DB like SQL or non-relational ones like your Hadoop cluster) in an automated and scheduled way, while performing the necessary data transforms (like setting certain data colums as dates vs. currency etc). Think ETL (Extract, Transform, Load) in the cloud. Event hub does the same for IoT type ingestion of data that streams in from lots of end points. The data brought in then can be persisted in flexible big data storage services like Data Lake and Azure SQL DW. You can then use a wide range of analytics services from Azure ML to Azure HDInsight to Azure Stream Analytics to analyze the data that are stored in the big data storage. This means you can create analytics services and models specific to your business need (say real time demand forecasting). The resultant analytics services and models created by taking these steps can then be surfaced as interactive dashboards and visualizations via Power BI These same analytics services and models created can also be integrated into various different UI (web apps or mobile apps or rich client apps) as well as via integrations with Cortana, so end users can naturally interact with them via speech etc., and so that end users can get proactively be notified by Cortana if the analytics model finds a new anomaly (unusual growth in certain product purchases- in the case of real time demand forecasting example given above) or whatever deserves the attention of the business users.
- Cluster Hadoop Hbase NoSQL (ex : Key/Value Store) Storm Real-time, basé sur des streams (des spouts et des bolts)
- Cortana Analytics Suite delivers an end-to-end platform with integrated and comprehensive set of tools and services to help you build intelligent applications that let you easily take advantage of Advanced Analytics. First Cortana Analytics Suite provides services to bring data in, so that you can analyze it. It provides information management capabilities like Azure Data Factory so that you can pull data from any source (relational DB like SQL or non-relational ones like your Hadoop cluster) in an automated and scheduled way, while performing the necessary data transforms (like setting certain data colums as dates vs. currency etc). Think ETL (Extract, Transform, Load) in the cloud. Event hub does the same for IoT type ingestion of data that streams in from lots of end points. The data brought in then can be persisted in flexible big data storage services like Data Lake and Azure SQL DW. You can then use a wide range of analytics services from Azure ML to Azure HDInsight to Azure Stream Analytics to analyze the data that are stored in the big data storage. This means you can create analytics services and models specific to your business need (say real time demand forecasting). The resultant analytics services and models created by taking these steps can then be surfaced as interactive dashboards and visualizations via Power BI These same analytics services and models created can also be integrated into various different UI (web apps or mobile apps or rich client apps) as well as via integrations with Cortana, so end users can naturally interact with them via speech etc., and so that end users can get proactively be notified by Cortana if the analytics model finds a new anomaly (unusual growth in certain product purchases- in the case of real time demand forecasting example given above) or whatever deserves the attention of the business users.