Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Build, train and deploy Machine Learning models on Amazon SageMaker (May 2019)

492 vues

Publié le

Talk @ AWS Summit London with British Airways, 08/05/2019

Publié dans : Technologie
  • Harness the potential of Amazon Sagemaker to solve the overwhelming challenge of deploying machine learning models into production as an easily accessible and scalable API endpoint. http://bit.ly/2lVyvwc
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Build, train and deploy Machine Learning models on Amazon SageMaker (May 2019)

  1. 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Build, train and deploy Machine Learning models on Amazon SageMaker Julien Simon Global Evangelist, AI & Machine Learning @julsimon Nils Mohr Flight Data Programmer Analyst British Airways
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon SageMaker 1 2 3
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Build, train and deploy models using SageMaker Business Problem ML problem framing Data collection Data integration Data preparation and cleaning Data visualization and analysis Feature engineering Model training and parameter tuning Model evaluation Monitoring and debugging Model deployment Predictions Are business goals met? YESNO Dataaugmentation Feature augmentation Re-training Neo Elastic inference Ground Truth
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Aircraft Information Intelligence British Airways Predictive Maintenance Platform Nils Mohr Flight Data Programmer Analyst British Airways S U M M I T
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T British Airways • 290+ aircraft • 46 million+ passengers per year • 200+ destinations in 75+ countries S U M M I T
  6. 6. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What is flight data?
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Timeseries data
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Timeseries data
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Report data
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Our dataset
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Reasons for the development
  13. 13. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T High level architecture British Airways Storage in Amazon S3 AWS Cloud On prem software/tools Transfer into AWS
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Our AWS Architecture AWS Cloud Amazon VPC Storage in Amazon S3 Data optimization and cleaning on Amazon Fargate Config and Metadata store Amazon SQS Metadata generation on Amazon Fargate Amazon SQS Problem evaluation on AWS Lambda Amazon SQS
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Traditional Event detection AWS Cloud Amazon VPC Amazon SQS Config and Metadata store Lambda function checks config and sends SQS messages Lambda function runs custom/bespoke script and creates an alert if necessary Amazon SQS
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Event detection with Amazon SageMaker AWS Cloud Amazon VPC Amazon SQS Config and Metadata store Lambda function checks config and creates endpoints Amazon SageMaker models trained on specific features Amazon SageMaker endpoint Lambda function creates dataset for inference Amazon SQS Amazon SQS Lambda function evaluates data and create alerts if needed
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Lessons learned Understand the problem Ensure data quality Ask questions Apply the easiest solution
  19. 19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Where do we go from here? Aircraft Engineers Create a training job in Amazon SageMaker Resulting model
  21. 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Where do we go from here? Aircraft Engineers
  22. 22. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  23. 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Model options Training code Factorization Machines Linear Learner Principal Component Analysis K-Means Clustering XGBoost And more Built-in Algorithms (17) No ML coding required No infrastructure work required Distributed training Pipe mode Bring Your Own Container Full control, run anything! R, C++, etc. No infrastructure work required Built-in Frameworks Bring your own code: script mode Open source containers No infrastructure work required Distributed training Pipe mode
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Built-in Deep Learning frameworks: just add your code • Built-in containers for training and prediction. • Code available on Github, e.g. https://github.com/aws/sagemaker-tensorflow-containers • Build them, run them on your own machine, customize them, etc. • Script mode: use the same code as on your laptop No infrastructure work required: simply define instance type and instance count Distributed training out of the box: zero setup Pipe mode: stream infinitely large datasets directly from Amazon S3
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS: The platform of choice to run TensorFlow 85% of all TensorFlow workloads in the cloud runs on AWS
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Training ResNet-50 with the ImageNet dataset using our optimized build of Tensorflow 1.11 on a c5.18xlarge instance type is 11x faster than training on the stock binaries. Optimizing Tensorflow for Amazon EC2 instances C5 instances (Intel Skylake) 65% 90% P3 instances (NVIDIA V100)
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Demo: Keras+Tensorflow Script mode Automatic model tuning Elastic inference https://gitlab.com/juliensimon/dlnotebooks/tree/master/keras/04-fashion- mnist-sagemaker-advanced
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T ApacheMXNet:DeepLearningforenterprisedevelopers • Gluon CV Gluon NLP ONNX 2x faster Java Scala
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Demo: Gluon CV State of the art models in just a few lines of code https://gluon-cv.mxnet.io/
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Getting started http://aws.amazon.com/free https://aws.amazon.com/sagemaker https://github.com/aws/sagemaker-python-sdk https://github.com/awslabs/amazon-sagemaker-examples https://medium.com/@julsimon https://gitlab.com/juliensimon/dlnotebooks
  31. 31. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Julien Simon Global Evangelist, AI and Machine Learning @julsimon
  32. 32. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×