Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

[ Meetup - Cloudreach - Paris - France - 19/09/2019]

189 vues

Publié le

[ Meetup - Cloudreach - Paris - France - 19/09/2019]

Publié dans : Technologie
  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

[ Meetup - Cloudreach - Paris - France - 19/09/2019]

  1. 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. MDR Learning Julien Simon Global Evangelist, AI & Machine Learning, AWS @julsimon Machine Learning Deep Learning Reinforcement Learning
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Artificial Intelligence: design software applications which exhibit human-like behavior, e.g. speech, natural language processing, reasoning or intuition Machine Learning: using statistical algorithms, teach machines to learn from featurized data without being explicitly programmed Deep Learning: using neural networks, teach machines to learn from complex data where features cannot be explicitly expressed First things first
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Supervised learning • Run an algorithm on a labeled data set. • The model learns how to correctly predict the right answer. • Regression and classification are examples of supervised learning. Unsupervised learning • Run an algorithm on an unlabeled data set. • The model learns patterns and organizes samples accordingly. • Clustering and topic modeling are examples of unsupervised learning. Types of Machine Learning
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://fr.slideshare.net/JulienSIMON5/an-introduction-to-machine-learning-with-scikitlearn-october-2018
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. http://scikit-learn.org/stable/tutorial/machine_learning_map/
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://fr.slideshare.net/JulienSIMON5/an-introduction-to-deep-learning-84214689
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activation functionsThe neuron ! "#$ % xi ∗ wi + b = u ”Multiply and Accumulate” Source: Wikipedia bias
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = 2 0 … 4 m labels, N2 categories 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks Buildingasimpleclassifier
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = 2 0 … 4 m labels, N2 categories Total number of predictions Accuracy = Number of correct predictions 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks Buildingasimpleclassifier
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Weights are initialized at random à the initial network predicts at random f(X1) = Y’1 A loss function measures the difference between the real label Y1 and the predicted label Y’1 error = loss(Y1, Y’1) For a batch of samples: ! "#$ %&'() *"+, loss(Yi, Y’i) = batch error The purpose of the training process is to minimize error by gradually adjusting weights. Neural networks Buildingasimpleclassifier
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training a neural network Training data set Training Trained neural network Batch size Learning rate Number of epochs Hyper parameters Backpropagation Forward propagation
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  13. 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a dataset is not always an option Large, complex problems Uncertain, chaotic environments Continuous learning Supply chain management, HVAC systems, industrial robotics, autonomous vehicles, portfolio management, oil exploration, etc.
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Types of Machine Learning Reinforcement learning (RL) Supervised learning Unsupervised learning AMOUNT OF TRAINING DATA REQUIRED SOPHISTICATIONOFMLMODELS
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Learning without any data: we’ve all done it!
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reinforcement Learning An agent interacts with its environment. The agent receives positive or negative rewards for its actions: rewards are computed by a user-defined function which outputs a numeric representation of the actions that should be incentivized. By trying to maximize the accumulation of rewards, the agent learns an optimal strategy (aka policy) for decision making. Source: Wikipedia
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training a RL model 1. Formulate the problem: goal, environment, state, actions, reward 2. Define the environment: real-world or simulator? 3. Define the presets 4. Write the training code and the reward function 5. Train the model
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://github.com/awslabs/amazon-sagemaker-examples/tree/master/reinforcement_learning/rl_roboschool_ray
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The players Simulation environment RL agent Initial model
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. At first, the agent can’t even stand up
  21. 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Actions and observations Simulation environment Actions Observations RL agent Initial model
  22. 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The model learns through actions and observations
  23. 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Interactions generate training data Simulation environment Actions Observation RL agent Training Data Observation, action, reward Training
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training results in model updates Simulation environment Actions Observation RL agent Model updates Training data Observation, action, reward Training
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The agent learns to stand and step
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multiple training episodes improve learning Simulation environment Actions Observation RL agent Model updates Training data Observation, action, reward Training
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Making progress
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. RL agents try to maximize rewards
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Eventually, the model learns how to walk and run
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Evaluate and deploy trained models Simulation environment Actions Observation Trained Model RL agent Model updates Training data Training visualization Observation, action, reward Deploy Model Training
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robotics
  33. 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Financial portfolio management Objective Maximize the value of a financial portfolio STATE Current stock portfolio, price history ACTION Buy, sell stocks REWARD Positive when return is positive Negative when return is negative https://github.com/awslabs/amazon-sagemaker- examples/tree/master/reinforcement_learning/rl_portfolio_management_coach_customEnv Jiang, Zhengyao, Dixing Xu, and Jinjun Liang « A deep reinforcement learning framework for the financial portfolio management problem. » arXiv:1706.10059 (2017)
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compressing deep learning models Objective Compress model without losing accuracy STATE Layers ACTION Remove or shrink a layer REWARD A combination of compression ratio and accuracy. https://github.com/awslabs/amazon-sagemaker- examples/tree/master/reinforcement_learning/rl_network_compression_ray_custom Ashok, Anubhav, Nicholas Rhinehart, Fares Beainy, and Kris M. Kitani "N2N learning: network to network compression via policy gradient reinforcement learning." arXiv:1709.06030 (2017).
  35. 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Vehicle routing Objective Fulfill customer orders STATE Current location, distance from homes ACTION Accept, pick up, and deliver order REWARD Positive when we deliver on time Negative when we fail to deliver on time https://github.com/awslabs/amazon-sagemaker- examples/tree/master/reinforcement_learning/rl_traveling_salesman_vehicle_routing_coach
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Autonomous driving AWS DeepRacer 1/18th scale autonomous vehicle Amazon RoboMaker
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Getting started http://aws.amazon.com/free https://ml.aws https://aws.amazon.com/sagemaker https://github.com/awslabs/amazon-sagemaker-examples https://aws.amazon.com/blogs/aws/amazon-sagemaker-rl-managed-reinforcement- learning-with-amazon-sagemaker/ https://aws.amazon.com/deepracer/ https://medium.com/@julsimon
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://amzn.to/2mp1Lf5
  39. 39. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Julien Simon Global Evangelist, AI & Machine Learning, AWS @julsimon

×