Contenu connexe Similaire à Racing with Artificial Intelligence (20) Plus de Daniel Zivkovic (20) Racing with Artificial Intelligence1. Wednesday, July 31, 2019
1. Intro & Activity Update
2. Community Open Mic
3. Alex Barbosa Coqueiro - Public
Sector SA Manager @ AWS
Canada: “Racing with AI”
4. Networking
1
ServerlessToronto.org Meetup Agenda
2. Why we do what we do?
2
Serverless is New Agile
Serverless Dev (gluing
other people’s APIs
and managed services)
We're obsessed
helping Startups (and
creating meaningful
MVPs & products)
We build bridges
between Serverless
Community (“Dev leg”),
and Front-end & Voice-
First folks (“UX leg”),
and empower UX
developers
Achieve agility NOT by
“sprinting” faster (like in
Scrum), but by working
smarter (by using
bigger building blocks
and less Ops)
5. Manning Publications 2019 giveaways:
1. www.manning.com/books/serverless-applications-with-nodejs
2. www.manning.com/livevideo/production-ready-serverless
3. www.manning.com/livevideo/production-ready-serverless
4. www.manning.com/livevideo/serverless-applications-with-AWS
5. www.manning.com/livevideo/serverless-applications-with-AWS
6. www.manning.com/books/serverless-architectures-on-aws
7. www.manning.com/books/http2-in-action
8. www.manning.com/books/event-streams-in-action
9. www.manning.com/books/the-design-of-everyday-apis
10. www.manning.com/livevideo/graphql-in-motion
11. www.manning.com/books/voice-applications-for-alexa-and-google-assistant
12. www.manning.com/livevideo/machine-learning-for-mere-mortals
13. www.manning.com/books/classic-computer-science-problems-in-python
14. www.manning.com/books/getting-mean-with-mongo-express-angular-and-node
5Check out MEAP program from our Learning Sponsor!
7. Catering Sponsor
7
Also an active Serverless Community member – check out
their “Data Pipelines using Serverless Architectures” talk!
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Racing with Artificial Intelligence
Alex Coqueiro
Head of Public Sector Solutions Architecture for Canada, Latin America and Caribbean
AWS
@alexbcbr
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rubik’s cube challenge
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43,252,003,274,489,856,000
43 QUINTILLION
UNIQUE COMBINATIONS
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Don’t code the patterns, let the
system learn through data
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F2 U' R' L F2 R L' U'
ModelData
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F2 U' R' L F2 R L' U'
Confidence
1%
accuracy
R U r U R U2 r U2%
accuracy
Training Models
Model
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Confidence
20%
accuracy
40%
accuracy
60%
accuracy
80%
accuracy
95%
accuracy
2%
accuracy
Training Models
Model
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Confidence
95%
accuracy
?
F2 R F R′ B′ D F D′ B D F
Inference
Model
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SOLVED IN 0.9 SECONDS
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s apply it into the business
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://youtu.be/xC-tikvEvzo
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
10,000+ customers | 2x customer references | 85% of TensorFlow projects in the cloud happen on
AWS
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
LEA (Robot Care Systems)
https://youtu.be/r-US8rs8EY0
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
tuSimple (Autonomous Vehicle)
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Show me how to do it
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.Use Case – Autonomous Driving
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our problem re-formulation
30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Non-labeled training data
31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Autonomous Driving Development
32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robocar (Donkey Car Project)
Donkey Car Project
https://github.com/sunilmallya/donkey/tree/master/sagetrain
http://awsrobocar.s3-website-us-east-1.amazonaws.com/
https://github.com/tescal2/donkey
33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SSD MultiBox — Real-Time Object Detection +
Behavioral Cloning
34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale
1
2
3
35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Unsupervised
learning
Reinforcement
Learning
Non-labeled training data
36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reinforcement learning in the real world
Reward positive
behavior
Don’t reward
negative
behavior
The result!
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Build machine learning models in Amazon
SageMaker
• Train, test, and iterate on the track using the AWS
DeepRacer 3D racing simulator
• Compete in the world’s first global autonomous
racing league, to race for prizes and a chance to
advance to win the coveted AWS DeepRacer Cup
AWS DeepRacer
A fully autonomous 1/18th-scale race car designed to help you learn about
reinforcement learning through autonomous driving
38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic autonomous
race car
DeepRacer: An exciting way for developers to get hands-on experience with
Reinforcement Learning
Racing LeagueVirtual simulator, to
train and experiment
39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Track components
TRACK CENTER
TRACK WALL
TRACK SURFACE aka ON-TRACK
FIELD aka OFF-TRACK
TRACK BOUNDARIES
40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Action space
41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The reward function in a race grid
S G = 2
GOALAGENT
42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Incentivizing centerline behavior
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
S 2 2 2 2 2 2 G = 2
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
REWARD FUNCTION
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
MAX VALUE OF EACH STATE
AFTER LOTS OF EXPLORING
Discount per step
0.9
43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Programming your own reward function
44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s go deeper
Let’s go deeper…
45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepRacer Neural Network Architecture
An overview of the network architecture that AWS DeepRacer uses:
Output
46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon
Sagemaker RL
47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reinforcement Learning Algorithms Compared
Value Approximation Policy Approximation
Advantages
More stable performance when it works, and tends to
converge on global optimum
Effective in continuous action spaces, can learn stochastic policies,
and faster convergence
Disadvantages
Difficult to converge if too many (state, action)
combinations, slower convergence in general, and can’t
learn stochastic properties
Typically converges to a local rather than global optimum, high
variance in estimating the gradient adversely affects stability, and
evaluating a policy is generally inefficient
Examples Q-Learning, Deep Q Network, Deep Double Q Network Policy Gradient, Proximal Policy Optimization (PPO)
48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hyper parameters control the training algorithm
49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Coordinate system and track waypoints
OUTER BOUNDARY WAYPOINTS
TRACK CENTER WAYPOINTS
INNER BOUNDARY WAYPOINTS
X
Y
TRACK WIDTH
CAR DIRECTION
50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepRacer Car Specifications
CAR 18th scale 4WD with monster truck chassis
CPU Intel Atom™ Processor
MEMORY 4GB RAM
STORAGE 32GB (expandable)
WI-FI 802.11ac
CAMERA 4 MP camera with MJPEG
DRIVE BATTERY 7.4V/1100mAh lithium polymer
COMPUTE BATTERY 13600mAh USB-C PD
SENSORS Integrated accelerometer and gyroscope
PORTS 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI
SOFTWARE Ubuntu OS 16.04.3 LTS, Intel® OpenVINO™
toolkit, ROS Kinetic
51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic Arms
International Space Station
Drones
Education
Water
Home
Self-Driving Vehicles
Autonomous Walker
Rover
Robot landscape
52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotics trends
in 2018
Robotics is undergoing fundamental
change in collaboration, autonomous
mobility, and increasing intelligence
Source: IDTechEx
• Logistics
• Construction
• Retail
• Hospitality
• Healthcare
Robots are being put to work every
day across many industries
• Agriculture
• Energy Management
• Oil and Gas
• Facilities Management
• Household chores
By 2023, it’s estimated that mobile autonomous robots will
emerge as the standard for logistic and fulfillment processes
By 2030, 70% of all mobile material
handling equipment will be autonomous
53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic development cycle
2) Develop
robotics
application
1) Select robotics
software
framework
1) Deploy and
manage
application
3) Test and
simulate
application
New application release and update
54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Most widely used software framework for teaching and learning about robotics – over 16 million .deb (Linux Debian)
packages downloaded in 2018, a 400% increase since 2014
Founded in Stanford labs over 10 year ago, now managed by the Open Source Robotics Foundation (OSRF)
Global open-source community supports two products—Robot Operating System (ROS) and Gazebo
ROS
A set of software libraries and tools, from drivers to algorithms,
that help developers build robot applications
Gazebo
Robust physics engine, high-quality graphics, and programmatic
and graphical interfaces to help developers simulate robots
Robot Operating System (ROS) primer
55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing AWS RoboMaker
A service that makes it easy for
developers to develop, test, and
deploy robotics applications, as
well as build intelligent robotics
functions using cloud services
56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RoboMaker service suite
Development
Environment
SimulationCloud Extensions for
ROS
Fleet
Management
57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RoboMaker
Sample Robot Applications
Hello
World
Navigation
and Person
Recognition
Voice
Commands
Robot
Monitoring
Object-
following using
RL
Self-
driving
using RL
59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ROS Msg Node
Stored File
ROS Nodes
Model
Optimizer
Video
M-JPEG
Web Server
Video
Inference
Results
Web
Server
Publisher
Autonomous
Drive
Control
Node
Optimized
Model
Media engine
Camera
Model
Inference
engine
Manual
Drive
Navigation
Node
Servo & Motor
AWS DeepRacer Software Architecture
60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DATA
61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We are not spectators,
but actors of the future
Herb Simon,
2000
62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ml.aws
@alexbcbr
65. Upcoming Serverless Toronto Meetups
2
1. August 12, 2019: 1. A look at Google Cloud Functions
– Matt Welke // 2. Firebase Crash Course – Kudz Murefu
2. September 12, 2019: Serverless Design Patterns and
Best Practices – Mike Apted, Startup SA at AWS
3. October, 2019: Serverless CI/CD pipelines with AWS
CodePipeline and CodeBuild, vs CircleCI, vs Travis, vs
Seed – Frank Wang & Jay V
4. November or December: Serverless Heroes (authors
of “Serverless Applications with Node.js”) Slobodan
Stojanović & Aleksandar Simović will be here!
5. December or January, 2020: re:Invent recap –
Jonathan Dion, Senior Technical Evangelist at AWS