"Making Cozmo See," a Presentation from Anki
•
6 j'aime•2,696 vues
For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-stein For more information about embedded vision, please visit: http://www.embedded-vision.com Andrew Stein, Lead Computer Vision Engineer at Anki, presents the "Making Cozmo See" tutorial at the May 2017 Embedded Vision Summit. In this presentation, Stein describes the vision capabilities of Cozmo, Anki's latest consumer robotics product. Cozmo is a sophisticated entertainment robot focused on personality, interactivity, and game play. It was one of the hottest toys of 2016, earning a spot amongst the top two best-selling premium toys of the holiday season. In addition to covering high-level architecture and functionality, Stein discusses some of the challenges of building a capable vision system at a scale and price point for the consumer market.
![Copyright © 2017 Anki, Inc. 2
Introducing Anki & Cozmo
• About Anki [http://www.anki.com]
• Consumer Robotics & AI Company, currently focused
on entertainment
• Two Products: Overdrive and Cozmo
• About Cozmo [http://www.anki.com/en-us/cozmo]
• App-driven, autonomous robot character
• Launched Fall 2016, $179
• Full-featured SDK (low- and high-level API)](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Recommandé
Contenu connexe
Plus de Edge AI and Vision Alliance
Plus de Edge AI and Vision Alliance (20)
Dernier
Dernier (20)
"Making Cozmo See," a Presentation from Anki
- 1. Copyright © 2017 Anki, Inc. 1 Andrew Stein May 2017 Making Cozmo See
- 2. Copyright © 2017 Anki, Inc. 2 Introducing Anki & Cozmo • About Anki [http://www.anki.com] • Consumer Robotics & AI Company, currently focused on entertainment • Two Products: Overdrive and Cozmo • About Cozmo [http://www.anki.com/en-us/cozmo] • App-driven, autonomous robot character • Launched Fall 2016, $179 • Full-featured SDK (low- and high-level API)
- 3. Copyright © 2017 Anki, Inc. 3 • Primary Goal: Bring Cozmo to Life • All about personality, via animation • Interactive, aware About Cozmo
- 4. Copyright © 2017 Anki, Inc. 4 Cozmo’s Personality
- 5. Copyright © 2017 Anki, Inc. 5 • Outputs: • 4 Mechanical DoF: Head, Lift, Treaded Wheels • Speaker • OLED face • Sensors: • Camera (VGA, 15fps) • Cliff sensor • Encoders and IMU • Battery Life: 1-2 hours • 3 Processors: comms, sensors/motor control, supervisor Cozmo’s Core Hardware Capabilities
- 6. Copyright © 2017 Anki, Inc. 6 Simple Linear Flow Script Animation Library I wish!
- 7. Copyright © 2017 Anki, Inc. 7 Cozmo Flow Reaction Triggers Game Logic World Model Path Planning Sensors Emotional State Controlled Movement Animation Library Behaviors
- 8. Copyright © 2017 Anki, Inc. 8 • Cozmo moves • Motion blur • Must not drive off cliffs • Odometry not reliable (treads!) • Cozmo gets moved • “Robot Kidnapping” not just academic with six-year-olds! • How to maintain and communicate world state? • Cozmo interacts: Faces, cubes, charger, etc. Not a Static Surveillance Camera
- 9. Copyright © 2017 Anki, Inc. 9 Vision Capabilities
- 10. Copyright © 2017 Anki, Inc. 10 • Detect/track/recognize: cubes, charger, faces, motion, etc. • Robust enrollment for face recognition • Make eye contact • Estimate 3D pose of objects • Maintain map of the world (even if picked up) • Rolling shutter correction • Flexible scheduling according to needs and computational budget Vision Capabilities
- 11. Copyright © 2017 Anki, Inc. 11 • Store up to 10 identities across play sessions, enrolled via app activity • Remember 3D poses for eye contact Face Detection, Tracking, & Recognition
- 12. Copyright © 2017 Anki, Inc. 12 Cat & Dog Face Detection
- 13. Copyright © 2017 Anki, Inc. 13 Markers: Cubes, Charger, & Custom Objects (SDK) Custom Markers for SDK
- 14. Copyright © 2017 Anki, Inc. 15 Markers: Design vs. Vision Vs.
- 15. Copyright © 2017 Anki, Inc. 16 Detecting “Interesting Stuff”
- 16. Copyright © 2017 Anki, Inc. 17 System Architecture
- 17. Copyright © 2017 Anki, Inc. 18 • Use Smart Device for heavy computation • Reduce robot cost • Better development environment System Architecture UI + Game Logic Robotics Engine (Vision / AI) Hardware Drivers Low-Level Controllers Comms Python SDK CLAD via Wifi CLAD via USB CLAD: Custom Cross-Platform Messaging (similar to protobufs) CLAD (Local)
- 18. Copyright © 2017 Anki, Inc. 19 • Animations don’t play instantaneously (must buffer audio) • Face Enrollment in “Meet Cozmo” activity (limited visual feedback) • State Mirroring (from engine to Robot / Game / SDK) • Latency for vision and control Architecture Complications
- 19. Copyright © 2017 Anki, Inc. 21 Under the Hood of the Full System
- 20. Copyright © 2017 Anki, Inc. 22 Under the Hood (Manual Drive)
- 21. Copyright © 2017 Anki, Inc. 23 Under the Hood (Autonomous)
- 22. Copyright © 2017 Anki, Inc. 24 Computer Vision in a Product at Scale
- 23. Copyright © 2017 Anki, Inc. 25 Computer Vision is Not the Product Realistic Expectations Product Requirements Computer Vision
- 24. Copyright © 2017 Anki, Inc. 26 Computer Vision at Scale: Calibration Manipulating cubes requires high precision • Camera variability (lens placement) • Mechanical tolerances Must calibrate each camera at factory • Verify camera placement inside head • Estimate intrinsic parameters
- 25. Copyright © 2017 Anki, Inc. 27 • Cannot control lighting in people’s homes • Cube lights complicate local binarization Computer Vision at Scale: Illumination
- 26. Copyright © 2017 Anki, Inc. 28 • Detecting and responding to failures is as important as being successful Failure Detection
- 27. Copyright © 2017 Anki, Inc. 30 • Cozmo: interactive, complex, full of personality • Computer vision is only a piece of the product, does not drive all decisions • Computer vision at scale, in a consumer product, comes with special challenges and considerations • Setting expectations for what is (not) possible with vision is important in a world of sci-fi TV & movies • Cozmo is a fun little robot to develop for Conclusions
- 28. Copyright © 2017 Anki, Inc. 31 What Cozmo is All About
- 29. Copyright © 2017 Anki, Inc. 32 Join us! We’re hiring!