Embodied Cognition - Booch HICSS50

2. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 2 Imagine unleashing Watson in the physical world. Give it eyes, ears, and touch, then let it act in that world with hands and feet and a face, not just as an action of force but also as an action of influence. This is embodied cognition: by placing the cognitive power of Watson in a robot, in an avatar, an object in your hand, or even in the walls of an operating room, conference room, or spacecraft, we take Watson's ability to understand and reason and draw it closer to the natural ways in which humans live and work. In so doing, we augment individual human senses and abilities, giving Watson the ability see a patient's complete medical condition, feel the flow of a supply chain, or orchestrate the tasks in a day in the life of an individual.

3. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 3 Embodied Cognition Defined § An embodied cognition must – Be in and of the world – Reason – Learn – Have identity § An embodied cognition is not – Simply an STT -> NLC -> action pipeline

4. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 4 Robotics Raw curiosity More computational power Advances in AI algorithms and architectures Better/smaller sensors and actuators Better batteries 3D printing Advances in neural implants New military doctrine Demographic changes Economic need Advances in neural implants Technology push Demand pull Raw curiosity

13. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 13 Thin Client Thick Client By way of context, Watson is largely a symbolically- organized thin client cognitive system Biologically- organized Symbolically- organized

17. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 17 Embodied Cognition 17 Use Cases Concierge (Robot, Avatar, Space) Where is the elevator? Retail (Robot, Avatar, Space) Do you wanna build a snowman? Elder Care (Robot, Avatar, Space) I’ve fallen and I can’t get up! Cobot (Robot) Get me a screwdriver. Manufacturing (Robot) Watch me do this. Transportation (Robot, Avatar, Space) Open the pod bay doors, Watson. Boardroom (Avatar, Space) Help me decide. Companion (Avatar, Device) Let’s play a game.

18. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 18 Self 18 Sensors & Percep,on Actuators Agents Goals & Planning Pla5orm Management Infrastructure Meta Management Models Voluntary Behavior & Skills Involuntary Behavior & Reﬂexes Others World Self Architecture

19. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 19 Points of Technical Confluence 19 Collaboration Who said that? Sensory Fusion I feel sad. Theory of Mind I need the Phillips head screwdriver. Context Give me that one; no, I mean that one! Embodied Conversation It’s good to see you again, Sandia! Learning I’ve just picked up a fault in the AE35 unit. Devices In what room was Alyssa working yesterday? Knowledge Representation

20. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 20 Embodied Cognition § Team ‒ Leader: Grady Booch; Co-Leader: Jeff Kephart ‒ Key Global Team Members: Sam Adams, Rachel Bellamy, Ray Chancey, Jon Connell, Stefan Hamer, Kohichi Kajitani, Sky Matthews, Nat Mills, Robert Moore, Anton Riabov, Francesca Rossi, Martin Rufli, Ryuki Tachibana, Hironobu Takagi, John Vergo § Theme ‒ Deliver a platform that makes the cognitive power of Watson manifest in the real world with sensors and actuators, serving as an embodied persona that orchestrates the agency of that system to augment and collaborate with humans using visual, vocal, and tactile forms of communication ‒ Deploy this platform in the form of robots, avatars, spaces, and objects ‒ Focus on higher levels of reasoning and learning ‒ Enable pluggable agents, goals, plans, skills, reflexes, classifiers, and models so as to facilitate a vibrant developer and solution ecosystem § Short Term Goal (one year) ‒ Go to market with the Watson Group and Watson IoT, delivering the Self kernel, services, and examplar solutions. § Long Term Goal (three years) ‒ Become the de facto industry standard for higher order embodied cognitive solutions § Gaps & Issues ‒ There’s more work to be done than IBM has skills or resources to do alone ‒ Maniacal focus is essential: there are many moving parts that cut across organizations boundaries 20 Executive Summary Sensors & Percep,on Actuators Agents Goals & Planning Pla5orm Management Infrastructure Meta Management Models Voluntary Behavior & Skills Involuntary Behavior & Reﬂexes Others World Self Self

21. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 21 Embodied Cognition Core Technology Topic Project Name Contact/Lab ~ Team Size Platform Self Booch/Almaden & Chancey/Austin 1 + 4 Goals & Planning Planning Riabov/Yorktown 2 Embodied Action Moore/Almaden 1 Multi-modal Contextual Interaction Bellamy/Yorktown 5 Embodied Decision Support Systems Rossi/Yorktown 5 Models of the World, Others, and Self Spatial Intelligence Rufli/Zurich 4 People Intelligence Hoory/Haifa 5 Spatial Awareness Kephart/Yorktown 4 Theory of Mind TBD Learning Learning by Example Connell/Yorktown 2 Sensors & Perception/Actuators Physical Interaction Tachibana/Tokyo 5 Edge Intelligence Hamer/Australia 2 Platform & Meta Management Tooling TBD 21 Solution Area Project Contact Spaces Watson for Buildings/Watson for Retail TBD/cross-laboratory Robots, Avatars, & Spaces Watson Labs Chancey/Austin Robots & Spaces Watson IoT Matthews/Durham Robots Industrial Robots Kajitani/Tokyo Spaces Elder Care Takagi/Tokyo & Adams/Raleigh Spaces M&A Vergo/Yorktown Spaces Boson Mills/Cambridge Objects Cognitive Objects Bellamy/Yorktown Project Portfolio

22. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 22 Embodied Cognition 22 Collaborators Solution Area Project Contact Robots Cooperative Robots Carnegie Mellon University Spaces Multimodal Interaction Rensselaer Polytechnic Institute Avatars Baby/X University of Auckland Robots TBD University of California, San Diego Robots TBD University of California, Berkeley Robots TBD Massachusetts Institute of Technology Robots TBD Stanford Research Institute

23. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 23 Background § In November 2015, Rob High, Ray Chancey, and I were brainstorming about a suitable grand challenge for the 2016 Embodied Cognition GTO. § We settled on a very grand one: build a system to support mission control, flight operations, and the scientific agenda of NASA’s Mission to Mars; the requirements of this mission lead to the concept of Watson in the walls (an embodied cognition for the Orion spacecraft and the Mars habitat) together with robots and other devices powered by Watson (namely, Robonaut and its peers). § Recognizing this was a 20+ year research effort, I set out to devise an architecture that would not only support this Mission to Mars but the journey of which would yield a middleware that would attend to a variety of use cases, from concierge to retail to elder care and many others. § Ray named this architecture “Self”.

24. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 24 Self Basic Principles § Augment human capabilities § Learn, don’t program § Theory of mind and social intelligence § Self understanding § Platform agnostic § Embodiment as a robot, an avatar, a device, or a space § Deployment as middleware with microservices in the cloud

25. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 25 Self Significant Design Decisions § Self is a hybrid architecture, encompassing explicit symbolic computation in the center together with neural networks at the edges. § Inspired by Minsky’s Society of Mind, behavior takes place in the context of multiple concurrent agents that communicate opportunistically via blackboards and deterministically via peer to peer connections. § Inspired by Brooks’s subsumption architecture, behavior takes place in a hierarchy of cognition, from involuntary reflexes to voluntary skills to goals and planning. § We maintain a clear separation of concerns among perception, actuating, models, and behavior. § As much as possible, behavior is either taught or is learned, not programmed. § As much as possible – driven by these separation of concerns, the needs of packaging, and performance – all components are made manifest as RESTful microservices. § As much as possible, plans, skills, and reflexes are extensible. § Self is intentionally full of strange loops: components of Self are also parts of the models of itself. § Self is intentionally fractal: an instance of Self may have models of others, which themselves are other instances of Self.

26. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 26 Self Markitecture IBM Confidential 26 Sensors & Perception Actuators Agents Goals & Planning Platform Management Infrastructure Meta Management Models Voluntary Behavior & Skills Involuntary Behavior & Reflexes Others World Self

32. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 32 Text + Audio This is the current state of an embodied Watson (the low hanging fruit, wherein platforms such as Pepper serve as animated speakers)

33. IBM Confidential | Do Not Distribute © 2016 IBM Corporation 33 Text + Audio There is much, much more that is possible (if we extent the embodiment to encompass greater perception and action in the world)

Embodied Cognition - Booch HICSS50

Vous êtes en train de lire un aperçu.

Consultez-les par la suite

Embodied Cognition - Booch HICSS50

Plus De Contenu Connexe

Diaporamas pour vous (20)

Les utilisateurs ont également aimé (9)

Similaire à Embodied Cognition - Booch HICSS50 (20)

Plus par diannepatricia (20)

Plus récents (20)