Keynote at Interact 2011 conference, Lisbon, September 2011. More info at http://www.yvision.com/

1. Natural User Interfaces using AntónioCâmara CEO Antão Almada Director, Strategic Sofware Development

2. For the past ten years, YDreams has been researching and developing new ways for humans to communicate with machines.

3. The goal is to transcend traditional interfaces and create experiences that would allow a more natural interaction with computational systems.

4. We take inspiration from the natural everyday environments that surround us, the world of objects which can be touched, pulled and pushed, and which are imbued with meaning in an appropriate context.

5. What is YVision? A development framework for NUI applications. Developed in YDreams’ R&D department. Integrates 10 years of experience on NUI apps. Used internaly by YDreams’ production in more than 300 projects.

6. Camera-based interaction

7. Camera-based interaction Video “Virtual Garden” (http://www.youtube.com/watch?v=YCuPU6fVGCo)

8. Camera-based interaction Video camera is a very rich sensor. Video is a stream of pictures. “A picture is worth a thousand words”. How to extract usefull information? Colors Shapes Movements Faces Sensor fusion Gestures

9. Sensor data processing pipeline Detection Interpretation Broker Simulation Capture

10. Natural User Interfaces Short duration interactions. No time/willingness to learn new paradigms. Interaction must be based on prior knowledge. Shape, Color, Texture, etc. Behavior NUI applications should support: Realistic real-time rendering Physics simulation Artificial intelligence

11. Component-based architecture Each object is composed of several components. Components expose services. Services allow the change of state. Objects are structured into hierarchies. Objects have access to its ancestors services. Facilitates decoupling => extensibility.

12. Object Behaviors NUI raises “new” challenges for object behaviors The “user” should not be a collection of “cursors”. The “user” is a blob (2D or 3D). Steering behaviors should be used Interpolated paths (motion tweens) -> Path finding Hard connections -> Flocking

13. Behavior Tree Actions are encapsulated into behaviors. Behaviors can have children. Behaviors have access to services of the associated object and ancestor behaviors. Behaviors return on each iteration: Running Succeded Failed Standard behaviors Sequence Step Sequence Parallel Stocastic Selector Suspend Status Control

14. Behavior Trees “Behavior trees replace the often intangible growing mess of state transitions of finite state machines (FSMs) with a more restrictive but also more structured traversal defining approach.” BjoernKnafla http://bjoernknafla.com/introduction-to-behavior-trees

15. Behavior Tree of a bubble

17. Cases

18. Augmented Reality using depth-sensing cameras

19. Video “3D Augmented Reality experience with depth-sensing cameras” (http://www.vimeo.com/11160661)

20. Chameleon Virtual intelligent characters as tour guides in an augmented reality environment

21. Video “VicTour, Virtual Interactive Character Tour Guide” (http://vimeo.com/22415896)

22. Audience Entertainment 20,000 people playing at Lyon Festival of Lights

23. Video “AE, Orange and 20,000 People play at Lyon Festival of Lights Dec 5 2009” (http://www.youtube.com/watch?v=U33BqjtRJ58)

24. Santander Interactive Guest Assistants

25. Video “The making-of a visitors center using advanced robotics and interactive interface design” (http://www.vimeo.com/11655898)

26. Free download at http://www.yvision.com/