“A New AI Platform Architecture for the Smart Toys of the Future,” a Presentation from Xperi
Signaler
2 Aug 2022•0 j'aime•40 vues
“A New AI Platform Architecture for the Smart Toys of the Future,” a Presentation from Xperi
2 Aug 2022•0 j'aime•40 vues
Télécharger pour lire hors ligne
Signaler
Technologie
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/08/a-new-ai-platform-architecture-for-the-smart-toys-of-the-future-a-presentation-from-xperi/ Gabriel Costache, Senior R&D Director at Xperi, presents the “New AI Platform Architecture for the Smart Toys of the Future” tutorial at the May 2022 Embedded Vision Summit. From a parent’s perspective, toys should be safe, private, entertaining and educational, with the ability to adapt and grow with the child. For natural interaction, a toy must see, hear, feel and speak in a human-like manner. Thanks to AI, we can now deliver near-human accuracy on computer vision, speech recognition, speech synthesis and other human interaction tasks. However, these technologies require very high computation performance, making them difficult to implement at the edge with today’s typical hardware. Cloud computing is not attractive for toys, due to privacy risks and the importance of low latency for human-like interaction. Xperi has developed a dedicated platform capable of executing multiple AI-based tasks in parallel at the edge with very low power and size requirements, enabling toys to incorporate sophisticated AI-based perception and communication. In this talk, Costache introduces this platform, which includes all of the hardware components required for next-generation toys.
Recommandé
Building a Thought Controlled DroneJim McKeeth
NUS-ISS Learning Day 2019-Building IoT solutions with the PiInstitute of Systems Science, National University of Singapore
Demystifying AR_VR_MRPK Mishra
GHK AiLibrary White paper03Gordon Kraft
Smart Finger Data SheetIDEX ASA
gkraft White paper05Gordon Kraft
Contenu connexe
Similaire à “A New AI Platform Architecture for the Smart Toys of the Future,” a Presentation from Xperi
AiLIbrary White paper05Gordon Kraft
Intel APJ Enterprise Day - Synopses of Demos at Intel Collaboration CenterIntelAPAC
Intel APJ Enterprise Day - Synopses of Demos at Intel Collaboration CenterIntelAPAC
The AI Takeover in Hollywood by Yves BergquistData Con LA
google glassVipin Sudhakar
NUX Presentation from TechMixer Birmingham 2011Michael Heydt
Similaire à “A New AI Platform Architecture for the Smart Toys of the Future,” a Presentation from Xperi(20)
Plus de Edge AI and Vision Alliance
“Efficient Neuromorphic Computing with Dynamic Vision Sensor, Spiking Neural ...Edge AI and Vision Alliance
May 2023 Embedded Vision Summit Opening Remarks (May 23)Edge AI and Vision Alliance
“Frontiers in Perceptual AI: First-person Video and Multimodal Perception,” a...Edge AI and Vision Alliance
“3D Sensing: Market and Industry Update,” a Presentation from the Yole GroupEdge AI and Vision Alliance
“Open Standards Unleash Hardware Acceleration for Embedded Vision,” a Present...Edge AI and Vision Alliance
“Responsible AI: Tools and Frameworks for Developing AI Solutions,” a Present...Edge AI and Vision Alliance
Dernier
Webinar: Discover the Power of SpiraTeam - A Jira Alternative To Revolutioniz...Inflectra
Knowledge graph use cases in natural language generationElena Simperl
Data Formats: Reading and writing JSON – XML - YAMLCSUC - Consorci de Serveis Universitaris de Catalunya
Prompt Engineering - an Art, a Science, or your next Job Title?Maxim Salnikov
Metadata & Discovery Group Conference 2023 - Day 2CILIP MDG
Industry 4.0.pdfTery Lockitski
“A New AI Platform Architecture for the Smart Toys of the Future,” a Presentation from Xperi
- 1. A New AI Platform Architecture for the Smart Toys of the Future Gabriel Costache Senior R&D Director XPERI
- 2. 40+ offices worldwide headquarters in San Jose, CA $1.5B + market cap public company, trading under XPER 1,600 + employees worldwide 1,500 + engineers 11,000 + patent assets 100B+ devices worldwide empowered by technologies delivered via Xperi brands
- 3. • Safe • Secure • Private • Enhances child development • Uses natural interaction • Monitors child cognitive load • Develops with the child • Long battery life • Re-usable Ideal Smart Toy 3 © 2022 XPERI
- 4. Smart Toy Examples 4 © 2022 XPERI
- 5. Privacy Issues 5 2022 XPERI
- 6. • Data privacy • Safety • Battery life • Fast response • AI technologies for children • Data bias in AI • Natural interaction with children • Multimodal: audio, imaging, sensing Smart Toy Challenges 6 © 2022 XPERI DTIF (Disruptive Technology Innovation Fund) D.A.V.I.D
- 7. DAVID will develop a “privacy by design” AI platform, capable of multi-modal, ultra-low power consumption, “data center” level processing of audio and vision data on-device, without the need to transmit any personal data to the cloud. What DAVID will deliver to the smart toy market: • A platform for a wide range of learning and interactive applications in the toy market • A smart, trusted proof-of-concept toy using this platform that helps children learn and develop, using XPERI imaging technology, Perceive® Ergo® chip and SoapBox Labs speech technology capabilities in collaboration with the National University of Ireland, Galway. • Cloud-free capabilities to ensure privacy and wonderfully immersive user experiences for children of all abilities. DAVID – Data-center Audio/Video Intelligence on Device 7 © 2022 XPERI All-in-one Chip/Platform Designed for Privacy Multi-modal Platform Communication Speech, Expressions, Emotions, Gesture, Context and others..
- 8. • Perception • Imaging/Vision • Face Analytics • Body Analytics • Hand Analytics • Video Compression • Thermal Imaging • Audio • Wake Words / VAD • Speech2Text / ASR • Voice Analytics / Biometrics • Sensing AI Technologies to be Considered 8 © 2022 XPERI • Interaction • Visual • Audio • Text2Speech • Sound Generation • Others • Language Models / Conversational Models • Multi Modal Intent • Cognitive and Behaviour Analysis • Personalization • Interactive Games
- 9. Perceive® Ergo® AI Processor 9 Source: A Reuther et al. MIT Lincoln Laboratory Supercomputing Center-arXiv:2009.00993 Ergo* *Note: Ergo uses a proprietary representation. Ergo is not INT8. © 2022 XPERI
- 10. DAVID Platform Design 10 © 2022 XPERI
- 11. • Interfaces: - I2S (Tx, Rx), I2C (Tx, Rx) – (HUB and Ergo) - MIPI and Parallel (Ergo) - SPI & QSPI (HUB & Ergo) - GPIO (HUB and Ergo) - FTDI (JTAG, UART) (HUB) - WiFi/BT (HUB) - USB OTG (HUB) • Computation Units: - 3 x Ergo (55 TOPS/Watt + Arc CPUDSP) - HUB STM32 MCU (Arm M7) - ESP32 (2x Xtensa LX6) • Memory: - 16MB QSPI Flash (Ergo) - 128MB QSPI Flash + 32MB SRAM (HUB) - 448 KB ROM + 520 KB SRAM (ESP32) - SDCard (HUB) DAVID Platform Specifications 11 © 2022 XPERI
- 12. DAVID Toy PoC 12 © 2022 XPERI microphones camera Thermal LCDs PIR Speaker Contacts Wireless charging Boards, battery & sensors
- 13. Current Ergo Vision Application 13 © 2022 XPERI Face, Body & Hand Detection Facial Analytics FR CNN Face Alignment ERGO x, y, w, h, confidence, trackID Facial Landmarks Face Orientation Face Expression Face Embedding FR x1,y1, x2,y2 …. Tx, Ty, Rot, Scale x, y, w, h Body Analytics Body Landmarks/Skeleton Hand Analytics Hand Gestures Video Encoder Encoded stream 1 2 3 4 5 6 Example Ergo Application • Frame rate 30 fps • Resolution 320x320 • Power ~100 mW
- 14. Fully neural video encoder (Ergo) and decoder (generic) • Trained end-to-end • Custom stream – data privacy • Extra security can be added • Y only currently but can be easily extended to color • Enabler of other image enhancement technologies: colorization, super resolution • Can enable smart monitoring Video Encoding 14 © 2022 XPERI ERGO Video Encoder Camera MIPI/Parallel Stream Packing Hub Streaming App Video Decoder ONNX, TFLite, NNAPI Mobile App Decoded Frame Hub
- 15. • Current Ergo board 3 application Text2Speech -> spectrogram generation + vocoder • Focus on comprehension, less on naturalness • Next focus on: voice adaptation, voice cloning • Extend to sound/music generation Speech/Audio Neural Synthesis 15 © 2022 XPERI
- 16. powers magical and joyful experiences for kids using speech technology that is engaging, fun, and frictionless. PLAY DAVID Partners NUIG C3I - Center for Computational, Cognitive & Connected Imaging © 2022 XPERI 16
- 17. • Smart Toy requirements: • Privacy • Battery life • Multimodal interaction • Platform requirements: • Dedicated NN unit with very high OPs/W • Communication unit • Multiple sensor support • Generic processing unit • DAVID platform and toy PoC • Available Q3/Q4 2022 for selected partners Conclusions 17 © 2022 XPERI
- 18. Resources
- 19. • Xperi – www.Xperi.com • Perceive, Ergo – www.perceive.io • SoapBox Labs – www.soapboxlabs.com • C3I, National University of Ireland, Galway - www.nuigalway.ie/c3i • Disruptive Technologies Innovation Fund – DTIF • STMicroelectronics STM32 MCU • Espressif Systems ESP32 Resources © 2022 XPERI 19