This document discusses using synthetic data generated from 3D models and simulations to train AI models. It provides examples of tools that can be used to generate synthetic data like Unity Perception, NVIDIA Omniverse Replicator, Three.js, and Playwright. It also discusses using synthetic data with edge devices and the Azure Percept platform to perform AI and computer vision tasks locally without sending data to the cloud.

1. Synthetic Data:
From 3D model to
AI on the Edge

2. SHERRY LIST
SENIOR PROGRAM MANAGER @ MICROSOFT
@sherrrylst
www.linkedin.com/in/sherrylist/

3. GORAN VUKSIC
ENGINEERING MANAGER @ PANDORA
Goran works as an Engineering manager for Pandora, he is Microsoft AI
MVP, he has 15 years of work experience in IT and wide knowledge about
various technologies and programming languages. He worked on various
projects for notable clients and projects he worked on have been featured
many times on web sites like Forbes, The Next Web, NVIDIA Developer,
TechCrunch, Macworld and others. Goran is a tech enthusiast, he writes
technical blog posts, and he likes to share his wide knowledge on different
workshops and talks.
@gvuksic
www.linkedin.com/in/goranvuksic/
Talks about: #ai, #iot, #azure, #innovation, and #technology

4. Computer vision is an interdisciplinary scientific field that deals with how
computers can gain high-level understanding from digital images or videos.
Common tasks:
• Image classification
• Object detection
• OCR (Optical Character Recognition)
• Facial recognition
• Pose estimation
• Etc.
COMPUTER VISION

5. OBJECT
DETECTION

6. OBJECT
DETECTION

7. Easily customize your own state-of-the-art computer vision
models for your unique use case.
Custom Vision is designed to be customized for your
scenario; you need to provide the data to train your model.
The Custom Vision service is optimized to quickly recognize
major differences between images, so you can start
prototyping your model with a small amount of data. We
recommend starting with 50 images per label. Depending on
the complexity of the problem and degree of accuracy
required, hundreds or thousands of samples may be
required for your final model.
AZURE
CUSTOM VISION

8. WHERE’S CHEWIE?

9. CREATE PROJECT

10. HOW IT WORKS
Test model
Train model
Tag images
Prepare images

11. UPLOAD IMAGES

12. TAG IMAGES

13. TRAIN MODEL

14. TEST MODEL

15. HOW IT WORKS
Test model
Train model
Tag images
Prepare images

16. TEST MODEL

17. HOW IT WORKS
Test model
Train model
Tag images
Prepare images

18. Synthetic data generated from computer simulations or
algorithms provides an inexpensive alternative to real-
world data that’s increasingly used to create accurate AI
models.
Source: NVIDIA Blog – What is Synthetic Data?
https://blogs.nvidia.com/blog/2021/06/08/what-is-synthetic-data/

19. Photorealistic images generated from 3D models and used for
AI training can simulate real-world scenarios and remove many
burdens, such as privacy and regulatory concerns, lack of
training data, and more.
• Shorten the time for data collection and tagging
• Minimize costs for data collection
• Reduce the bias in your training data
• Get more accurate AI detections
BENEFITS OF
SYNTHETIC DATA

20. SYNTHETIC DATA IS THE
FUTURE OF AI
Source: Gartner Research - Maverick* Research: Forget About Your Real Data — Synthetic Data Is the Future of AI
https://www.gartner.com/en/documents/4002912-maverick-research-forget-about-your-real-data-synthetic-

21. The Perception package provides a toolkit for generating large-scale
datasets for computer vision training and validation.
UNITY PERCEPTION
Unity Perception package: https://github.com/Unity-Technologies/com.unity.perception

23. Omniverse Replicator is a simulation framework that produces physically
accurate synthetic data to accelerate training of deep neural networks for AI
applications. NVIDIA has created Omniverse Replicators for DRIVE Sim -
for training of AI perception networks for autonomous vehicles - and for
Isaac Sim, for training robots.
NVIDIA
OMNIVERSE
REPLICATOR

25. Are there some other ways to generate synthetic
data? Can we create it on our own
programmatically?

26. Three.js is a cross-browser JavaScript library and
application programming interface used to create and
display animated 3D computer graphics in a web browser
using WebGL.
The source code is hosted in a repository on GitHub.
Website: https://threejs.org/
GitHub: https://github.com/mrdoob/three.js/
THREE.JS

27. Playwright is a framework for Web Testing and Automation.
It allows testing Chromium, Firefox and WebKit with a single
API. Playwright is built to enable cross-browser web
automation that is ever-green, capable, reliable and fast.
• Cross-browser.
• Cross-platform.
• Cross-language.
• Test Mobile Web.
Website: https://playwright.dev/
GitHub: https://github.com/microsoft/playwright
PLAYWRIGHT

28. PLAYWRIGHT - CODE

29. R2D2 MODEL TRAINING

30. EXPORT MODEL
TensorFlow is an end-to-end open source platform for machine learning. It has a
comprehensive, flexible ecosystem of tools, libraries and community resources that lets
researchers push the state-of-the-art in ML and developers easily build and deploy ML
powered applications.

31. IoT scenarios in many verticals require a combination of
intelligence at the edge, and the power of cloud.
Instead of sending all the data to the cloud, the edge
device sensors process the data right at the source.
INTELLIGENCE
AT THE EDGE

32. Azure Percept is a family of hardware, software, and services designed to
accelerate business transformation using IoT and AI at the edge.
Start your proof of concept in minutes with hardware accelerators built to
integrate seamlessly with Azure AI and Azure IoT services.
Website: https://azure.microsoft.com/en-us/services/azure-percept/
AZURE PERCEPT

33. AZURE PERCEPT
DEVELOPMENT KIT

34. Azure Percept is a comprehensive, easy-to-use platform
with added security for creating edge AI solutions.
Start your proof of concept in minutes with hardware
accelerators built to integrate seamlessly with Azure AI and
Azure IoT services.
Azure Percept works out of the box with Azure Cognitive
Services, Azure Machine Learning, and other Azure
services to deliver vision and audio insights in real time.
AZURE
PERCEPT STUDIO

35. VISION MODELS

36. VOICE ASSISTANT TEMPLATES

37. R2D2 MODEL TESTING

38. R2D2 MODEL TESTING

39. syntheticAIdata

41. With syntheticAIdata you can generate synthetic datasets on a
large scale and accelerate vision AI model training. Generating
large synthetic datasets will bring great cost savings to your
projects, and it will reduce the risk of human error.
Synthetic data is generated with three simple steps:
1. Upload your 3D model
2. Configure options
3. Download generated data
LinkedIn: www.linkedin.com/company/syntheticaidata/
How it works
syntheticAIdata
Follow for updates