Slide deck from the first Robotics & Artificial Intelligence Virtual User Group, with theme "Computer Vision". Includes a presentation of Azure Kinect.
https://www.meetup.com/robotics-artificial-intelligence-meetup-group/
13. WHAT’S INSIDE
1 MP depth sensor with wide and narrow field-
of-view (FOV) options that help you optimise
for your application
7-microphone array for far-field speech and
sound capture
12 MP RGB video camera for an additional
colour stream that’s aligned to the depth
stream
Accelerometer and gyroscope (IMU) for sensor
orientation and spatial tracking
External sync pins to easily synchronise sensor
streams from multiple Kinect devices
1
2
3
4
5
15. COLOUR CAMERA + AZURE
COGNITIVE SERVICES
Computer Vision
Image classification
Object detection
Text extraction (OCR)
Face Recognition
Face detection
Face identification
Face matching
Azure Kinect is a cutting-edge spatial computing developer kit with sophisticated computer vision and speech models, advanced AI sensors and a range of powerful SDKs that can be connected to Azure cognitive services.
Using Azure Kinect, manufacturing, retail, healthcare and media enterprises are leveraging spatial data and context to enhance operational safety, increase performance, improve outcomes and revolutionise the customer experience.
Deliver improved safety and performance
Use advanced body tracking to monitor and analyse worker movement and behaviour. Design safer, more ergonomic workstations that can mitigate long-term injury risk while improving productivity through task analysis and process optimisation.
Pair real-time depth sensor data with cognitive services for AI-driven part identification and real-time anomaly detection. Monitor assembly line workers to catch part or process issues straight away for a just-in-time fix.
Gain new environmental understanding
Use depth sensing to automate depalletisation and palletisation processes. Streamline repetitive tasks such as stripping down a pallet to improve productivity while optimising storage space in smart ways.
Mobilise robot workers with spatial data. Automate new tasks by bringing robots into human spaces, from picking online orders and tracking shelf inventory to maintaining the spatial data of digital twin models.
Transform experiences and accelerate innovation
Use dimensional analysis to maximise transport storage space and optimise delivery logistics. Move from pre-defined delivery routes to real-time routes based on inventory analysis and subsequent last-mile efficiency.
Create a connected retail experience by mapping and monitoring retail stores in real time. Gain actionable customer insights and create alerts leveraging spatial data.
Reimagine communication and engagement
Add computer vision and speech capabilities to smart signage. Transform smart signage into interactive, personalised kiosks that can enhance brand engagement.
Use depth sensing to capture high-quality volumetric video for use in interactive 3D environments. Leverage 3D volumetric video for cutting-edge immersive storytelling, delivered through interactive XR experiences.
Develop safer solutions and strengthen engagement
Use computer vision to track and monitor exercise movements and overall form. Create a smart gym experience that can enhance results, or an accessible and effective virtual rehabilitation solution.
Pair real-time depth sensor data with AI-driven insights. Prevent and mitigate potential patient accidents and injuries in care environments with predictive alerts.
The Sensor SDK has the following features that work once installed and run on the Azure Kinect DK:
Depth camera access and mode control (a passive IR mode, plus wide and narrow field-of-view depth modes)
RGB camera access and control (for example, exposure and white balance)
Motion sensor (gyroscope and accelerometer) access
Synchronized Depth-RGB camera streaming with configurable delay between cameras
External device synchronization control with configurable delay offset between devices
Camera frame meta-data access for image resolution, timestamp, etc.
Device calibration data access
Azure Kinect DK is a developer kit with advanced AI sensors that provide sophisticated computer vision and speech models. Kinect contains a depth sensor, spatial microphone array with a video camera, and orientation sensor as an all in-one small device with multiple modes, options, and software development kits (SDKs).
The Azure Kinect DK development environment consists of the following multiple SDKs:
Sensor SDK for low-level sensor and device access.
Body Tracking SDK for tracking bodies in 3D.
Speech Cognitive Services SDK for enabling microphone access and Azure cloud-based speech services.
In addition, Cognitive Vision services can be used with the device RGB camera.
The Azure Kinect DK depth camera implements the Amplitude Modulated Continuous Wave (AMCW) Time-of-Flight (ToF) principle.
The camera casts modulated illumination in the near-IR (NIR) spectrum onto the scene. It then records an indirect measurement of the time it takes the light to travel from the camera to the scene and back.
These measurements are processed to generate a depth map. A depth map is a set of Z-coordinate values for every pixel of the image, measured in units of millimeters.
Along with a depth map, we also obtain a so-called clean IR reading. The value of pixels in the clean IR reading is proportional to the amount of light returned from the scene. The image looks similar to a regular IR image. The figure below shows an example depth map (left) and a corresponding clean IR image (right).
Azure Kinect body tracking can track multiple human bodies at the same time.
Joint coordinates
The position and orientation of each joint form its own joint coordinate system. All joint coordinate systems are absolute coordinate systems relative to the depth camera 3D coordinate system.
Joint hierarchy
A skeleton includes 32 joints with the joint hierarchy flowing from the center of the body to the extremities. Each connection (bone) links the parent joint with a child joint. The figure illustrates the joint locations and connection relative to the human body.