Day 2 of my workshop on hacking the Kinect.

1. OPEN FRAMEWORKS +
KINECT PART II
@GAFFTA
Thursday, March 7, 13

2. The Kinect for XBOX - released in
November 2010
Thursday, March 7, 13

3. why it matters
http://memo.tv/archive/kinect_why_it_matters
Affordable Depth Sensor
Calibrated Depth + RGB Sensor
Nothing new technically, just making it accessible to a MUCH
larger audience.
Thursday, March 7, 13

4. Microphone Array has advanced voice capabilities
Motorized tilt helps it to adjust to multiple environments
RGB camera for other data ( Facial Tracking, Color tracking )
Thursday, March 7, 13

5. what does the kinect see ?
http://www.youtube.com/watch?v=dTKlNGSH9Po
Laser Projector casts a grid of infrared dots
Infrared lens interprets the size of the dots as distance and turns
it into a “depth map”
This depth map makes it easy to separate parts of a scene
Computer learning is based off of these depth maps and large
data sets
Thursday, March 7, 13

6. Hack The Kinect Method #1
libfreenect is the driver / library
ofxKinect is the OF addon
https://github.com/ofTheo/ofxKinect
Simplest wrapper for kinect. ( also the first )
Good at getting raw image + point cloud data + motor control
Extremely useful if combined with openCV
cross platform - PC / Mac / Linux
Thursday, March 7, 13

7. What is OpenCV ?
Open Source Computer Vision Library
A library that allows for real-time image manipulation and feature
extraction. Including :
Gesture recognition
Facial recognition
2D + 3D features
Motion Tracking
Stereopsis
Originally developed by Intel Labs in 1999 for CPU intensive
programs.
Released to the public in 2006. Now developed by Willow Garage
Thursday, March 7, 13

8. OpenCV with a webcam
Thursday, March 7, 13

9. OpenCV with a webcam
First there is a color image from a camera.
That color image is converted to grayscale so that it can be
processed it faster. ( less channels = less data )
The background has a grayscale “snapshot” taken
A difference is applied to new frames after the background
snapshot
OpenCV ContourFinder returns: a shape with many points, a
bounding box, and centroid
Thursday, March 7, 13

10. OpenCV with a kinect
Thursday, March 7, 13

11. OpenCV with a kinect
There is already a grayscale depth feed, that can be
used to isolate a range using OpenCV Threshold
Running OpenCV contour finder on this thresholded
gray image gets us OpenCV blobs.
In addition to blob tracking we also get the 3d position
of each point in space. Also known as a “point cloud”
Thursday, March 7, 13

12. LET’S START CODING
https://github.com/benMcChesney/OF_Kinect_Tutorials
Thursday, March 7, 13

13. ofxKinect + ofxUI
Thursday, March 7, 13