Presentations used in the 2013 version for the RIAtec master session

3. Bruno Capuano
Innovation Manager
@elbruno
http://www.elbruno.co
m

4. I also work here
I live /
work
here
I was born here

9. IR Emitter Color Sensor
IR Depth Sensor
Tilt Motor
Microphone Array

11. • http://kinectforwindows.org

12. DEMO

16. DEMO

20. DEMO

22. D D D D D D D D D D D D D P P P
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

24. Feet 1.3’ 2.6’ 9.8’ 13.1’ 26.2’
Meters .4 .8 3 4 8
Default
Mode
Near
Mode
Unknown Normal

25. Feet 1.3’ 2.6’ 9.8’ 13.1’ 26.2’
Meters .4 .8 3 4 8
Default
Mode
Near
Mode
Unknown Too Near Normal Too Far

26. short[] allData = new short[depthFrame.PixelDataLength];
Get depth for a point (X:160, Y:120, Width: 320)
int index = (width * y) + x; // 38,560
int // 200
int //15
short depthPoint = allData[index];
int depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth; // 2,000
int player = depthPoint & DepthImageFrame.PlayerIndexBitmask; // 1

27. WORKING WITH DEPTH DATA
DEMO

29. Max of two skeletons per Kinect
6 Player Proposals

30. Joints in meters from camera Positive
Positive Negative
Negative

31. • Each joint has associated tracking state
• Tracked, Not tracked, or Inferred
• Inferred - Occluded, clipped, or low confidence joints
• Use TransformSmoothParameters to smooth joint
data to reduce jitter
Skeleton.ClippedEdges = FrameEdges.Top;
http://en.wikipedia.org/wiki/Holt-Winters

32. Fatigue Kills Use Coding4Fun ScaleTo
Gestures extension method

33. DEMO

35. -50 0 +50

38. <!-- Confirmation_Yes._value: string ["Yes"]
-->
<rule id="Confirmation_Yes" scope="public">
<example> yes </example> var grammar = new Choices();
<example> yes please </example> grammar.Add(“yes please");
<one-of>
<item> yes </item>
grammar.Add(“yes");
<item> yeah </item> grammar.Add(“yeah");
<item> yep </item> grammar.Add(“ok");
<item> ok </item>
</one-of>
<item repeat="0-1"> please </item>
<tag> out._value = "Yes";</tag>

39. Engagement model
Mic indicator for speech-enabled menus
Keyword to engage
Feedback and confirmation

40. DEMO

42. • Real-time tracking of multiple faces
• 2D Mesh and Points
• 3D Head Pose
• Animation Units

43. • 100 2D Points
• 87 in picture
• 13 additional for center
of eye, corners of mouth,
center of nose, head
bounding box

45. Neutral • Six Animation Units
• Uses Candide3
model
Jaw Lowerer
Lip Corner Depressor

46. DEMO

59. What we want to do with Kinect for
Windows
• Provide common Kinect UI interactions to developers
so that they can easily build Kinect apps
• Solve the most common app tasks and UI requirements
• E.g., engagement, targeting and selection, X/Y navigation
• Ensure that K4W interactions and controls are intuitive
and delightful for end users
• Advance the capabilities of Kinect
• Show progression!

60. Kinect interactions
Kinect for Windows Prototype controls
DEMO