Copy of the Presentation used at a talk at the State of North Carolina Undergraduate Research Symposium at UNC Charlotte in November 2013

1. Leveraging the Kinect SDK to
Control a Remote Device
Akhil Acharya and Sean Freemerman
Summer Ventures in Science and Mathematics
Appalachian State University

2. Computing Today

4. Objective
● Original research on novel interaction
techniques
● Develop new methodologies to interface the
Kinect Software Development Kit with an
Infrared (IR) transmitter
● Investigate the technology used by the
Kinect to accurately track body parts
● Determine the viability of the Kinect platform

5. Tools
● Kinect
○ Tracks movement
● IR Toy
○ Sends IR signals
● WinLIRC software package
○ "Middle man" between Kinect Application and IR toy
● 2 AirSwimmer Remote Controlled Balloons
○ System scalable to any IR device

6. Position Data
Kinect SDK Application
Pass command to
WinLIRC
IR signals
WinLIRC (Always Running)
Serial
information over
USB

8. Microsoft Kinect
●
●
●
●
Announced: 2009
Released: 2010
Full body motion controller for Xbox 360
Scatters multiple IR beams
○ Readings at discrete points
○ Saves on processing power
● Tracks depth by measuring depth of focus
● Kinect SDK allows developers to create
applications using Kinect

9. Why Kinect?
● Novel form of human-computer interaction
● Relatively cheap
○ $150 to get started
● Hands-Free
● 3D Capabilities

10. Position Data
Kinect SDK Application
Pass command to
WinLIRC
IR signals
WinLIRC (Always Running)
Serial
information
over USB

11. Application Design
● Built using example application "SkeletonBasics WPF"
● Communicates with the Kinect

12. Control System (Kinect)
● Relative distance measured
○ Distance between right shoulder and right hand
○ User doesn't need to stand in the center of the
Kinect's FOV
● Radius of 0.2 units - "null space"
○ Better differentiate commands
○ Space to rest hand without performing action

13. Position Data
Kinect SDK Application
System
Pass command to
WinLIRC
IR signals
WinLIRC (Always Running)
Serial
information
over USB

14. Control System (WinLIRC)
● If X and Y values exceed "null space"
boundaries, command is sent to WinLIRC
○ Done every 30 Frames (1 second)
■ Prevents WinLIRC from being overloaded
○ Commands
■ Left/Right (X Values)
■ Up/Down (Y Values)
● All commands defined as bytes in
AirSwimmers.cfg file.

16. AirSwimmer
● Two models
○ Shark ("Bruce")
○ Clownfish ("Nemo")
● Lightweight control system
○
○
○
○
Microcontroller with IR receiver
Weighted ballast
Servo
IR remote

17. Challenges
● Translating information
● Debugging
● Documentation

18. Results
● It works!
● Movement occurs in near real time
● Caveats:
○ Balloon requires line of sight
■ Limited movement capability
○ Remotes avoid this by having higher power LEDs

19. Next Steps
● Higher emission IR transmitter
● Full-on voice control
○ Partially implemented already
● Replicate with other motion control devices
○ Second Generation Kinect (Late 2013)
○ Leap Motion
● Untapped potential
○ Controlling non-IR devices
○ Potential to control any device

20. Position Data
Kinect SDK Application
Pass command to
WinLIRC
IR signals
WinLIRC (Always Running)
Serial
information over
USB

21. Position Data
Kinect SDK Application
Pass command to
WinLIRC
IR signals
WinLIRC (Always Running)
Serial
information over
USB

22. Position Data
Kinect SDK Application

23. Lessons
● Value of documentation
● Powerful Kinect API
● Potential of Gesture-based computing

24. Acknowledgements
● Mentors: Dr. Rahman Tashakkori, Mr. Luke
Rice, Ms. Bahar Akhtar
● Appalachian State University
● AirSwimmer and IR Toy donated by Dan
Thyer.
● UNC-Charlotte and the State of North
Carolina Undergraduate Research
Symposium

25. Thank You