1. Technology Overview
Chris Slaughter
President
Prof. Sriram Vishwanath
Chief Scientist

2. Our Team 2
Prof. Sriram Vishwanath
• 9 years, Prof., UT Austin
• Information Theory, Entrepreneurship
• 2 Robotics Labs: MA coordination and 3D
Perception
Chris Slaughter
• PhD. Candidate, Electrical Engineering
• 1.5 years consulting T.O. of Austin Startup
• Research Lead, UT Perception Laboratory
Ongoing Collaborations/Partnerships
• Lockheed Martin
• HKS
• NVIDIA
• UC Berkeley

3. Our Team 3
• Computer Vision: Multi-view geometry and stereo; tracking; 3d reconstruction
• High Performance Computing: General purpose graphics programming
(GPGPU), parallel/distributed computing, heterogeneous computation
• Statistics and Learning: Large scale clustering problems, compressive motion
analysis, graphical inference
• Embedded Systems: Board design, multi-processor
interaction, interfaces, power/weight/form factor

4. Mission Statement 4
To Teach Unmanned Vehicles to See as Humans Do
Applications:
• Absolute localization in GPS denied scenarios
• Visual tracking odometry
• Landmark detection and landmark-based navigation
• Terrain mapping and change detection for IED disposal
• Immersive visualization for situational awareness
• Disaster response

5. System Architecture 4
Server Nodes
Producer Nodes Consumer Nodes

6. System Architecture 5
Server Nodes
Producer Nodes Consumer Nodes

7. Producer Nodes

8. Producer Nodes 7
Computing Element:
Embedded GPGPU processor
ARM multi-core
Heterogeneous architecture
Optoelectronic Device:
Active Stereo (IR)
Passive Stereo
Laser / TOF

9. Producer Nodes 8
Key Features:
• Visual Odometry ( < 3 mm )
• Mapping
• Landmark Extraction
• High Data Rates ( 9.2M pts / sec )
• Volumetric / Manifold Reconstruction
• Source Compression for Uplink
Compatibility:
• Low power ( < 4.5 W )
• Low weight ( cell phone – battery )
• Low cost ( COTS sensors )

10. Producer Nodes 9
Signal Processing
• Fundamental task in vision-based
algorithms
• Most algorithms too slow even for
desktops
• Bilateral filters
• Pyramid computation
• Depth-RGB conversion
• SIFT descriptors
• Object recognition

11. Image Filtering

12. Pyramid Computation

13. Frame Matching

14. Producer Nodes 1
3
Visual Odometry
• RGB-based
• 30 FPS
• Accuracy in mm
• Compressive sensing solution
• Range-based
• 9.2M pts / sec
• 90T comps / sec
• Towards drift free?

15. Vision-Only Tracking

16. RANSAC (1983)

17. RANSAC (1983)
SOLO (2010)

18. Producer Nodes 1
7
Can a mobile device map 3D environments to produce dense
3D data points rather than sparse landmarks?
Dense Reconstruction
• Landmark-based mapping (1960’s – present) – SOLVED
• DARPA grand challenge
• Efficient global alignment (1999 – 2005) – SOLVED
• Multiple view geometry + dense reconstruction (1980’s – present)
• Live dense reconstruction (2006 – present)
• KinectFusion
• Range-based SLAM
• Dense Tracking and Mapping

19. Volumetric Reconstruction

20. Point Cloud Reconstruction

21. Patchwork Reconstruction

22. Depth Data

23. Producer Nodes 2
2
Patchwork Reconstructions
• More memory efficient than volumes
• Faster integration and tracking
• Efficient caching interplay
• Global refinement
• Runs on embedded device

24. Server Nodes

25. Server Nodes 2
4
Global Refinement
• Maps must be globally consistent
• Dense reconstruction doesn’t allow
for this refinement
• Patchwork generalizes to this
functionality
• Inherently multi-core problem
• ARM architecture for GraDeS

26. No Refinement
HOGMAN (2007)

27. Full Visualization 2
6
Custom Visualization
• Key task: visualization
• Interactivity (XBOX controller)
• Full support for 3d / 2d streams
• Event processing
• Interoperability with GPGPU (CUDA)
• Compression for consumer nodes
• Fully scalable
Rendering Pipeline
• Custom rendering pipeline for
Patchwork
• Volumetric and Point Cloud
• Raycasting with lighting sources

28. Consumer Nodes

29. Consumer Nodes 2
8
Our technology can produce maps at high speeds and unprecedented fidelities
But.. What to do with this content?
Visual Localization Situational Awareness
• Localization a major problem in GPS- • Visualize mapping assets in real time
denied scenarios from cell phones
• “Urban canyons” • Coordinate with server and receive
• Indoor environments compressed video stream
• MAV / UGV coordination • Back-end models dynamics of
• Existing solutions based mostly on adversaries
state estimation • Extensible visualizer: new
• Possible to query large maps for tags, models, data sources
location?

30. Visual Absolute Localization

31. Feasible Path Infeasible Path

32. Positional Decoding

33. Conclusion 3
2
Current trends in computer vision and robotics:
• High performance computing
• Live dense reconstruction
• Range-based tracking and mapping
Our architecture:
• Producer nodes:
• COTS sensors
• Commodity computational unit
• Dense tracking and mapping
• Server nodes
• Combine producer data into large maps
• Serve consumer nodes
• Consumer nodes
• Visual absolute localization and remote visualization

34. Thanks!