Tutorial 5: Augmented Reality on Glass The Glass Class at HIT Lab NZ Learn how to program and develop for Google Glass. https://www.youtube.com/watch?v=smtvWeXLgJg&list=PLsIGb72j1WOlLFoJqkhyugDv-juTEAtas http://arforglass.org http://www.hitlabnz.org

1. The Glass Class:
AR on Glass
Feb 17th – 21st 2014
Mark Billinghurst, Gun Lee
HIT Lab NZ
University of Canterbury

2. http://www.arforglass.org

3. Demo
- NyARToolkit and Vuforia
- CityViewAR for Glass

4. Outdoor AR Framework
 Outdoor AR Visualization
 OpenGL ES based scene rendering
 GPS, inertial, compass
 Component based Framework
 Easily customizable UI
 Supports various styles of apps
 Android (+ Web Tools)
 Free download on our website
 http://www.hitlabnz.org/mobileAR

5. Outdoor AR Browsers
Wikitude Layar Junaio

6. SW Dev. Tools & Frameworks
 High-level content description
 AR Browsers
 Hard to customize app logic and UI
 Low-level functional modules
 Programming libraries
 Lack of higher-level abstraction of AR scene
and content (BYO)

7. Outdoor AR Framework
 High-level abstraction and low-level
programming support in a seamless way.
 Low-level functional modules
- Flexible and highly customizable
 Ready-to-use high level components
- Focus on building the interface, behavior and
content of the application rather than system
integration
 The high-level and low-level tools share the
common scene (or POI) data structure
- Both skilled and novice developers can work
together under the same framework.

8. Apps built with OAR Framework
[CityViewAR 2011] [CCDU 3D 2012]
[GeoBoids 3D 2012] [AntarcticAR 2012]

9. Outdoor AR Framework
 Covers full tier of ready-to-use
components, yet highly customizable
Mobile Device (Client)
Mobile OS
Mobile AR Application
Scene
data
Media
data
Mobile SW Library
Server
Server OS
Application Service
Scene
data
Media
data
Server SW Library
Data
Communication
(Optional)

10. Server Software and Tools
Web Client
(Browser)
ServerMobile Client
(App)
HTMLWeb-based
Authoring Tool
RESTful API
Scene
Database
AR View
Data
Manager
Local
cache
Javascript

11. Server Software and Tools
 Web-based Authoring Tool

12. Outdoor AR Library
Functional Component Layer
Service Component Layer
Map
Component
Scene Data Manager
AR
Component
List
Component
Tracking Data Manager
Scene Data
Structures
Tracking
Sensors
3D Graphics
Rendering
UI Views
Tools &
Utilities
3D
Sound
Map View AR View List ViewViews
Controllers
Models

13. Outdoor AR Library
 Customization by extending components
 Customized AR scenes
- Customize scene data structure
 Customized user interface
- Customize UI layout
- Customize UI behavior (e.g. onSceneSelected)
- Motion and touch gesture based interaction

14. Step-by-step Guide
 Setup the development environment
 Create an Android app
 Using the Outdoor AR Library in your app
 Adding outdoor AR scenes with 3D models
 Customizing UI
 Run on a Google Glass

15. Setup the dev environment
 Prerequisite
 Android Development Environment
- http://developer.android.com/sdk/index.html
- Eclipse, Android SDK
 Google Play Services Library (for maps)
 Download the Outdoor AR Library
 http://www.hitlabnz.org/mobileAR
- Download, Tutorials, Forum

16. Setup the dev environment
 Import the library and sample projects
 OutdoorARLibrary
 SampleOAComponent
 Dependency
 Google (not Android) API 4.1.2 (API Level 16)
 Google Play Services Library
- extras/google/google_play_services/libproject
 Android Support Library 4
- extras/android/compatibility/v4/
android-support-v4.jar

17. Create an Android app
 Create a new Android Application project
 Blank Activity
- “An activity is a single, focused thing that the user
can do.” – Android SDK API Reference
 Anatomy of an Android App project
 src
 res & gen
 assets
 AndroidManifest.xml

18. Using the OA Library in your app
 Add the Outdoor AR Library to your
project’s build path
 Add permissions in the manifest
 Add a new AR view (activity)
 Create a custom class inheriting
OAARComponentBase
 Add activity to the manifest
- Full screen landscape orientation
 Wire a button to start the AR view

19. Adding scenes with 3D models
 Preparing 3D model
 OBJ/MTL with JPEG or PNG textures
 OpenGL Coordinate frame
- X-right, Y-up, Z-out from screen
- North = -Z
 Placing the 3D model in the real world
 Latitude, Longitude
- http://itouchmap.com/latlong.html
 Optionally, elevation from the ground plane
 SketchUp – 3D model on Google Earth

20. Adding scenes with 3D models
 Adding 3D models to the project
 assets/OutdoorAR/3Dmodels/name/model.obj
 Adding scenes to the data manager
 Override onSetupScenes()
 Handling the scene selection event
 Override onSceneSelected()
 Using a mock location
 Override onCreate()
 Sensor manager’s enableMockLocation()

21. Loading and managing scenes
 Add scenes by code – difficult to manage
 Load from assets or local file storage
 OADataManagerAssets
 OADataManagerLocal
 SQLite or XML file
 SQLite Database Browser
 Default path
- (assets/)OutdoorAR/scenes.db
- (assets/)OutdoorAR/scenes.xml

22. Custom scene class
 OAScene
 Lat/Lon, 3D model, Name, Description,
Category
 Custom scene classes extend OAScene
 Override OADataManager’s loadScene()
to load in custom properties of your scene
 Cast OAScene to your custom class in
onSceneSelected() or onTouchedScene()
 Different scene classes can live together
- if(scene instanceof MyCustomScene)

23. Customizing UI
 Add layout resource
 Override setupUILayout()
 LayoutInflater - read in the layout res file
 addContentView() - overlay UI layout on top
of AR view
 findViewById() - get reference to the UI
elements
 onSceneSelected() vs. onTouchedScene()

24. Map and List views
 OAMapComponentBase, OAListComponentBase
 Similar class structure with OAARComponentBase
- Load scenes from a data manager
- setupXYZ() methods
- onSceneSelected() callback
 Map components needs API key
 Google Maps Android API v2
- OAGoogleMapsV2ComponentBase
 https://developers.google.com/maps/documentation/an
droid/start

25. Run your app on Google Glass
 Startup with AR
 Set AR view as a launcher activity in the app
manifest file
 Aim to select
 Use Timer to trigger touch events at the
center of the screen
 Override onResume() and onPause() to
start/stop the timer

26. Other tips for Google Glass
 Touch gestures
 onGenericMotionEvent()
 Swipe forward/backward/up
 Tap, double tap, tap and hold, etc.
 Head motion
 Add orientation listener to sensor manager
- getSensorManager().addSensorListener()
 Avoid conflicts with AR scene viewing motions
- Time-outs

27. Other tips for Google Glass
 GPS not on Glass
 Pair with Phone then use remote provider
 Stream location from a phone through
Bluetooth
 Mock location
 Turning off camera background
 Application dependent
 Override setupOptions()
 enableCameraBackground()
http://developer.android.com/guide/topics/connectivity/bluetooth.html

28. CityViewAR for Glass
Demo Revisited

29. More Information
 Website
 http://www.hitlabnz.org/mobileAR
 http://arforglass.org
 Gun Lee
 gun.lee@hitlabnz.org
 Mark Billinghurst
 mark.billinghurst@hitlabnz.org