Tutorial 2: Mirror API The Glass Class at HIT Lab NZ Learn how to program and develop for Google Glass. https://www.youtube.com/watch?v=nml8qE6SF9k&list=PLsIGb72j1WOlLFoJqkhyugDv-juTEAtas http://arforglass.org http://www.hitlabnz.org

1. The Glass Class: Mirror API
July 7th – 11th 2014
Mark Billinghurst, Gun Lee
HIT Lab NZ
University of Canterbury

2. An Introduction to
Glassware Development
Mirror API
using Java Servlet
Gun Lee
* Images in the slides are from variety of sources,
including http://developer.android.com and http://developers.google.com/glass

3. Glassware Development
 Mirror API
 Server programming, online/web application
 Static cards / timeline management
 GDK
 Android programming, Java (+ C/C++)
 Live cards & Immersions
 https://developers.google.com/glass/

4. Mirror API
 REST API
 Java servlet, PHP, Go,
Python, Ruby, .NET
 Timeline based apps
 Static cards
- Text, HTML, media attachment (image & video)
- Standard and custom menu items
 Manage timeline
- Subscribe to timeline notifications
- Sharing with contacts
- Location based services
https://developers.google.com/glass/develop/mirror/index

5. Mirror API based Web App
3. Insert a static card
User sees the card
Glassware Web app
https://developers.google.com/glass/develop/mirror/stories

6. Develop with Mirror API
 Prepare a web server
 https callback required for OAuth 2.0
 Minimum storage for credentials
 Create a Google APIs Console project
 Enable Mirror API and get Client ID & secret
 Create a web application
 Java servlet, PHP, Go, Python, Ruby, .NET
 Implement OAuth 2.0 authorization
 Use Mirror API to make REST API calls
- Wrapper classes/methods provided

7. Google App Engine
 Free web server space!
 You can skip this step if you have another server
 Upgrade to paid service based on storage and traffic
needed
 https://developers.google.com/appengine/
 Python, Java, PHP, Go
 Cloud Storage
 Easy integration with Google APIs

8. Google APIs Console
 http://console.developers.google.com
 Login with your own Google account
 Create a New Project
 The Project ID becomes your URL
- http://your-project-id.appspot.com

9. Google APIs Console
 In the project’s APIs & auth section,
enable Google Mirror API

10. Google APIs Console
 APIs & auth > Credentials > OAuth: Create
New Client ID
 Choose "Web Application"
 Type in your website and OAuth callback
address
 https://your-app-id.appspot.com
 https://your-app-id.appspot.com/oauth2callback

11. Google APIs Console
 Note both Client ID and Client secret

12. Google APIs Console
 APIs & auth > Consent screen
 Fill in the Product Name and Homepage URL
 This information is shown when users
authorizing your web app

13. Live Demo
- Setup Google App Engine
- Setup Google APIs Console

14. Dev. Env. Setup
 ADT Bundle (Eclipse + Andriod SDK)
 http://developer.android.com/sdk/index.html
 GAE plugin for Eclipse
 http://dl.google.com/eclipse/plugin/4.2
- Google App Engine Tools for Android
- Google Plugin for Eclipse
- SDK > Google App Engine Java SDK
 Sign in on Eclipse GAE plugin
 Use the Google account that will host your
web app on GAE

15. Create a Web App for GAE
 File > New > (Other > Google > )
Web Application Project
 Fill in the Project name and Java package
namespace to use
 Check on option Use Google App Engine
 Uncheck option Use Google Web Toolkit
 Select option Use App Id then click Browse
button to choose your app id
 Check on Generate project sample code
 Click Finish

16. Anatomy of a Web App Project
 Web App Project
 src (source folder)
- Java classes organized in namespaces
 war (web application archive)
- WEB-INF (settings and JAR libraries)
• appengine-web.xml
• web.xml
• lib
- index.html
- favicon.ico

17. Deploying to GAE
 Right click on the Project
 Google > Deploy to App Engine
 Uncheck option Launch app in browser ...
 Click Deploy
 Open your GAE site on a browser
 http://your-app-id.appspot.com

18. Live Demo
- Creating a new Web App project
- Anatomy of a Web App project
- Deploy to GAE

19. Adding Mirror API library
 Right click on the Project
 Google > Add Google APIs
 Choose Google Mirror API
 Click Finish

20. Authorization with OAuth 2.0
 Authorizing your web app to access timeline on user’s
Google account
Programming Google Glass - The Mirror API by Eric Redmond
http://pragprog.com/book/erpgg/programming-google-glass

21. Authorization with OAuth 2.0
 Enable HTTP sessions on GAE
 Force https access only
 Filter out inappropriate access
 Implement OAuth 2.0 callback servlets
 Code from Quick Start sample project
- https://developers.google.com/glass/develop/mirror
/quickstart/index
 Add a new servlet to logout
https://developers.google.com/glass/develop/mirror/authorization

22. Enable sessions on GAE
 war/WEB-INF/appengine-web.xml
<appengine-web-app ...>
...
<sessions-enabled>true</sessions-enabled>
</appengine-web-app>

23. Force https access only
 war/WEB-INF/web.xml
<web-app ...>
...
<!-- force https only -->
<security-constraint>
<web-resource-collection>
<web-resource-name>Protected Area</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
...
</web-app>

24. Define Auth Filters
 war/WEB-INF/web.xml
<web-app ...>
...
<!-- filters -->
<filter>
<filter-name>authFilter</filter-name>
<filter-class>com.google.glassware.AuthFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>authFilter</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
<filter>
<filter-name>reauthFilter</filter-name>
<filter-class>com.google.glassware.ReauthFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>reauthFilter</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
</web-app>

25. Define Auth Servlet
 war/WEB-INF/web.xml
<webapp ...>
…
<servlet>
<servlet-name>oauth2callback</servlet-name>
<servlet-class>com.google.glassware.AuthServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>oauth2callback</servlet-name>
<url-pattern>/oauth2callback</url-pattern>
</servlet-mapping>
...
</webapp>

26. Add Java classes for Auth
 Create com.google.glassware package in
the src folder
 Right click on src folder > New > Package
 Copy Java classes for Auth
 Drag and drop the following .java files into the
created package
 AuthFilter.java, AuthServlet.java,
AuthSettings.java, AuthUtil.java,
ReauthFilter.java, WebUtil.java

27. Add Java classes for Auth
 In AuthSettings.java
 Fill in your CLIENT_ID and CLIENT_SECRET
strings you’ve noted from Google APIs
Console’s credential section
public static String CLIENT_ID =
"567615950876-14k9b9sggrtpulhu9s72le4vsejjscak.apps.googleusercontent.com";
public static String CLIENT_SECRET = "lo9hajhpQFneXP5K8YR0gEVK";

28. Add a New Servlet for Logout
 Add LogoutServlet class
 Right click on your app package in src folder
 New > Class
 Name: LogoutServlet
 Superclass: javax.servlet.http.HttpServlet

29. Add a New Servlet for Logout
 Implement LogoutServlet class
 Open LogoutServlet.java
 Source > Override/Implement Methods ...
 Check on doGet(…) the click OK
 Call AuthUtil.clearUserId(req) in the method
// log out
AuthUtil.clearUserId(req);
// print out results on the web browser
resp.setContentType("text/html; charset=utf-8");
resp.getWriter().println(
"<html><head><meta http-equiv="refresh" content="3;url=/index.html"></head> " +
"<body>Good bye!<br></body></html>" );

30. Add a New Servlet for Logout
 Add servelt tag to war/WEB-INF/web.xml
<webapp ...>
…
<servlet>
<servlet-name>Logout</servlet-name>
<servlet-class>org.hitlabnz.hw.LogoutServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>Logout</servlet-name>
<url-pattern>/logout</url-pattern>
</servlet-mapping>
...
</webapp>

31. Add a New Servlet for Logout
 Add a link to the LogoutServlet in
index.html
<a href="logout">Logout</a>

32. Live Demo
- Enable HTTP sessions on GAE
- Filter out inappropriate access
- Implement OAuth 2.0 callback
- Add logout servlet

33. Get Access to Mirror API
 In HelloWorldServlet class
 Add method getMirror(req)
private Mirror getMirror(HttpServletRequest req) throws IOException {
// get credential
Credential credential = AuthUtil.getCredential(req);
// build access to Mirror API
return new Mirror.Builder( new UrlFetchTransport(),
new JacksonFactory(), credential)
.setApplicationName("Hello Glass!")
.build();
}

34. Insert a Card to Timeline
 In HelloWorldServlet class
 Add code in doGet(...) method
// get access to Mirror API
Mirror mirror = getMirror(req);
// get access to the timeline
Timeline timeline = mirror.timeline();
// create a timeline item (card)
TimelineItem timelineItem = new TimelineItem()
.setText( "Hello World!" )
.setDisplayTime(new DateTime(new Date()))
.setNotification(new NotificationConfig().setLevel("DEFAULT"));
// insert the card into the timeline
timeline.insert(timelineItem).execute();

35. Insert a Card to Timeline
 In HelloWorldServlet class
 Modify code in doGet(...) method for richer
response on the web browser
// print out results on the web browser
resp.setContentType("text/html; charset=utf-8");
resp.getWriter().println(
"<html><head>“ +
"<meta http-equiv="refresh"content="3;url=/index.html">" +
"</head> " +
"<body>A card is inserted to your timeline.<br></body></html>" );

36. Live Demo
- Access to Mirror API
- Insert a Card into Timeline

37. HTML in a Card
 Follows CSS for the Glass UI
 https://mirror-api-playground.appspot.com/assets/css/base_style.css
String html = "<article><section><p class="text-auto-size">" +
"<em class="yellow">Hello World!</em><br>" +
"Welcome to the <strong class="blue">Glass Class</strong> at HIT Lab NZ." +
"</p></section></article>";
timelineItem.setHtml(html);

38. Google Mirror API Playground
 https://developers.google.com/glass/tools-
downloads/playground
 To try it with your Glass
 Go to Google API console
- https://console.developers.google.com
- Login and open your project
 Create a new Client ID with the following URL
- https://mirror-api-playground.appspot.com
 Use the generated Client ID to authorize
access on the Playground site

39. Multiple Pages in a Card
 Auto paginate
 Manually paginate
 Multiple articles
<article class="auto-paginate">
Very long article ...
</article>
<article class="cover-only">
<section>
<p class="text-auto-size">This is the cover
page.</p>
</section>
</article>
<article>
<section>
This is second page.
</section>
</article>
<article>
<section>
Third page.
</section></article>
https://developers.google.com/glass/develop/mirror/timeline

40. Grouping Multiple Cards
 Bundle – a set of cards grouped under a
cover card (or the latest card)
timelineItem.setBundleId( “UniqueBundleID” )
timelineItem.setIsBundleCover( true );
https://developers.google.com/glass/develop/mirror/timeline

41. Built-in Actions with Menu
 READ_ALOUD, DELETE, OPEN_URI, PLAY_VIDEO,
TOGGLE_PINNED, VOICE_CALL, NAVIGATE, REPLY,
SHARE
List<MenuItem> menuItemList = new ArrayList<MenuItem>();
menuItemList.add(new MenuItem().setAction("READ_ALOUD"));
timelineItem.setSpeakableText("Hello World!");
menuItemList.add(new MenuItem().setAction("TOGGLE_PINNED"));
menuItemList.add(new MenuItem().setAction("DELETE"));
timelineItem.setMenuItems(menuItemList);
https://developers.google.com/glass/develop/mirror/static-cards#creating_menu_items

42. Live Demo
- Cards with HTML
- Add Menu Items

43. Say hello to me
 Add web form in index.html
 Override doPost(...) method and get
parameter from the request
<form action="helloworld" method="post">
Hello <input type="text" name="custom_name" value="World"> !
<input type="submit" value="Submit">
</form>
String custom_name = req.getParameter("custom_name");
String message = "Hello " + custom_name = "!";

44. Image attachment
 Adding static resources to the server
 Edit war/WEB-INF/web.xml
 Add a folder named static under the war
folder and add image files in it
<web-app ...>
...
<static-files>
<include path="/static/*" />
</static-files>
...
</web-app>

45. Image attachment
 Attached image will be used as the
background of the card
 In low-level HTTP Multipart attachment
// get an image to use with the card
URL url = new URL("http://hello-world-mirror.appspot.com/static/hitlabnz.jpg");
InputStreamContent attachment =
new InputStreamContent("image/jpeg", url.openStream());
// insert the card into the timeline
timeline.insert(timelineItem, attachment).execute();
https://developers.google.com/glass/develop/mirror/static-cards#inserting_a_timeline_item_with_media

46. Live Demo
- Input from web forms
- Image attachment

47. Managing Timeline Items
 List
 Delete
https://developers.google.com/glass/v1/reference/timeline/list
// request for list of the timeline items
Mirror.Timeline.List listRequest = timeline.list();
listRequest.setMaxResults(5L);
TimelineListResponse listResponse = listRequest.execute();
// retrieve results
List<TimelineItem> timelineItemList = listResponse.getItems();
mirror.timeline().delete(deleteItemId).execute();

48. Update Pinned Timeline Item
 Simulating Live Cards with Mirror API
 Ask the user to pin the item
 Update the item regularly
TimelineItem timelineItem = getSavedTimelineItem();
// If a saved item exists, isn't deleted, and is pinned
if (timelineItem != null
&& !(timelineItem.getIsDeleted() != null && timelineItem.getIsDeleted()) &&
(timelineItem.getIsPinned() != null && timelineItem.getIsPinned()))
{
// update the pinned item
timelineItem.setHtml( new_html );
timeline.update( timelineItem.getId(), timelineItem ).execute();
} else {
// create a new one and save the id for latter use
}
https://developers.google.com/glass/v1/reference/timeline/update#examples

49. Live Demo
- List Timeline Items
- Delete Timeline Items

50. Location
 Get latest known location
 Draw maps in your card
Location location = mirror.locations().get("latest").execute();
double latitude = location.getLatitude();
double longitude = location.getLongitude();
https://developers.google.com/glass/develop/mirror/location
<article>
<figure>
<img src="glass://map?w=240&h=360&marker=0;42.369590,-71.107132"
height="360" width="240">
</figure>
<section>
<div class="text-auto-size">
<p>Map shown on the left</p>
</div>
</section>
</article>

51. Navigate Menu Item
 Set Location to the timeline item and add the
NAVIGATE menu item
...
timelineItem.setLocation(
new Location()
.setLatitude( -43.522087 )
.setLongitude( 172.582823 )
.setAddress("University of Canterbury, Ilam, Christchurch")
.setDisplayName("HIT Lab NZ") );
// add menu items with built-in actions
List<MenuItem> menuItemList = new ArrayList<MenuItem>();
menuItemList.add(new MenuItem().setAction("NAVIGATE"));
timelineItem.setMenuItems(menuItemList);
// insert the card to the timeline
timeline.insert(timelineItem).execute();

52. Live Demo
- Locate My Glass
- Add a Navigation Card

53. Subscriptions
 Mirror API (server) calls back your web app with
notification POST
 Your web app can subscribe to
 Location updates
 Timeline updates
- Share, reply, delete, custom menu item, launch with voice
 Your web app must respond with a 200 OK
HTTP status code if no error occurred.
 If not, Mirror API might resend notifications.
https://developers.google.com/glass/develop/mirror/static-cards#subscriptions

54. Subscribe
 Add subscription to the mirror api
 Better check if already subscribing
Subscription subscription = new Subscription();
subscription.setCollection("locations"); // or "timeline"
subscription.setCallbackUrl("https://your_web_app/notification");
subscription.setUserToken( AuthUtil.getUserId(req) );
mirror.subscriptions().insert(subscription).execute();

55. Unsubscribe
 Add subscription to the mirror api
// find subscription to delete
SubscriptionsListResponse subslist_resp =
mirror.subscriptions().list().execute();
List<Subscription> subsclist = subslist_resp.getItems();
String subscription_id_to_delete = null;
for (Subscription subsc : subsclist) {
if (subsc.getId().equals("locations")) { // or “timeline"
subscription_id_to_delete = subsc.getId();
break;
}
}
// delete the subscription
mirror.subscriptions().delete(subscription_id_to_delete).execute();

56. Handle Notification Callback
 Create a callback Servlet that
 handles notification POST
 responds with a 200 OK HTTP status code
 Make sure your callback Servlet URL is
not filtered in AuthFilter.java
 Use the user ID provided by
notification.getUserToken() to call Mirror
APIs subsequently

57. Handle Notification Callback
protected void doPost(HttpServletRequest req, HttpServletResponse resp) {
// Respond with OK and status 200 to prevent redelivery
resp.setContentType("text/html");
Writer writer = resp.getWriter();
writer.append("OK");
writer.close();
// extract notification object
JsonFactory jsonFactory = new JacksonFactory();
Notification notification =
jsonFactory.fromInputStream(req.getInputStream(), Notification.class);
// Figure out the impacted user and get their credentials for API calls
String userId = notification.getUserToken();
Credential credential = AuthUtil.getCredential(userId);
Mirror mirror = new Mirror.Builder(new UrlFetchTransport(),
new JacksonFactory(), credential)
.setApplicationName("Hello World")
.build();
if (notification.getCollection().equals("locations")) { // or “timeline“
...

58. Handle Notification Callback
 Handling location updates
 Handling timeline updates
 User action type
- REPLY, DELETE, CUSTOM, LAUNCH, SHARE
// get notified location
Location location = mirror.locations().get(notification.getItemId()).execute();
location.getLatitude();
location.getLongitude();
...
// Get the timeline item which triggered notification
TimelineItem notifiedItem = mirror.timeline().get(notification.getItemId()).execute();
if(notification.getUserActions().contains(new UserAction().setType("REPLY")))
{
String message = "Hello " + notifiedItem.getText() + "!";
...
https://developers.google.com/glass/develop/mirror/subscriptions

59. Live Demo
- Subscribe to Location Updates
- Subscribe to Timeline Updates:
REPLY

60. Contact
 LAUNCH and SHARE user actions need
contact of your web app
Contact helloWorldContact = new Contact();
helloWorldContact.setId("org.hitlabnz.helloworld");
helloWorldContact.setDisplayName("Hello World");
ArrayList<String> imageUrlList = new ArrayList<String>();
imageUrlList.add("http://hello-world-mirror.appspot.com/static/hitlabnz.jpg");
helloWorldContact.setImageUrls(imageUrlList);
ArrayList<Command> commandList = new ArrayList<Command>();
commandList.add(new Command().setType("POST_AN_UPDATE")); // TAKE_A_NOTE
helloWorldContact.setAcceptCommands(commandList);
https://developers.google.com/glass/develop/mirror/contacts
https://developers.google.com/glass/develop/mirror/subscriptions

61. Live Demo
- Subscribe to LAUNCH

62. DIY
- CUSTOM menu item
- Subscribe to SHARE
https://developers.google.com/glass/develop/mirror/contacts
https://developers.google.com/glass/develop/mirror/subscriptions

63. More Tips
 Templates
 JSP
 http://freemarker.org
 Scheduled jobs on server
 CRON settings on GAE
 Programming Google Glass - The Mirror API
 By Eric Redmond
 http://pragprog.com/book/erpgg/programming-google-glass
 Quick Start Samples
 https://developers.google.com/glass/develop/mirror/quickstart/in
dex

64. Sample Code Online
 Mirror-QuickStartSample-Java-GAE
 https://glass-mirror-java-sample.appspot.com
 Mirror0-HelloGlass
 https://hello-glass-mirror.appspot.com
 Mirror1-HelloWorld
 https://hello-world-mirror.appspot.com

65. More Information
 Website
 https://developers.google.com/glass
 http://arforglass.org
 http://www.hitlabnz.org
 Gun Lee
 gun.lee@hitlabnz.org
 Mark Billinghurst
 mark.billinghurst@hitlabnz.org