Description of our experience developing TouchActive using Google App Engine and Google Web Toolkit

1. Google App Engine &
Google Web Toolkit
our experience
developing
TouchActive
Andrew Mackenzie
Daniel Ruiz
BCNTouch
http://ta.bcntouch.com

2. The purpose of this talk
● Share our experiences using GAE and GWT
● Help guide you if you are considering similar
technologies
● Provide ideas if you are already developing
with these technologies
● Provide feedback to Google on things to
maintain, improve and renew/drop

3. Contents
● Solution
Our Project ● PaaS choice
● GAE overview
● GWT overview
GAE + GWT ● GAE + GWT debugging
● TouchActive
implementation
Implementation ● Addit
● Optimization tools
● Phonegap experiment
Summary ● Avoiding lock-in
● What would we do
differently
Feedback ● Feedback to Google

4. Our Project

5. TouchActive Platform
- what we built
Brand
A platform to put Using tagged
consumers and products that the
brands in touch with consumers were
each other interested in
Consumer

6. Mobile apps to read tagged products
- NFC, QR, barcode
Scalable store in the
cloud with product
Cloud service and data and
data storage synchronization of
user data
Mobile apps read
products tagged
with various
technologies
Get product
information, help
remember, organize
in lists, etc.
Focus on easy and intuitive interaction with
no data entry. NFC the best.

7. Get information about products
from tagged objects

8. Management console for brand
- multi-tenant, groups, roles
Access control on
changes
Cloud service and
data storage
Products findable in
open queries from
mobile devices

9. Consumer Web Interface
- Lists, Products, History, etc
Mobile web version for mobile
platforms without a native app
Addit "Web Clipper" to capture
products from web pages
User data synchronized
with native mobile app Cloud service and
data storage

10. Social networks, Identity and
Sharing
Leverage existing
user identities for
easy sign-up
(web and mobile)
- OpenId and
OAuth
Sharing of Products,
Lists via:
● Social Networks
● EMail
● NFC Tags
● NFC peer-to-peer
● QR codes
● Bump

11. TouchActive Solution Overview
- what we built

12. TouchActive Solution Overview
- what we built
Desktop and Mobile Identity
Management consumer web Providers
console
Addit Sharing (web
and mobile)
over net and
Cloud backend
face-to-face
Mobile Apps
Tagged
products

13. Context for our choices
2+ years ago
● Start Small
○ pre-start-up: build prototype
○ uncertain financing (probably small)
○ small team of engineers
● Think big, or the possibility of big!
● Uncertainties
○ Mobile platforms (especially NFC)
○ Cloud computing platforms

14. Cooling
Power / UPS
Security
Buildings

15. LAMP stack on owned Amazon Web Services Google App Engine
or hosted HW (AWS) (GAE)
small start-up, low Newer options:
capital investment. ● OpenStack (Open SW, multiple service
Pay as you go, don't providers)
buy servers. ● Windows Azure
● Amazon features for auto scaling, etc.
unknown/unexpected
demand
we
don't spend people time and money on managing
servers, runtimes, etc. Focus on the application. chose
GAE
Cooling
Power / UPS
Security But we didn't want to get "locked-in"
Buildings to a single service provider

16. Applications developed
with GWT
AdWords AdSense Orkut
Wave Health Blogger
And many more, see:
http://gwtreferencelist.appspot.com
https://developers.google.com/web-toolkit/developer_spotlight
http://gwtgallery.appspot.com/
http://www.quora.com/What-web-applications-use-Google-Web-Toolkit-(GWT)

17. Google Web Toolkit
Complex UI
Complex UI with
graphical Data
representation

18. GAE + GWT
10

19. Google App Engine Features
● Free developer / Pay-as-you go costs
● Platform not Infrastructure
○ No need to architect the SW stack for scalability
○ No need to manage the SW stack
○ No machine administration
○ Reasonable set of platform services
● Scalability
○ Instances created automatically based on demand
○ Scalable, distributed, datastore
○ Content Distribution Network (CDN)
○ High-performance Image service

20. Google App Engine
Tools and Languages
Language choices at the time: Java and Python
○ Now "GO" is also available now
We chose Java
● Developing J2ME and Android mobile apps in Java
at the same time
Eclipse IDE
○ Local development server
○ Java debugging of server code
Appstats for server performance analysis (RPC)
Management dashboard
Command line tools and ANT tasks for automation

21. Google App Engine
Datastore Datastore
Megatable
BigTable
GFS2
Advantages Drawbacks
● Scalable ● Pay-per-use
● Time vary with result set ● NoSql & pre-built
size, not search set size indexes
● Compatible with existing ○ No JOIN operations
frameworks ○ No inequality filters on more
○ Java Data Objects (JDO) than one "field"
○ Java Persistence API (JPA) ○ No filtering of data based on
● Distributed, replicated subqueries
across data centers ○ No Grouping of results
● Write rate limited

22. Google App Engine
Platform Services
Standard Implementations Non-Standard
● Mail (JMail) ● Cron Tasks
● Datastore (JPA/JDO) ● Task Queues
● MemCache (JSR107) ● Back-ends
● Logging (Java Logging) ● Users (Identity)
● Blob Store
● Image Service
● Chat (XMPP/Jabber)
with Presence
● Channels API (Push)

23. Web Client Development
Google Web Toolkit
● Develop Javascript AJAX client with Java
○ Strongly, statically, typed language.
○ Many errors caught at compile time
○ Full IDE support (completion, etc)
○ Internationalization (i18n) built-in
○ Generates optimized JavaScript, client bundles, code splitting, etc.
○ Hides browser differences
■ Compiles a "permutation" for each language and browser you
support
● Integrated Server and Client development project
○ Java across Server and Client
○ Shared classes between Server and Client
○ "Transparent" RPCs between server and client (takes care of

24. Web Client Development
Google Web Toolkit
● Deferred binding with "properties"
○ Used for loading script variants based on browser, language, form
factor, etc
● Reasonable Widget set
○ Has improved a lot
○ Other widget sets available on top (GXT from Sencha, SmartGWT,
Vadim, etc)
● "DevMode"
○ Java debug experience in IDE for client side code
○ Combined Client/Server/RPC debug experience
○ No-compile, fast turn-around, hot code replace
○ Memory intensive! :-(

25. Web Client Development
Google Web Toolkit
image
Encourages reusable
modular code development
● Custom widgets
● Code in defined "modules"
(server/client/shared code,
RPCs, html, css, images, etc)
● Bundle modules into GWT
libraries (.jar)

26. Web Client Development
Google Web Toolkit
Ease of implementing
Design Patterns
● Model-View-Presenter with
late-binding of views
● Same Presenter, different
Views
● Detect Form Factor from
UA and serve the most
appropriate view at run-
time

27. Google Web Toolkit
Late-Binding of views with ClientFactory
TouchActive.java
TouchActive.gwt.
xml

28. Google Web Toolkit
Late-Binding of views with ClientFactory
TouchActive.java
Desktop Mobile

29. Google Web Toolkit
MVP, Activities & Places
Event-Bus across client code for
subscribe/broadcast event
programming
Application Activity and Place
management via browser history
History Token

30. Google Web Toolkit
UiBinder (XML) for declarative layout
With standard or custom widgets

31. GAE/GWT Debug experience:
An example
Click Event on
Facebook sign
in button

32. GAE/GWT Debug experience:
Client debugging

33. GAE/GWT Debug experience:
Server debugging

34. GAE/GWT Debug experience:
Client debugging

35. Implementation
20

36. Our implementation of TouchActive
with GAE + GWT
● Authentication / Identity management
● Architecture overview
● Data Persistence Layer
● Platform API
● Portability
● CMS Integration
● Addit

37. Authentication:
Own, portable, Identity management
● Created own User Object and related to standard
external accounts
○ OAuth (FB, Twitter, LikedIn)
○ OpenID (Google, Yahoo, MySpace, ...)
○ Google Apps domains for company users
● Username + Password also
● Avoid forcing users having to have a Google account
● Use OAuth & OpenId on mobile apps transparently for
account creation and sign-up
Avoided lock-in, didn't use Google App Engine Users
Service

38. Architecture Overview
Request
● Headers, cookies, ...
Servlet Filters
Request App Logic
Context
Platform API
Data Persistence Layer Platform Specific Services

39. Servlet Filters
● Flexible, just map filters to servlets with a config file
using paths
○ /api
○ /rpc
○ /cms
○ /static
● Keeps servlet code clean and modular
● Used for:
○ Language selection
○ Desktop/Mobile pages while maintaining constant URL
○ Authentication + assignment user to their Multi-Tenancy namespace
○ Query logging for analytics

40. Servlet Filters:
EntryPoint Filter (The Special One)
Request
● Headers, cookies, ...
EntryPoint Filter Servlet Filters
Request
Context
● Different types of EntryPoint filters that set different permissions and tokens
○ Initializes the Request Context used in your app and DataManager
○ Use of Thread variables to make available to all code but multi-threaded
● Authenticates the user and sets the correct "Namespace" in the request context
● Defined the appropriate filter for each request based on Access Control rules in
web.xml

41. Data Persistence Layer
Request
Context
DataManager I/F to rest of app
Data Manager
JDO portable library across multiple
datastore implementations
Google Datastore MySQL
Google AppEngine AWS MySQL Server

42. Data Persistence Layer
DataManager Interface DataManager I/F to rest of app
● Created an API to manage data
objects for reading and writing Data Manager
● DataManager layer is used by all
app, app is kept 100% portable
JDO
Data Objects
● ACL, Protected, Global, etc...
● Enforce visibility and Google Datastore MySQL
permissions according to filters
used on request and object type
MySQL
Multi-Tenancy Google AppEngine AWS
Server
● Create data areas in the same
datastore
● Avoided using Google Datastore
Namespaces

43. Platform Specific Services
App Logic
Defined a Platform API for platform dependant services
GAE Implementation Other Implementation
Cron Tasks Cron Tasks
Task Queues Task Queues
Blobstore Blobstore
Image Service Image Service
Custom build process
Chat (XMPP) Chat (XMPP)
outside IDE
● ANT tasks to build, deploy, test, Channels (Push) Channels (Push)
release
● Run manually from Command
Line or scripts / cron
● Significant effort

44. Additional Need:
Web content beyond the app
Editable by non-programmers
No need for application release on each change
Localized to multiple languages
Use a CMS
GWT
CMS Content

45. Integrating a CMS
Chose to serve content, scripts and RPCs from same domain
○ Deploy CMS to same GAE "appID" as our app
○ This could maybe be avoided with subdomains...
Chose Vosao GAE/J based CMS
○ A bit immature at the time....still < 1.0
○ Uses Apache Velocity (Java based) scripting language
○ Java plugin can integrate tightly with your app and call app code directly
from within CMS web pages
○ Has locked us in to GAE at CMS level, but our code is still portable
○ Manual integration of config files, with help from Vosao team it could be
automated

46. Internationalization (i18N)
● Client GWT UI
○ Built-in i18N support in GWT, strings in property files
○ Permutation of script compiled for each language and correct
language loaded at run time
● Server messages
○ We have avoided as much as possible
○ Java Message Bundle + Properties files
● CMS content
○ Core feature of CMS. Individual strings or entire pages can be
localized with common template

47. Other Features
30

48. Other features implemented
● Chat functionality with server (XMPP)
○ used for testing at the moment
● "Presence" detection of users (XMPP)
● Push messages to browser using the GAE Channel API
○ GWT Client JSNI Wrapper and EventBus connection
● "Addit" Bookmarklet and "web-clipper"

49. Addit -
Objectives
● Help users create products in TouchActive
by capturing product from any web page
● Reduce need for data entry by capturing as
much data as possible automatically
○ Use eCommerce/Search meta data when available
● Relatively simple and easy to use
● Broad usage
○ Support all main web browsers
○ Support as many web sites as possible
○ Desktop, Tablet and Mobile versions

50. Product meta-data
meta and micro-formats used by eCommerce
sites to provide data to search engines or
crawlers:
Various Standards Exist:
● Facebook
○ OpenGraph "product"
● Schema.org
● GoodRelations
● GoogleProducts
○ Google Merchant Center /
Google Product Search
● hProduct

51. Google Product Search

52. Addit Installation
Drag and drop bookmarklet
from web-page to
bookmarks bar or folder
Install extension directly
● From Chrome Web Store
● Chrome, Safari, Firefox from web page

53. User clicks either the
Addit Use Bookmarklet or the Extension
button (FF, Safari, Chrome)

54. Addit Detect
language from
Load localized script from
cloud server and injected
Script loading page contents into the page
Keeping script out of
the bookmarklet or
extention allows us to
update the script
continually, without any
re-install or
redistribution to users
Script NOT from Same-
Origin as page and
security concerns and
barriers to overcome

55. Addit
Fetch user lists and detect currency
Fetches users lists from
their TouchActive
account using a Cross-
Domain "JSONP" RPC
call
Select default currency
based on page
language, page
contents and user
settings retrieved with
lists

56. Addit
product meta-data parsing
Parse DOM and detect
micro-format product
markup elements.
Prepares products for
user automatically
If multiple products
detected, a simple
product selector UI is
shown initially
If no product detected,
uses page-level markup
(meta, OpenGraph) or
general page
information (Title, etc)

57. Addit
Personalize and save to account
Users take the suggested
texts, or customize them.
Select image, price, etc.
Select one of their own lists
they would like to add the
product to
Add a personal note
... and "Addit" to their
TouchActive account
Will be automatically synced with their mobile app

58. Addit
Different layouts for different form factors
Desktop Mobile
Wider, no vertical scrolling, smaller mouse targets Narrower (adjust to Viewport),
vertical scrolling, larger touch targets

59. All Developed in GWT
● GWT XSI Cross-Site linker for CORS script loading
● Wrote own JSONP library for Cross-Site RPC
○ in Java with some JSNI
● GWT "late run-time binding" of scripts for Localization
and Form Factor permutations using custom "Property
Providers"
○ Detect "best guess" language from host page using multiple methods
(poor standards implementations)
○ Detect form factor using a combination of User Agent and display size
● Avoid older Layout Widgets that compile down to HTML
Tables (Box Model and CSS problems...)
● Fast product meta-data parsing from DOM using
"querySelectorAll()" and CSS3 selectors

60. Difficulties - mainly due to injecting a
script into a page you don't control
● Unknown, uncontrolled CSS / DOCTYPE / QuirksMode
○ Can't just do a "CSS Reset"
○ CSS properties influence complex layouts
○ DOCTYPES affect browser behavior (a nightmare on IE 7/8/9)
● "Difficult" sites
○ Don't follow W3C standard for language selection
○ iFrames/Frames/Frameset based
○ Flash sites!
○ Broken image links
● Different implementations of product markup standards
○ Especially prices "," vs "." EUR vs € before/after/middle (1€25)
● JavaScript image galleries and tricks
○ Images loaded but not visible
○ Images being pre-cached (not always in DOM as image)

61. IE and DOCTYPE and Quirks
For masochists only!
http://blogs.msdn.com/b/askie/archive/2009/03/23/understanding-compatibility-modes-in-internet-
explorer-8.aspx

62. Phonegap Experiment
Q Can you take a GWT compiled app (HTML, CSS, JS,
Images etc) and create a "native" app for multiple
platforms?
A Kind-of...
● GWT RPCs from Phonegap client to server!
○ Must be able to go cross-domain
○ We wrapped the RPC class so that it remaps the RPC endpoint to
default server when running "offline"
○ Version problems expected with serialization in the future!!
● Phonegap has problems with relative file paths
○ Processed GWT comiled output to use absolute paths
● Prototype produced from existing GWT app
○ Automated with ANT tasks

63. Optimization
40

64. Optimization Tools Available
Chrome Developer Tools "Audit"
PageSpeed
AppStats for server GAE internal RPC
SpeedTracer
SpeedTracer with AppStats

65. Chrome Audit

66. Page Speed

67. GAE AppStats
understand AppEngine RPC usage

68. AppStats
drill-down to calls, duration and latency

69. SpeedTracer
Understanding client side performance

71. SpeedTracer
Network resource usage from client
Resource Loads
● HTML
● CSS
● JavaScript

72. SpeedTracer (client) integrated with
AppStats (server)
RPC request to server
Server side trace and timing
from AppStats on server

73. What's new in GWT2.5
● Super Dev Mode (experimental)
○ Replaces DevMode, uses "SourceMaps" in Chrome
to enable debugging in language JS was written in
(Java, Dart...)
● New compiler optimizations
○ Smaller, faster JavaScript code
○ With slightly longer compile times
● Elemental
○ Lightweight JavaScript library
● Updated ARIA support
● UIBinder enhancements
● Validation enhancements

74. GWT Compiled code
size reductions
GWT 2.4
100%
GWT 2.5
93%
GWT 2.5
Closure Compiler
87%

75. Summary
40

76. Did we avoid provider "Lock-in"?
Portable data layer built on JDO
● Maybe not the most optimal when running on GAE
● Have to work with limitations of JDO/GAE subset
● Over time, actually moving the data becomes a problem
Platform API has kept rest of code portable
● needs work to get running on additional platforms
GAE based CMS has increased it
● outside of the code, but still important for the project
● decouple by moving to multiple domain approach

77. Things we would do differently if we
started again
New competitors to GAE ● New AWS services bring it
closer to PaaS
● OpenStack from multiple
Analytics providers
● Analyze more closely and
prototype if time permits
Phonegap app
API and directory
structure
CMS

78. Things we would do differently if we
started again
New competitors to GAE
Analytics ● Avoid programming a lot of
that, and falling short
Phonegap app ● Expose analytics data and
use external tools
● Give business users top-
API and directory class tools with flexibility of
structure
analysis, drill-down etc
CMS

79. Things we would do differently if we
started again
New competitors to GAE
Analytics
Phonegap app ● Design and structure web-app
for that from day one
API and directory
structure
CMS

80. Things we would do differently if we
started again
New competitors to GAE
Analytics
Phonegap app
● Assume you will have to
API and directory structure version your external API
● Design app structure early on
○ statics, resources, API,
CMS CMS, RPC,...

81. Things we would do differently if we
started again
New competitors to GAE
Analytics
Phonegap app
API and directory
● Explore using other CMS
structure
outside of GAE
● Sub-domains for "Single-
CMS Origin" content, RPCs

82. Possible Future Work
● Optimize
○ Using tools to focus our efforts
○ More use of memcache for DS accesses
● Explore use JDO3 (Polymorphism)
● Prototype alternative CMS
○ Run on sub-domain in parallel
● Write platform code for other platforms
○ Test on AWS or OpenStack?
● Phonegap app for other platforms
○ Blackberry, Windows Phone...
● Investigate Continuous Integration Support
○ Jenkins on GAE

83. GAE + GWT
The Good
● Scalable, without knowing about or managing the stack
● Free for developers, our costs have been pretty low
● Java across mobile, backend and client
● Programming large app "all together" as client-server
○ But avoid that permitting spaghetti, structure APIs and code
○ Hierarchy of reusable modules of client, server and shared code
○ Drop-in GTW libraries with client, server and shared code
● Excellent debug experience when it works!
○ including hot code replacement (server + client)
● Model-View-Presenter architecture combined with late-
binding
○ extend your app to mobile, tablet views more easily
● UI Binder for declarative UI layout, code for behavior

84. GAE + GWT
The Bad
DevMode problems
● Lack of maintenance on browser plug-ins
● Memory intensive
You have to put in effort to avoid "lock-in"
JDO
● Some people reporting long start-up times (meta-data related)
○ Has worked pretty well for us
● Some features missing: e.g. No Async writes

85. Feedback
44

86. Feedback for Google
App Engine
● Don't introduce features or ways to use features that
increase lock-in.
○ If you do, we will go to "standard" stacks on AWS or OpenStack...
○ e.g. threads, etc
● More transparency on Billing and scheduler behaviour
● Need better attention to developers in forums
○ Paying customers!
○ Especially around scheduling and billing - invest people hours in this
● Channels API
○ Support multiple listeners per page and broadcast
○ Google supported JSNI GWT library

87. Feedback for Google
GWT Debugging
Don't break the debug experience!
● Combined server/RPC/client debug experience in a
single IDE is fantastic and unique (?) in developing an
AJAX client - cloud server application
○ Make it better! More maintenance of plug-in
○ Support at least on Chrome / WebKit browser
○ Less of a memory hog
● GWT 2.5 SuperDevMode does not replace it IMHO
○ list of limitations of seems to outweigh the advantages
○ main advantages (plug-in maintenance) are for Google

88. Feedback for Google
GWT - others
● Allow client-only code and server-only code in same file
○ E.g. Via @annotations
○ Avoid having to complicate the class hierarchy to
have shared/server-only/client-only code for a class
● Java Calendar emulation in GWT and allow for
shared-code date handling

89. Thank You!
Questions?
Andrew Mackenzie
andrew@mackenzie-serres.net
andrew.mackenzie@bcntouch.com
http://www.linkedin.com/in/andrewdmackenzie
Daniel Ruiz Giménez
druizgimenez@gmail.com
daniel.ruiz@bcntouch.com
http://www.linkedin.com/in/danielruizgimenez
http://www.slideshare.
net/andrewdmackenzie/developing-
touchactive-with-gae-and-gwt-google-
50 devfest-barcelona-2012