appengine ja night #24 Google Cloud Endpoints and BigQuery (English)

Google Cloud Endpoints
& BigQuery

appengine ja night #24
Ryo Yamasaki @vierjp

1

Self Introduction
Ryo Yamasaki
•  Blog
•  http://blog.vier.jp/
•  Twitter
•  @vierjp
•  Work
•  I mainly develop server side applications by Java.
•  I was a technical leader for the smart phone service provider
company in the previous job.
•  I used the products of Google Cloud Platform, such as App
Engine, Cloud Storage and BigQuery.
•  Publishing Android apps
•  WiFi Tunes Sync
•  WiFi Tunes Sync Pro

2


Overview

3

In one sentence…
Cloud Endpoints make it possible to use App
Engine more easily as the backend of client
applications.

4

You don’t need to write codes for communicating with servers anymore.

Range
which
Cloud
Endpoints
supports

クライアント
サーバー側

Get
values

Set
GET
or

which
Send
Receive

POST

required
to
request
request
Parameters

execute
API

Process

Server
Side

API
using
the

Client
Library
request

Framework
parameters

Convert

Parse
json

Process
objects
to

and

using
the
Receive
json
format

Mapping

results
response
and

values
to

obtained
return
json

your
objects

string

5

You can generate client libraries for Android, iOS, and JavaScript.

App
Engine

Cloud

Endpoints

Android
iOS
JavaScript

6

Authentication support with OAuth 2.0

7

You can use all of the services and features available in App Engine.

Cloud

Endpoints

AppEngine

Google
Cloud

Datastore
Url
Fetch
Mail
Task
Queues

Storage

8

APIs that was created with Endpoints can be execute from the APIs Explorer

•  Comply with “Google APIs Discovery Service”
•  You can execute APIs with OAuth2 authentication
Execute
View

List
View

9


The Development Process

10

Requirements (Java)
•  Eclipse
•  AppEngine SDK 1.7.5 or newer
•  Google Plugin for Eclipse (newer)

11

The Development Process
1.  Create your Eclipse Project
2.  Write your backend app, and add annotations
3.  Test your API using the APIs Explorer
4.  Generating the client library
5.  Write your client app, using the client library

12

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

If
you
make
Android
If
you
do
not
make

clients
Android
clients

2.Right-‐click
on
the
New-‐>Project-‐>

1.Create
an
Android
Android
project
and
Google
-‐>
project
select
"Generate
“Web
ApplicaWon

AppEngine
Backend".
Project”

13

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

@Api(name = "appData", version = "v1", description = "AppData API description v1")
public class AppDataV1Endpoint {
Process which acquires one data
@ApiMethod(name = "data.get", path = "data/get/{id}”) specified by ID and returns it.
public AppDataV1 get(String id) throws ServiceException {
@Named("id") String id) throws ServiceException {
try {
// Get Entity
Key key = KeyFactory.createKey("AppData", id);
DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Entity entity = ds.get(key);
// Generate a DTO to return, refill the value of Entity.
Point
AppDataV1 result = entity2Dto(entity); - Annotations: @Api, @ApiMethod, @Named
return result; - Exception is thrown to notify an error to a client.
} } catch (EntityNotFoundException e) {
// Error handling in case the data corresponding to ID is not found.
throw new NotFoundException("id:" + id + " is not found.");
} catch (Exception e) {
// Error handling in case an unexpected error occurs.
throw new InternalServerErrorException(e);
}
}

14

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Annotation offered by Endpoints
•  @Api
•  Defined for the class
•  Declare API (ex. appData API v1)
•  If you define multiple versions of the same API, create multi classes.
•  @ApiMethod
•  Defined for the method
•  Declare an individual function (ex. ”data.get”)
•  @Named
•  Defined for the method argument
•  If you specify @Nullable, @Named is required.
•  @Nullable
•  Defined for the method argument
•  Specify if you want to allow Null

15

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  @Api
@Api(name = "appData", version = "v1", description = "AppData API description v1”)

・JavaScript
gapi.client.appData.data.get({'id':$scope.id}).execute(function(response,rawResponseJson) {
// callback
});

・Android
AppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(), new
GsonFactory(), null);
AppData appDataApi = CloudEndpointUtils.updateBuilder(builder).build();
AppDataV1 response = appDataApi.data().get(id).execute();
return response;

16

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  @ApiMethod
@ApiMethod(name = "data.get", path = "data/get/{id}”)
@ApiMethod(name = "data.put", path = "data/put”)

・JavaScript
// callback
});

・Android
AppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(), new
GsonFactory(), null);
return response;

17

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

Exception classes that can be used for notification of errors
BadRequestExcepWon

• For
input
errors
(h[p
status
code:
400)

UnauthorizedExcepWon

• For
authenWcaWon
errors
(h[p
status
code:
401)

ForbiddenExcepWon

• For
permission
errors
(h[p
status
code:
403)

NotFoundExcepWon

• For
resource
not
found
errors
(h[p
status
code:
404)

InternalServerErrorExcepWon

• For
system
errors
(h[p
status
code:
500)

ServiceUnavailableExcepWon

• For
service
unavailable
errors
(h[p
status
code:
503)

18

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  How to create your exception class
•  Extends
“com.google.api.server.spi.ServiceException”

•  Sample Code
public class ConflictException extends ServiceException{
public ConflictException(String message){
super(409, message);
}
}

19

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Point
•  API name (value of name attribute of @API) should
start with lower case.
•  Can’t return a literal as the return value.
•  Can't return Entity type of App Engine as the return
value.
•  By using CollectionResponse, paging can be
implemented easily.
•  The following types can be obtained if defined as the
method argument.
•  javax.servlet.http.HttpServletRequest
•  javax.servlet.ServletContext

20

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Tips: Unit testing for server side with JUnit
•  You can write unit test codes as well as the
other test codes of AppEngine app general.
•  Simplified by AppEngineTestCase of Slim3.

21

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Testing using the APIs Explorer
•  Input items are
automatically reflected on
form.
•  Required items are
displayed in red.
•  The contents of a request
can be referred to.
•  The contents of a response
can be referred to.
•  You can execute APIs with
OAuth2 authentication

22

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Use Google Plugin for Eclipse

•  From the command line
sh
appengine-‐java-‐sdk-‐1.7.7/bin/endpoints.sh
get-‐java-‐client-‐lib

jp.vier.sample.endpoints.sample.v1.SampleEndpoint

23

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Point
•  If the code is incorrect, an error occurs in the
process of generating client libraries.
•  Error informations are displayed on "Error
Log" view.
(Notice: It isn't a "Problems" view.)

24

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  JavaScript: Initialize (with AngularJS)
<script type="text/javascript">
var ENDPOINTS_ROOT = 'https://[your_app_id].appspot.com/_ah/api';

function init() {
// Initialization process for using the API of Endpoints.
// Specify the API name that is defined by @API on the server side to the first argument.

gapi.client.load(’appData', 'v1', function() {
// initialize AngularJS
angular.bootstrap(document,['ui.bootstrap','my.filter']);
}, ENDPOINTS_ROOT);
}
</script>
<script src="https://apis.google.com/js/client.js?onload=init"></script>

25

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  JavaScript: Execute Method (with AngularJS)
$scope.getAppData=function() {
// Execute API
gapi.client.appData.data.get({'id':$scope.id}).execute(function(response) {
// To process the response received
$scope.appData = response;
$scope.$apply();
});

・By calling the function of JavaScript, you can get the results from the API on the server.
・Calling "apply" is needed with AngularJS.

26

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  JavaScript: Error Handling (with AngularJS)
$scope.getAppData=function() {
// Execute API
// for dev server
if(response===false){
var rawResponse = JSON.parse(rawResponseJson);
// Display an error dialog
$scope.openErrorModal(rawResponse[0].error.message);
$scope.$apply();
// for production
}else if(response.error && response.error.data && response.error.data.length > 0){
// Display an error dialog
$scope.openErrorModal(response.error.data[0].message);
$scope.$apply();
// success ・You can get the http status code and error message.
}else{ ・Errors behavior is different between the dev server and production
environment. (Issue 9171)
$scope.appData = response;
・"rawResponseJson" is normally not necessary, but It is needed to error
$scope.$apply();
handling on dev server.
}
});
}
27

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Android: Execute Method
AppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(),
new GsonFactory(), null);

・By calling the method of Client Library, you can get the results from the API on the
server.
・It will be executed using the AsyncTask normally.

28

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Android: Error Handling
@Override

protected
AppDataV1
doInBackground(Void...
unused)
{

try
{

AppData.Builder
builder
=
new
AppData.Builder(AndroidH[p.newCompaWbleTransport(),
new
GsonFactory(),
null);

AppData
appDataApi
=
CloudEndpointUWls.updateBuilder(builder).build();

AppDataV1
response
=
appDataApi.data().get(id).execute();

return
response;

}
catch
(ﬁnal
GoogleJsonResponseExcepWon
e)
{

handler.post(new
Runnable()
{

@Override

public
void
run()
{

//
Get
the
error
message
that
was
passed
to
the
excepWon
instance
on
the
server
side.

String
resMessage
=
e.getDetails().getMessage();

Toast.makeText(context,
"Error:
"
+
resMessage,
Toast.LENGTH_LONG).show();

return;

・You can catch the error returned from the server as

}
GoogleJsonResponseException.

});
・You can get the http status code and error message.

return
null;

}
catch
(IOExcepWon
e)
{
・IOException should be treated as a communication

//
CommunicaWon
error
error.
handler.post(new
Runnable()
{

@Override

public
void
run()
{

Toast.makeText(context,
“A
communicaWon
error
has
occurred.
Please
check
the
network
status.",
Toast.LENGTH_LONG).show();

return;

}

});

return
null;

}

}

29

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

•  Point
•  By calling the function or method of Client
Library, you can get the results from the API
on the server.
•  How to use the client library is almost same
as that of the Google API.
•  Therefore, when you are in trouble, searching
how to use the Google API might be a good idea.

30

Create
Eclipse
Write
your
Test
using
the
GeneraWng
Write
your

Project
Backend
App
APIs
Explorer
Client
Libraries
Client
App

Tips:How to connect to the dev server from Android.
•  Use “CloudEndpointUtils” on Android.
•  It is generated by the “Generate AppEngine Backend”
•  How to use: Show ”Android: Execute Method”
•  Set true to CloudEndpointUtils#LOCAL_ANDROID_RUN.

•  Change the settings of your dev server.
•  Debug Configurations -> “Arguments” Tab
-> Program arguments
•  Add “ --address=0.0.0.0”

31


OAuth 2.0 authentication

32

OAuth2 : Create Client ID
APIs Console -> API Access
For JavaScript For Android

33

OAuth2: Implementation of Backend
/** ClientId for Web */
private static final String CLIENT_ID_WEB = "10*********09-7ufji**********k2a91.apps.googleusercontent.com";
/** ClientId for Android */
private static final String CLIENT_ID_ANDROID = "10*********09-
o4e8d**********gt2ts3.apps.googleusercontent.com";
/** Audience for Android */
private static final String AUDIENCE_ANDROID = CLIENT_ID_WEB;
- Definition of Client ID is required for using the OAuth2 authentication.
- When it is accessed using the token that is obtained by authentication in
@ApiMethod( the ClientID that is not defined, the user argument will be null.
name = "data.put", (In this case the request is treated as not authenticated)
- "Audience" is required if you are supporting Android clients.
path = "data/put",
clientIds = {CLIENT_ID_WEB, CLIENT_ID_ANDROID },
audiences = AUDIENCE_ANDROID)
public AppDataV2 put(AppDataV2 appDataV2, User user) throws ServiceException {
// Authentication check
- When you define the argument of
com.google.appengine.api.users.User type,
if (user == null) { user information is set in the variable in the case of
throw new UnauthorizedException(“Login is required. "); OAuth2 authenticated.

}
- An authentication state can be judged by whether user is null.
// Registration of data - If authentication is required and user is null, throw
}
UnauthorizedException.

34

OAuth2: Implementation of JavaScript
var
mainCtrl
=

funcWon($scope)
{

var
CLIENT_ID
=
'10*********09-‐7uti**********k2a91.apps.googleusercontent.com';

var
SCOPES
=
'h[ps://www.googleapis.com/auth/userinfo.email’;

//
OAuthライブラリをloadして自動で認証を試みる

- Use Client Id for Web

gapi.client.load('oauth2',
'v2',
funcWon(response)
{
- At minimum, you should request access

$scope.signin(true,
$scope.userAuthed);
to the userinfo.email scope.

});

//
OAuth認証関数
- Execute the authentication process

$scope.signin=funcWon(mode)
{
-‐
Specify
the
"immediate:true"
for
automaWcally

gapi.auth.authorize({client_id:
CLIENT_ID,
authenWcate.

scope:
SCOPES,
immediate:
mode,
-‐
Specify
the
"immediate:false"
for
explicitly

response_type:
'token
id_token'},
authenWcate.

$scope.userAuthed);

};
- Get a Token and user informations in the callback

//
認証後にユーザー情報を取得するcallback関数
which is called after the authentication process.

$scope.userAuthed
=
funcWon()
{

var
request
=
gapi.client.oauth2.userinfo.get().execute(funcWon(response)
{

if
(!response.code)
{

var
token
=
gapi.auth.getToken();

token.access_token
=
token.id_token;
-‐
For
sign
out
,
call
gapi.auth.setToken(null);

gapi.auth.setToken(token);

-‐
How
to
sign
in
explicitly

//
User情報を取得
$scope.user=response;
<a
ng-‐click="signin(false);">
Sign
In
</a>

//
反映させる
$scope.$apply();

}

});

}

}

35

OAuth2: Implementation of Android (Manage Credential)
private
staWc
ﬁnal
String
CLIENT_ID
=

"10*********09-‐7uti**********k2a91.apps.googleusercontent.com";

Define the clientId specified as the Audience on the
server side.
private
GoogleAccountCredenWal
credenWal;

Define a Credential
@Override

protected
void
onCreate(Bundle
savedInstanceState)
{

super.onCreate(savedInstanceState);

setContentView(R.layout.acWvity_main);

//
Create
a
credenWal
credenWal
=
GoogleAccountCredenWal.usingAudience(this,
"server:client_id:"
+
CLIENT_ID);

}

private
void
setAccountName(String
accountName)
{

//
set
account
name
to
credenWal

credenWal.setSelectedAccountName(accountName);

}

36

OAuth2: Implementation of Android (Choose Account)
//
Display
Account
Picker
private
void
chooseAccount()
{

startAcWvityForResult(credenWal.newChooseAccountIntent(),
REQUEST_ACCOUNT_PICKER);

}

@Override

protected
void
onAcWvityResult(int
requestCode,
int
resultCode,
Intent
data)
{

super.onAcWvityResult(requestCode,
resultCode,
data);

switch
(requestCode)
{

//
Get
the
selecWon
result
of
Account
Picker
case
REQUEST_ACCOUNT_PICKER:

if
(data
!=
null
&&
data.getExtras()
!=
null)
{

Bundle
bundle
=
data.getExtras();

String
accountName
=
bundle.getString(AccountManager.KEY_ACCOUNT_NAME);

if
(accountName
!=
null)
{

//
Set
account
name
to
credenWal
setAccountName(accountName);

}

}

break;

}

}

37

OAuth2: Implementation of Android (Execute Method)
AppData.Builder
builder
=
new
AppData.Builder(

AndroidH[p.newCompaWbleTransport(),

new
GsonFactory(),

credenWal);

AppData
appDataApi
=
CloudEndpointUWls.updateBuilder(builder).build();

To execute API using authentication information, credential is specified as an argument.
If authentication information is not required, you specify null.

38

OAuth2: Two kinds of token
Id_token
access_token
Used
by
oﬃcial
sample
code
Used
by
API
Explorer
(*1)

Documented
Not
Documented
(*2)

You
can
get

only

a
NickName
and
an
Email
You
can
get

the
UserId,
the
Email,
NickName,
and

Address

from
variable
of
User
Type.
(*3)
AuthDomain

from
variable
of
User
Type.
Perhaps,
on
the
server
side,
it
is
not
possible
to
On
the
server
side,
it
is
possible
to
exercise
the

exercise
the
privileges
of
authenWcated
users.
privileges
of
authenWcated
users.
(*4)
AutomaWcally
check
the
validity
of
ClientID
that
(same
as
on
the
le|)
was
used
to
generate
the
Token.

1.  Regarding authentication, Behavior is different in the API Explorer and JS-Android 　
implementation written in the document.
2.  It is not clear whether the way of using access token will be supported continuously.
Because it is not written in the document.
3.  It may be fixed as a bug because the status of issue has been accepted.
4.  It seems that there is no structure to get a token or a credential as a function of
Endpoints.
You need to get access token directly from HttpServletRequest#getHeader.

39

Demo of JavaScript & Android
•  APIs on backend
•  Get messages
•  Post a message (require authentication)
•  Client
•  List messges
•  Post a message
•  OAuth2 authntication

40

Useful links
•  Official documentation
•  http://goo.gl/6wp7R
•  Official Sample (tictactoe)
•  Android - http://goo.gl/xjX1B
•  Backend & JS - http://goo.gl/V8MEV
•  API Explorer
•  https://developers.google.com/apis-explorer/#s/
•  Google APIs Discovery Service
•  https://developers.google.com/discovery/?hl=ja
•  APIs Console
•  https://code.google.com/apis/console/

41

BigQuery
Demo of the new features
•  Small JOIN &Big JOIN
•  GROUP BY &GROUP EACH BY

42

BigQuery
Useful links
•  BigQuery Official documentation
•  https://developers.google.com/bigquery/
•  BigQuery Query Reference (JOIN)
•  http://goo.gl/LKEZq
•  BigQuery Query Reference (GROUP BY)
•  http://goo.gl/EI2aK
•  Google Developers Blog (BigQuery gets big new
features to make data analysis even easier)
•  http://goo.gl/Unwmq

43

Google Cloud Endpoints & BigQuery

Thanks for reading!

44

appengine ja night #24 Google Cloud Endpoints and BigQuery (English)

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (14)

Similaire à appengine ja night #24 Google Cloud Endpoints and BigQuery (English)

Similaire à appengine ja night #24 Google Cloud Endpoints and BigQuery (English) (20)

Dernier

Dernier (20)

appengine ja night #24 Google Cloud Endpoints and BigQuery (English)