Half-hour tech talk given at user groups or technical conferences to introducing developers to integrating with Google (Cloud) APIs from Python or JavaScript.
ABSTRACT
Want to integrate Google technologies into the web+mobile apps that you build? Google has various open source libraries & developer tools that help you do exactly that. Users who have run into roadblocks like authentication or found our APIs confusing/challenging, are welcome to come and make these non-issues moving forward. Learn how to leverage the power of Google technologies in the next apps you build!!
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring Google (Cloud) APIs with Python & JavaScript
1. Exploring Google (Cloud) APIs
with Python & JavaScript
Wesley Chun
Developer Advocate, Google
Adjunct CS Faculty, Foothill College
G Suite Dev Show
goo.gl/JpBQ40
About the speaker
Developer Advocate, Google Cloud
● Mission: enable current and future
developers everywhere to be
successful using Google Cloud and
other Google developer tools & APIs
● Videos: host of the G Suite Dev Show
on YouTube
● Blogs: developers.googleblog.com &
gsuite-developers.googleblog.com
● Twitters: @wescpy, @GoogleDevs,
@GSuiteDevs
Previous experience / background
● Software engineer & architect for 20+ years
● One of the original Yahoo!Mail engineers
● Author of bestselling "Core Python" books
(corepython.com)
● Technical trainer, teacher, instructor since
1983 (Computer Science, C, Linux, Python)
● Fellow of the Python Software Foundation
● AB (Math/CS) & CMP (Music/Piano), UC
Berkeley and MSCS, UC Santa Barbara
● Adjunct Computer Science Faculty, Foothill
College (Silicon Valley)
4. &
Google APIs client
libraries for many
languages; demos in
developers.google.com/api-
client-library
cloud.google.com/apis/docs
/cloud-client-libraries
02
Google Cloud
5. Google Compute Engine, Google Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google Apps Script
Salesforce1/force.com
G Suite (Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
Google BigQuery, Cloud SQL,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Cloud Platform vs. G Suite
G Suite
APIs
GCP
APIs
6. 03
Google APIs &
developer tools
(focusing on Cloud)
GCP data & storage services
Build, innovate, and scale with Google Cloud Platform
7. Storing Data: Cloud SQL
SQL servers in the cloud
High-performance, fully-managed
600MB to 416GB RAM; up to 64 vCPUs
Up to 10 TB storage; 40,000 IOPS
Types:
MySQL
Postgres
SQLServer (2019)
cloud.google.com/sql
Storing Data: Cloud Datastore
Cloud Datastore: a fully-
managed, highly-scalable
NoSQL database for your web
and mobile applications
cloud.google.com/datastore
8. Storing Data: Firebase
Firebase data is stored
as JSON & synchronized in
real-time to every
connected client; other
tools + FB == v2 mobile
development platform
firebase.google.com
Storing Data: Cloud Firestore
The best of both worlds: the
next generation of Cloud
Datastore (w/product rebrand)
plus features from the
Firebase realtime database
(For choosing between Firebase & Cloud Firestore: see
firebase.google.com/docs/firestore/rtdb-vs-firestore)
cloud.google.com/firestore
9. Cloud Firestore data model
collections subcollections
Cloud Firestore straightforward querying
github.com/GoogleCloudPlatform/python-docs-samples/blob/master/firestore/cloud-client/snippets.py
10. BigQuery: querying Shakespeare words
from google.cloud import bigquery
TITLE = "The most common words in all of Shakespeare's works"
QUERY = '''
SELECT LOWER(word) AS word, sum(word_count) AS count
FROM `bigquery-public-data.samples.shakespeare`
GROUP BY word ORDER BY count DESC LIMIT 10
'''
rsp = bigquery.Client().query(QUERY).result()
print('n*** Results for %r:n' % TITLE)
print('t'.join(col.name.upper() for col in rsp.schema)) # HEADERS
print('n'.join('t'.join(str(x) for x in row.values()) for row in rsp)) # DATA
Top 10 most common Shakespeare words
$ python bq_shake.py
*** Results for "The most common words in all of Shakespeare's works":
WORD COUNT
the 29801
and 27529
i 21029
to 20957
of 18514
a 15370
you 14010
my 12936
in 11722
that 11519
11. ● BigQuery public data sets: cloud.google.com/bigquery/public-data
● Other public data sets: cloud.google.com/public-datasets (Google
Cloud) and research.google/tools/datasets (Google Research)
● COVID-19
○ Read the blog post announcement
○ How to use our data sets
○ JHU Coronavirus COVID-19 Global Cases data set
○ List of all COVID-19 data sets
● BigQuery sandbox (1TB/mo): cloud.google.com/bigquery/docs/sandbox
● Cloud Healthcare API: cloud.google.com/healthcare (read blog post)
BigQuery public data sets
GCP machine learning
Bringing AI/ML to developers of all skillsets
12. ML "building block" APIs
● Gain insights from data using GCP's
pre-trained machine learning models
● Leverage the same technology as Google
Translate, Photos, and Assistant
● Requires ZERO prior knowledge of ML
● If you can call an API, you can use AI/ML!
● cloud.google.com/products/ai/building-blocks
Vision Video
Intelligence
Speech
(S2T & T2S)
Natural
Language
Translation
from google.cloud import vision
image_uri = 'gs://cloud-samples-data/vision/using_curl/shanghai.jpeg'
client = vision.ImageAnnotatorClient()
image = vision.types.Image()
image.source.image_uri = image_uri
response = client.label_detection(image=image)
print('Labels (and confidence score):')
print('=' * 30)
for label in response.label_annotations:
print(f'{label.description} ({label.score*100.:.2f}%)')
Vision: label annotation/object detection
13. $ python3 label-detect.py
Labels (and confidence score):
==============================
People (95.05%)
Street (89.12%)
Mode of transport (89.09%)
Transport (85.13%)
Vehicle (84.69%)
Snapshot (84.11%)
Urban area (80.29%)
Infrastructure (73.14%)
Road (72.74%)
Pedestrian (68.90%)
Vision: label annotation/object detection
g.co/codelabs/vision-python
Cloud Vision
exercise
g.co/codelabs/vision-
python
(others at gcplab.me)
14. ● What is it, and how does it work?
○ Google Cloud ML APIs use pre-trained models
○ Perhaps those models less suitable for your data
○ Further customize/train our models for your data
○ Without sophisticated ML background
○ Translate, Vision, Natural Language, Video Intelligence, Tables
○ cloud.google.com/automl
● Steps
a. Prep your training data
b. Create dataset
c. Import items into dataset
d. Create/train model
e. Evaluate/validate model
f. Make predictions
Cloud AutoML
Full Spectrum of AI & ML Offerings
App developer Data scientist,
developer
Data scientist, Researcher
(w/infrastructure access &
DevOps/SysAdmin skills)
ML EngineAuto ML
Build custom models,
use OSS SDK on fully-
managed infrastructure
ML APIs
App developer,
data scientist
Use/customize pre-built
models
Use pre-built/pre-
trained models
Build custom models, use/
extend OSS SDK, self-manage
training infrastructure
15. Google Cloud compute & serverless
4 opinionated logic-hosting containers in the cloud
>
Google Compute Engine configurable
VMs of all shapes & sizes, from
"micro" to 416 vCPUs, 11.75 TB RAM,
64 TB HDD/SSD plus Google Cloud
Storage for blobs/cloud data lake
(Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise Linux,
Ubuntu, FreeBSD; Windows Server 2008 R2, 2012 R2, 2016)
cloud.google.com/compute
cloud.google.com/storage
Yeah, we got VMs & big disk… but why*?
16. Serverless: what & why
● What is serverless?
○ Misnomer
○ "No worries"
○ Developers focus on writing code & solving business problems*
● Why serverless?
○ Fastest growing segment of cloud... per analyst research*:
■ $1.9B (2016) and $4.25B (2018) ⇒ $7.7B (2021) and $14.93B (2023)
○ What if you go viral? Autoscaling: your new best friend
○ What if you don't? Code not running? You're not paying.
* in USD; source:Forbes (May 2018), MarketsandMarkets™ & CB Insights (Aug 2018)
Why does App Engine exist?
● Focus on app not DevOps
○ Web app
○ Mobile backend
○ Cloud service
● Enhance productivity
● Deploy globally
● Fully-managed
● Auto-scaling
● Pay-per-use
● Familiar languages
17. Hello World (Python "MVP")
app.yaml
runtime: python37
main.py
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
return 'Hello World!'
requirements.txt
Flask==1.0.2
Deploy:
$ gcloud app deploy
Access globally:
PROJECT_ID.appspot.com
Quickstart tutorial and open source repo at
cloud.google.com/appengine/docs/standard/python3/quickstart
Why does Cloud Functions exist?
● Don't have entire app?
○ No framework "overhead" (LAMP, MEAN...)
○ Deploy microservices
● Event-driven
○ Triggered via HTTP or background events
■ Pub/Sub, Cloud Storage, Firebase, etc.
○ Auto-scaling & highly-available; pay per use
● Flexible development environment
○ Cmd-line or developer console (in-browser)
● Cloud Functions for Firebase
○ Mobile app use-cases
● Available runtimes
○ JS/Node.js 6, 8, 10
○ Python 3.7
○ Go 1.11, 1.12
○ Java 8
18. main.py
def hello_world(request):
return 'Hello World!'
Deploy:
$ gcloud functions deploy hello --runtime python37 --trigger-http
Access globally (curl):
curl -X POST https://GCP_REGION-PROJECT_ID.cloudfunctions.net/hello
-H "Content-Type:application/json"
Access globally (browser):
GCP_REGION-PROJECT_ID.cloudfunctions.net/hello
Hello World (Python "MVP")
Quickstart tutorial and open source repo at
cloud.google.com/functions/docs/quickstart-python
Cloud Run
Fully-managed serverless
ContainerizedApps-aaS
No cluster to manage
Pay for what you use
Cloud Run on GKE
Serverless experience
Access custom nodes, GPUs, VPC
Simplicity of Cloud Run
With flexibility of GKE
Fully-managed Kubernetes cluster
Serverless containers, where you want them
Self-managed
Kubernetes cluster
Cloud Run compatible
With Knative open API
Runs on-prem or in other cloud on
Self-managed Kubernetes cluster
cloud.google.com/run
21. Accessing maps from
spreadsheets?!?
goo.gl/oAzBN9
This… with help from Google Maps & Gmail
function sendMap() {
var sheet = SpreadsheetApp.getActiveSheet();
var address = sheet.getRange("A2").getValue();
var map = Maps.newStaticMap().addMarker(address);
GmailApp.sendEmail('friend@example.com', 'Map',
'See below.', {attachments:[map]});
}
JS
Codelab and open source repo at
g.co/codelabs/apps-script-intro
22. G Suite & other Google APIs
Collaborate and be more productive with G Suite
G Suite: Google Drive
Drive API allows developers to read,
write, control permissions/sharing,
import/export files, and more!
developers.google.com/drive
23. Try our Node.js customized reporting tool codelab:
g.co/codelabs/sheets
Why use the Sheets API?
data visualization
customized reports
Sheets as a data source
Migrate SQL data to a Sheet
# read SQL data then create new spreadsheet & add rows into it
FIELDS = ('ID', 'Customer Name', 'Product Code',
'Units Ordered', 'Unit Price', 'Status')
cxn = sqlite3.connect('db.sqlite')
cur = cxn.cursor()
rows = cur.execute('SELECT * FROM orders').fetchall()
cxn.close()
rows.insert(0, FIELDS)
DATA = {'properties': {'title': 'Customer orders'}}
SHEET_ID = SHEETS.spreadsheets().create(body=DATA,
fields='spreadsheetId').execute().get('spreadsheetId')
SHEETS.spreadsheets().values().update(spreadsheetId=SHEET_ID, range='A1',
body={'values': rows}, valueInputOption='RAW').execute()
Migrate SQL data
to Sheets
goo.gl/N1RPwC
24. G Suite: Google Docs & Slides
Docs & Slides APIs give you access
to read or write documents and
presentations programmatically so
you can auto-generate them with data
integrated from various sources
developers.google.com/docs
developers.google.com/slides
● Not just for conversations
● Create microservice utilities
● Build chat bots to...
○ Automate workflows
○ Query for information
○ Other heavy-lifting
● Plain text or rich UI "cards"
● Very flexible ("any")
development environment
○ POST to HTTP port
Hangouts Chat bots
(bot framework & API)
25. "Hello World" (echo bot)
JavaScript: Google Apps Script
function onMessage(m) {
return {
'text': 'Hi ' + m.sender.displayName + '. You sent: ' + m.text,
'thread': {'name': m.thread.name}
};
}
Hangouts Chat bots
goo.gl/jt3FqK
Other Google APIs & platforms
● Firebase (mobile development platform + RT DB)
○ firebase.google.com
● Google Data Studio (data visualization, dashboards, etc.)
○ datastudio.google.com/overview
○ goo.gle/datastudio-course
● Actions on Google/Assistant/DialogFlow (voice apps)
○ developers.google.com/actions
● YouTube (Data, Analytics, and Livestreaming APIs)
○ developers.google.com/youtube
● Google Maps (Maps, Routes, and Places APIs)
○ developers.google.com/maps
● Flutter (native apps [Android, iOS, web] w/1 code base[!])
○ flutter.dev
27. Gmail message processing with GCP
Gmail
Cloud
Pub/Sub
Cloud
Functions
Cloud
Vision
G Suite GCP
Star
message
Message
notification
Trigger
function
Extract
images
Categorize
images
28. Inbox augmented with Cloud Function
● Gmail API: sets up notification forwarding to Cloud Pub/Sub
● developers.google.com/gmail/api/guides/push
● Pub/Sub: triggers logic hosted by Cloud Functions
● cloud.google.com/functions/docs/calling/pubsub
● Cloud Functions: "orchestrator" accessing GCP (and G Suite) APIs
● Combine all of the above to add custom intelligence to Gmail
● Deep dive code blog post
● cloud.google.com/blog/products/application-development/
adding-custom-intelligence-to-gmail-with-serverless-on-gcp
● Application source code
● github.com/GoogleCloudPlatform/cloud-functions-gmail-nodejs
App summary
29. Big data analysis to slide presentation
Access GCP tools from G Suite
31. Supercharge G Suite with GCP
G Suite GCP
BigQuery
Apps Script
Slides Sheets
Application
request
Big data
analytics
App summary
● Leverage GCP and build the "final mile" with G Suite
● Driven by Google Apps Script
● Google BigQuery for data analysis
● Google Sheets for visualization
● Google Slides for presentable results
● "Glued" together w/G Suite serverless
● Build this app (codelab): g.co/codelabs/bigquery-sheets-slides
● Video and blog post: bit.ly/2OcptaG
● Application source code: github.com/googlecodelabs/bigquery-sheets-slides
● Presented at Google Cloud NEXT (Jul 2018 [DEV229] & Apr 2019 [DEV212])
● cloud.withgoogle.com/next18/sf/sessions/session/156878
● cloud.withgoogle.com/next/sf/sessions?session=DEV212
32. 05
Wrap-up
Summary & resources
Session Summary
● Google provides more than just apps
○ You may know Google for search, YouTube, Android, Chrome, and Gmail, but...
○ We've made much of our technology available to developers through our APIs
○ To help prevent all those wheel reinventions!
● Tour of Google APIs & developer tools
○ G Suite: not just a set of productivity apps… you can code them too!
○ GCP: compute, storage, networking, security, data & machine learning tools
○ Google Cloud serverless frees developers from infrastructure
■ 4 unique solutions so you can focus on building solutions
● Interesting possibilities using both Cloud platforms (GCP + G Suite)
● Many other Google developer products
33. ● Documentation
○ GCP: cloud.google.com/{docs,appengine,functions,vision,automl,sql,compute.storage,
language,speech,text-to-speech,translate,video-intelligence,firestore,bigquery,filestore}
○ G Suite: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script}
● Introductory "codelabs" ([free] self-paced, hands-on tutorials)
○ G Suite REST APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API)
○ Apps Script: g.co/codelabs/apps-script-intro
○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python
○ Cloud Vision: g.co/codelabs/vision-python (or C# or Ruby)
○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions
○ All others: gcplab.me (GCP) and g.co/codelabs (non-GCP)
● Videos: youtube.com/GoogleCloudPlatform (GCP), goo.gl/JpBQ40 (G Suite), and all others
● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/gsuitedevs (G Suite)
● GCP Free trial (new users) and Always Free (daily/monthly tier) - cloud.google.com/free
Compare with GCP at cloud.google.com/docs/compare/aws
Online resources
Thank you! Questions?
Wesley Chun
@wescpy
Progress bars: goo.gl/69EJVw
Slides: bit.ly/2SHybQg