quick introduction to CI & CD with Google cloud platform
- App Engine Development
- Devops: CI & CD (Docker, gitlab, GCP)
- Tips & Study information
repo: https://gitlab.com/cage1016/gae-flask
6. Who is using GAE?
Angry Birds
Khan Academy
Royal Wedding
Snapchat
EA Games
Forbes
Bestbuy
https://cloud.google.com/customers/ Appengine
7. Google Cloud Platform | Compute
PaasIaaS
Application
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Application
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Saas
Application
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Infrastructure as a Service Platform as a Service Service as a Service
You manage Google manage
12. 2.
Devops: CI & CD
(Docker, gitlab,
GCP)
Let’s start with the second set of
slides
13. Devops
DevOps (a clipped compound of
development and operations) is
a culture, movement or practice
that emphasizes the
collaboration and
communication of both software
developers and other
information-technology (IT)
professionals while automating
the process of software delivery
and infrastructure changes.[1][2]
It aims at establishing a culture
and environment where
building, testing, and releasing
software, can happen rapidly,
frequently, and more reliably -
DevOps - Wikipedia
[image](https://goo.gl/pMC6UZ)
14. Devops: CI & CD (Docker, gitlab, GCP)
[image](https://goo.gl/pMC6UZ)
15. Origin imgae from and modified [DevOps:持續整合&持續交付(Docker、CircleCI、AWS)](http://blog.amowu.com/2015/04/devops-
continuous-integration-delivery-docker-circleci-aws-beanstalk.html)
16. Simple flow
◎ Python (gae-flask)
○ Local Development
○ Local Test
◎ Gitlab
◎ Code Review with GitHub(Gitlab) Flow
◎ gitlab-ci-multi-runner
○ Local runner exec test docker
○ Gitlab runner test docker
◎ Google App Engine
○ Google Cloud Source Repositories
○ Stackdriver Debugger
17. Local Development / Test | gae-flask
// clone repo
$ git clone git@gitlab.com:cage1016/gae-flask.git
// make a new virtualenv
$ mkvirtualenv gae-flask
// pip install requirements.test.txt or link env as lib folder
(gae-flask-gitlab-ci) $ pip install -r requirements.testing.txt -t lib
// or
(gae-flask-gitlab-ci) $ pip install git+https://github.com/ze-phyr-us/linkenv.git
(gae-flask-gitlab-ci) $ linkenv env/lib/python2.7/site-packages lib
// local run app engine project
(gae-flask-gitlab-ci) $ dev_appserver.py app.yaml
// local run test
(gae-flask-gitlab-ci) $ python ./runtests.py
18. Local Development / mian.py
import flask
import requests
app = flask.Flask(__name__)
@app.route("/")
def hello():
return "hello world"
if __name__ == "__main__":
app.run()
19. Local Development / test_main.py
import pytest
import main
@pytest.fixture
def app():
return main.app
@pytest.fixture
def test_client(app):
return app.test_client()
def test_hello(test_client):
response = test_client.get("/")
assert response.data.decode("utf-8") == "hello world"
25. Local runner exec test docker
// [install gitlab-ci-multi-runner](https://goo.gl/a99Aa4)
// list and register gitlab-ci-runner
$ gitlab-ci-multi-runner list
Listing configured runners ConfigFile=/Users/cage/.gitlab-runner/config.toml
gae-flask-gitlab-ci Executor=docker Token=95d56b26104856b9f3852f2d82b1e5 URL=https://gitlab.com/ci
// prepare a docker-machine for gitlab-ci-multi-runner
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
runner - virtualbox Running tcp://192.168.99.100:2376 v1.11.0
// run a gitlab-ci-multi-runner test via docker
$ gitlab-ci-multi-runner --debug exec docker test --env "GCLOUD_KEY=$GCLOUD_KEY
GCLOUD_ACCOUNT=$GCLOUD_ACCOUNT GCP_PROJECT_VERSION=$GCP_PROJECT_VERSION"
// push git
(gae-flask-gitlab-ci) $ git push master
26. Docker cage1016/gitlab-ci-gcp:v1.5
FROM buildpack-deps:jessie
# remove several traces of debian python
RUN apt-get purge -y python.*
RUN apt-get update && apt-get install -y -qq --no-install-recommends unzip openssh-client python-openssl
# install "virtualenv", since the vast majority of users of this image will want it
RUN pip install --no-cache-dir virtualenv
# Install the Google Cloud SDK.
# archive gcloud sdk https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-105.0.0-
linux-x86_64.tar.gz
ENV HOME /
ENV CLOUDSDK_PYTHON_SITEPACKAGES 1
RUN wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-102.0.0-linux-x86_64.
tar.gz && tar -zxvf google-cloud-sdk-102.0.0-linux-x86_64.tar.gz && rm google-cloud-sdk-102.0.0-linux-
x86_64.tar.gz
RUN google-cloud-sdk/install.sh --usage-reporting=true --path-update=true --bash-completion=true --rc-
path=/.bashrc --additional-components app-engine-java app-engine-python app kubectl alpha beta
...
https://github.com/cage1016/gitlab-ci-gcp/blob/master/Dockerfile
31. Google Cloud Source Repositories repo
$ git remote add google https://source.developers.google.com/p/[PROJECT_ID]/r/default
// generates a file named source-context.json, which contains information about the version of the
source code used to build the application
$ gcloud preview app gen-repo-info-file
// push code to google cloud source repositories
$ git push google master
[/ - GAE Labs](https://goo.gl/c99Dz8)
35. “
5G Google cloud storage for App
engine for FREE
<project-id>.appspot.com
staging.<project-id>.appspot.com
36. “
App Engine Managed VMs is
renamed to App Engine flexible
environment. March 24, 2016
[Python Release Notes - Python — Google Cloud Platform](https://cloud.google.com/appengine/docs/python/release-notes#1.9.35)
37. “
App Engine Admin API
Cloud Source Repositories API
Google Cloud Storage
38. “
At the moment, the Go app engine SDK is still based
on Go 1.4, even if you use the go1.6beta version for
your app. This means that the SDK doesn't
understand 1.6-style vendoring, and so doesn't
correctly upload the vendored dependencies.
This will be fixed when 1.6 becomes the stable (go1)
version and is used as the basis of the SDK. Stay
tuned!
[gcpug.tw.go/.travis.yml at master · gcpug-tw/gcpug.tw.go](https://github.com/gcpug-tw/gcpug.tw.go/blob/master/.travis.yml)
[Vendorization for deployment - Google Groups](https://groups.google.com/forum/#!topic/google-appengine-go/umcPV_4tAm8)
39. Study information
◎ Gitlab
○ [Quick Start](http://doc.gitlab.com/ce/ci/quick_start/README.
html)
○ [Configuration of your builds with .gitlab-ci.yml](http://doc.gitlab.
com/ee/ci/yaml/README.html)
○ [GitLab.org / gitlab-ci-multi-runner · GitLab](https://gitlab.
com/gitlab-org/gitlab-ci-multi-runner)
◎ Deploying to App Engine or Managed VMs from Travis CI
○ [Deploying to App Engine or Managed VMs from Travis CI - YouTube]
(https://www.youtube.com/watch?v=7U4jjRw_AJk&feature=youtu.
be)
○ [GoogleCloudPlatform/continuous-deployment-demo](https:
//github.com/GoogleCloudPlatform/continuous-deployment-
demo)
40. Study information cont.
◎ GCP
○ [Cloud Source Repositories - Private Git Repositories — Google
Cloud Platform](https://cloud.google.com/source-repositories/)
○ [Stackdriver Debugger - Production Debugging - Google Cloud
Platform — Google Cloud Platform](https://cloud.google.
com/debugger/)
○ [Managed VMs lab · kaichu.io](http://kaichu.io/2015/09/managed-
vms-lab/)