Then! You've successfully implemented a few Web Services, and you're wondering how to deploy, scale, secure and monitor them. We'll explain the the "SideCar" and "API Gateway" patterns of microservices architectures principles, and apply these principles to the deployment of scalable and secure Chat Bots with the Caddy Server. Join this session to learn about microservices architecture patterns and implementation options (whether you're a bot builder or not !). DEVNET_1871 https://www.ciscolive.com/us/learn/sessions/session-catalog/?search=1871

1. #CLUS
Stève Sfartz, API Evangelist
DEVNET-1871
Microservices
and Serverless
architecture principles
applied

2. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
/Cisco/DevNet/SteveSfartz
• API Evangelist @CiscoDevNet
• API Design,Architecture and Deployment
• WebexTeams & xAPI developer
• hosted @CiscoROI: Paris Innovation Center
• Node.js mainly, a bit of #golang
• Europe and all over the world
• github: //ObjectIsAdvantag
DEVNET-1871
“vision without
execution is
hallucination”
-- Thomas Edison
mailto: stsfartz@cisco.com
twitter: @SteveSfartz
2

3. Agenda
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
• ApplyingTwelve Factors
• Scaling Monoliths
• MicroserviceArchitecture Design
Principles
• Componentization via Services
• Adaptative Routing
• Healthchecks
• API Gateways
DEVNET-1871 3

4. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Cisco WebexTeams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session
Find this session in the Cisco Live Mobile App
Click “Join the Discussion”
Install Webex Teams or go directly to the team space
Enter messages/questions in the team space
How
Webex Teams will be moderated
by the speaker until June 18, 2018.
cs.co/ciscolivebot#DEVNET-1871
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
1
2
3
4
4

5. ApplyingTwelve Factors in
the Enterprise
- for developers: building Web
services and applications
- for ops engineers: who deploy or
manage such apps

6. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Twelve Factors
https://12factor.net/
DEVNET-1871 6

7. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS DEVNET-1871
Twelve factors applied to Spark Bots
https://github.com/CiscoDevNet/botkit-template/blob/master/bot.js#L49
7

8. Twelve factors applied to Spark Bots
« One code base to rule them all »
local dev env demo

9. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
The ‘config file’ versus ‘env variables’ dilemna
• Where to put all not env specific info, nevertheless configurable
• Think defaults for PORTS, HTTP ROUTES, INSTANCE NAME
• Common practice to use a ‘.env’ file to load variables to ENV
• Ruby: https://github.com/bkeepers/dotenv
• Node: https://github.com/motdotla/dotenv
• ALWAYS read from ENV and assume env variables precedence
• over-ride defaults, the execution environment always has the last world
• env may be injected by CI/CD tools, your production platform
• environment managed by Puppet or Chef
• ‘heroku config’ CLI command, and console environment
• env field in a Kubernetes PodYAML configuration
DEVNET-1871 9

10. Twelve factors applied to Spark Bots
« One code base to rule them all »
Heroku & Glitch Demos

11. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Twelve Factor sum up
• Take time to iterate on your code’s launch code to read from the
environment:
• local dev machines, integration, demo, production platforms
• when appropriate, dig into each platform specifics
• Add ops teams feedback to your backlog
• Total launch time
• Maintenance mode
• Enrich logs
• Add healthchecks
• …
DEVNET-1871 11

12. Applying Microservices
Design Principles

13. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Monolithic versus Microservices
http://martinfowler.com/articles/microservices.html
DEVNET-1871 13

14. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Scaling the Monolith
• Several instances behind a load balancer
• lack of modularity
14DEVNET-1871
Load
Balancer
Client
Client
Client
- Complexity to route traffic
- Very large code base
- Change cycles tied together
- Limited scalability
+ Quickest path to scale
+ High availability

15. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Making the Monolith more modular
• Specialized instances of a single codebase
• Modular invocation
DEVNET-1871
Load
Balancer Real time
business logic
Client
Client
Client
Monolith
MonolithAuthentication
Monolith
Batches
Load
Balancer
+ ‘Renovation’ as an alternative to a rewrite
+ Generally does the trick
+ Tip: engage an iterative API-fication
15

16. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
The End of Monoliths ?
http://www.stavros.io/posts/microservices-cargo-cult
DEVNET-1871 16

17. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Microservices (2014)
Martin Fowler, James Lewis
• In short, the microservice architectural style is an approach to developing a
single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API.
• These services are built around business capabilities and independently
deployable by fully automated deployment machinery.
• There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
DEVNET-1871 17

18. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Monolithic versus Microservices
http://martinfowler.com/articles/microservices.html
DEVNET-1871

19. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Common Characteristics
http://martinfowler.com/articles/microservices.html
 Componentization via Services
 Organized around Business Capabilities
 Products not Projects
 Smart endpoints and dumb pipes
 Decentralized governance
 Decentralized data management
 Infrastructure automation
 Design for failure
 Evolutionary design
DEVNET-1871 19

20. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Organized around Business Capabilities
DEVNET-1871
Any organization that
designs a system will
produce a design
whose structure is a
copy of the
organization's
communication
structure.
Melvyn Conway,
1967.
20

21. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Organized around Business Capabilities
• Microservices to solve Conway’s anti-pattern
DEVNET-1871
Cross-functional teams…. … organised around capabilities
21

22. Lessons learnt building the event chatbot

23. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Componentization via Services
 Services as components rather than libraries
 Services avoid tight coupling by using explicit remote call mechanisms.
 Services are independently deployable and scalable
DEVNET-1871 23

24. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS DEVNET-1871
Componentization via Services
https://creations.devnetcloud.com/detail?cid=1e4a9f92-deea-11e7-9d2f-aed03b187700
Events API
Bot Admin
Public
catalog
https://devnet-events-api.herokuapp.com/
CiscoDevNet
@sparkbot.io Postman JSON
Voice
Responder
Tropo
24

25. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Smart endpoints and dumb pipes
 Be as decoupled and as cohesive as possible
 own domain logic,
 act more as filters in the classical Unix sense
 using simple RESTish protocols and lightweight messaging
 Smarts live in the services, not in-between the endpoints
 No central tool / bus that includes sophisticated routing, transformations, process,
business rules
 Pre-requisite : turn the chatty in-process communication of the monolith
into coarser-grained network messaging
DEVNET-1871 25

26. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Componentization via Services
Design Best Practices
 ThinkAPI first = 1 microservice
 Treat each API consumer as a new independent channel = multiple
consumers
DEVNET-1871
deployment
delivery
versioning
contracts
scaling/autoscaling
service discovery
load balancing
routing / adaptive routing
health checking
configuration
circuit breaking
TTL/deadlining
latency tracing
service causal tracing
distributed logging
metrics exposure, collection
26

27. Adaptive Routing

28. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
When ONE backend becomes latent
DEVNET-1871 28

29. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS DEVNET-1871
At 50+ req/s, all request threads can block in seconds
29

30. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
NetFlix Hystrix
• Latency and FaultTolerance library
• designed to isolate points of access to remote systems, services and 3rd party
libraries,
• stop cascading failure and enable resilience where failure is inevitable.
• real time monitoring via Dashboard
DEVNET-1871 30

31. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Hystrix Dashboard
DEVNET-1871 31

32. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Healthchecks
• Passive healthcheck
• Monitoring of your app responses (ie, checks status codes)
• Active healthcheck
• Basically GET / returns a 200 OK
• Provide extra JSON meaning full data
• Used for monitoring, diagnose but also traffic optimization
• Platform dependent: Nginx active healthckech, Amazon LB healthcheck
configuration, Kubernetes liveness probes, Envy
• Types of failure: use 50x error codes
• Tip: include Maintenance, Recovery states info
DEVNET-1871 32

33. Demo: Botkit template healthcheck

34. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Healthchecks Best Practices
• Always provide an healthcheck
• You cannot assume how your service will be deployed
• Make it configurable
• On / off toggle, secured via credentials
• Include maintenance, recovery states
• Provide extra JSON meaningful data
• Start small with 200, think big via adapters to address various platforms
• Nginx active healthcheck, Amazon LB healthcheck configuration, Kubernetes
liveness probes, Envoy healthcheck
• FAIL FAST (12-factor)
DEVNET-1871 34

35. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Resources
• Amazon healthcheck configuration
• https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-
healthchecks.html
• Nginx active healthecks
• https://www.nginx.com/resources/admin-guide/http-health-check/#hc_active
• Kubernetess liveness probes
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-
readiness-probes
DEVNET-1871 35

36. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Common Characteristics
http://martinfowler.com/articles/microservices.html
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized governance
• Decentralized data management
• Infrastructure automation
• Design for failure
• Evolutionary design
DEVNET-1871 36

37. ‘Sidecar’ pattern

38. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
SideCar pattern
• Complement an existing service with extra capabilities via a distinct service
rather than in-process
• Ex: SSL endpoint, outgress routing, traffic priorization
• Typically reverse proxy or smart routing
DEVNET-1871 38

39. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
CaddyWeb Server
• Production-ready
• Secure by default: HTTPS by default, hardenedTLS stack
• Written in go, self-contained
• Single executable with no dependency
• https://caddyserver.com/download
• Multi-platform, container friendly
• Plugin-architecture
DEVNET-1871 39

40. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
WebexTeams Widgets
HTTPS required for Audio/Video calls
40DEVNET-1871
Caddyfile
https://github.com/CiscoDevNet/widget-samples

41. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Resources
• Caddy
• https://caddyserver.com/
• Cisco SparkWidget Samples
• https://github.com/CiscoDevNet/widget-samples
• SparkAffinity: scalingWebhooks
• https://github.com/ObjectIsAdvantag/spark-webhook-affinity
DEVNET-1871 41

42. Programmable
API Gateways

43. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Decentralized Centralized
DEVNET-1871 43

44. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong
• OpensourceAPI Gateway
• Consists in Nginx plugins written in LUA, leverages OpenResty
• Cluster configuration: persists data to Cassandra or PostgreSQL
• The API Gateway can be deployed on bare-metal, cloud or on-premises,
docker/container friendly
• Microservices management building block
• NEW: Kubernetes Ingress Controller for Kong
• https://konghq.com/blog/kubernetes-ingress-controller-for-kong/
https://getkong.org
44DEVNET-1871

45. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS DEVNET-2896

46. Kong demo
https://github.com/ObjectIsAdvantag/kong-101/tree/master/docker

47. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: launching with docker
PostGreSQL backend
DEVNET-1871
docker network create kong-net
docker run -d --name kong-database
--network=kong-net
-p 5432:5432
-e "POSTGRES_USER=kong“
-e "POSTGRES_DB=kong“
postgres:9.6
47

48. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: launching with docker
PostGreSQL backend
DEVNET-1871
docker run --rm
--network=kong-net
-e "KONG_DATABASE=postgres"
-e "KONG_PG_HOST=kong-database"
kong:latest kong migrations up
48

49. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: launching with docker
Reverse proxy
DEVNET-1871
docker run -d --name kong --network=kong-net
-e "KONG_DATABASE=postgres" -e "KONG_PG_HOST=kong-database"
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout"
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout"
-e "KONG_PROXY_ERROR_LOG=/dev/stderr"
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr"
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl"
-p 8000:8000 -p 8443:8443 -p 8001:8001 -p 8444:8444
kong:latest
49

50. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: launching with docker
• Proxy at port :8000
• Administration at :8001
• Exposes :8443 and :8444 for SSL support
• Healthcheck at admin root :8001/ and :8444/
Reverse proxy
DEVNET-1871 50

51. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: Registering HTTP Endpoint
Via admin API
DEVNET-1871
curl http://localhost:8001/services/
--data 'name=minispark'
--data 'url=https://mini-spark.herokuapp.com/'
51

52. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Kong: Rate Limitation
https://getkong.org/plugins/rate-limiting
DEVNET-2871
curl http://localhost:8001/services/{{_service}}/plugins
--data "name=rate-limiting"
--data "config.second=1"
--data "config.minute=5"
--data "config.limit_by=ip"
--data "config.policy=local"
52

53. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
API Gateways use cases
DEVNET-1871 53

54. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Resources
• Kong
• https://getkong.org
• Tyk
• https://tyk.io
• Envoy
• https://www.envoyproxy.io
• Ambassador
• https://www.getambassador.io
DEVNET-1871 54

55. Wrapup

56. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Applying Microservices Principles
• Make your code easy to run and scale
• Embrace environment variables and configuration best practices
• Docker-ready, Kubernetes-ready
• Leverage the infrastructure, don’t ignore it !!!
• Get familiar with Load Balancers, Reverse Proxies, API Gateways
• Practice regularly: local tests, demo environments
• Expose healthchecks
as an Application Developer
DEVNET-1871 56

57. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Furthers considerations
Lessons learnt from Microservices journeys
• Solid communications
• Fault tolerant pipes
• Gateways and Service discovery
• Committed teams
• Devops culture
• Code/Test/Deploy/Support, 24/7
• Automation, automation, automation
• Ownership
• Organisation aligned with the overall strategy
• Operational insights (monitoring, logging infrastructure, real time analysis)
DEVNET-1871 57

58. Complete your online session evaluation
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS
Give us your feedback to be entered into a
Daily Survey Drawing.
Complete your session surveys through the
Cisco Live mobile app or on
www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available for viewing on
demand after the event at www.CiscoLive.com/Online.
DEVNET-1871

59. Q&A

60. Thank you
#CLUS

61. #CLUS