SpringOne Platform 2018 Cloud-Native Streaming Platform: Running Apache Kafka on PKS (Pivotal Container Service) Prasad Radhakrishnan, Pivotal; Viktor Gamov, Confluent

1. Cloud-Native Streaming Platform:
Running Apache Kafka on PKS
Prasad Radhakrishnan, @prasad_0101, Pivotal
Viktor Gamov, @gamussa, Confluent

2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
https://twitter.com/kelseyhightower/status/963413508300812295

3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Who we are
3
Prasad
Radhakrishnan
Manager, Data
Engineering
Viktor Gamov
Developer Advocate,
Confluent
@prasad_0101
@gamussa

4. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Agenda
• Why do you need a streaming platform
• Kafka on Kubernetes
• Pivotal ❤ Kafka
• Confluent Platform on PKS
• Demo
4


5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Agenda
• Why do you need a streaming platform
• Kafka on Kubernetes
• Pivotal ❤ Kafka
• Confluent Platform on PKS
• Demo
5


6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/

7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/

8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/

9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Is this streaming?
Streaming

10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
A data for events: The Streaming Platform

11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Apache Kafka Streaming Platform

12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Apache Kafka Streaming Platform

13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Apache Kafka Streaming Platform

14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Apache Kafka Streaming Platform

15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Apache Kafka Streaming Platform

16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Platform Reference Architecture
16

17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/

18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Agenda
• Why do you need a streaming platform
• Kafka on Kubernetes
• Pivotal ❤ Kafka
• Confluent Platform on PKS
• Demo
1
8


19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Time check: What year is it? https://twitter.com/sahrizv/status/1018184792611827712

20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Orchestration
● Compute
● Networking
● Storage
● Service Discovery

21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Platform Reference Architecture
21

22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/ 22
Bare minimum

23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Key Constructs
for Stateful Workloads

24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Refresher - Kubernetes Architecture
Master
etcd
Workers or
Minions
kube-controller-mana
ger
kube-api-server
kube-scheduler
Kubelet
Kube-Proxy
Container
Runtime
Kubectl

25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Pod
Basic Unit of Deployment in
Kubernetes
A collection of containers
sharing:
• Namespace
• Network
• Volumes
Pod
Containers
Volumes

26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Controller
Brain behind Kubernetes resources (core and
custom)
• e.g. replication controller, namespace controller
etc.
Controller subscribes to a ‘queue’ and will handle
3 types of events: Add, Update and Delete

27. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Storage
Persistent Volume (PV) & Persistent Volume
Claim (PVC)
• Both PV and PVC are ‘resources’
• PV is a piece of storage that is
provisioned dynamic or static with a
lifecycle independent of any individual
pod that uses the PV
• PVC is a request for storage by a User.
• PVCs consume PV
Pod
PVC
PV
Static PV
Dynamic
PV
via
Storage Class

28. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Storage
Persistent Volume (PV) &
Persistent Volume Claim
(PVC)
• PV and PVC are
‘resources’
Pod
PVC
PV
Static PV
Dynamic
PV
via
Storage Class

29. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Storage
PV is a piece of storage
that is provisioned
dynamic or static with a
lifecycle independent of
any individual pod that
uses the PV
Pod
PVC
PV
Static PV
Dynamic
PV
via
Storage Class

30. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Storage
Pod
PVC
PV
Static PV
Dynamic
PV
via
Storage Class
PVC is a request for
storage by a User

31. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Storage
PVCs consume PV Pod
PVC
PV
Static PV
Dynamic
PV
via
Storage Class

32. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Represents a set of
Pods with unique,
persistent identities
and stable
hostnames
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

33. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
StatefulSets use an
ordinal index for the
identity and
ordering of their
Pods
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

34. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes
Pods are deployed
in sequential order
and are terminated
in reverse ordinal
order

35. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
When specification
is changed, its Pods
are gracefully
stopped and
recreated in an
ordered way
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

36. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Represents a set of Pods with unique,
persistent identities and stable
hostnames
StatefulSets use an ordinal index for
the identity and ordering of their Pods
Pods are deployed in sequential order
and are terminated in reverse ordinal
order
When specification is changed, its
Pods are gracefully stopped and
recreated in an ordered way
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

37. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Rely on Headless Service to provide
network identity
Ideal for highly available stateful
workloads
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

38. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Rely on Headless
Service to provide
network identity
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

39. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
StatefulSet
Ideal for highly
available stateful
workloads
Headless Service
Pod-0
Containers
Volumes
Pod-1
Containers
Volumes
Pod-2
Containers
Volumes

40. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
Every Pod has an IP address
IPs are routable
All Pods can reach each other without
NAT
All containers in a Pod share the same
network resources - IP, localhost and
IPC
Overlay networks using CNI plugins
defines tunnels between Worker
Nodes

41. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
Every Pod has an IP
address

42. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
IPs are routable

43. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
All Pods can reach
each other without
NAT

44. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
All containers in a
Pod share the same
network resources -
IP, localhost and IPC

45. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Networking
Overlay networks
using CNI plugins
defines tunnels
between Worker
Nodes

46. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Custom Resource Definition(CRD)
Extend existing Kubernetes API
Usually works together Custom
Controller (operator pattern)
Users can create and access
Customer Resources(CRs) with
kubectl, just as they do for built-in
resources like pods.
StatefulSet ReplicaSet CRD...API
StatefulSet
Controller
ReplicaSet
Controller
Custom
Controller
Controller
StatefulSet ReplicaSet
Custom
Resource
Instance
...
...

47. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Workload Deployments

48. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Package Manager
• Package multiple K8s resources
into one deployment unit: Chart
Deployment Management
• Repeatable deployment
• Dependency management
(sub-charts)
• Multiple Configurations (templates
with multiple values file)
• Update, rollback and test
deployment (releases)

49. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Package Manager
• Package multiple
K8s resources into
one deployment
unit: Chart

50. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Deployment Management
• Repeatable deployment
• Dependency management
(sub-charts)
• Multiple Configurations
(templates with multiple
values file)
• Update, rollback and test
deployment (releases)

51. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Deployment
Management
• Repeatable
deployment

52. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Deployment
Management
• Dependency
management
(sub-charts)

53. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Deployment
Management
• Multiple
Configurations
(templates with
multiple values file)

54. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts
Deployment
Management
• Update, rollback
and test
deployment
(releases)

55. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Helm Charts

56. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes Operator
Custom Resource Definition (CRDs)
• Extending K8s Controller
Embedded with operational
knowledge of both data software and
Kubernetes
• Backup/restore
• Scale up/down
• Rebalance data
• Regular health checks
Observe
Analyze
Act

57. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/ 57
Bare minimum

58. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
ZK & Kafka install
ZooKeeper Kafka
PVC for Storage
StatefulSet for 3-node zk
Optional Pod Anti-Affinity to
spread the ZK ensemble
across nodes.
Headless Service
ConfigMap for Prometheus
JMX exporter
Uses ZK Headless Svc
PVC for Storage
StatefulSet for n-node
Kafka
A group of NodePort
Services for external traffic
ConfigMap for Prometheus
JMX exporter

59. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes Operator
ObserveCustom Resource
Definition (CRDs)
• Extending K8s
Controller
Analyze
Act

60. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes Operator
Observe
Analyze
Act
Embedded with operational
knowledge of both data software
and Kubernetes
• Backup/restore
• Scale up/down
• Rebalance data
• Regular health checks

61. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
BOSH
GCP
Service
Broker
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
VMware GCP Azure Openstack AWS
PKSController

62. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
That’s Kubernetes, what’s PKS?

63. Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
OpenServiceBrokerAPI
Repair
— CVEs
Repave Rotate
— Credhub
Pivotal Container Service (PKS)

64. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Multi-Cloud with BOSH + CPI

65. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Agenda
• Why do you need a streaming platform
• Kafka on Kubernetes
• Pivotal ❤ Kafka
• Confluent Platform on PKS
• Demo
6
5


66. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Kafka Kubernetes journey: Step 1
https://cnfl.io/cp-helm

67. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Let’s see some yml!

68. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Agenda
• Why do you need a streaming platform
• Kafka on Kubernetes
• Pivotal ❤ Kafka
• Confluent Platform on PKS
• Demo
6
8


69. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Step 2: Confluent Operator
Deploy and Manage your production streaming platform with Confluent Operator.
• Automated Provisioning
• Platform Operations
• Resiliency
• Monitoring
6
9


70. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Platform Reference Architecture
70
Each Confluent Platform component has specific characteristics:
• Security (SSL certificates)
• DNS names and zones
• Host selection
• Fault tolerance
• Scaling

71. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Operator: Automated Provisioning
7
1

> helm install
–f kafka.yaml
--name kafka
# kafka.yaml
replicas: 3
version: 5.0.1
storage:
provisioner: kubernetes.io/псу-зв
…
resources:
cpu: 200m
memory: 1Gi
external_network:
enabled: true
domain: acme.com

72. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Operator: Scale Horizontally
72

> helm upgrade
-f kafka_new.yaml
--name kafka
# kafka_new.yaml
replicas: 5
version: 5.0.1
storage:
…
Automate scaling:
• Spin up new broker pod(s)
• Distribute partitions to the
new broker(s)
• Determine balancing
plan
• Execute balancing plan
• Monitor resources

73. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Confluent Operator: Rolling Upgrade
73

> helm upgrade
-f kafka_new.yaml
--name kafka
# kafka_new.yaml
replicas: 5
version: 5.1.0
Automated rolling upgrade
with no downtime for Kafka.
• Stop broker
• Wait for leader election to
complete
• Start broker with new
version
• Wait for zero
under-replicated-partitions
• Repeat

74. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Will it fly? Let’s see

75. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Automate Deployment and Management of Apache Kafka®
on Pivotal Container Service (PKS)
Confluent Operator enables you to:
confluent.io/kubernetes

76. > Stay Connected.
https://cnfl.io/cp-helm
https://cnfl.io/k8s
https://pivotal.io/pks
#springone@s1p
@gamussa @prasad_0101

77. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
STOP! Download Fonts Now
PLEASE DOWNLOAD AND INSTALL PROXIMA NOVA FONTS BEFORE CREATING YOUR PRESENTATION. You can
download the fonts here…
https://brandfolder.com/pivotal Password: keepitsimple
Fonts included in the ZIP file:
Proxima Nova (headline and body text)
http://www.fontspring.com/support/installing/how-do-i-install-fonts-on-my-mac
http://www.fontspring.com/support/installing/how-do-i-install-fonts-on-my-windows-pc
77

78. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Disclaimer

79. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Safe Harbor Statement
The following is intended to outline the general direction of Pivotal's offerings. It is intended for information
purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal
offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is
subject to change. This information is provided without warranty or any kind, express or implied, and is not a
commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing
decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently
available. The development, release, and timing of any features or functionality described for Pivotal's offerings
in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking
information in this presentation.
79

80. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Slide Templates

81. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Body Slide - Dark Background
All body text is Proxima Nova Regular
• Subhead
• Level Two
• Level Three
• Level Four
81

82. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Two Columns – Dark Background
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque
8
2
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque

83. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Body Slide - Light Background
All body text is Proxima Nova Regular
• Subhead
• Level Two
• Level Three
• Level Four
8
3


84. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Two Columns – Light Background
8
4
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque

85. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Design Assets

86. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Brand Colors
SpringOne Platform
2018 Primary Colors
86
Spring
Brand Color
Pivotal
Brand Color

87. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Code Slide
87
// This is Roboto Mono: 42pt or higher please
public class TransferServiceImpl implements TransferService {
public TransferServiceImpl(AccountRepository ar) {
this.accountRepository = ar;
}
…
}

88. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Pivotal Logos on Dark Background
88
Looking for more Pivotal logos, PCF services icons, or OSS logos?
Visit: brandfolder.com/pivotal-assets

89. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Pivotal Logos on Light Background
Looking for more Pivotal logos, PCF services icons, or OSS logos?
Visit: brandfolder.com/pivotal-assets
8
9


90. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Event Logos
90
On dark Horizontal Vertical
On light
On mint

91. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Spring Logo and Project Icons
91
Spring
Framework
Spring
Security
Spring
Data
Spring
Batch
Spring
Integration
Project
Reactor
Spring
AMQP
Spring
Hateoas
Spring
Mobile
Spring
Android
Spring
Social
Spring
Web Services
Spring
Web Flow
Spring
XD
Spring
Boot
Spring
LDAP
Spring Tool
Suite
Spring Cloud
Data Flow
Spring
Kafka
Spring
Cloud

92. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Sample Table
92
2007 2008 2009 2010
Category 1 2.4 6.4 4.1 6.6
Category 2 8.2 4.5 3.2 3.8
Category 3 4.6 3.2 1.9 9.6
Category 4 6.7 3.3 3.4 2.2
Category 5 4.3 5.6 7.1 3.4

93. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license:
http://creativecommons.org/licenses/by-nc/3.0/
Sample Table
9
3
2007 2008 2009 2010
Category 1 2.4 6.4 4.1 6.6
Category 2 8.2 4.5 3.2 3.8
Category 3 4.6 3.2 1.9 9.6
Category 4 6.7 3.3 3.4 2.2
Category 5 4.3 5.6 7.1 3.4