We will walk through how to build serverless event streaming applications as functions running in a function mesh on kubernetes with cloud native messaging via Apache Pulsar. In this talk, you will deploy ML functions to transform real-time data on Kubernetes. This talk was given by Timothy Spann for DoK Day Europe @ KubeCon 2022.

1. Timothy Spann
Developer Advocate
Serverless Event Streaming
Applications as Functions on K8
16 May 2022 I Valencia, Spain

2. Timothy Spann
Developer Advocate
FLiP(N) Stack = Flink, Pulsar and NiFi Stack
Streaming Systems & Data Architecture Expert
Experience:
15+ years of experience with streaming technologies including
Pulsar, Flink, Spark, NiFi, Kafka, Big Data, Cloud, MXNet, IoT
and more.
Today, he helps to grow the Pulsar community sharing rich
technical knowledge and experience at both global
conferences and through individual conversations.

3. Accessing historical as well as real-time data
Pub/sub model enables event streams to be sent from multiple
producers, and consumed by multiple consumers
To process large amounts of data in a highly scalable way
When is Messaging and Streaming used?

4. Building
Microservices
Asynchronous
Communication
Building Real Time
Applications
Highly Resilient
Tiered storage
Pulsar Benefits

5. ● “Bookies”
● Stores messages and cursors
● Messages are grouped in
segments/ledgers
● A group of bookies form an “ensemble” to
store a ledger
● “Brokers”
● Handles message routing and
connections
● Stateless, but with caches
● Automatic load-balancing
● Topics are composed of
multiple segments
●
● Stores metadata for both Pulsar
and BookKeeper
● Service discovery
Store
Messages
Metadata & Service
Discovery
Metadata & Service
Discovery
Metadata Store
(ZK, RocksDB, etcd, …)
Pulsar Cluster

6. Pulsar Cluster
CLUSTER
Global ZooKeeper
Configuration
Store
ZK3
ZK2
ZK1
Local
ZooKeeper
Quorum
Bookie
0
Bookie
1
Bookie
2
BookKeeper
Bookie
Ensemble
Pulsar
Broker 0
Pulsar
Broker 1
Pulsar
Broker 2

7. Apache Pulsar
Apache BookKeeper
Offloader & Tiered Storage
Offloading
Broker 0
Producer Consumer
Broker 1 Broker 2
Bookie 0 Bookie 1 Bookie 2 Bookie 3 Bookie 4
S S S S
T
1
T
2
T
3
T
4
T
0
S

8. CONFIDENTIAL. DO NOT SHARE.
Apache Pulsar adoption is
being driven by
organizations seeking
cloud-native architectures
and new uses cases.
Microservices
Apache Pulsar - Built for Containers / Modern Cloud
Cloud Native
Hybrid & Multi-Cloud Containers

9. ● Consume messages from one
or more Pulsar topics.
● Apply user-supplied
processing logic to each
message.
● Publish the results of the
computation to another topic.
● Support multiple
programming languages
(Java, Python, Go)
● Can leverage 3rd-party
libraries
Pulsar Functions

10. import java.util.function.Function;
public class MyFunction implements Function<String, String> {
public String apply(String input) {
return doBusinessLogic(input);
}
}
Entire Function
Why Pulsar
Functions?

11. Function Mesh
Pulsar Functions, along with Pulsar IO/Connectors, provide a powerful API for ingesting, transforming,
and outputting data.
Function Mesh, another StreamNative project, makes it easier for developers to create entire
applications built from sources, functions, and sinks all through a declarative API.

12. K8 Deploy

13. Function Execution

14. Apache Pulsar Kafka K8
https://docs.streamnative.io/platform/v1.3.0/quickstart

15. FLiP Stack Weekly
This week in Apache Flink, Apache Pulsar, Apache
NiFi, Apache Spark, Elasticsearch and open source
friends.
https://bit.ly/32dAJft

16. StreamNative Solution
Application Messaging Data Pipelines Real-time Contextual Analytics
Tiered Storage
APP Layer
Computing
Layer
Storage
Layer
StreamNative
Platform
IaaS Layer
Micro
Service
Notification Dashboard Risk Control Auditing
Payment ETL

17. Webinar Series:
Building Microservices
with Pulsar
Watch now Read more
Learn how Pulsar Functions, can be
leveraged to build a message bus for
event-driven microservices

18. Apache Pulsar Resources
● https://pulsar.apache.org/docs/en/kubernetes-helm/
● https://pulsar.apache.org/charts
● https://streamnative.io/streamnativecloud/
● https://medium.com/@_oleksii_/how-to-deploy-apache-puls
ar-cluster-in-kubernetes-808ecdfd87
Academy.StreamNative.io

19. Visit My Booth #S91
Stop By
Scan the QR Code to learn more about
StreamNative and our exclusive booth offerings.