Timothy Spann
Developer Advocate
Serverless Event Streaming
Applications as Functions on K8
16 May 2022 I Valencia, Spain

Apache Pulsar is a Cloud-Native Messaging
and Event-Streaming Platform.

Key Pulsar Concepts: Messaging vs
Streaming
Message Queueing - Queueing
systems are ideal for work queues
that do not require tasks to be
performed in a particular order.
Streaming - Streaming works
best in situations where the
order of messages is important.

Pulsar Cluster
CLUSTER
Global ZooKeeper
Configuration Store
ZK3
ZK2
ZK1
Local
ZooKeeper
Quorum
Bookie 0 Bookie 1 Bookie 2
BookKeeper
Bookie
Ensemble
Pulsar Broker 0 Pulsar Broker 1 Pulsar Broker 2

● Consume messages from one
or more Pulsar topics.
● Apply user-supplied
processing logic to each
message.
● Publish the results of the
computation to another topic.
● Support multiple
programming languages
(Java, Python, Go)
● Can leverage 3rd-party
libraries
Pulsar Functions

import java.util.function.Function;
public class MyFunction implements Function<String, String> {
public String apply(String input) {
return doBusinessLogic(input);
}
}
Entire Function
Why Pulsar
Functions?

from pulsar import Function
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
import json
class Chat(Function):
def __init__(self):
pass
def process(self, input, context):
fields = json.loads(input)
sid = SentimentIntensityAnalyzer()
ss = sid.polarity_scores(fields["comment"])
row = { }
row['id'] = str(msg_id)
if ss['compound'] < 0.00:
row['sentiment'] = 'Negative'
else:
row['sentiment'] = 'Positive'
row['comment'] = str(fields["comment"])
json_string = json.dumps(row)
return json_string
Entire Function
Pulsar Python
NLP Function
https://github.com/tspannhw/pulsar-pychat-function

Function Mesh
Pulsar Functions, along with Pulsar IO/Connectors, provide a powerful API for ingesting, transforming,
and outputting data.
Function Mesh, another StreamNative project, makes it easier for developers to create entire
applications built from sources, functions, and sinks all through a declarative API.

K8 Deploy

Function Execution

Apache Pulsar Kafka K8
https://docs.streamnative.io/platform/v1.3.0/quickstart

Webinar Series:
Building Microservices
with Pulsar
Watch now Read more
Learn how Pulsar Functions, can be
leveraged to build a message bus for
event-driven microservices

Apache Pulsar Resources
● https://pulsar.apache.org/docs/en/kubernetes-helm/
● https://pulsar.apache.org/charts
● https://streamnative.io/streamnativecloud/
● https://medium.com/@_oleksii_/how-to-deploy-apache-puls
ar-cluster-in-kubernetes-808ecdfd87
Academy.StreamNative.io

Visit My Booth #S91
Stop By
Scan the QR Code to learn more about
StreamNative and our exclusive booth offerings.

StreamNative Solution
Application Messaging Data Pipelines Real-time Contextual Analytics
Tiered Storage
APP Layer
Computing
Layer
Storage
Layer
StreamNative
Platform
IaaS Layer
Micro
Service
Notification Dashboard Risk Control Auditing
Payment ETL

Timothy Spann
Developer Advocate
FLiP(N) Stack = Flink, Pulsar and NiFi Stack
Streaming Systems & Data Architecture Expert
Experience:
15+ years of experience with streaming technologies including
Pulsar, Flink, Spark, NiFi, Kafka, Big Data, Cloud, MXNet, IoT
and more.
Today, he helps to grow the Pulsar community sharing rich
technical knowledge and experience at both global
conferences and through individual conversations.

FLiP Stack Weekly
This week in Apache Flink, Apache Pulsar, Apache
NiFi, Apache Spark, Elasticsearch and open source
friends.
https://bit.ly/32dAJft

Accessing historical as well as real-time data
Pub/sub model enables event streams to be sent from multiple
producers, and consumed by multiple consumers
To process large amounts of data in a highly scalable way
When is Messaging and Streaming used?

CONFIDENTIAL. DO NOT SHARE.
Apache Pulsar adoption is
being driven by
organizations seeking
cloud-native architectures
and new uses cases.
Microservices
Apache Pulsar - Built for Containers / Modern Cloud
Cloud Native
Hybrid & Multi-Cloud Containers

● “Bookies”
● Stores messages and cursors
● Messages are grouped in
segments/ledgers
● A group of bookies form an “ensemble” to
store a ledger
● “Brokers”
● Handles message routing and
connections
● Stateless, but with caches
● Automatic load-balancing
● Topics are composed of
multiple segments
●
● Stores metadata for both Pulsar
and BookKeeper
● Service discovery
Store
Messages
Metadata & Service
Discovery
Metadata & Service
Discovery
Metadata Store
(ZK, RocksDB, etcd, …)
Pulsar Cluster

Building
Microservices
Asynchronous
Communication
Building Real Time
Applications
Highly Resilient
Tiered storage
Pulsar Benefits

Apache Pulsar
Apache BookKeeper
Offloader & Tiered Storage
Offloading
Broker 0
Producer Consumer
Broker 1 Broker 2
Bookie 0 Bookie 1 Bookie 2 Bookie 3 Bookie 4
S S S S
T
1
T
2
T
3
T
4
T
0
S