Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Event Streaming in the Telco Industry
Use Cases, Architectures and Technologies
Kai Waehner
Technology Evangelist
contact@...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Telcos’ current IT architectures are not able to sat...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Telco need to transform everything
“The biggest hurd...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
The market development of the SMS
https://disruptive...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
OTT (Over The Top) - aka Telco-OTT
OTT Applications ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Four scenarios
for 2030
-
Where do you
want to go?
h...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
https://www2.deloitte.com/content/dam/Deloitte/it/Do...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Event Streaming for
traditional and new innovative u...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Event streaming platform –
The commit log
Time
P
C1 ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Event streaming platform –
A distributed system for ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Apache Kafka (kafka.apache.org) includes
Kafka Conne...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
An event streaming platform
is the underpinning of a...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
STREAM
PROCESSING
Create and store
materialized view...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Apache Kafka at scale at tech giants
> 7 trillion me...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Hotstar – OTT for millions of cricket fans in India
...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Kafka Connect
Kafka Cluster
CRM
Integration
Domain-D...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
From Monoliths to decoupled, independent services
ht...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Mission-critical
How to
deploy Event
Streaming
24/7,...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Disaster Recovery – RPO and RTO
RPO = Recovery Point...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Digital service at the edge
Single
Kafka Broker
or C...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Hybrid Architectures in the Telco Industry
Digital
S...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Kafka Clusters
can Stretch over Regions
Zero Downtim...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Example of a Multi-Region Cluster in a Telco
Large T...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Global Event Streaming
Aggregate Small Footprint
Edg...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Traditional monitoring in OSS
SNMP poll
SNMP respons...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Challenges
• Vendor-specific management
information ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Unified telemetry with an Event Streaming platform
S...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
OSM - Open Source MANO
(Management and Orchestration...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
40
Customer 360
-
Timely and accurate view of a
cust...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Realtime recommendations
Act at the right time with ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Telco services at the heart of augmented reality
42
...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Augmented reality with picture enhancement
Pre-Proce...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Spoilt for choice!
ESB MQ
Storage
Streaming
Engine
M...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Independent, scalable, reliable components (server a...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Why Apache Kafka versus traditional middleware?
Even...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Don’t use your “ESB knowledge” with Kafka!
https://w...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Event Streaming platform and traditional middleware
...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Legacy IMS (IP Multimedia Core Network Subsystem)
an...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Kafka + ESB / ETL à Blog, slides, video recording…
h...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
1. The Evolution of the Telco Industry
2. Apache Kaf...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Replace old mainframe with Confluent
(instead of buy...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Cloud
Adoption
Journey from mainframe
to hybrid and ...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Strangler design pattern
https://paulhammant.com/201...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Year 0: Direct communication between mainframe and a...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Year 1: Kafka for decoupling between mainframe and a...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Year 2 to 4: New projects and applications
Applicati...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Year 5: Mainframe replacement
Application
Microservi...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
What about transactions?
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
IBM mainframe database and transaction managers
62
I...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
“Transactions” in Apache Kafka
Exactly-Once Semantic...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Kafka + mainframe à Blog, slides, video recording…
h...
Why Confluent
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
Confluent Platform
Fully Managed Cloud ServiceSelf M...
Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
I N V E S T M E N T & T I M E
VALUE
3
4
5
1
2
Event ...
Kai Waehner
Technology Evangelist
contact@kai-waehner.de
@KaiWaehner
www.kai-waehner.de
www.confluent.io
LinkedIn
Question...
Prochain SlideShare
Chargement dans…5
×

Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Mainframe, etc.)

1 894 vues

Publié le

Real-time data streaming is a hot topic in the Telecommunications Industry / Telecom Sector. As telecommunications companies strive to offer high speed, integrated networks with reduced connection times, connect countless devices at reduced latency, and transform the digital experience worldwide, more and more companies are turning to Apache Kafka’s data stream processing solutions to deliver a scalable, real-time infrastructure for OSS and BSS scenarios. Enabling a combination of on-premise data centers, edge processing, and multi-cloud architectures is becoming the new normal in the Telco Industry. This combination is enabling accelerated growth from value-added services delivered over mobile networks.

Join Kai Waehner, Technology Evangelist at Confluent, for this session which explores various telecommunications use cases, including data integration, infrastructure monitoring, data distribution, data processing and business applications. Different architectures and components from the Kafka ecosystem are also discussed.

This talk explores:
- Overcome challenges for building a modern hybrid telco infrastructure
- Build a real time infrastructure to correlate relevant events
- Connect thousands of devices, networks, infrastructures, and people

- Work together with different companies, organisations and business models

- Leverage open source and fully managed solutions from the Apache Kafka ecosystem, Confluent Platform and Confluent Cloud

Publié dans : Logiciels

Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Mainframe, etc.)

  1. 1. Event Streaming in the Telco Industry Use Cases, Architectures and Technologies Kai Waehner Technology Evangelist contact@kai-waehner.de LinkedIn @KaiWaehner www.confluent.io www.kai-waehner.de
  2. 2. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  3. 3. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  4. 4. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Telcos’ current IT architectures are not able to satisfy business needs because of their high complexity, lack of flexibility and low level of automation. https://www2.deloitte.com/content/dam/Deloitte/it/Documents/technology-media-telecommunications/Next%20Gen%20Telco%20Architecture_2017_final.pdf
  5. 5. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Telco need to transform everything “The biggest hurdle to overcome with digital transformation is to understand that it isn’t just a simple technology challenge – it covers every part of the telco business.” https://www.huawei.com/en/about-huawei/publications/winwin-magazine/plus-intelligence/telco-digital-transformation
  6. 6. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de The market development of the SMS https://disruptive.asia/p2p-sms-2022-ovum/ https://dis-blog.thalesgroup.com/mobile/2014/10/06/three-issues-mobile-network-operators-must-face-to-reverse-the-slide-of-sms-revenue/ SMS has become a large commercial industry, earning $114.6 billion globally in 2010. Costs between 10 cents to 20 cents per message were the normal in many countries.
  7. 7. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de OTT (Over The Top) - aka Telco-OTT OTT Applications use the existing Telecommunication Infrastructure and provide better cost / features / convenience. Example: Messaging applications • SMS (short message service) by telco providers: Text format message. • WhatsApp (by Facebook): Above + group chat, gif/stickers, photos, videos, audio, location, contact information and ‘walkie-talkie’ services. • WeChat (by Tencent): Above + payment + various other partner integrations Options for the Telco provider: • Partnership: No control + potentially damage their reputation and customer relationships • Development of own services: No skills + too late • Blocking OTT services: Losing revenue for traffic and customers The answer from the telco providers: RCS (rich communication services)… What do you use today?
  8. 8. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Four scenarios for 2030 - Where do you want to go? https://www2.deloitte.com/content/dam/Deloitte/pl/ Documents/Reports/pl_Deloitte_TMT_Telco_2030.pdf
  9. 9. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de https://www2.deloitte.com/content/dam/Deloitte/it/Documents/technology-media-telecommunications/Next%20Gen%20Telco%20Architecture_2017_final.pdf
  10. 10. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  11. 11. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Event Streaming for traditional and new innovative use cases in the telecommunications industry Real Time Processing Digital Transformation Strategic Goals Move data from OSS fixed, mobile, cable into various systems Mainframe offloading and replacement Proactive network monitoring CDR (Call Detail Record) processing Fleet management and connected car solutions Customer service (e.g. chat bots) and social Integration Smarter network simulation, planning and deployment Personalized messages and recommendations Fraud detection Predictive Maintenance Across Networks and Infrastructure Nationwide security infrastructure (SIEM / SOAR) Augmented reality Spanning different business units (TV, analytics, Video platform, Sales and Services) OSS-BSS Integration and Middleware Modernization Regulatory reporting Customer churn and cross- / up-selling Merger after acquisitions Omni-Channel Customer 360 Converged billing solution
  12. 12. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de
  13. 13. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Event streaming platform – The commit log Time P C1 C2 C3
  14. 14. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Event streaming platform – A distributed system for 24/7 and zero data Loss Broker 1 Topic1 partition1 Broker 2 Broker 3 Broker 4 Topic1 partition1 Topic1 partition1 Leader Follower Topic1 partition2 Topic1 partition2 Topic1 partition2 Topic1 partition3 Topic1 partition4 Topic1 partition3 Topic1 partition3 Topic1 partition4 Topic1 partition4
  15. 15. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Apache Kafka (kafka.apache.org) includes Kafka Connect and Kafka Streams Kafka Streams Your app sinksource KafkaConnect KafkaConnect
  16. 16. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de An event streaming platform is the underpinning of an event-driven architecture Microservices Mainframes SaaS apps Mobile Customer 360 Real-time fraud detection Data warehouse Producers Consumers Database change Microservices events SaaS data Customer experience s Streams of real time events Stream processing apps Connectors Connectors Stream processing apps
  17. 17. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de STREAM PROCESSING Create and store materialized views Filter Analyze in-flight Time C CC Event Streaming
  18. 18. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Apache Kafka at scale at tech giants > 7 trillion messages / day > 6 Petabytes / day “You name it” * Kafka Is not just used by tech giants ** Kafka is not just used for big data
  19. 19. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Hotstar – OTT for millions of cricket fans in India https://www.confluent.io/kafka-summit-san-francisco-2019/scaling-for-indias-cricket-hungry-population/
  20. 20. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Kafka Connect Kafka Cluster CRM Integration Domain-Driven Design (DDD) and decoupled Telco services Legacy Integration Proactive Monitoring Mainframe Connector Java / C++ / Go / ksqlDB Schema Registry Event Streaming Platform BSS Domain Legacy Customer Domain OSS Domain Audit Logs, RBAC, etc.
  21. 21. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de From Monoliths to decoupled, independent services https://www2.deloitte.com/content/dam/Deloitte/it/Documents/technology-media-telecommunications/Next%20Gen%20Telco%20Architecture_2017_final.pdf (Network Functions Virtualization)
  22. 22. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  23. 23. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Mission-critical How to deploy Event Streaming 24/7, including Disaster Recovery?
  24. 24. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Disaster Recovery – RPO and RTO RPO = Recovery Point Objective RTO = Recovery Time Objective
  25. 25. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Digital service at the edge Single Kafka Broker or Cluster Digital Service Self-managed or certified OEM Hardware Kafka Cluster in DC / Cloud Replicator SNMP Monitoring Syslog Troubleshooting TCP Command & Control
  26. 26. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Hybrid Architectures in the Telco Industry Digital Service
  27. 27. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Kafka Clusters can Stretch over Regions Zero Downtime + Zero Data loss (RPO=0 and RTO=0) e.g. Stretched over US East + Mid + West Automated Disaster Recovery Sync or Async Replication per Topic Offset Preserving Automated Client Failover without Custom Code Multi-Region Cluster (Only available in Confluent Platform)
  28. 28. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Example of a Multi-Region Cluster in a Telco Large Telco Customer Transaction Log Transaction Log Location Location synchronous asynchronous ● ‘Payment’ transactions enter from us-east and us-west with fully synchronous replication ● ‘Log’ and ‘Location’ information in the same cluster use async - optimized for latency ● Automated disaster recovery (zero downtime, zero data loss) Result: Clearing time from ‘deposit’ to ‘available’ goes from 5 days to 5 seconds (including security checks) (Only available in Confluent Platform)
  29. 29. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Global Event Streaming Aggregate Small Footprint Edge Deployments with Replication (Aggregation) Simplify Disaster Recovery Operations with Multi-Region Clusters with RPO=0 and RTO=0 Stream Data Globally with Replication and Cluster Linking
  30. 30. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  31. 31. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Traditional monitoring in OSS SNMP poll SNMP response EMS (Cisco) Syslog event NMS (Huawei) SNMP poll SNMP response EMS (Juniper) EMS = Element Management System NMS = Network Management System OSS = Operation Support System Router Soft switch Services gateway (audio, video) OSS
  32. 32. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Challenges • Vendor-specific management information base (MIB) result in multiple, incompatible data semantics and naming • Vendor-specific EMS often required resulting in inefficient “swivel-chair” troubleshooting of issues. • Realtime analytics across a heterogenous environment becomes impossible. • Migration to modern streaming technologies means even more segmentation of visibility.
  33. 33. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Unified telemetry with an Event Streaming platform SNMP Telemetry via Protobuf Telemetry via gRPC Normalize telemetry streams to a common schema Apply real time analytics across ALL sources via the normalized telemetry stream Single source of insight into the performance and availability of the entire network infrastructure.
  34. 34. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de OSM - Open Source MANO (Management and Orchestration) Addressing challenges with orchestration, interoperability and performance optimization Based on ETSI (European Telecommunication Standards Institute) NFV standards and Information Model Interoperability among different service provider NFV infrastructures or OSS systems Apache Kafka asynchronous communication between OSM components and decoupling introduction of new pluggable modules https://osm.etsi.org/news-events/blog/30-open-source-mano-addressing-interoperability-challenge-in-nfv
  35. 35. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  36. 36. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 40 Customer 360 - Timely and accurate view of a customer in its entirety Challenges • Multi-Play Communication Service Providers (CSP) with different systems or stacks for mobile, fixed line, broadband and add-value services • Various technology stacks (3/4/5G, fiber/copper etc.) • Another dimension is added by mergers or acquisitions, where the above challenges immediately multiply • No single point of view, not real time, not context-specific https://admin.salesforce.com/blog/2018/unlock-customer-data-with-salesforce-customer-360%EF%BB%BF
  37. 37. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Realtime recommendations Act at the right time with the right next best offer or action
  38. 38. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Telco services at the heart of augmented reality 42 http://hyper-reality.co/ • CRM integration • Real time product inventory • Predictive recommendations • Geo-location • Payment integration • …
  39. 39. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Augmented reality with picture enhancement Pre-Processing (Kafka Streams / ksqlDB) Mobile App Receive enhanced picture Payment Service including Fraud Detection using Machine Learning (Kafka Streams) Picture Enhancement e.g. using image processing software (external service) Send picture
  40. 40. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  41. 41. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Spoilt for choice! ESB MQ Storage Streaming Engine Messaging: Kafka Core Storage: Kafka Core Caching: Kafka Core Real-Time, Batch: Kafka Clients Integration: Kafka Connect Stream Processing: Kafka Streams / KSQL Request-Response: REST Proxy ”Eat your own dog food” vs. Enemy 1 Enemy 2 Enemy 3 Enemy 4 But be aware: More components, clusters, technologies means more conflicts, incompatibility, operations burden! Enemy 5
  42. 42. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Independent, scalable, reliable components (server and client side!) read, write App (Kafka Streams) Kafka (data) More Apps (KSQL, Connect, Python, REST, “You-name-it”) BookingsTeam FraudTeam … MobileTeam …
  43. 43. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Why Apache Kafka versus traditional middleware? Event Streaming Platform The core is event-based Supports real time stream processing But also Fire-and-Forget, Publish / Subscribe, Request-Response / RPC, Batch, … Single infrastructure Messaging, storage, processing Small and high throughputs (up to Gigabytes per second) Elastic Scalability (up and down) Reliability and zero downtime High availability Exactly-once semantics Rolling upgrades and dynamic configuration changes Backwards compatibility Decoupling of clients Agile microservices Dumb pipes, smart endpoints Handling backpressure No vendor lock-in
  44. 44. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Don’t use your “ESB knowledge” with Kafka! https://www.thoughtworks.com/radar/techniques/recreating-esb-antipatterns-with-kafka !
  45. 45. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Event Streaming platform and traditional middleware are complementary! Orders Customers Infrastructure Status REST Connect ESB JMS Database Mainframe SOAP … Kafka Kafka Kafka KafkaRPC API Management OSS BSSMiddleware Kafka Connect Syslog SNMP … Mobile App CRM
  46. 46. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Legacy IMS (IP Multimedia Core Network Subsystem) and the need for open, scalable, reliable middleware https://www.youtube.com/watch?v=IwOEcYlUmwwExample: VMware vIMS
  47. 47. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Kafka + ESB / ETL à Blog, slides, video recording… https://www.kai-waehner.de/blog/2019/03/07/apache-kafka-middleware-mq-etl-esb-comparison/
  48. 48. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de 1. The Evolution of the Telco Industry 2. Apache Kafka and Confluent 3. Edge, Hybrid and Cloud Architectures 4. Event Streaming in OSS 5. Event Streaming in BSS 6. Middleware Integration and Replacement 7. Case Study Agenda
  49. 49. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Replace old mainframe with Confluent (instead of buying a new mainframe) https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/destination-z1/2019/12/23/capacity-is-king-how-the-ibm-z14-addresses-telecom-volume-concerns
  50. 50. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Cloud Adoption Journey from mainframe to hybrid and cloud PHASE 3 Hybrid Cloud Cloud-First Development PHASE 2 PHASE 1 Case Study - Telco CEO “This is the last 5-year $50M IBM contract. Get rid off the mainframe!”
  51. 51. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Strangler design pattern https://paulhammant.com/2013/07/14/legacy-application-strangulation-case-studies/ https://martinfowler.com/bliki/StranglerFigApplication.html “The most important reason to consider a strangler fig application over a cut-over rewrite is reduced risk.” Martin Fowler
  52. 52. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Year 0: Direct communication between mainframe and app Application 1) Direct Legacy Mainframe Communication to App Date Amount 1/27/2017 $4.56 1/22/2017 $32.14 Customer Service System ‘1980’ (Mainframe)
  53. 53. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Year 1: Kafka for decoupling between mainframe and app Application 1) Direct Legacy Mainframe Communication to App 2) Kafka for Decoupling between Mainframe and App Date Amount 1/27/2017 $4.56 1/22/2017 $32.14 Mainframe Integration - Change Data Capture (IIDR) - Kafka Connect (JMS, MQ, JDBC) - REST Proxy - Kafka Client - 3rd Party Tool Customer Service System ‘1980’ (Mainframe)
  54. 54. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Year 2 to 4: New projects and applications Application Microservices Agile, Lightweight (but Scalable, Robust) Applications Big Data Project (Elastic, Spark, AWS Services, …) 1) Direct Legacy Mainframe Communication to App 2) Kafka for Decoupling between Mainframe and App 3) New Projects and Applications External Solution Date Amount 1/27/2017 $4.56 1/22/2017 $32.14 Mainframe Integration - Change Data Capture (IIDR) - Kafka Connect (JMS, MQ, JDBC) - REST Proxy - Kafka Client - 3rd Party Tool Customer Service System ‘1980’ (Mainframe)
  55. 55. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Year 5: Mainframe replacement Application Microservices Agile, Lightweight (but Scalable, Robust) Applications Big Data Project (Elastic, Spark, AWS Services, …) 1) Direct Legacy Mainframe Communication to App 2) Kafka for Decoupling between Mainframe and App 3) New Projects and Applications 4) Mainframe Replacement External Solution Date Amount 1/27/2017 $4.56 1/22/2017 $32.14 Customer Service System ‘2020’ (Modern Technology)
  56. 56. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de What about transactions?
  57. 57. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de IBM mainframe database and transaction managers 62 IMS • Hierarchical Database • Transaction Manager • Supports Cobol, Assembler, PL/1, Java • IMS Connect for Integration with WebSphere MQ, SOAP, … DB2 • Relational Database CICS • Transaction Manager • Database “Lite” (VSAM Datasets) • Integration and Application Programming Capabilities similar to IMS, but much easier to use • Advanced Features like Transaction Prioritization The Heart of your Business App
  58. 58. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de “Transactions” in Apache Kafka Exactly-Once Semantics (EOS) available since Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging https://www.confluent.io/kafka-summit-london18/dont-repeat-yourself-introducing-exactly-once-semantics-in-apache-kafka/
  59. 59. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Kafka + mainframe à Blog, slides, video recording… https://www.kai-waehner.de/blog/2020/04/24/mainframe-offloading-replacement-apache-kafka-connect-ibm-db2-mq-cdc-cobol/
  60. 60. Why Confluent
  61. 61. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de Confluent Platform Fully Managed Cloud ServiceSelf Managed Software FREEDOM OF CHOICE COMMITTER-DRIVEN EXPERTISE PartnersTrainingProfessional Services Enterprise Support Apache Kafka EFFICIENT OPERATIONS AT SCALE PRODUCTION- STAGE PREREQUISITES UNRESTRICTED DEVELOPER PRODUCTIVITY SQL-based Stream Processing KSQL (ksqlDB) Rich Pre-built Ecosystem Connectors | Hub | Schema Registry Multi-language Development non-Java clients | REST Proxy GUI-driven Mgmt & Monitoring Control Center Flexible DevOps Automation Operator | Ansible Dynamic Performance & Elasticity Auto Data Balancer | Tiered Storage Enterprise-grade Security RBAC | Secrets | Audit logs Data Compatibility Schema Registry | Schema Validation Global Resilience Multi-Region Clusters | Replicator Developer Operator Architect Open Source | Community licensed PARTNERSHIP FOR BUSINESS SUCCESS Complete Engagement Model Revenue / Cost / Risk Impact TCO / ROI Executive Buyer
  62. 62. Apache Kafka in the Telco Industry – @KaiWaehner - www.kai-waehner.de I N V E S T M E N T & T I M E VALUE 3 4 5 1 2 Event Streaming Maturity Model Initial Awareness / Pilot (1 Kafka Cluster) Start to Build Pipeline / Deliver 1 New Outcome (1 Kafka Cluster) Mission-Critical Deployment (Stretched, Hybrid, Multi-Region) Build Contextual Event-Driven Apps (Stretched, Hybrid, Multi-Region) Central Nervous System (Global Kafka) Product, Support, Training, Partners, Technical Account Management...
  63. 63. Kai Waehner Technology Evangelist contact@kai-waehner.de @KaiWaehner www.kai-waehner.de www.confluent.io LinkedIn Questions? Feedback? Let’s connect!

×