The Many Faces of Apache Kafka: How Is Kafka Used in Practice

•

3 j'aime•757 vues

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1EvA7jo. Neha Narkhede discusses how companies are using Apache Kafka and where it fits in the Big Data ecosystem. Filmed at qconnewyork.com. Neha Narkhede is co-founder and head of engineering at Confluent. Previously, she was responsible for LinkedIn’s petabyte scale streaming infrastructure supporting hundreds of billions of events per day. She is also one of the initial authors of Apache Kafka and serves as a PMC member and committer for the project.

Technologie

The Many Faces of
Apache Kafka: Leveraging
Real-time Data at Scale
Neha Narkhede, Confluent

InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/kafka-big-data

Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

• Mission: Make Kafka-based stream data platform a
practical reality everywhere
• Product
• Stream processing integration for Kafka
• Connectors for streaming data flow for common systems
• Monitor end-to-end data flow
• Schemas and metadata management
• First release - Confluent Platform 1.0
confluent

Thank you
@nehanarkhede
http://confluent.io/careers

Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/kafka-big-
data

Recommandé

Fraud Detection for Israel BigThings MeetupGwen (Chen) Shapira

Confluent building a real-time streaming platform using kafka streams and k...Thomas Alex

Kafka connect-london-meetup-2016Gwen (Chen) Shapira

Kafka at scale facebook israelGwen (Chen) Shapira

Streaming Data Integration - For Women in Big Data MeetupGwen (Chen) Shapira

Streaming Data Ingest and Processing with Apache KafkaAttunity

Protecting your data at rest with Apache Kafka by Confluent and Vormetricconfluent

101 ways to configure kafka - badly (Kafka Summit)Henning Spjelkavik

Recommandé

Fraud Detection for Israel BigThings MeetupGwen (Chen) Shapira

Confluent building a real-time streaming platform using kafka streams and k...Thomas Alex

Kafka connect-london-meetup-2016Gwen (Chen) Shapira

Kafka at scale facebook israelGwen (Chen) Shapira

Streaming Data Integration - For Women in Big Data MeetupGwen (Chen) Shapira

Streaming Data Ingest and Processing with Apache KafkaAttunity

Protecting your data at rest with Apache Kafka by Confluent and Vormetricconfluent

101 ways to configure kafka - badly (Kafka Summit)Henning Spjelkavik

Deep Dive into Apache Kafkaconfluent

Confluent Enterprise Datasheetconfluent

Kafka Reliability - When it absolutely, positively has to be thereGwen (Chen) Shapira

Leveraging Kafka for Big Data in Real Time Bidding, Analytics, ML & Campaign ...Helena Edelson

Break Free From Oracle with Attunity and MicrosoftAttunity

Building Reactive Distributed Systems For Streaming Big Data, Analytics & Mac...Helena Edelson

Introducing Kafka Streams, the new stream processing library of Apache Kafka,...Michael Noll

What's new in Confluent 3.2 and Apache Kafka 0.10.2 confluent

Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media

Next Generation Client APIs in Envoy MobileC4Media

Software Teams and Teamwork Trends Report Q1 2020C4Media

Understand the Trade-offs Using Compilers for Java ApplicationsC4Media

Kafka Needs No KeeperC4Media

High Performing Teams Act Like OwnersC4Media

Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media

Service Meshes- The Ultimate GuideC4Media

Shifting Left with Cloud Native CI/CDC4Media

CI/CD for Machine LearningC4Media

Fault Tolerance at SpeedC4Media

Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media

ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media

Build Your Own WebAssembly CompilerC4Media

Contenu connexe

En vedette

Deep Dive into Apache Kafkaconfluent

Confluent Enterprise Datasheetconfluent

Kafka Reliability - When it absolutely, positively has to be thereGwen (Chen) Shapira

Leveraging Kafka for Big Data in Real Time Bidding, Analytics, ML & Campaign ...Helena Edelson

Break Free From Oracle with Attunity and MicrosoftAttunity

Building Reactive Distributed Systems For Streaming Big Data, Analytics & Mac...Helena Edelson

Introducing Kafka Streams, the new stream processing library of Apache Kafka,...Michael Noll

What's new in Confluent 3.2 and Apache Kafka 0.10.2 confluent

En vedette (8)

Deep Dive into Apache Kafka

Confluent Enterprise Datasheet

Kafka Reliability - When it absolutely, positively has to be there

Leveraging Kafka for Big Data in Real Time Bidding, Analytics, ML & Campaign ...

Break Free From Oracle with Attunity and Microsoft

Building Reactive Distributed Systems For Streaming Big Data, Analytics & Mac...

Introducing Kafka Streams, the new stream processing library of Apache Kafka,...

What's new in Confluent 3.2 and Apache Kafka 0.10.2

Plus de C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media

Next Generation Client APIs in Envoy MobileC4Media

Software Teams and Teamwork Trends Report Q1 2020C4Media

Understand the Trade-offs Using Compilers for Java ApplicationsC4Media

Kafka Needs No KeeperC4Media

High Performing Teams Act Like OwnersC4Media

Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media

Service Meshes- The Ultimate GuideC4Media

Shifting Left with Cloud Native CI/CDC4Media

CI/CD for Machine LearningC4Media

Fault Tolerance at SpeedC4Media

Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media

ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media

Build Your Own WebAssembly CompilerC4Media

User & Device Identity for Microservices @ Netflix ScaleC4Media

Scaling Patterns for Netflix's EdgeC4Media

Make Your Electron App Feel at Home EverywhereC4Media

The Talk You've Been Await-ing ForC4Media

Future of Data EngineeringC4Media

Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreC4Media

Plus de C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video

Next Generation Client APIs in Envoy Mobile

Software Teams and Teamwork Trends Report Q1 2020

Understand the Trade-offs Using Compilers for Java Applications

Kafka Needs No Keeper

High Performing Teams Act Like Owners

Does Java Need Inline Types? What Project Valhalla Can Bring to Java

Service Meshes- The Ultimate Guide

Shifting Left with Cloud Native CI/CD

CI/CD for Machine Learning

Fault Tolerance at Speed

Architectures That Scale Deep - Regaining Control in Deep Systems

ML in the Browser: Interactive Experiences with Tensorflow.js

Build Your Own WebAssembly Compiler

User & Device Identity for Microservices @ Netflix Scale

Scaling Patterns for Netflix's Edge

Make Your Electron App Feel at Home Everywhere

The Talk You've Been Await-ing For

Future of Data Engineering

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

Dernier

Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software

WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2

FWD Group - Insurer Innovation Award 2024The Digital Insurer

Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya

DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity

Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays

Apidays New York 2024 - The value of a flexible API Management solution for O...apidays

Corporate and higher education May webinar.pptxRustici Software

presentation ICT roal in 21st century educationjfdjdjcjdnsjd

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Angeliki Cooney

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays

Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz

Platformless Horizons for Digital AdaptabilityWSO2

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub

How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software

Dernier (20)

Strategies for Landing an Oracle DBA Job as a Fresher

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

WSO2's API Vision: Unifying Control, Empowering Developers

FWD Group - Insurer Innovation Award 2024

Artificial Intelligence Chap.5 : Uncertainty

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Corporate and higher education May webinar.pptx

presentation ICT roal in 21st century education

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Platformless Horizons for Digital Adaptability

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

How to Troubleshoot Apps for the Modern Connected Worker

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

The Many Faces of Apache Kafka: How Is Kafka Used in Practice

1. The Many Faces of Apache Kafka: Leveraging Real-time Data at Scale Neha Narkhede, Confluent

2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /kafka-big-data

3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60. • Mission: Make Kafka-based stream data platform a practical reality everywhere • Product • Stream processing integration for Kafka • Connectors for streaming data flow for common systems • Monitor end-to-end data flow • Schemas and metadata management • First release - Confluent Platform 1.0 confluent

61. Thank you @nehanarkhede http://confluent.io/careers

62. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/kafka-big- data