Kafka, Killer of Point-to-Point Integrations, Lucian Lita

•

3 j'aime•1,453 vues

With 60+ products and over 24% of the US GDP flowing through it, system integration is a tough problem for Intuit. Seasonality, scale, and massive peaks in products like TurboTax, QuickBooks, and Mint.com add extra layers of difficulty when building shared data services around transaction and user graphs, clickstream processing, a/b testing, and personalization. To reduce complexity and latency, we’ve implemented Kafka as the backbone across these data services. This allows us to asynchronously trigger relevant processing, elegantly scaling up and down as needed around peaks, all without the need for point-to-point integrations. In this talk, we share what we’ve learned about Kafka at Intuit and describe our data services architecture. We found that Kafka is invaluable in achieving a scalable, clean architecture, allowing engineering teams to focus less on integration and more on product development.

Kafka, Killer of Point-to-Point Integrations
Lucian Lita
@datariver
Intuit Data Team

@datariver

@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
Data PlaBorm & Services
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
login
login
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
login
change descrip1on (1,2)
descrip1on
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
login
invoice
change descrip1on (1,2,3)
invoice
@datariver

1.2B
800M
400M
Tax ﬁling clickstream ac1vity in TurboTax
Requirements?
ü  high volume – billions of events / day
ü  hybrid: cloud & colo
ü  robust & reliable / HA
ü  near real 1me
ü  secure: encryp1on, access
ü  scalable: up and down
-  seasonal: traﬃc >10,00x
-  high burst: up to 250k tps
Oct
April 15th
early Feb
Dec
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
Star1ng State: Service Fragmenta1on
10+
15+
5+
10+
7+
5+
15+
0
10+
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
Star1ng State: Point-to-Point Integra1ons
10+
15+
5+
10+
7+
5+
15+
0
10+
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
Target State
@datariver

Company Proﬁles
Deep Analy1cs
A/B Tes1ng
Predic1ve Modeling
User Proﬁles
Graph
Search
Real Time Analy1cs
Data Capture
Core Data PlaBorm & Services
@datariver

Zoom Out: Small Business
Accoun1ng
Payroll
Payments
Inventory
Invoices
Repor1ng
3rd Party Apps
Small Business

Small Business
Tax
Personal Finance
Accountants
Zoom Out: Intuit
@datariver

@datariver

Thank you!
Intuit Data Team
Lucian Lita
@datariver

Contenu connexe

Tendances

For many industries the need to group together related events based on a period of activity or inactivity is key. Advertising businesses, content producers are just a few examples of where session windows can be used to better understand user behavior. While such sessionization has been possible in Apache Kafka up to this point, implementing it has been rather complex and required leveraging low-level APIs. In the most recent release of Kafka, however, new capabilities have been added making session windows much easier to implement. In this online talk, we’ll introduce the concept of a session window, talk about common use cases, and walk through how Apache Kafka can be used for session-oriented use cases.

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

Using Apache Kafka to Analyze Session Windows

Using Apache Kafka to Analyze Session Windows

Using Apache Kafka to Analyze Session Windows

At Stripe, we operate a general ledger modeled as double-entry bookkeeping for all financial transactions. Warehousing such data is challenging due to its high volume and high cardinality of unique accounts. aFurthermore, it is financially critical to get up-to-date, accurate analytics over all records. Due to the changing nature of real time transactions, it is impossible to pre-compute the analytics as a fixed time series. We have overcome the challenge by creating a real time key-value store inside Pinot that can sustain half million QPS with all the financial transactions. We will talk about the details of our solution and the interesting technical challenges faced.

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

HostedbyConfluent

At Under Armour Connected Fitness, we’ve built an event streaming platform on top of Kafka and the Confluent stack that makes it easy for developers to produce and consume schema-based events without requiring direct knowledge of Kafka. We are constantly trying to improve the developer experience. The platform consists of multiple federated Kafka clusters, a schema registry, a topology service, an archiver and specialized client libraries and Web / CLI tools that assist developers with producer and consumer workflows. In this talk, we will take a deeper dive into the design and implementation of a Scala/Java implementation of our client library that allows developers to produce or consume events without worrying about the underlying infrastructure and their location while enjoying the benefits of data compatibility through schemas. We’ll also look at an HTTP based client proxy that exposes the same API but for languages without our native support. Finally, we’ll walk through Web and CLI tools we built to make working with the platform easier. The content of this talk will be primarily aimed at software developers looking for ideas on how to build Kafka client tools that allow producer/consumer interactions protected by schema-based event definitions while hiding details of the underlying infrastructure.

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Today, many companies that have lots of data are still struggling to derive value from machine learning (ML) and data science investments. Why? Accessing the data may be difficult. Or maybe it’s poorly labeled. Or vital context is missing. Or there are questions around data integrity. Or standing up an ML service can be cumbersome and complex. At Nuuly, we offer an innovative clothing rental subscription model and are continually evolving our ML solutions to gain insight into the behaviors of our unique customer base as well as provide personalized services. In this session, I’ll share how we used event streaming with Apache Kafka® and Confluent Cloud to address many of the challenges that may be keeping your organization from maximizing the business value of machine learning and data science. First, you’ll see how we ensure that every customer interaction and its business context is collected. Next, I’ll explain how we can replay entire interaction histories using Kafka as a transport layer as well as a persistence layer and a business application processing layer. Order management, inventory management, logistics, subscription management – all of it integrates with Kafka as the common backbone. These data streams enable Nuuly to rapidly prototype and deploy dynamic ML models to support various domains, including pricing, recommendations, product similarity, and warehouse optimization. Join us and learn how Kafka can help improve machine learning and data science initiatives that may not be delivered to their full potential.

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

“The mainframe is going away” is as true now as it was 10, 20 and 30 years ago. Mainframes are still crucial in handling critical business transactions, they were however built for an era where batch data movement was the norm and can be difficult to integrate into today’s data-driven, real-time, analytics-focused business processes as well as the environments that support them. Until now. Join experts from Confluent, Attunity, and Capgemini for a one-hour online talk session where you’ll learn how to: Unlock your mainframe data with unique change data capture (CDC) functionality without incurring the complexity and expense that come with sending ongoing queries into the mainframe database How using CDC benefits advanced analytics approaches such as deep machine learning and predictive analytics Deliver ongoing streams of data in real-time to the most demanding analytics environments Ensure that your analytics environment includes the broadest possible range of data sources and destinations while ensuring true enterprise-grade functionality Identify use cases that can help you get started delivering value to the business moving from POC to Pilot to Production

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

One of the key metrics to monitor when working with Apache Kafka, as a data pipeline or a streaming platform, is Consumer Groups Lag. Lag is the delta between the last produced message and the last committed message of a partition. In other words, lag indicates how far behind your application is in processing up-to-date information. For a long time, we used our own service to keep track of these metrics, collect them and visualize them. But this didn’t scale well. You had to perform many manual operations, redeploy it and to do other tedious manual tasks, but most importantly, the biggest gap for us, was that its output was represented in absolute numbers (e.g - your lag is 30K), which basically tells you nothing as a human being. We understood that we had to find a more suitable solution that will give us better visibility and will allow us to measure the lag in a time-based format that we all understand. In this talk, I’m going to go over the core concepts of Kafka offsets and lags, and explain why lag even matters and is an important KPI to measure. I’ll also talk about the kind of research we did to find the right tool, what the options in the market were at the time, and eventually why we chose Linkedin’s Burrow as the right tool for us. And finally, I’ll take a closer look at Burrow, its building blocks, how we build and deploy it, how we monitor better with it, and eventually the most important improvement - how we transformed its output from numbers to time-based metrics.

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

HostedbyConfluent

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Time series-analysis-using-an-event-streaming-platform -_v3_final

Time series-analysis-using-an-event-streaming-platform -_v3_final

Time series-analysis-using-an-event-streaming-platform -_v3_final

PayPal currently processes tens of billions of signals per day from different sources in batch and streaming mode. The data processing platform is the one powering these different analytical needs and use cases, not just at PayPal but our adjacencies like Venmo, Hyperwallet and iZettle. End users of this platform demand access to data insights with as much flexibility as possible to explore it with low processing latency. One such use case is where our Switchboard(data de-multiplexer) platform where we process approximately 20 billion events daily and provide data to different teams and platforms with-in PayPal and also to platform outside PayPal for more insights. When we started building this platform Kafka was just another asynchronous message processing platform for us but we have seen it evolving to a place where its adds value not just in terms of event processing but also for platform resiliency and scalability. Takeaway for the audience: Most people work with and have knowledge about data. With this talk I want to present information which is relevant and meaningful to the audience. Information and examples which will make it easier for attendees to understand our complex system and hopefully have some practical takeaways to use Kafka for similar problems on their hand.

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Over 100 million subscribers from over 190 countries enjoy the Netflix service. This leads to over a trillion events, amounting to 3 PB, flowing through the Keystone infrastructure to help improve customer experience and glean business insights. The self-serve Keystone stream processing service processes these messages in near real-time with at-least once semantics in the cloud. This enables the users to focus on extracting insights, and not worry about building out scalable infrastructure. I’ll share the details about this platform, and our experience building it.

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

Activision Data team has been running a data pipeline for a variety of Activision games for many years. Historically we used a mix of micro-batch microservices coupled with classic Big Data tools like Hadoop and Hive for ETL. As a result, it could take up to 4-6 hours for data to be available to the end customers. In the last few years, the adoption of data in the organization skyrocketed. We needed to de-legacy our data pipeline and provide near-realtime access to data in order to improve reporting, gather insights faster, power web and mobile applications. I want to tell a story about heavily leveraging Kafka Streams and Kafka Connect to reduce the end latency to minutes, at the same time making the pipeline easier and cheaper to run. We were able to successfully validate the new data pipeline by launching two massive games just 4 weeks apart.

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

HostedbyConfluent

Several different frameworks have been developed to draw data from Kafka and maintain standard SQL over continually changing data. This provides an easy way to query and transform data - now accessible by orders of magnitude more users. At the same time, using Standard SQL against changing data is a new pattern for many engineers and analysts. While the language hasn’t changed, we’re still in the early stages of understanding the power of SQL over Kafka - and in some interesting ways, this new pattern introduces some exciting new idioms. In this session, we’ll start with some basic use cases of how Standard SQL can be effectively used over events in Kafka- including how these SQL engines can help teams that are brand new to streaming data get started. From there, we’ll cover a series of more advanced functions and their implications, including: - WHERE clauses that contain time change the validity intervals of your data; you can programmatically introduce and retract records based on their payloads! - LATERAL joins turn streams of query arguments into query results; they will automatically share their query plans and resources! - GROUP BY aggregations can be applied to ever-growing data collections; reduce data that wouldn't even fit in a database in the first place. We'll review in-production examples where each of these cases make unmodified Standard SQL, run and maintain over data streams in Kafka, and provide the functionality of bespoke stream processors.

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

HostedbyConfluent

To remain competitive, organizations need to democratize access to fast analytics, not only to gain real-time insights on their business but also to power smart apps that need to react in the moment. In this session, you will learn how Kafka and SingleStore enable modern, yet simple data architecture to analyze both fast paced incoming data as well as large historical datasets. In particular, you will understand why SingleStore is well suited process data streams coming from Kafka.

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

HostedbyConfluent

This is a story about what happens when a distributed system becomes a big part of a small team's infrastructure. This distributed system was Kafka and the team size was one engineer. I will discuss my failures along with my journey of deploying Kafka at scale with very little prior distributed systems experience. In this presentation, we will discuss how unique insights in the following organization culture, engineering and metrics created tailwinds and headwinds. This presentation will be a tactical approach to conquering a complex system with an understaffed team while your business is growing fast. I will discuss how the use case and resilience requirements for our Kafka cluster change as the user base grew from 100K users to over 6 million.

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

DataOps challenges us to build data experiences in a repeatable way. For those with Kafka, this means finding a means of deploying flows in an automated and consistent fashion. The challenge is to make the deployment of Kafka flows consistent across different technologies and systems: the topics, the schemas, the monitoring rules, the credentials, the connectors, the stream processing apps. And ideally not coupled to a particular infrastructure stack. In this talk we will discuss the different approaches and benefits/disadvantages to automating the deployment of Kafka flows including Git operators and Kubernetes operators. We will walk through and demo deploying a flow on AWS EKS with MSK and Kafka Connect using GitOps practices: including a stream processing application, S3 connector with credentials held in AWS Secrets Manager.

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

HostedbyConfluent

http://flink-forward.org/kb_sessions/flink-and-beam-current-state-roadmap/ It is no secret that the Dataflow model, which evolved from Google’s MapReduce, Flume, and MillWheel, has been a major influence to Apache Flink’s streaming API. The essentials of this model are captured in Apache Beam. Beam provides the Dataflow API with the option to deploy to various backends (e.g. Flink, Spark). In this talk we will examine the current state of the Flink Runner. Beam’s Runners manage the translation of the Beam API into the backend API. The Beam project itself has made an effort to summarize the capabilities of each Runner to provide an overview of the supported API concepts. From all open sources backends, Flink is currently the Runner which supports the most features. We will look at the supported Beam features and their counterpart in Flink. Further, we will look at potential improvements and upcoming features of the Flink Runner.

Maximilian Michels - Flink and Beam

Maximilian Michels - Flink and Beam

Maximilian Michels - Flink and Beam

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Tendances (20)

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

user Behavior Analysis with Session Windows and Apache Kafka's Streams API

Using Apache Kafka to Analyze Session Windows

Using Apache Kafka to Analyze Session Windows

Using Apache Kafka to Analyze Session Windows

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Kafka Summit NYC 2017 - Every Message Counts: Kafka as a Foundation for Highl...

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Maximize the Business Value of Machine Learning and Data Science with Kafka (...

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Kafka Summit NYC 2017 - Stream it Together: 3 Realities of Modern Programming

Time series-analysis-using-an-event-streaming-platform -_v3_final

Time series-analysis-using-an-event-streaming-platform -_v3_final

Time series-analysis-using-an-event-streaming-platform -_v3_final

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

AWS Re-Invent 2017 Netflix Keystone SPaaS - Monal Daxini - Abd320 2017

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

SingleStore & Kafka: Better Together to Power Modern Real-Time Data Architect...

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

DataOps Automation for a Kafka Streaming Platform (Andrew Stevenson + Spiros ...

Maximilian Michels - Flink and Beam

Maximilian Michels - Flink and Beam

Maximilian Michels - Flink and Beam

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

En vedette

In this talk, you’ll learn how we’re taking Comcast’s Technology and Product group’s massive, heterogeneous set of data collection systems and centralizing on a single platform built around Kafka. These data collection systems are used for everything from business analytics, to near-real time operations, to executive reporting. We’ll go over what it takes to wrangle streaming data across an enterprise, including the need for, and our approaches to: Schema management, both at schema creation time and when schema evolution is required Data ingest and cleansing Multi-datacenter collection and failover How we use the same data stream for many different purposes, across many different teams

Towards A Stream Centered Enterprise, Gabriel Commeau

Towards A Stream Centered Enterprise, Gabriel Commeau

Towards A Stream Centered Enterprise, Gabriel Commeau

In the last few years, Apache Kafka has been used extensively in enterprises for real-time data collecting, delivering, and processing. In this presentation, Jun Rao, Co-founder, Confluent, gives a deep dive on some of the key internals that help make Kafka popular. - Companies like LinkedIn are now sending more than 1 trillion messages per day to Kafka. Learn about the underlying design in Kafka that leads to such high throughput. - Many companies (e.g., financial institutions) are now storing mission critical data in Kafka. Learn how Kafka supports high availability and durability through its built-in replication mechanism. - One common use case of Kafka is for propagating updatable database records. Learn how a unique feature called compaction in Apache Kafka is designed to solve this kind of problem more naturally.

Deep Dive into Apache Kafka

Deep Dive into Apache Kafka

Deep Dive into Apache Kafka

We are at the dawn of an age where listening to data drives the difference between unicorns and dinosaurs. Learn how HomeAway leverages Kafka as the “dial tone for data” enabling enterprises to connect disparate data sources “supercharging” them for a variety of use cases: SLA monitoring, A/B Testing, Visitor Segmentation, Fraud Detection, real-time ETL and more. The secret? Come find out.

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

In the financial industry, losing data is unacceptable. Financial firms are adopting Kafka for their critical applications. Kafka provides the low latency, high throughput, high availability, and scale that these applications require. But can it also provide complete reliability? As a system architect, when asked “Can you guarantee that we will always get every transaction,” you want to be able to say “Yes” with total confidence. In this session, we will go over everything that happens to a message – from producer to consumer, and pinpoint all the places where data can be lost – if you are not careful. You will learn how developers and operation teams can work together to build a bulletproof data pipeline with Kafka. And if you need proof that you built a reliable system – we’ll show you how you can build the system to prove this too.

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

www.confluent.io/online-talks/ This talk will introduce Kafka Streams and help you understand how to map practical data problems to stream processing and how to write applications that process streams of data at scale using Kafka Streams. We will also cover what is stream processing, why one should care about stream processing, where Apache Kafka and Kafka Streams fit in, the hard parts of stream processing, and how Kafka Streams solves those problems; along with a concrete example of how these ideas tie together in Kafka Streams and in the big picture of your data center.

Demystifying Stream Processing with Apache Kafka

Demystifying Stream Processing with Apache Kafka

Demystifying Stream Processing with Apache Kafka

While we frequently talk about how to build interesting products on top of machine and event data, the reality is that collecting, organizing, providing access to, and managing this data is where most people get stuck. Many organizations understand the use cases around their data – fraud detection, quality of service and technical operations, user behavior analysis, for example – but are not necessarily data infrastructure experts. In this session, we’ll follow the flow of data through an end to end system built to handle tens of terabytes an hour of event-oriented data, providing real time streaming, in-memory, SQL, and batch access to this data. We’ll go into detail on how open source systems such as Hadoop, Kafka, Solr, and Impala/Hive are actually stitched together; describe how and where to perform data transformation and aggregation; provide a simple and pragmatic way of managing event metadata; and talk about how applications built on top of this platform get access to data and extend its functionality. Attendees will leave this session knowing not just which open source projects go into a system such as this, but how they work together, what tradeoffs and decisions need to be addressed, and how to present a single general purpose data platform to multiple applications. This session should be attended by data infrastructure engineers and architects planning, building, or maintaining similar systems.

Building an Event-oriented Data Platform with Kafka, Eric Sammer

Building an Event-oriented Data Platform with Kafka, Eric Sammer

Building an Event-oriented Data Platform with Kafka, Eric Sammer

At Dropbox we are currently handling approximately 10,000,000 messages per second at peak across our handful of Kafka clusters. The largest of which has hit throughputs of 7,000,000 per second (~30 Gbps) on only 20 nodes. We’ll walk you through the steps we took to get where we are, the design that works for us — and those that didn’t. We’ll talk about the tooling we had to build and what we want to see exist. We’ll dive deeper into configuration and provide a blueprint you can follow. We’ll talk about the trials and tribulations of using Kafka — including ways we’ve set our clusters on fire, ways we’ve lost data, ways we’ve turned our hairs gray, and ways we’ve heroically saved the day for our users. Finally, we’ll spend time on some of the work we’re doing to handle consumer coordination across our many different systems and to integrate Kafka into a well established corporate infrastructure. (I.e., making Kafka “”play nice”” with everybody.)

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

The concept of stream processing has been around for a while and most software systems continuously transform streams of inputs into streams of outputs. Yet the idea of directly modeling stream processing in infrastructure systems is just coming into its own after a few decades on the periphery. At its core, stream processing is simple: read data in, process it, and maybe emit some data out. So why are there so many stream processing frameworks that all define their own terminology? And are the components of each even comparable? Why do I need to know about spouts or DStreams just to process a simple sequence of records? Depending on your application’s requirements, you may not need a framework. This talk will be delivered by one of the creators of the popular stream data systems Apache Kafka and will abstract away the details of individual frameworks while describing the key features they provide. These core features include scalability and parallelism through data partitioning, fault tolerance and event processing order guarantees, support for stateful stream processing, and handy stream processing primitives such as windowing. Based on our experience building and scaling Kafka to handle streams that captured hundreds of billions of records per day — this presentation will help you understand how to map practical data problems to stream processing and how to write applications that process streams of data at scale.

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Building a real-time pipeline from scratch that is able to handle billion+ transactions per day, store, analyze and visualize it all in real-time has never been easier. In this build-as-we-go talk, we’ll create a front-to-back architecture that does exactly that. * we’ll start with a simple producer emitting a few messages and publishing them onto a Kafka queue * on consuming end of the queue a Spark-based Streamliner process will pick them up and store in MemSQL * ZoomData will connect to MemSQL for real-time visualization where we’ll be able to ask various questions and see answers change as data is flowing through the system * we’ll quickly make the entire pipeline more complex by increasing the amount of data as well as complexity of the data, until reaching 100K transactions per second As we walk through this demo, we will touch on cross data-center Kafka and MemSQL set-ups, speed limitations if any as well as echo back to real-life use cases of a similar set-up used in Goldman’s Asset Management division for the purposes of Portfolio Management & Trading.

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Securing Kafka

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of continuously changing data in real time? The answer is stream processing, and one system that has become a core hub for streaming data is Apache Kafka. This presentation will give a brief introduction to Apache Kafka and describe its usage as a platform for streaming data. It will explain how Kafka serves as a foundation for both streaming data pipelines and applications that consume and process real-time data streams. It will introduce some of the newer components of Kafka that help make this possible, including Kafka Connect, a framework for capturing continuous data streams, and Kafka Streams, a lightweight stream processing library. This is talk 1 out of 6 from the Kafka Talk Series. http://www.confluent.io/apache-kafka-talk-series/introduction-to-stream-processing-with-apache-kafka

Introduction To Streaming Data and Stream Processing with Apache Kafka

Introduction To Streaming Data and Stream Processing with Apache Kafka

Introduction To Streaming Data and Stream Processing with Apache Kafka

Many companies are adopting Apache Kafka to power their data pipelines, including LinkedIn, Netflix, and Airbnb. Kafka’s ability to handle high throughput real-time data makes it a perfect fit for solving the data integration problem, acting as the common buffer for all your data and bridging the gap between streaming and batch systems. However, building a data pipeline around Kafka today can be challenging because it requires combining a wide variety of tools to collect data from disparate data systems. One tool streams updates from your database to Kafka, another imports logs, and yet another exports to HDFS. As a result, building a data pipeline can take significant engineering effort and has high operational overhead because all these different tools require ongoing monitoring and maintenance. Additionally, some of the tools are simply a poor fit for the job: the fragmented nature of the data integration tools ecosystem lead to creative but misguided solutions such as misusing stream processing frameworks for data integration purposes. We describe the design and implementation of Kafka Connect, Kafka’s new tool for scalable, fault-tolerant data import and export. First we’ll discuss some existing tools in the space and why they fall short when applied to data integration at large scale. Next, we will explore Kafka Connect’s design and how it compares to systems with similar goals, discussing key design decisions that trade off between ease of use for connector developers, operational complexity, and reuse of existing connectors. Finally, we’ll discuss how standardizing on Kafka Connect can ultimately lead to simplifying your entire data pipeline, making ETL into your data warehouse and enabling stream processing applications as simple as adding another Kafka connector. eventbrite_kafka_summit_event_logo_v3-035858-edited.png

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Presentation by Nick Dearden, Direct, Product and Engineering, Confluent It’s 3 am. Do you know how your Kafka cluster is doing? With over 150 metrics to think about, operating a Kafka cluster can be daunting, particularly as a deployment grows. Confluent Control Center is the only complete monitoring and administration product for Apache Kafka and is designed specifically for making the Kafka operators life easier. Join Confluent as we cover how Control Center is used to simplify deployment, operability, and ensure message delivery. Watch the recording: https://www.confluent.io/online-talk/monitoring-and-alerting-apache-kafka-with-confluent-control-center/

Monitoring Apache Kafka with Confluent Control Center

Monitoring Apache Kafka with Confluent Control Center

Monitoring Apache Kafka with Confluent Control Center

With the introduction of connect and streams API in 2016, Apache Kafka is becoming the defacto solution for anyone looking to build a streaming platform. The community continues to add additional capabilities to make it the complete solution for streaming data. Join us as we review the latest additions in Apache Kafka 0.10.2. In addition, we’ll cover what’s new in Confluent Enterprise 3.2 that makes it possible for running Kafka at scale.

What's new in Confluent 3.2 and Apache Kafka 0.10.2

What's new in Confluent 3.2 and Apache Kafka 0.10.2

What's new in Confluent 3.2 and Apache Kafka 0.10.2

Presentation by Ewen Cheslack-Postava, Engineer, Apache Kafka Committer, Confluent In streaming workloads, often times data produced at the source is not useful down the pipeline or it requires some transformation to get it into usable shape. Similarly, where sensitive data is concerned, filtering of topics is helpful to ensure that the wrong data doesn't get to the wrong place. The newest release of Apache Kafka now offers the ability to do transformations on individual messages, making is possible to implement finer grained transformations customized to your unique needs. In this session we’ll talk about the new single message transform capabilities, how to use them to implement things like data masking and advanced partitioning, and when you’ll need to use more complex tools like the Kafka Streams API instead.

Data Pipelines Made Simple with Apache Kafka

Data Pipelines Made Simple with Apache Kafka

Data Pipelines Made Simple with Apache Kafka

Microservices in the Apache Kafka Ecosystem

Microservices in the Apache Kafka Ecosystem

Microservices in the Apache Kafka Ecosystem

Connected Homes is at the forefront of IoT in the UK. Spun out of British Gas in 2012, its expanding Hive IoT product range & its access to the largest pool of UK smart meter data uniquely positions it as a key player in the UK market. We will share with you how Apache Kafka has become a strategic technology used throughout the business and explore some of our use cases. We will give a brief overview of Connected Homes and why Apache Kafka is being adopted in teams for operational, feature and realtime data science purposes. Deeper technical insights will be shown around smart meter customers and how we use Apache Kafka to provide realtime alerting.

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

En vedette (20)

Towards A Stream Centered Enterprise, Gabriel Commeau

Towards A Stream Centered Enterprise, Gabriel Commeau

Towards A Stream Centered Enterprise, Gabriel Commeau

Deep Dive into Apache Kafka

Deep Dive into Apache Kafka

Deep Dive into Apache Kafka

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

Kafka, the "DialTone for Data": Building a self-service, scalable, streaming ...

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

When it Absolutely, Positively, Has to be There: Reliability Guarantees in Ka...

Demystifying Stream Processing with Apache Kafka

Demystifying Stream Processing with Apache Kafka

Demystifying Stream Processing with Apache Kafka

Building an Event-oriented Data Platform with Kafka, Eric Sammer

Building an Event-oriented Data Platform with Kafka, Eric Sammer

Building an Event-oriented Data Platform with Kafka, Eric Sammer

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

Deploying Kafka at Dropbox, Mark Smith, Sean Fellows

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Introducing Kafka Streams: Large-scale Stream Processing with Kafka, Neha Nar...

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

Securing Kafka

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Introduction To Streaming Data and Stream Processing with Apache Kafka

Introduction To Streaming Data and Stream Processing with Apache Kafka

Introduction To Streaming Data and Stream Processing with Apache Kafka

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka Connect: Real-time Data Integration at Scale with Apache Kafka, Ewen Ch...

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Monitoring Apache Kafka with Confluent Control Center

Monitoring Apache Kafka with Confluent Control Center

Monitoring Apache Kafka with Confluent Control Center

What's new in Confluent 3.2 and Apache Kafka 0.10.2

What's new in Confluent 3.2 and Apache Kafka 0.10.2

What's new in Confluent 3.2 and Apache Kafka 0.10.2

Data Pipelines Made Simple with Apache Kafka

Data Pipelines Made Simple with Apache Kafka

Data Pipelines Made Simple with Apache Kafka

Microservices in the Apache Kafka Ecosystem

Microservices in the Apache Kafka Ecosystem

Microservices in the Apache Kafka Ecosystem

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

Never at Rest - IoT and Data Streaming at British Gas Connected Homes, Paul M...

Similaire à Kafka, Killer of Point-to-Point Integrations, Lucian Lita

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

A Feature Store enables machine learning (ML) features to be registered, discovered, and used as part of ML pipelines, thus making it easier to transform and validate the training data that is fed into machine learning systems. Feature stores can also enable consistent engineering of features between training and inference, but to do so, they need a common data processing platform. The first Feature Stores, developed at hyperscale AI companies such as Uber, Airbnb, and Facebook, enabled feature engineering using domain specific languages, providing abstractions tailored to the companies’ feature engineering domains. However, a general purpose Feature Store needs a general purpose feature engineering, feature selection, and feature transformation platform. In this talk, we describe how we built a general purpose, open-source Feature Store for ML around dataframes and Apache Spark. We will demonstrate how data engineers can transform and engineers features from backend databases and data lakes, while data scientists can use PySpark to select and transform features into train/test data in a file format of choice (.tfrecords, .npy, .petastorm, etc) on a file system of choice (S3, HDFS). Finally, we will show how the Feature Store enables end-to-end ML pipelines to be factored into feature engineering and data science stages that each can run at different cadences. Bio: Fabio Buso is the head of engineering at Logical Clocks AB, where he leads the Feature Store development. Fabio holds a master's degree in cloud computing and services with a focus on data intensive applications, awarded by a joint program between KTH Stockholm and TU Berlin. Topics: feature store, MLOps.

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

Data Science Milan

If there were a buzzword of the hour, it would certainly be "data mesh"! This new architectural paradigm unlocks analytic data at scale and enables rapid access to an ever-growing number of distributed domain datasets for various usage scenarios. As such, the data mesh addresses the most common weaknesses of the traditional centralized data lake or data platform architecture. And the heart of a data mesh infrastructure must be real-time, decoupled, reliable, and scalable. This presentation explores how Apache Kafka, as an open and scalable decentralized real-time platform, can be the basis of a data mesh infrastructure and - complemented by many other data platforms like a data warehouse, data lake, and lakehouse - solve real business problems. There is no silver bullet or single technology/product/cloud service for implementing a data mesh. The key outcome of a data mesh architecture is the ability to build data products; with the right tool for the job. A good data mesh combines data streaming technology like Apache Kafka or Confluent Cloud with cloud-native data warehouse and data lake architectures from Snowflake, Databricks, Google BigQuery, et al.

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

Rabobank is a worldwide food- and agri-bank from the Netherlands. Rabobank wants to make a substantial contribution to welfare and prosperity in the Netherlands and to feeding the world sustainably. Rabobank Group operates through Rabobank and its subsidiaries in 40 countries. Rabobank is active in both retail and wholesale banking. For our wholesale clients we provide real-time business insight information by making use of Cloudera and Hortonworks technology. An example is our recently launched service that gives insight in market performance of Rabobank customers, starting with the dairy farmers market segment, by making use of benchmark information. Our current technology stack contains Hortonworks Data Flow (HDF) and Cloudera Hadoop (CDH). Our real-time data stream is implemented by making use of Kafka and Nifi from HDF. Cloudera is used to store the data needed for the business insight information, mainly in HDFS and HBase. During our presentation we will provides insight about the project approach, the architecture and actual implementation. Speaker Jeroen Wolffensperger, Solution Architect Data, Rabobank Martijn Groen, Delivery Manager Data , Rabobank Netherlands

From an experiment to a real production environment

From an experiment to a real production environment

From an experiment to a real production environment

DataWorks Summit

There is an increasing need to unleash analytical capabilities directly to the end-users to democratize decision-making. User-Facing Analytics is a new frontier that will shape the products of tomorrow and push the limits of existing technology. It demands a solution that will scale to millions of users to provide fast, real-time insights. In this session, Xiang will talk about his journey to build Apache Pinot to tackle the analytics problem space with the architectural changes and technology inventions made over the past decade. He will also talk about how other big data companies such as LinkedIn, Uber, and Stripe power their user-facing analytical applications.

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Modern data-driven business infrastructure is not as effective as it should be when it comes to critical decision making. Eng-to-End Data pipelines are composed out of fundamentally diverse pieces of tech, each focusing on a specific frontend (e.g., DataFrames, Tensors, Streams) and running in total isolation, thus, being highly unoptimised and complex to integrate with event-based business logic. Our research group has been looking into ways we can use advanced systems theory to compile, optimise and execute distributed functions in unison across the whole spectrum of data-driven programming, leading to a unified way to combine analytics and services all the way down to hardware execution and make continuous intelligence a reality. Key Takeaways Introducing the concept of Continuous Intelligence and why we are not there yet. Pinpointing weaknesses in the current way we structure data-driven pipelines today Explaining the potential of an Intermediate Representation (IR) and Shared Hardware Execution support to solve the problem. Presenting our vision on how this new tech can be used to radically change the way we declare and distill knowledge from data in a fast-changing world.

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Set Your Data In Motion - CTO Roundtable

Set Your Data In Motion - CTO Roundtable

Set Your Data In Motion - CTO Roundtable

Technical thought leadership presentation to discuss how leading organizations move to real-time architecture to support business growth and enhance customer experience. This is a forum to discuss use cases with your peers to understand how other digital-native companies are utilizing data in motion to drive competitive advantage. Agenda: - Data in Motion with Event Streaming and Apache Kafka - Streaming ETL Pipelines - IT Modernisation and Hybrid Multi-Cloud - Customer Experience and Customer 360 - IoT and Big Data Processing - Machine Learning and Analytics

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

2018 Oracle Impact 발표자료: Oracle Enterprise AI

2018 Oracle Impact 발표자료: Oracle Enterprise AI

2018 Oracle Impact 발표자료: Oracle Enterprise AI

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Paris Data Engineers !

"Data mesh is a relatively recent architectural innovation, espoused as one of the best ways to fix analytic data. We renegotiate aged social conventions by focusing on treating data as a product, with a clearly defined data product owner, akin to that of any other product. In addition, we focus on building out a self-service platform with integrated governance, letting consumers safely access and use the data they need to solve their business problems. Data mesh is prescribed as a solution for _analytical data_, so that conventionally analytical results (think weekly sales or monthly revenue reports) can be more accurately and predictably computed. But what about non-analytical business operations? Would they not also benefit from data products backed by self-service capabilities and dedicated owners? If you've ever provided a customer with an analytical report that differed from their operational conclusions, then this talk is for you. Adam discusses the resounding successes he has seen from applying data mesh _off-label_ to both analytical and operational domains. The key? Event streams. Well-defined, incrementally updating data products that can power both real-time and batch-based applications, providing a single source of data for a wide variety of application and analytical use cases. Adam digs into the common areas of success seen across numerous clients and customers and provides you with a set of practical guidelines for implementing your own minimally viable data mesh. Finally, Adam covers the main social and technical hurdles that you'll encounter as you implement your own data mesh. Learn about important data use cases, data domain modeling techniques, self-service platforms, and building an iteratively successful data mesh."

Off-Label Data Mesh: A Prescription for Healthier Data

Off-Label Data Mesh: A Prescription for Healthier Data

Off-Label Data Mesh: A Prescription for Healthier Data

HostedbyConfluent

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenzhong XU | Current 2022 If you are a data scientist or a platform engineer, you probably can relate to the pains of working with the current explosive growth of Data/ML technologies and toolings. With many overlapping options and steep learning curves for each, it’s increasingly challenging for data science teams. Many platform teams started thinking about building an abstracted ML platform layer to support generalized ML use cases. But there are many complexities involved, especially as the underlying real-time data is shifting into the mainstream. In this talk, we’ll discuss why ML platforms can benefit from a simple and ""invisible"" abstraction. We’ll offer some evidence on why you should consider leveraging streaming technologies even if your use cases are not real-time yet. We’ll share learnings (combining both ML and Infra perspectives) about some of the hard complexities involved in building such simple abstractions, the design principles behind them, and some counterintuitive decisions you may come across along the way. By the end of the talk, I hope data scientists can walk away with some tips on how to evaluate ML platforms, and platform engineers learned a few architectural and design tricks.

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

HostedbyConfluent

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Twitter is powered by thousands of microservices running on an internal cloud platform consisting of a suite of multitenant platform services that offer compute, storage, messaging, monitoring, etc. as a service. These platforms have thousands of tenants and run atop hundreds of thousands of servers both on-premises and in the public cloud. The scale of diversity in Twitter’s multitenant infrastructure services makes it extremely difficult to effectively forecast capacity, compute resource utilization, and cost and drive efficiency. Vinu Charanya explains how she and her team are building a system that captures, defines, provisions, meters, and charges infrastructure resources, redefining how systems are built atop Twitter infrastructure. The infrastructure resources include primitive bare metal servers and VMs in the public cloud and abstract resources offered by multitenant services such as a compute platform (powered by Apache Aurora and Mesos), storage (Manhattan for key-value, cache, RDBMS), and observability. Along the way, Vinu shares how Twitter used this data to better plan capacity and drive a cultural change in engineering that helped improve overall resource utilization and led to significant savings in infrastructure spend.

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

Building a real-time monitoring service that handles millions of custom events per second while satisfying complex rules, varied throughput requirements, and numerous dimensions simultaneously is a complex endeavor. Sumeet Singh and Mridul Jain explain how Yahoo approached these challenges with Apache Storm Trident, Kafka, HBase, and OpenTSDB and discuss the lessons learned along the way. Sumeet and Mridul explain scaling patterns backed by real scenarios and data to help attendees develop their own architectures and strategies for dealing with the scale challenges that come with real-time big data systems. They also explore the tradeoffs made in catering to a diverse set of daily users and the associated usability challenges that motivated Yahoo to build a self-serve, easy-to-use platform that requires minimal programming experience. Sumeet and Mridul then discuss event-level tracking for debugging and troubleshooting problems that our users may encounter at this scale. Over the course of their talk, they also address building infrastructure and operational intelligence with anomaly detection, alert correlation, and trend analysis based on the monitoring platform.

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Confluent kafka meetupseattle jan2017

Confluent kafka meetupseattle jan2017

Confluent kafka meetupseattle jan2017

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

Modern Data Architectures for Real Time Analytics & Engagement

Modern Data Architectures for Real Time Analytics & Engagement

Modern Data Architectures for Real Time Analytics & Engagement

Amazon Web Services

Spark and machine learning in microservices architecture

Spark and machine learning in microservices architecture

Spark and machine learning in microservices architecture

Stepan Pushkarev

Twitter is powered by thousands of microservices that run on our internal Cloud platform which consists of a suite of multi-tenant platform services that offer Compute, Storage, Messaging, Monitoring, etc as a service. These platforms have thousands of tenants and run atop hundreds of thousands of servers, across on-prem & the public cloud. The scale & diversity in multi-tenant infrastructure services make it extremely difficult to effectively forecast capacity, compute resource utilization & cost and drive efficiency. In this talk, I would like to share how my team is building a system (Kite - A unified service manager) to help define, model, provision, meter & charge infrastructure resources. The infrastructure resources include primitive bare metal servers / VMs on the public cloud and abstract resources offered by multi-tenant services such as our Compute platform (powered by Apache Aurora/Mesos), Storage (Manhattan for key/val, Cache, RDBMS), Observability. Along with how we solved this problem, I also intend to share a few case-studies on how we were able to use this data to better plan capacity & drive a cultural change in engineering that helped improve overall resource utilization & drive significant savings in infrastructure spending.

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

Similaire à Kafka, Killer of Point-to-Point Integrations, Lucian Lita (20)

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

MLOps with a Feature Store: Filling the Gap in ML Infrastructure

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

The Heart of the Data Mesh Beats in Real-Time with Apache Kafka

From an experiment to a real production environment

From an experiment to a real production environment

From an experiment to a real production environment

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Data Democracy: Journey to User-Facing Analytics - Pulsar Summit SF 2022

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Continuous Intelligence - Intersecting Event-Based Business Logic and ML

Set Your Data In Motion - CTO Roundtable

Set Your Data In Motion - CTO Roundtable

Set Your Data In Motion - CTO Roundtable

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

2018 Oracle Impact 발표자료: Oracle Enterprise AI

2018 Oracle Impact 발표자료: Oracle Enterprise AI

2018 Oracle Impact 발표자료: Oracle Enterprise AI

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Delta Lake OSS: Create reliable and performant Data Lake by Quentin Ambard

Off-Label Data Mesh: A Prescription for Healthier Data

Off-Label Data Mesh: A Prescription for Healthier Data

Off-Label Data Mesh: A Prescription for Healthier Data

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

Considerations for Abstracting Complexities of a Real-Time ML Platform, Zhenz...

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

[Velocity Conf 2017 NY] How Twitter built a framework to improve infrastructu...

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...

Confluent kafka meetupseattle jan2017

Confluent kafka meetupseattle jan2017

Confluent kafka meetupseattle jan2017

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

TIBCO Advanced Analytics Meetup (TAAM) - June 2015

Modern Data Architectures for Real Time Analytics & Engagement

Modern Data Architectures for Real Time Analytics & Engagement

Modern Data Architectures for Real Time Analytics & Engagement

Spark and machine learning in microservices architecture

Spark and machine learning in microservices architecture

Spark and machine learning in microservices architecture

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

[Kubecon 2017 Austin, TX] How We Built a Framework at Twitter to Solve Servic...

Plus de confluent

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

El Stream processing es un requisito previo de la pila de data streaming, que impulsa aplicaciones y pipelines en tiempo real. Permite una mayor portabilidad de datos, una utilización optimizada de recursos y una mejor experiencia del cliente al procesar flujos de datos en tiempo real. En nuestro taller práctico híbrido, aprenderás cómo filtrar, unir y enriquecer fácilmente datos en tiempo real dentro de Confluent Cloud utilizando nuestro servicio Flink sin servidor.

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Our talk will explore the transformative impact of integrating Confluent, HiveMQ, and SparkPlug in Industry 4.0, emphasizing the creation of a Unified Namespace. In addition to the creation of a Unified Namespace, our webinar will also delve into Stream Governance and Scaling, highlighting how these aspects are crucial for managing complex data flows and ensuring robust, scalable IIoT-Platforms. You will learn how to ensure data accuracy and reliability, expand your data processing capabilities, and optimize your data management processes. Don't miss out on this opportunity to learn from industry experts and take your business to the next level.

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

La arquitectura impulsada por eventos (EDA) será el corazón del ecosistema de MAPFRE. Para seguir siendo competitivas, las empresas de hoy dependen cada vez más del análisis de datos en tiempo real, lo que les permite obtener información y tiempos de respuesta más rápidos. Los negocios con datos en tiempo real consisten en tomar conciencia de la situación, detectar y responder a lo que está sucediendo en el mundo ahora.

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Traditional data pipelines often face scalability issues and challenges related to cost, their monolithic design, and reliance on batch data processing. They also typically operate under the premise that all data needs to be stored in a single centralized data source before it's put to practical use. Confluent Cloud on Amazon Web Services (AWS) provides a fully managed cloud-native platform that helps you simplify the way you build real-time data flows using streaming data pipelines and Apache Kafka.

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

An in depth look at how Confluent is being used in the financial services industry. Gain an understanding of how organisations are utilising data in motion to solve common problems and gain benefits from their real time data capabilities. It will look more deeply into some specific use cases and show how Confluent technology is used to manage costs and mitigate risks. This session is aimed at Solutions Architects, Sales Engineers and Pre Sales, and also the more technically minded business aligned people. Whilst this is not a deeply technical session, a level of knowledge around Kafka would be helpful.

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Transforming applications built with traditional messaging solutions such as TIBCO, MQ and Solace to be scalable, reliable and ready for the move to cloud How can applications built with traditional messaging technologies like TIBCO, Solace and IBM MQ be modernised and be made cloud ready? What are the advantages to Event Streaming approaches to pub/sub vs traditional message queues? What are the strengeths and weaknesses of both approaches, and what use cases and requirements are actually a better fit for messaging than Kafka?

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Vous apprendrez également à : • Créer plus rapidement des produits et fonctionnalités à l’aide d’une suite complète de connecteurs et d’outils de gestion des flux, et à connecter vos environnements à des pipelines de données • Protéger vos données et charges de travail les plus critiques grâce à des garanties intégrées en matière de sécurité, de gouvernance et de résilience • Déployer Kafka à grande échelle en quelques minutes tout en réduisant les coûts et la charge opérationnelle associés

Data In Motion Paris 2023

Data In Motion Paris 2023

Data In Motion Paris 2023

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

Plus de confluent (20)

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Data In Motion Paris 2023

Data In Motion Paris 2023

Data In Motion Paris 2023

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

Dernier

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking Escorts Service Available Whatsapp SABANA ☎️ : [+91-7001035870] Escorts Service are always ready to make their clients happy. Their exotic looks and sexy personalities are sure to turn heads. You can enjoy with them, including massages and erotic encounters. Our area Escorts are young and sexy, so you can expect to have an exotic time with them. They are trained to satiate your naughty nerves and they can handle anything that you want. They are also intelligent, so they know how to make you feel comfortable and relaxed Independent Escorts Service They know all the sex positions and can satisfy you in any way that you desire. They can even give you erotic massages to help you relax before your session. This is essential, because a man who is stressed won’t be receptive to the pleasures of sex. They also know how to play with your sexy organs, so you’ll have plenty of foreplay and cuddling. P252024SS SERVICE ✅ ❣️ ⭐➡️HOT & SEXY MODELS // COLLEGE GIRLS HOUSE WIFE RUSSIAN , AIR HOSTES ,VIP MODELS . AVAILABLE FOR COMPLETE ENJOYMENT WITH HIGH PROFILE INDIAN MODEL AVAILABLE HOTEL & HOME ★ SAFE AND SECURE HIGH CLASS SERVICE AFFORDABLE RATE ★ SATISFACTION,UNLIMITED ENJOYMENT. ★ All Meetings are confidential and no information is provided to any one at any cost. ★ EXCLUSIVE PROFILes Are Safe and Consensual with Most Limits Respected ★ Service Available In: - HOME & HOTEL Star Hotel Service .In Call & Out call SeRvIcEs : ★ A-Level (star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob)Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without condom ★ANAL SERVICES.

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Booking Booking Now open +91- 7737669865 Why you Choose Us- +91- 7737669865 HOT⇄ 7737669865 Mr ashu ji Call Mr ashu Ji +91- 7737669865 (V020524]N) 𝐇𝐨𝐭𝐞𝐥 𝐑𝐨𝐨𝐦𝐬 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐑𝐚𝐭𝐞 𝐒𝐡𝐨𝐭𝐬/𝐇𝐨𝐮𝐫𝐲🆓 .█▬█⓿▀█▀ 𝐈𝐍𝐃𝐄𝐏𝐄𝐍𝐃𝐄𝐍𝐓 𝐆𝐈𝐑𝐋 𝐕𝐈𝐏 𝐄𝐒𝐂𝐎𝐑𝐓 Hello Guys ! High Profiles young Beauties and Good Looking standard Profiles Available , Enquire Now if you are interested in Hifi Service and want to get connect with someone who can understand your needs. Service offers you the most beautiful High Profile sexy independent female Escorts in genuine ✔✔✔ To enjoy with hot and sexy girls ✔✔✔ ★providing:- • Models • vip Models • Russian Models

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Built environment is known for its capacity, capability, role, relevance and importance to change the quality of life of the occupants and communities. Presentation focuses on options which need to be leveraged to make buildings sustainable, cost-effective, energy efficient, resource efficient, qualitative over its entire life-cycle through designing, construction, operation. It calls for making buildings green and sustainable.

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

JIT KUMAR GUPTA

data_management_and _data_science_cheat_sheet.pdf

data_management_and _data_science_cheat_sheet.pdf

data_management_and _data_science_cheat_sheet.pdf

The purpose of Integrated Test Rig is to assess performance of Fuel Cooled Oil Cooler (FCOC), HP Pump & Oil Pump prior to fitment on Aero Engine. The design of rig is such that it can cater to individual testing of individual LRU or integrated testing of all the LRU’s installed together (in combination). The pumps under test shall be mounted directly on the high-speed electric motor flange through adapter quill shaft assembly. Variable speed drives shall be provided for all the pumps under testing. The rig shall cater for real time monitoring and recording of performance parameters. KEY FEATURES Key features of this Rig are as follows: - 1. It has separate power packs for both the fluids i.e. ATF Power pack for ATF Fluid, OX Power Pack for OX-27 Oil. 2. This Rig is designed for high temperature Application. It works on 200 deg. Celsius. 3. Two Dedicated heating systems have been integrated with this Rig to raise the temperature of OX-27 Oil. 4. In both Power Packs insulated tanks are being used, which is good for safe operation and safety of operator. 5. This rig is designed in such a way, that it’s OX power pack is in its left side, ATF Power pack is in right Side and test bed is in Centre. 6. A big Test bed have been provided for smooth testing of UUT’s. 7. All piping is being insulated for safe operation and maintenance. 8. It comprises with DAQ System. 9. This rig can test several components at a time, together or separately. Application All the UUT’s are mounted on HTFE-25 Engine after successful performance testing of those. Application of this Rig are as follows: - 1. One of its application is to test LP Pump of HTFE-25. It is designed for functional testing of LP Pump. a. Free Flow Test b. Load Test 2. It is used to test HP Pump of HTFE-25. It is designed for functional testing of HP Pump. a. Free Flow Test b. Load Test 3. It is used to test Oil Pump of HTFE-25. It is designed for functional testing of Oil Pump. 4. It is used to test FCOC of HTFE-25.

Integrated Test Rig For HTFE-25 - Neometrix

Integrated Test Rig For HTFE-25 - Neometrix

Integrated Test Rig For HTFE-25 - Neometrix

Neometrix_Engineering_Pvt_Ltd

KubeKraft presentation @CloudNativeHooghly

KubeKraft presentation @CloudNativeHooghly

KubeKraft presentation @CloudNativeHooghly

sanyuktamishra911

ONLINE FOOD ORDER SYSTEM is a website designed primarily for use in the food delivery industry. This system will allow hotels and restaurants to increase scope of business by reducing the labor cost involved. The system also allows to quickly and easily manage an online menu which customers can browse and use to place orders with just few clicks. Restaurant employees then use these orders through an easy to navigate graphical interface for efficient processing.

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

Call Girl Aurangabad Indira Call Now: 8617697112 Aurangabad Escorts Booking Contact Details WhatsApp Chat: +91-8617697112 Aurangabad Escort Service includes providing maximum physical satisfaction to their clients as well as engaging conversation that keeps your time enjoyable and entertaining. Plus they look fabulously elegant; making an impressionable. Independent Escorts Aurangabad understands the value of confidentiality and discretion - they will go the extra mile to meet your needs. Simply contact them via text messaging or through their online profiles; they'd be more than delighted to accommodate any request or arrange a romantic date or fun-filled night together. We provide –

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

Call Girls in Nagpur High Profile Call Girls

Work-Permit-Receiver-in-Saudi-Aramco.pptx

Work-Permit-Receiver-in-Saudi-Aramco.pptx

Work-Permit-Receiver-in-Saudi-Aramco.pptx

JuliansyahHarahap1

notes on Evolution Of Analytic Scalability.ppt

notes on Evolution Of Analytic Scalability.ppt

notes on Evolution Of Analytic Scalability.ppt

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking Booking Now open +91- 7737669865 Why you Choose Us- +91- 7737669865 HOT⇄ 7737669865 Mr ashu ji Call Mr ashu Ji +91- 7737669865 (V020524]N) 𝐇𝐨𝐭𝐞𝐥 𝐑𝐨𝐨𝐦𝐬 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐑𝐚𝐭𝐞 𝐒𝐡𝐨𝐭𝐬/𝐇𝐨𝐮𝐫𝐲🆓 .█▬█⓿▀█▀ 𝐈𝐍𝐃𝐄𝐏𝐄𝐍𝐃𝐄𝐍𝐓 𝐆𝐈𝐑𝐋 𝐕𝐈𝐏 𝐄𝐒𝐂𝐎𝐑𝐓 Hello Guys ! High Profiles young Beauties and Good Looking standard Profiles Available , Enquire Now if you are interested in Hifi Service and want to get connect with someone who can understand your needs. Service offers you the most beautiful High Profile sexy independent female Escorts in genuine ✔✔✔ To enjoy with hot and sexy girls ✔✔✔ ★providing:- • Models • vip Models • Russian Models

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Employee leave management system project.

Employee leave management system project.

Employee leave management system project.

A Study of Urban Area Plan for Pabna Municipality

A Study of Urban Area Plan for Pabna Municipality

A Study of Urban Area Plan for Pabna Municipality

Morshed Ahmed Rahath

GAS POWER CYCLES Cycles: Otto, Diesel, Dual, Brayton - Calculation of mean effective pressure - Air standard efficiency - Comparison of cycles INTERNAL COMBUSTION ENGINES Classification - Components and their function - valve timing diagram and port timing diagram - actual and theoretical p-v diagram of two stroke and four stroke engines – carburettor - diesel pump and injector system - battery and magneto ignition system - principles of combustion and detonation in CI engines - lubrication and cooling systems - performance parameters and calculations.

Thermal Engineering Unit - I & II . ppt

Thermal Engineering Unit - I & II . ppt

Thermal Engineering Unit - I & II . ppt

DineshKumar4165

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking Escorts Service Available Whatsapp SABANA ☎️ : [+91-7001035870] Escorts Service are always ready to make their clients happy. Their exotic looks and sexy personalities are sure to turn heads. You can enjoy with them, including massages and erotic encounters. Our area Escorts are young and sexy, so you can expect to have an exotic time with them. They are trained to satiate your naughty nerves and they can handle anything that you want. They are also intelligent, so they know how to make you feel comfortable and relaxed Independent Escorts Service They know all the sex positions and can satisfy you in any way that you desire. They can even give you erotic massages to help you relax before your session. This is essential, because a man who is stressed won’t be receptive to the pleasures of sex. They also know how to play with your sexy organs, so you’ll have plenty of foreplay and cuddling. P252024SS SERVICE ✅ ❣️ ⭐➡️HOT & SEXY MODELS // COLLEGE GIRLS HOUSE WIFE RUSSIAN , AIR HOSTES ,VIP MODELS . AVAILABLE FOR COMPLETE ENJOYMENT WITH HIGH PROFILE INDIAN MODEL AVAILABLE HOTEL & HOME ★ SAFE AND SECURE HIGH CLASS SERVICE AFFORDABLE RATE ★ SATISFACTION,UNLIMITED ENJOYMENT. ★ All Meetings are confidential and no information is provided to any one at any cost. ★ EXCLUSIVE PROFILes Are Safe and Consensual with Most Limits Respected ★ Service Available In: - HOME & HOTEL Star Hotel Service .In Call & Out call SeRvIcEs : ★ A-Level (star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob)Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without condom ★ANAL SERVICES.

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

Generative AI or GenAI technology based PPT

Generative AI or GenAI technology based PPT

Generative AI or GenAI technology based PPT

2016EF22_0 solar project report rooftop projects

2016EF22_0 solar project report rooftop projects

2016EF22_0 solar project report rooftop projects

Call Girls In Mahipalpur (Delhi) Call or Whataap {+91-8377877756} Escorts Provide 24×7 Available With Room ☆☆TIMINGS 24 HOURS OPENS☆☆☆ Booking Now Gentleman Only:-Call Now Best High Class Normal Call Girls Escorts Service In Delhi NCR 24-7 Hours Available Service I provide In Delhi NCR Female Escorts Sex Service 100% Customers Satisfaction Guarantee VIP Profiles Top Grade Service 100% Cooperative All round Service ꧁❤8377877756❤꧂ InCall: – You Can Reach At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation OutCall: – Service For Out Call You Have To Come Pick The Girl From My Place. We Also Provide Door Step Services. Note: – Pic Collectors Time Passers And Bargainers Stay Away As We Respect The Value Of Your Money And Time And Expect The Same From You. 8377877756. Hygienic: – Full Ac Neat And Clean Rooms Available In Hotel 24 * 7 Hrs In Delhi NCR 8377877756. Place: – South Extension Nehru Place Saket Malviya Nagar Munirka Vasant Kunj Safdarjung Katwaria Sarai Lajpat Nagar Kalkaji Hauz Khas Mahipalpur Dwarka Karol Bagh Noida Gurgaon Faridabad All Outcall Only Hotel Service In Delhi Ncr 8377877756. We are providing. : – House Wife’s : – Private Independent House Wife’s : – Private Independent Collage Going Girls : – Corporate MNC Working Profiles : – Call Center Girls : – Live Band Girls : – Foreigners and Many More: – Independent Models Service type : – Incall Short 2500 : – Outcall Short 3500 : – Incall Night 8500 : – Outcall Night 95,00 For Pictures And Other Details Pls Whatsapp Me Otherwise Call Me Any Time Incall Outcall Both Are Services Available Door Step, House, Apartment, Guest House, Flate, All Star Hotel Available ꧁❤8377877756❤꧂ THANKS FOR VISITING. Booking 24×7 HRS

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

dollysharma2066

“HOSTEL MANAGEMENT SYSTEM” is a software developed for managing various activities in the hostel. For the past few years the number of educational institutions are increasing rapidly. Thereby the number of hostels are also increasing for the accommodation of the students studying in this institution. And hence there is a lot of strain on the person who are running the hostel and software’s are not usually used in this context. This particular project deals with the problems on managing a hostel and avoids the problems which occur when carried manually.

Hostel management system project report..pdf

Hostel management system project report..pdf

Hostel management system project report..pdf

Dernier (20)

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

COST-EFFETIVE and Energy Efficient BUILDINGS ptx

data_management_and _data_science_cheat_sheet.pdf

data_management_and _data_science_cheat_sheet.pdf

data_management_and _data_science_cheat_sheet.pdf

Integrated Test Rig For HTFE-25 - Neometrix

Integrated Test Rig For HTFE-25 - Neometrix

Integrated Test Rig For HTFE-25 - Neometrix

KubeKraft presentation @CloudNativeHooghly

KubeKraft presentation @CloudNativeHooghly

KubeKraft presentation @CloudNativeHooghly

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

ONLINE FOOD ORDER SYSTEM PROJECT REPORT.pdf

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7

Work-Permit-Receiver-in-Saudi-Aramco.pptx

Work-Permit-Receiver-in-Saudi-Aramco.pptx

Work-Permit-Receiver-in-Saudi-Aramco.pptx

notes on Evolution Of Analytic Scalability.ppt

notes on Evolution Of Analytic Scalability.ppt

notes on Evolution Of Analytic Scalability.ppt

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking

Employee leave management system project.

Employee leave management system project.

Employee leave management system project.

A Study of Urban Area Plan for Pabna Municipality

A Study of Urban Area Plan for Pabna Municipality

A Study of Urban Area Plan for Pabna Municipality

Thermal Engineering Unit - I & II . ppt

Thermal Engineering Unit - I & II . ppt

Thermal Engineering Unit - I & II . ppt

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking

Generative AI or GenAI technology based PPT

Generative AI or GenAI technology based PPT

Generative AI or GenAI technology based PPT

2016EF22_0 solar project report rooftop projects

2016EF22_0 solar project report rooftop projects

2016EF22_0 solar project report rooftop projects

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

Hostel management system project report..pdf

Hostel management system project report..pdf

Hostel management system project report..pdf

Kafka, Killer of Point-to-Point Integrations, Lucian Lita

1. Kafka, Killer of Point-to-Point Integrations Lucian Lita @datariver Intuit Data Team

4. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture Data PlaBorm & Services @datariver

5. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture login login @datariver

6. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture login change descrip1on (1,2) descrip1on @datariver

7. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture login invoice change descrip1on (1,2,3) invoice @datariver

8. 1.2B 800M 400M Tax ﬁling clickstream ac1vity in TurboTax Requirements? ü  high volume – billions of events / day ü  hybrid: cloud & colo ü  robust & reliable / HA ü  near real 1me ü  secure: encryp1on, access ü  scalable: up and down -  seasonal: traﬃc >10,00x -  high burst: up to 250k tps Oct April 15th early Feb Dec @datariver

9. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture Star1ng State: Service Fragmenta1on 10+ 15+ 5+ 10+ 7+ 5+ 15+ 0 10+ @datariver

10. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture Star1ng State: Point-to-Point Integra1ons 10+ 15+ 5+ 10+ 7+ 5+ 15+ 0 10+ @datariver

11. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture Target State @datariver

12. Company Proﬁles Deep Analy1cs A/B Tes1ng Predic1ve Modeling User Proﬁles Graph Search Real Time Analy1cs Data Capture Core Data PlaBorm & Services @datariver

13. Zoom Out: Small Business Accoun1ng Payroll Payments Inventory Invoices Repor1ng 3rd Party Apps Small Business

14. Small Business Tax Personal Finance Accountants Zoom Out: Intuit @datariver

16. Thank you! Intuit Data Team Lucian Lita @datariver