The Ecosystem is too damn big

•

5 j'aime•29,915 vues

DataWorks Summit/Hadoop Summit

The Ecosystem is too damn big

The Ecosystem
is Too Damn
Big
Andrew Brust
Senior Director, Market Strategy and Analysis
Datameer

there are too many vendors
Credit: Matt
Turck, Firstmark
Capital

ON‐PREMISES CLOUD
HADOOP AS A SERVICE OTHER
Cloudera
Hortonworks
MapR
IBM
Amazon Web Services
Microsoft Azure
Google Cloud Platform
Qubole
Altiscale
ArenaData
InfoSys
there are too many distributions

there are too many engines

there are too many projects

the problem with churn

problems
analysis paralysis
big bang platform decisions
one-size-fits-all imprecision
inevitable obsolescence
“engine choice dread”
on every single freakin’ job

too many...
SQL on Hadoop solutions
project iterations
vendor serving Apache projects
programming language choices

buzzwords
actionable insights
days instead of weeks
weeks instead of months
for the business user
data scientists
citizen data scientists
the 4th “V”
all your data in one place
data lake

MODULAR LOCKED
IN OR OUTPRODUCTS
LOOK FOR DON'T BE
NO
HARD CODING
IT'S ALL ABOUT
BALANCE
you'll need to fend for yourself

Top 6 Solutions

Industry:
build consortia
06

Vendors:
stop copying
each other’s
messages05

VCs:
stop enabling bad
marketing, funding
companies that
don’t add value04

Analyst firms:
decode things,
don’t confuse
them03

Technologists:
realize the
value of
standards02

Customers:
demand a consistent
core stack, do your
research, buy on the
merits01

thank you.andrew.brust@datameer
@andrewbrust

Contenu connexe

Tendances

Power BI as a storyteller

Power BI as a storyteller

Power BI as a storyteller

Berkovich Consulting

Today a Data Platform is expected to process and analyze a multitude of sources spanning batch files, streaming sources, backend databases, REST APIs, and more. There is clearly a need for standardizing the platform that scales and be flexible letting data engineers and data scientists focus on the business problems rather than managing the infrastructure and backend services. Another key aspect of the platform is multi-tenancy to isolate the workloads and able to track cost usage per tenant. In this talk, Richa Singhal and Esha Shah will cover how to build a scalable Data Platform using Databricks and deploy your data pipelines effectively while managing the costs. The following topics will be covered: Key tenets of a Data Platform Setup multistage environment on Databricks Build data pipelines locally and test on Databricks cluster CI/CD for data pipelines with Databricks Orchestrating pipelines using Apache Airflow – Change Data Capture using Databricks Delta Leveraging Databricks Notebooks for Analytics and Data Science teams

Scaling and Modernizing Data Platform with Databricks

Scaling and Modernizing Data Platform with Databricks

Scaling and Modernizing Data Platform with Databricks

Data engineering

Data engineering

Data engineering

Parimala Killada

In this presentation, we will study a recent use case we implemented recently. In this use case we are working with a large, metropolitan fire department. Our company has already created a complete analytics architecture for the department based upon Azure Data Factory, Databricks, Delta Lake, Azure SQL and Azure SQL Server Analytics Services (SSAS). While this architecture works very well for the department, they would like to add a real-time channel to their reporting infrastructure. This channel should serve up the following information: •The most up-to-date locations and status of equipment (fire trucks, ambulances, ladders etc.) • The current locations and status of firefighters, EMT personnel and other relevant fire department employees • The current list of active incidents within the city The above information should be visualized through an automatically updating dashboard. The central component of the dashboard will be map which automatically updates with the locations and incidents. This view should be as real-time as possible and will be used by the fire chiefs to assist with real-time decision-making on resource and equipment deployments. In this presentation, we will leverage Databricks, Spark Structured Streaming, Delta Lake and the Azure platform to create this real-time delivery channel.

Build Real-Time Applications with Databricks Streaming

Build Real-Time Applications with Databricks Streaming

Build Real-Time Applications with Databricks Streaming

This Edureka Spark Hadoop Tutorial will help you understand how to use Spark and Hadoop together. This Spark Hadoop tutorial is ideal for both beginners as well as professionals who want to learn or brush up their Apache Spark concepts. Below are the topics covered in this tutorial: 1) Spark Overview 2) Hadoop Overview 3) Spark vs Hadoop 4) Why Spark Hadoop? 5) Using Hadoop With Spark 6) Use Case - Sports Analytics (NBA)

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

It can be quite challenging keeping up with the frequent updates to the Microsoft products and understanding all their use cases and how all the products fit together. In this session we will differentiate the use cases for each of the Microsoft services, explaining and demonstrating what is good and what isn't, in order for you to position, design and deliver the proper adoption use cases for each with your customers. We will cover a wide range of products such as Databricks, SQL Data Warehouse, HDInsight, Azure Data Lake Analytics, Azure Data Lake Store, Blob storage, and AAS as well as high-level concepts such as when to use a data lake. We will also review the most common reference architectures (“patterns”) witnessed in customer adoption.

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Embarking on building a modern data warehouse in the cloud can be an overwhelming experience due to the sheer number of products that can be used, especially when the use cases for many products overlap others. In this talk I will cover the use cases of many of the Microsoft products that you can use when building a modern data warehouse, broken down into four areas: ingest, store, prep, and model & serve. It’s a complicated story that I will try to simplify, giving blunt opinions of when to use what products and the pros/cons of each.

Building a modern data warehouse

Building a modern data warehouse

Building a modern data warehouse

Tech talk on what Azure Databricks is, why you should learn it and how to get started. We'll use PySpark and talk about some real live examples from the trenches, including the pitfalls of leaving your clusters running accidentally and receiving a huge bill ;) After this you will hopefully switch to Spark-as-a-service and get rid of your HDInsight/Hadoop clusters. This is part 1 of an 8 part Data Science for Dummies series: Databricks for dummies Titanic survival prediction with Databricks + Python + Spark ML Titanic with Azure Machine Learning Studio Titanic with Databricks + Azure Machine Learning Service Titanic with Databricks + MLS + AutoML Titanic with Databricks + MLFlow Titanic with DataRobot Deployment, DevOps/MLops and Operationalization

Databricks for Dummies

Databricks for Dummies

Databricks for Dummies

Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Airbnb, Comcast, GrubHub, Facebook, FINRA, LinkedIn, Lyft, Netflix, Twitter, and Uber, in the last few years Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments over Object Stores, HDFS, NoSQL and RDBMS data stores.

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Want to see a high-level overview of the products in the Microsoft data platform portfolio in Azure? I’ll cover products in the categories of OLTP, OLAP, data warehouse, storage, data transport, data prep, data lake, IaaS, PaaS, SMP/MPP, NoSQL, Hadoop, open source, reporting, machine learning, and AI. It’s a lot to digest but I’ll categorize the products and discuss their use cases to help you narrow down the best products for the solution you want to build.

Azure data platform overview

Azure data platform overview

Azure data platform overview

Tomer Shiran est le fondateur et chef de produit (CPO) de Dremio. Tomer était le 4e employé et vice-président produit de MapR, un pionnier de l'analyse du Big Data. Il a également occupé de nombreux postes de gestion de produits et d'ingénierie chez IBM Research et Microsoft, et a fondé plusieurs sites Web qui ont servi des millions d'utilisateurs. Il est titulaire d'un Master en génie informatique de l'Université Carnegie Mellon et d'un Bachelor of Science en informatique du Technion - Israel Institute of Technology. Le Modern Data Stack meetup est ravi d'accueillir Tomer Shiran. Depuis Apache Drill, Apache Arrow maintenant Apache Iceberg, il ancre avec ses équipes des choix pour Dremio avec une vision de la plateforme de données “ouverte” basée sur des technologies open source. En plus, de ces valeurs qui évitent le verrouillage de clients dans des formats propriétaires, il a aussi le souci des coûts qu’engendrent de telles plateformes. Il sait aussi proposer un certain nombre de fonctionnalités qui transforment la gestion de données grâce à des initiatives telles Nessie qui ouvre la route du Data As Code et du transactionnel multi-processus. Le Modern Data Stack Meetup laisse “carte blanche” à Tomer Shiran afin qu’il nous partage son expérience et sa vision quant à l’Open Data Lakehouse.

From Data Warehouse to Lakehouse

From Data Warehouse to Lakehouse

From Data Warehouse to Lakehouse

Modern Data Stack France

Databricks is a Software-as-a-Service-like experience (or Spark-as-a-service) that is a tool for curating and processing massive amounts of data and developing, training and deploying models on that data, and managing the whole workflow process throughout the project. It is for those who are comfortable with Apache Spark as it is 100% based on Spark and is extensible with support for Scala, Java, R, and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib). It has built-in integration with many data sources, has a workflow scheduler, allows for real-time workspace collaboration, and has performance improvements over traditional Apache Spark.

Introduction to Azure Databricks

Introduction to Azure Databricks

Introduction to Azure Databricks

Solr vs. Elasticsearch - Case by Case

Solr vs. Elasticsearch - Case by Case

Solr vs. Elasticsearch - Case by Case

Alexandre Rafalovitch

Introduction to Apache Spark

Introduction to Apache Spark

Introduction to Apache Spark

Anastasios Skarlatidis

Introducing Databricks Delta

Introducing Databricks Delta

Introducing Databricks Delta

TechEvent Databricks on Azure

TechEvent Databricks on Azure

TechEvent Databricks on Azure

How to build a successful Data Lake

How to build a successful Data Lake

How to build a successful Data Lake

DataWorks Summit/Hadoop Summit

More and more organizations are moving their ETL workloads to a Hadoop based ELT grid architecture. Hadoop`s inherit capabilities, especially it`s ability to do late binding addresses some of the key challenges with traditional ETL platforms. In this presentation, attendees will learn the key factors, considerations and lessons around ETL for Hadoop. Areas such as pros and cons for different extract and load strategies, best ways to batch data, buffering and compression considerations, leveraging HCatalog, data transformation, integration with existing data transformations, advantages of different ways of exchanging data and leveraging Hadoop as a data integration layer. This is an extremely popular presentation around ETL and Hadoop.

A Reference Architecture for ETL 2.0

A Reference Architecture for ETL 2.0

A Reference Architecture for ETL 2.0

DataWorks Summit

Are you using the fastest query tool for Hadoop? Provide and discuss the latest performance results of the industry standard TPC_H benchmarks executed across an assortment of open source query tools such as Hive (using MR, TEZ, LLAP, SPARK), SparkSQL, Presto, and Drill. Additionally, the performance tests will utilize a variety of data sizes and popular storage formats such as ORC, Parquet and Text and compression codecs.

Hadoop Query Performance Smackdown

Hadoop Query Performance Smackdown

Hadoop Query Performance Smackdown

DataWorks Summit

Tendances (20)

Power BI as a storyteller

Power BI as a storyteller

Power BI as a storyteller

Scaling and Modernizing Data Platform with Databricks

Scaling and Modernizing Data Platform with Databricks

Scaling and Modernizing Data Platform with Databricks

Data engineering

Data engineering

Data engineering

Build Real-Time Applications with Databricks Streaming

Build Real-Time Applications with Databricks Streaming

Build Real-Time Applications with Databricks Streaming

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

Spark Hadoop Tutorial | Spark Hadoop Example on NBA | Apache Spark Training |...

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Differentiate Big Data vs Data Warehouse use cases for a cloud solution

Building a modern data warehouse

Building a modern data warehouse

Building a modern data warehouse

Databricks for Dummies

Databricks for Dummies

Databricks for Dummies

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...

Azure data platform overview

Azure data platform overview

Azure data platform overview

From Data Warehouse to Lakehouse

From Data Warehouse to Lakehouse

From Data Warehouse to Lakehouse

Introduction to Azure Databricks

Introduction to Azure Databricks

Introduction to Azure Databricks

Solr vs. Elasticsearch - Case by Case

Solr vs. Elasticsearch - Case by Case

Solr vs. Elasticsearch - Case by Case

Introduction to Apache Spark

Introduction to Apache Spark

Introduction to Apache Spark

Introducing Databricks Delta

Introducing Databricks Delta

Introducing Databricks Delta

TechEvent Databricks on Azure

TechEvent Databricks on Azure

TechEvent Databricks on Azure

How to build a successful Data Lake

How to build a successful Data Lake

How to build a successful Data Lake

A Reference Architecture for ETL 2.0

A Reference Architecture for ETL 2.0

A Reference Architecture for ETL 2.0

Hadoop Query Performance Smackdown

Hadoop Query Performance Smackdown

Hadoop Query Performance Smackdown

Similaire à The Ecosystem is too damn big

Adoption is the only option hadoop is changing our world and changing yours f...

Adoption is the only option hadoop is changing our world and changing yours f...

Adoption is the only option hadoop is changing our world and changing yours f...

DataWorks Summit

Big Data - Part II

Big Data - Part II

Big Data - Part II

Thanuja Seneviratne

This is a power point presentation on Hadoop and Big Data. This covers the essential knowledge one should have when stepping into the world of Big Data. This course is available on hadoop-skills.com for free! This course builds a basic fundamental understanding of Big Data problems and Hadoop as a solution. This course takes you through: • This course builds Understanding of Big Data problems with easy to understand examples and illustrations. • History and advent of Hadoop right from when Hadoop wasn’t even named Hadoop and was called Nutch • What is Hadoop Magic which makes it so unique and powerful. • Understanding the difference between Data science and data engineering, which is one of the big confusions in selecting a carrier or understanding a job role. • And most importantly, demystifying Hadoop vendors like Cloudera, MapR and Hortonworks by understanding about them. This course is available for free on hadoop-skills.com

Hadoop for beginners free course ppt

Hadoop for beginners free course ppt

Hadoop for beginners free course ppt

Introduction to Big Data An analogy between Sugar Cane & Big Data

Introduction to Big Data An analogy between Sugar Cane & Big Data

Introduction to Big Data An analogy between Sugar Cane & Big Data

Jean-Marc Desvaux

HadoopWorkshopJuly2014

HadoopWorkshopJuly2014

HadoopWorkshopJuly2014

Dieter De Witte

After three decades of prominence, Relational Database Management Systems (RDBMS) are being challenged by a raft of new technologies. While enjoying a position of incumbency, newer data management approaches are benefitting from a vibrancy powered by the effects of Moore's Law and Big Data. Hadoop and NoSQL offerings were designed for the cloud, but are finding a place in enterprise architecture. In fact, Hadoop has already made a dent in the burgeoning field of analytics, previously the realm of data warehouses and analytical (relational) platforms.

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Gail Zhou, MBA, PhD

In today’s context, the big data market is rapidly undergoing contortions that define market maturity, such as consolidation. Big data refers to large volumes of data. This can be both structured and unstructured data. Big data is data that is huge in size and grows exponentially with time. As the data is too large and complex, traditional data management tools are not sufficient for storing or processing it efficiently. But analyzing big data is crucial to know the patterns and trends to be adopted to improve your business.

Top 10 renowned big data companies

Top 10 renowned big data companies

Top 10 renowned big data companies

The Briefing Room with Robin Bloor and Pervasive Software Slides from the Live Webcast on May 1, 2012 The old methods of delivering data for analysts and other business users will simply not scale to meet new demands. Hadoop is rapidly emerging as a powerful and economic platform for storing and processing Big Data. And yet, the biggest obstacle to implementing Hadoop solutions is the scarcity of Hadoop programming skills. Check out this episode of The Briefing Room to learn from veteran Analyst Robin Bloor, who will explain why modern information architectures must embrace the new, massively parallel world of computing as it relates to several enterprise roles: traditional business analysts, data scientists, and line-of-business workers. He'll be briefed by David Inbar and Jim Falgout of Pervasive Software, who will explain how Pervasive RushAnalyzer™ was designed to accommodate the new reality of Big Data. For more information visit: http://www.insideanalysis.com Watch us on YouTube: http://www.youtube.com/playlist?list=PL5EE76E2EEEC8CF9E

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Inside Analysis

Big Data – From Strategy to Production

Big Data – From Strategy to Production

Big Data – From Strategy to Production

Semantic Web Company

Cloud as a Data Platform

Cloud as a Data Platform

Cloud as a Data Platform

Building a data lake involves more than installing Hadoop or putting data into AWS. The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This tutorial covers design assumptions, design principles, and how to approach the architecture and planning for multi-use data infrastructure in IT. Long: The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This session will discuss hidden design assumptions, review design principles to apply when building multi-use data infrastructure, and provide a reference architecture to use as you work to unify your analytics infrastructure. The focus in our market has been on acquiring technology, and that ignores the more important part: the larger IT landscape within which this technology lives and the data architecture that lies at its core. If one expects longevity from a platform then it should be a designed rather than accidental architecture. Architecture is more than just software. It starts from use and includes the data, technology, methods of building and maintaining, and organization of people. What are the design principles that lead to good design and a functional data architecture? What are the assumptions that limit older approaches? How can one integrate with, migrate from or modernize an existing data environment? How will this affect an organization's data management practices? This tutorial will help you answer these questions. Topics covered: * A brief history of data infrastructure and past design assumptions * Categories of data and data use in organizations * Data architecture * Functional architecture * Technology planning assumptions and guidance

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

02 a holistic approach to big data

02 a holistic approach to big data

02 a holistic approach to big data

Cloud computing

Cloud computing

Cloud computing

Richard Claassens CIPPE

Top Business Intelligence Trends for 2016 by Panorama Software

Top Business Intelligence Trends for 2016 by Panorama Software

Top Business Intelligence Trends for 2016 by Panorama Software

Panorama Software

Predictive analytics is touching more and more lives every day. Machine Learning lets you predict and change the future. Do you know that Microsoft products like Xbox and Bing integrate some machine learning capabilities in their workflows? Come to the session and take a look of the new cloud-based machine learning platform called AzureML from a BI architect perspective, without all the data scientist knowledge.

AzureML Welcome to the future of Predictive Analytics

AzureML Welcome to the future of Predictive Analytics

AzureML Welcome to the future of Predictive Analytics

Ruben Pertusa Lopez

Big Data on Public Cloud

Big Data on Public Cloud

Big Data on Public Cloud

Forecast of Big Data Trends

Forecast of Big Data Trends

Forecast of Big Data Trends

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Analysis Patterns: Tying real world use cases to strategies for analysis using big data technologies and tools. Big data is ushering in a new era for analytics with large scale data and relatively simple algorithms driving results rather than relying on complex models that use sample data. When you are ready to extract benefits from your data, how do you decide what approach, what algorithm, what tool to use? The answer is simpler than you think. This session tackles big data analysis with a practical description of strategies for several classes of application types, identified concretely with use cases. Topics include new approaches to search and recommendation using scalable technologies such as Hadoop, Mahout, Storm, Solr, & Titan.

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Similaire à The Ecosystem is too damn big (20)

Adoption is the only option hadoop is changing our world and changing yours f...

Adoption is the only option hadoop is changing our world and changing yours f...

Adoption is the only option hadoop is changing our world and changing yours f...

Big Data - Part II

Big Data - Part II

Big Data - Part II

Hadoop for beginners free course ppt

Hadoop for beginners free course ppt

Hadoop for beginners free course ppt

Introduction to Big Data An analogy between Sugar Cane & Big Data

Introduction to Big Data An analogy between Sugar Cane & Big Data

Introduction to Big Data An analogy between Sugar Cane & Big Data

HadoopWorkshopJuly2014

HadoopWorkshopJuly2014

HadoopWorkshopJuly2014

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Relational Technologies Under Siege: Will Handsome Newcomers Displace the St...

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Top 10 renowned big data companies

Top 10 renowned big data companies

Top 10 renowned big data companies

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and...

Big Data – From Strategy to Production

Big Data – From Strategy to Production

Big Data – From Strategy to Production

Cloud as a Data Platform

Cloud as a Data Platform

Cloud as a Data Platform

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

02 a holistic approach to big data

02 a holistic approach to big data

02 a holistic approach to big data

Cloud computing

Cloud computing

Cloud computing

Top Business Intelligence Trends for 2016 by Panorama Software

Top Business Intelligence Trends for 2016 by Panorama Software

Top Business Intelligence Trends for 2016 by Panorama Software

AzureML Welcome to the future of Predictive Analytics

AzureML Welcome to the future of Predictive Analytics

AzureML Welcome to the future of Predictive Analytics

Big Data on Public Cloud

Big Data on Public Cloud

Big Data on Public Cloud

Forecast of Big Data Trends

Forecast of Big Data Trends

Forecast of Big Data Trends

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Day LA 2016/ Use Case Driven track - How to Use Design Thinking to J...

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Big Data Analysis Patterns with Hadoop, Mahout and Solr

Plus de DataWorks Summit/Hadoop Summit

Running Apache Spark & Apache Zeppelin in Production

Running Apache Spark & Apache Zeppelin in Production

Running Apache Spark & Apache Zeppelin in Production

DataWorks Summit/Hadoop Summit

State of Security: Apache Spark & Apache Zeppelin

State of Security: Apache Spark & Apache Zeppelin

State of Security: Apache Spark & Apache Zeppelin

DataWorks Summit/Hadoop Summit

Unleashing the Power of Apache Atlas with Apache Ranger

Unleashing the Power of Apache Atlas with Apache Ranger

Unleashing the Power of Apache Atlas with Apache Ranger

DataWorks Summit/Hadoop Summit

Enabling Digital Diagnostics with a Data Science Platform

Enabling Digital Diagnostics with a Data Science Platform

Enabling Digital Diagnostics with a Data Science Platform

DataWorks Summit/Hadoop Summit

Revolutionize Text Mining with Spark and Zeppelin

Revolutionize Text Mining with Spark and Zeppelin

Revolutionize Text Mining with Spark and Zeppelin

DataWorks Summit/Hadoop Summit

Double Your Hadoop Performance with Hortonworks SmartSense

Double Your Hadoop Performance with Hortonworks SmartSense

Double Your Hadoop Performance with Hortonworks SmartSense

DataWorks Summit/Hadoop Summit

Hadoop Crash Course

Hadoop Crash Course

Hadoop Crash Course

DataWorks Summit/Hadoop Summit

Data Science Crash Course

Data Science Crash Course

Data Science Crash Course

DataWorks Summit/Hadoop Summit

Apache Spark Crash Course

Apache Spark Crash Course

Apache Spark Crash Course

DataWorks Summit/Hadoop Summit

Dataflow with Apache NiFi

Dataflow with Apache NiFi

Dataflow with Apache NiFi

DataWorks Summit/Hadoop Summit

Many Organizations are currently processing various types of data and in different formats. Most often this data will be in free form, As the consumers of this data growing it’s imperative that this free-flowing data needs to adhere to a schema. It will help data consumers to have an expectation of about the type of data they are getting and also they will be able to avoid immediate impact if the upstream source changes its format. Having a uniform schema representation also gives the Data Pipeline a really easy way to integrate and support various systems that use different data formats. SchemaRegistry is a central repository for storing, evolving schemas. It provides an API & tooling to help developers and users to register a schema and consume that schema without having any impact if the schema changed. Users can tag different schemas and versions, register for notifications of schema changes with versions etc. In this talk, we will go through the need for a schema registry and schema evolution and showcase the integration with Apache NiFi, Apache Kafka, Apache Storm.

Schema Registry - Set you Data Free

Schema Registry - Set you Data Free

Schema Registry - Set you Data Free

DataWorks Summit/Hadoop Summit

There is increasing need for large-scale recommendation systems. Typical solutions rely on periodically retrained batch algorithms, but for massive amounts of data, training a new model could take hours. This is a problem when the model needs to be more up-to-date. For example, when recommending TV programs while they are being transmitted the model should take into consideration users who watch a program at that time. The promise of online recommendation systems is fast adaptation to changes, but methods of online machine learning from streams is commonly believed to be more restricted and hence less accurate than batch trained models. Combining batch and online learning could lead to a quickly adapting recommendation system with increased accuracy. However, designing a scalable data system for uniting batch and online recommendation algorithms is a challenging task. In this talk we present our experiences in creating such a recommendation engine with Apache Flink and Apache Spark.

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

DataWorks Summit/Hadoop Summit

DeepLearning is not just a hype - it outperforms state-of-the-art ML algorithms. One by one. In this talk we will show how DeepLearning can be used for detecting anomalies on IoT sensor data streams at high speed using DeepLearning4J on top of different BigData engines like ApacheSpark and ApacheFlink. Key in this talk is the absence of any large training corpus since we are using unsupervised machine learning - a domain current DL research threats step-motherly. As we can see in this demo LSTM networks can learn very complex system behavior - in this case data coming from a physical model simulating bearing vibration data. Once draw back of DeepLearning is that normally a very large labaled training data set is required. This is particularly interesting since we can show how unsupervised machine learning can be used in conjunction with DeepLearning - no labeled data set is necessary. We are able to detect anomalies and predict braking bearings with 10 fold confidence. All examples and all code will be made publicly available and open sources. Only open source components are used.

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

DataWorks Summit/Hadoop Summit

QE automation for large systems is a great step forward in increasing system reliability. In the big-data world, multiple components have to come together to provide end-users with business outcomes. This means, that QE Automations scenarios need to be detailed around actual use cases, cross-cutting components. The system tests potentially generate large amounts of data on a recurring basis, verifying which is a tedious job. Given the multiple levels of indirection, the false positives of actual defects are higher, and are generally wasteful. At Hortonworks, we’ve designed and implemented Automated Log Analysis System - Mool, using Statistical Data Science and ML. Currently the work in progress has a batch data pipeline with a following ensemble ML pipeline which feeds into the recommendation engine. The system identifies the root cause of test failures, by correlating the failing test cases, with current and historical error records, to identify root cause of errors across multiple components. The system works in unsupervised mode with no perfect model/stable builds/source-code version to refer to. In addition the system provides limited recommendations to file/open past tickets and compares run-profiles with past runs.

Mool - Automated Log Analysis using Data Science and ML

Mool - Automated Log Analysis using Data Science and ML

Mool - Automated Log Analysis using Data Science and ML

DataWorks Summit/Hadoop Summit

Improving business performance is never easy! The Natixis Pack is like Rugby. Working together is key to scrum success. Our data journey would undoubtedly have been so much more difficult if we had not made the move together. This session is the story of how ‘The Natixis Pack’ has driven change in its current IT architecture so that legacy systems can leverage some of the many components in Hortonworks Data Platform in order to improve the performance of business applications. During this session, you will hear: • How and why the business and IT requirements originated • How we leverage the platform to fulfill security and production requirements • How we organize a community to: o Guard all the players, no one gets left on the ground! o Us the platform appropriately (Not every problem is eligible for Big Data and standard databases are not dead) • What are the most usable, the most interesting and the most promising technologies in the Apache Hadoop community We will finish the story of a successful rugby team with insight into the special skills needed from each player to win the match! DETAILS This session is part business, part technical. We will talk about infrastructure, security and project management as well as the industrial usage of Hive, HBase, Kafka, and Spark within an industrial Corporate and Investment Bank environment, framed by regulatory constraints.

How Hadoop Makes the Natixis Pack More Efficient

How Hadoop Makes the Natixis Pack More Efficient

How Hadoop Makes the Natixis Pack More Efficient

DataWorks Summit/Hadoop Summit

HBase hast established itself as the backend for many operational and interactive use-cases, powering well-known services that support millions of users and thousands of concurrent requests. In terms of features HBase has come a long way, overing advanced options such as multi-level caching on- and off-heap, pluggable request handling, fast recovery options such as region replicas, table snapshots for data governance, tuneable write-ahead logging and so on. This talk is based on the research for the an upcoming second release of the speakers HBase book, correlated with the practical experience in medium to large HBase projects around the world. You will learn how to plan for HBase, starting with the selection of the matching use-cases, to determining the number of servers needed, leading into performance tuning options. There is no reason to be afraid of using HBase, but knowing its basic premises and technical choices will make using it much more successful. You will also learn about many of the new features of HBase up to version 1.3, and where they are applicable.

HBase in Practice

HBase in Practice

HBase in Practice

DataWorks Summit/Hadoop Summit

There has been an explosion of data digitising our physical world – from cameras, environmental sensors and embedded devices, right down to the phones in our pockets. Which means that, now, companies have new ways to transform their businesses – both operationally, and through their products and services – by leveraging this data and applying fresh analytical techniques to make sense of it. But are they ready? The answer is “no” in most cases. In this session, we’ll be discussing the challenges facing companies trying to embrace the Analytics of Things, and how Teradata has helped customers work through and turn those challenges to their advantage.

The Challenge of Driving Business Value from the Analytics of Things (AOT)

The Challenge of Driving Business Value from the Analytics of Things (AOT)

The Challenge of Driving Business Value from the Analytics of Things (AOT)

DataWorks Summit/Hadoop Summit

In this talk, we will present a new distribution of Hadoop, Hops, that can scale the Hadoop Filesystem (HDFS) by 16X, from 70K ops/s to 1.2 million ops/s on Spotiy's industrial Hadoop workload. Hops is an open-source distribution of Apache Hadoop that supports distributed metadata for HSFS (HopsFS) and the ResourceManager in Apache YARN. HopsFS is the first production-grade distributed hierarchical filesystem to store its metadata normalized in an in-memory, shared nothing database. For YARN, we will discuss optimizations that enable 2X throughput increases for the Capacity scheduler, enabling scalability to clusters with >20K nodes. We will discuss the journey of how we reached this milestone, discussing some of the challenges involved in efficiently and safely mapping hierarchical filesystem metadata state and operations onto a shared-nothing, in-memory database. We will also discuss the key database features needed for extreme scaling, such as multi-partition transactions, partition-pruned index scans, distribution-aware transactions, and the streaming changelog API. Hops (www.hops.io) is Apache-licensed open-source and supports a pluggable database backend for distributed metadata, although it currently only support MySQL Cluster as a backend. Hops opens up the potential for new directions for Hadoop when metadata is available for tinkering in a mature relational database.

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

DataWorks Summit/Hadoop Summit

In high-risk manufacturing industries, regulatory bodies stipulate continuous monitoring and documentation of critical product attributes and process parameters. On the other hand, sensor data coming from production processes can be used to gain deeper insights into optimization potentials. By establishing a central production data lake based on Hadoop and using Talend Data Fabric as a basis for a unified architecture, the German pharmaceutical company HERMES Arzneimittel was able to cater to compliance requirements as well as unlock new business opportunities, enabling use cases like predictive maintenance, predictive quality assurance or open world analytics. Learn how the Talend Data Fabric enabled HERMES Arzneimittel to become data-driven and transform Big Data projects from challenging, hard to maintain hand-coding jobs to repeatable, future-proof integration designs. Talend Data Fabric combines Talend products into a common set of powerful, easy-to-use tools for any integration style: real-time or batch, big data or master data management, on-premises or in the cloud.

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

DataWorks Summit/Hadoop Summit

While you could be tempted assuming data is already safe in a single Hadoop cluster, in practice you have to plan for more. Questions like: "What happens if the entire datacenter fails?, or "How do I recover into a consistent state of data, so that applications can continue to run?" are not a all trivial to answer for Hadoop. Did you know that HDFS snapshots are handling open files not as immutable? Or that HBase snapshots are executed asynchronously across servers and therefore cannot guarantee atomicity for cross region updates (which includes tables)? There is no unified and coherent data backup strategy, nor is there tooling available for many of the included components to build such a strategy. The Hadoop distributions largely avoid this topic as most customers are still in the "single use-case" or PoC phase, where data governance as far as backup and disaster recovery (BDR) is concerned are not (yet) important. This talk first is introducing you to the overarching issue and difficulties of backup and data safety, looking at each of the many components in Hadoop, including HDFS, HBase, YARN, Oozie, the management components and so on, to finally show you a viable approach using built-in tools. You will also learn not to take this topic lightheartedly and what is needed to implement and guarantee a continuous operation of Hadoop cluster based solutions.

Backup and Disaster Recovery in Hadoop

Backup and Disaster Recovery in Hadoop

Backup and Disaster Recovery in Hadoop

DataWorks Summit/Hadoop Summit

Plus de DataWorks Summit/Hadoop Summit (20)

Running Apache Spark & Apache Zeppelin in Production

Running Apache Spark & Apache Zeppelin in Production

Running Apache Spark & Apache Zeppelin in Production

State of Security: Apache Spark & Apache Zeppelin

State of Security: Apache Spark & Apache Zeppelin

State of Security: Apache Spark & Apache Zeppelin

Unleashing the Power of Apache Atlas with Apache Ranger

Unleashing the Power of Apache Atlas with Apache Ranger

Unleashing the Power of Apache Atlas with Apache Ranger

Enabling Digital Diagnostics with a Data Science Platform

Enabling Digital Diagnostics with a Data Science Platform

Enabling Digital Diagnostics with a Data Science Platform

Revolutionize Text Mining with Spark and Zeppelin

Revolutionize Text Mining with Spark and Zeppelin

Revolutionize Text Mining with Spark and Zeppelin

Double Your Hadoop Performance with Hortonworks SmartSense

Double Your Hadoop Performance with Hortonworks SmartSense

Double Your Hadoop Performance with Hortonworks SmartSense

Hadoop Crash Course

Hadoop Crash Course

Hadoop Crash Course

Data Science Crash Course

Data Science Crash Course

Data Science Crash Course

Apache Spark Crash Course

Apache Spark Crash Course

Apache Spark Crash Course

Dataflow with Apache NiFi

Dataflow with Apache NiFi

Dataflow with Apache NiFi

Schema Registry - Set you Data Free

Schema Registry - Set you Data Free

Schema Registry - Set you Data Free

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...

Mool - Automated Log Analysis using Data Science and ML

Mool - Automated Log Analysis using Data Science and ML

Mool - Automated Log Analysis using Data Science and ML

How Hadoop Makes the Natixis Pack More Efficient

How Hadoop Makes the Natixis Pack More Efficient

How Hadoop Makes the Natixis Pack More Efficient

HBase in Practice

HBase in Practice

HBase in Practice

The Challenge of Driving Business Value from the Analytics of Things (AOT)

The Challenge of Driving Business Value from the Analytics of Things (AOT)

The Challenge of Driving Business Value from the Analytics of Things (AOT)

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

From Regulatory Process Verification to Predictive Maintenance and Beyond wit...

Backup and Disaster Recovery in Hadoop

Backup and Disaster Recovery in Hadoop

Backup and Disaster Recovery in Hadoop

Dernier

MS Copilot expands with MS Graph connectors

MS Copilot expands with MS Graph connectors

MS Copilot expands with MS Graph connectors

Nanddeep Nachan

DBX First Quarter 2024 Investor Presentation

DBX First Quarter 2024 Investor Presentation

DBX First Quarter 2024 Investor Presentation

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

GenAI Risks & Security Meetup 01052024.pdf

GenAI Risks & Security Meetup 01052024.pdf

GenAI Risks & Security Meetup 01052024.pdf

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Architecting Cloud Native Applications

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

The Digital Insurer

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

The Digital Insurer

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

A Beginners Guide to Building a RAG App Using Open Source Milvus

A Beginners Guide to Building a RAG App Using Open Source Milvus

A Beginners Guide to Building a RAG App Using Open Source Milvus

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Dernier (20)

MS Copilot expands with MS Graph connectors

MS Copilot expands with MS Graph connectors

MS Copilot expands with MS Graph connectors

DBX First Quarter 2024 Investor Presentation

DBX First Quarter 2024 Investor Presentation

DBX First Quarter 2024 Investor Presentation

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

GenAI Risks & Security Meetup 01052024.pdf

GenAI Risks & Security Meetup 01052024.pdf

GenAI Risks & Security Meetup 01052024.pdf

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Powerful Google developer tools for immediate impact! (2023-24 C)

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Architecting Cloud Native Applications

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

FWD Group - Insurer Innovation Award 2024

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

AXA XL - Insurer Innovation Award Americas 2024

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Exploring the Future Potential of AI-Enabled Smartphone Processors

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

A Beginners Guide to Building a RAG App Using Open Source Milvus

A Beginners Guide to Building a RAG App Using Open Source Milvus

A Beginners Guide to Building a RAG App Using Open Source Milvus

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

The Ecosystem is too damn big

1. The Ecosystem is Too Damn Big Andrew Brust Senior Director, Market Strategy and Analysis Datameer

2. there are too many vendors Credit: Matt Turck, Firstmark Capital

3. ON‐PREMISES CLOUD HADOOP AS A SERVICE OTHER Cloudera Hortonworks MapR IBM Amazon Web Services Microsoft Azure Google Cloud Platform Qubole Altiscale ArenaData InfoSys there are too many distributions

4. there are too many engines

5. there are too many projects

6. the problem with churn

7. problems analysis paralysis big bang platform decisions one-size-fits-all imprecision inevitable obsolescence “engine choice dread” on every single freakin’ job

8. too many... SQL on Hadoop solutions project iterations vendor serving Apache projects programming language choices

9. buzzwords actionable insights days instead of weeks weeks instead of months for the business user data scientists citizen data scientists the 4th “V” all your data in one place data lake

10. MODULAR LOCKED IN OR OUTPRODUCTS LOOK FOR DON'T BE NO HARD CODING IT'S ALL ABOUT BALANCE you'll need to fend for yourself

11. Top 6 Solutions

12. Industry: build consortia 06

13. Vendors: stop copying each other’s messages05

14. VCs: stop enabling bad marketing, funding companies that don’t add value04

15. Analyst firms: decode things, don’t confuse them03

16. Technologists: realize the value of standards02

17. Customers: demand a consistent core stack, do your research, buy on the merits01

18. thank you.andrew.brust@datameer @andrewbrust