OpenStack Data Processing ("Sahara") project update - December 2014

•

1 j'aime•850 vues

Sergey Lukjanov

Logiciels

Sergey Lukjanov
PTL
OpenStack Sahara
Data Processing Update

To provide a scalable data processing stack
and associated management interfaces Sahara
● provisions & operates data proc. clusters
● schedules & operates data proc. jobs /
workloads

Elastic Data Processing (EDP) is Sahara’s take
on data processing workflow management.

Juno release overview
● 540+ code commits from 50 people
● 32 blueprints implemented
● ~5000 code reviews
● ~168 bugs fixed
details: https://launchpad.net/sahara/juno

Sahara UI completely merged into Horizon

Vanilla Apache Hadoop 2.4.1 support
Cloudera Distribution of Apache Hadoop 5.X
support
Apache Spark 0.9.1 and 1.0.0 support

Kilo plans
● New versions for CDH, HDP, etc.
● Dashboard UX improvements
● Better Heat integration
● Ironic support

OpenStack Data Processing ("Sahara") project update - December 2014

Recommandé

Savanna project update Jan 2014Sergey Lukjanov

Atlanta OpenStack Summit: The State of OpenStack Data Processing: Sahara, Now...Sergey Lukjanov

Savanna - Elastic Hadoop on OpenStackSergey Lukjanov

Hong Kong OpenStack Summit: Savanna - Hadoop on OpenStackSergey Lukjanov

Atlanta OpenStack Summit: Technical Deep Dive: Big Data Computations Using El...Sergey Lukjanov

Data Processing Updates - Juno EditionOpenStack Foundation

Hadoop on OpenStack - Sahara @DevNation 2014spinningmatt

Hadoop and OpenStack - Hadoop Summit San Jose 2014spinningmatt

Recommandé

Savanna project update Jan 2014Sergey Lukjanov

Atlanta OpenStack Summit: The State of OpenStack Data Processing: Sahara, Now...Sergey Lukjanov

Savanna - Elastic Hadoop on OpenStackSergey Lukjanov

Hong Kong OpenStack Summit: Savanna - Hadoop on OpenStackSergey Lukjanov

Atlanta OpenStack Summit: Technical Deep Dive: Big Data Computations Using El...Sergey Lukjanov

Data Processing Updates - Juno EditionOpenStack Foundation

Hadoop on OpenStack - Sahara @DevNation 2014spinningmatt

Hadoop and OpenStack - Hadoop Summit San Jose 2014spinningmatt

The state of the art for OpenStack Data Processing (Hadoop on OpenStack) - At...spinningmatt

Lessons Learned from Building an Enterprise Big Data Platform from the Ground...DataWorks Summit

State of Spark in the cloud (Spark Summit EU 2017)Nicolas Poggi

Savanna: Hadoop on OpenStackMirantis

Hadoop and OpenStackDataWorks Summit

20150425 experimenting with openstack sahara on dockerWei Ting Chen

OpenStack Trove Day (19 Aug 2014, Cambridge MA) - Saharaspinningmatt

Tachyon and Apache Sparkrhatr

Performance Troubleshooting Using Apache Spark MetricsDatabricks

Using Apache Spark in the Cloud—A Devops Perspective with Telmo OliveiraSpark Summit

Apache Kylin: Speed Up Cubing with Apache Spark with Luke Han and Shaofeng ShiDatabricks

Apache Spark on K8S Best Practice and Performance in the CloudDatabricks

Scale-Out Using Spark in Serverless Herd Mode!Databricks

IEEE International Conference on Data Engineering 2015Yousun Jeong

20150314 sahara intro and the future plan for open stack meetupWei Ting Chen

Apache Spark OverviewairisData

Storage Engine Considerations for Your Apache Spark Applications with Mladen ...Spark Summit

Improved Applications with IPv6: an overviewCisco DevNet

Hello OpenStack, Meet HadoopDataWorks Summit

Building a Business Logic Translation Engine with Spark Streaming for Communi...Spark Summit

Benchmarking sahara based big data as a service solutionsZhidong Yu

Sahara presentation latest - Codemotion Rome 2015Codemotion

Contenu connexe

Tendances

The state of the art for OpenStack Data Processing (Hadoop on OpenStack) - At...spinningmatt

Lessons Learned from Building an Enterprise Big Data Platform from the Ground...DataWorks Summit

State of Spark in the cloud (Spark Summit EU 2017)Nicolas Poggi

Savanna: Hadoop on OpenStackMirantis

Hadoop and OpenStackDataWorks Summit

20150425 experimenting with openstack sahara on dockerWei Ting Chen

OpenStack Trove Day (19 Aug 2014, Cambridge MA) - Saharaspinningmatt

Tachyon and Apache Sparkrhatr

Performance Troubleshooting Using Apache Spark MetricsDatabricks

Using Apache Spark in the Cloud—A Devops Perspective with Telmo OliveiraSpark Summit

Apache Kylin: Speed Up Cubing with Apache Spark with Luke Han and Shaofeng ShiDatabricks

Apache Spark on K8S Best Practice and Performance in the CloudDatabricks

Scale-Out Using Spark in Serverless Herd Mode!Databricks

IEEE International Conference on Data Engineering 2015Yousun Jeong

20150314 sahara intro and the future plan for open stack meetupWei Ting Chen

Apache Spark OverviewairisData

Storage Engine Considerations for Your Apache Spark Applications with Mladen ...Spark Summit

Improved Applications with IPv6: an overviewCisco DevNet

Hello OpenStack, Meet HadoopDataWorks Summit

Building a Business Logic Translation Engine with Spark Streaming for Communi...Spark Summit

Tendances (20)

The state of the art for OpenStack Data Processing (Hadoop on OpenStack) - At...

Lessons Learned from Building an Enterprise Big Data Platform from the Ground...

State of Spark in the cloud (Spark Summit EU 2017)

Savanna: Hadoop on OpenStack

Hadoop and OpenStack

20150425 experimenting with openstack sahara on docker

OpenStack Trove Day (19 Aug 2014, Cambridge MA) - Sahara

Tachyon and Apache Spark

Performance Troubleshooting Using Apache Spark Metrics

Using Apache Spark in the Cloud—A Devops Perspective with Telmo Oliveira

Apache Kylin: Speed Up Cubing with Apache Spark with Luke Han and Shaofeng Shi

Apache Spark on K8S Best Practice and Performance in the Cloud

Scale-Out Using Spark in Serverless Herd Mode!

IEEE International Conference on Data Engineering 2015

20150314 sahara intro and the future plan for open stack meetup

Apache Spark Overview

Storage Engine Considerations for Your Apache Spark Applications with Mladen ...

Improved Applications with IPv6: an overview

Hello OpenStack, Meet Hadoop

Building a Business Logic Translation Engine with Spark Streaming for Communi...

En vedette

Benchmarking sahara based big data as a service solutionsZhidong Yu

Sahara presentation latest - Codemotion Rome 2015Codemotion

20151027 sahara + manila finalWei Ting Chen

از نماینده ایران در WSIS Prizes 2016 حمایت کنید ... متشکریم ...Leila Esmaeili

20150704 benchmark and user experience in sahara weitingWei Ting Chen

Sahara Updates - Kilo EditionOpenStack Foundation

آشنایی با جرم‌یابی قانونی رایانه‌ایRamin Najjarbashi

Cloud Security and Risk ManagementMorteza Javan

The Evolution of OpenStack – From Infancy to EnterpriseRackspace

Big Data on OpenStackNati Shalom

En vedette (10)

Benchmarking sahara based big data as a service solutions

Sahara presentation latest - Codemotion Rome 2015

20151027 sahara + manila final

از نماینده ایران در WSIS Prizes 2016 حمایت کنید ... متشکریم ...

20150704 benchmark and user experience in sahara weiting

Sahara Updates - Kilo Edition

آشنایی با جرم‌یابی قانونی رایانه‌ای

Cloud Security and Risk Management

The Evolution of OpenStack – From Infancy to Enterprise

Big Data on OpenStack

Similaire à OpenStack Data Processing ("Sahara") project update - December 2014

Sahara Updates - Liberty EditionOpenStack Foundation

Hortonworks tech workshop in-memory processing with sparkHortonworks

Creating the Internet of Your ThingsDataWorks Summit/Hadoop Summit

Bigdataappliance datasheet-1883358Ory Chhean

[Hadoop Meetup] Apache Hadoop 3 community update - Rohith SharmaNewton Alex

Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...Precisely

Apache Ambari BOF - OpenStack - Hadoop Summit 2013Hortonworks

Spark and Object Stores —What You Need to Know: Spark Summit East talk by Ste...Spark Summit

Hadoop Now, Next and BeyondDataWorks Summit

Hadoop Summit Dublin 2016: Hadoop Platform at Yahoo - A Year in Review Sumeet Singh

Hadoop Platform at YahooDataWorks Summit/Hadoop Summit

Empower Data-Driven Organizations with HPE and HadoopDataWorks Summit/Hadoop Summit

Apache Spark and Object StoresSteve Loughran

Introduction to the Hadoop EcoSystemShivaji Dutta

Big data architecture on cloud computing infrastructuredatastack

Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformHortonworks

Cloudera + MicrosoftでHadoopするのがイイらしい。 #CWT2016Cloudera Japan

Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...OpenShift Origin

HPC DAY 2017 | FlyElephant Solutions for Data Science and HPCHPC DAY

Azure Big data Michel HUBERT

Similaire à OpenStack Data Processing ("Sahara") project update - December 2014 (20)

Sahara Updates - Liberty Edition

Hortonworks tech workshop in-memory processing with spark

Creating the Internet of Your Things

Bigdataappliance datasheet-1883358

[Hadoop Meetup] Apache Hadoop 3 community update - Rohith Sharma

Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...

Apache Ambari BOF - OpenStack - Hadoop Summit 2013

Spark and Object Stores —What You Need to Know: Spark Summit East talk by Ste...

Hadoop Now, Next and Beyond

Hadoop Summit Dublin 2016: Hadoop Platform at Yahoo - A Year in Review

Hadoop Platform at Yahoo

Empower Data-Driven Organizations with HPE and Hadoop

Apache Spark and Object Stores

Introduction to the Hadoop EcoSystem

Big data architecture on cloud computing infrastructure

Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform

Cloudera + MicrosoftでHadoopするのがイイらしい。 #CWT2016

Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...

HPC DAY 2017 | FlyElephant Solutions for Data Science and HPC

Azure Big data

Plus de Sergey Lukjanov

[Mirantis Day 2015] Проект Sahara - BigData на OpenStackSergey Lukjanov

Courses: concurrency #2Sergey Lukjanov

Twitter StormSergey Lukjanov

Java Agents and Instrumentation techtalkSergey Lukjanov

Java Bytecode techtalkSergey Lukjanov

Kotlin techtalkSergey Lukjanov

Plus de Sergey Lukjanov (6)

[Mirantis Day 2015] Проект Sahara - BigData на OpenStack

Courses: concurrency #2

Twitter Storm

Java Agents and Instrumentation techtalk

Java Bytecode techtalk

Kotlin techtalk

Dernier

why an Opensea Clone Script might be your perfect match.pdfjoe51371421

TECUNIQUE: Success Stories: IT Service providermohitmore19

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.

Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01

Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...OnePlan Solutions

Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Delhi Whatsup 9873940964 Enjoy Unlimited Pleasure

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...harshavardhanraghave

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️Delhi Call girls

How To Troubleshoot Collaboration Apps for the Modern Connected WorkerThousandEyes

How To Use Server-Side Rendering with Nuxt.jsAndolasoft Inc

Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.

Professional Resume Template for Software DevelopersVinodh Ram

Software Quality Assurance Interview QuestionsArshad QA

Optimizing AI for immediate response in Smart CCTVshikhaohhpro

CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️anilsa9823

A Secure and Reliable Document Management System is Essential.docxComplianceQuest1

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Steffen Staab

Right Money Management App For Your Financial GoalsJhone kinadey

Microsoft AI Transformation Partner Playbook.pdfWilly Marroquin (WillyDevNET)

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...ICS

Dernier (20)

why an Opensea Clone Script might be your perfect match.pdf

TECUNIQUE: Success Stories: IT Service provider

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...

Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...

Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...

Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

How To Troubleshoot Collaboration Apps for the Modern Connected Worker

How To Use Server-Side Rendering with Nuxt.js

Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...

Professional Resume Template for Software Developers

Software Quality Assurance Interview Questions

Optimizing AI for immediate response in Smart CCTV

CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️

A Secure and Reliable Document Management System is Essential.docx

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Right Money Management App For Your Financial Goals

Microsoft AI Transformation Partner Playbook.pdf

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

OpenStack Data Processing ("Sahara") project update - December 2014

1. Sergey Lukjanov PTL OpenStack Sahara Data Processing Update

2. To provide a scalable data processing stack and associated management interfaces Sahara ● provisions & operates data proc. clusters ● schedules & operates data proc. jobs / workloads

3. Elastic Data Processing (EDP) is Sahara’s take on data processing workflow management.

4. Juno release overview ● 540+ code commits from 50 people ● 32 blueprints implemented ● ~5000 code reviews ● ~168 bugs fixed details: https://launchpad.net/sahara/juno

5. Moved to specs for new features

6. Sahara UI completely merged into Horizon

7. Pluggable framework-agnostic EDP

8. Vanilla Apache Hadoop 2.4.1 support Cloudera Distribution of Apache Hadoop 5.X support Apache Spark 0.9.1 and 1.0.0 support

9. Ceilometer integration

10. Heat resources for Sahara available

11. Auto security groups creation

12. Kilo plans ● New versions for CDH, HDP, etc. ● Dashboard UX improvements ● Better Heat integration ● Ironic support