Hadoop and subsystems in livedoor #Hcj11f

•

11 j'aime•3,054 vues

SATOSHI TAGOMORI

Hadoop and
Subsystems
in
livedoor
Hadoop Conference Japan
2011 Fall
2011/09/26
tagomoris
2011 9 26

2011 9 26

we are hiring!

2011 9 26

what's livedoor?

2011 9 26

2011 9 26

large scale web services

2800+ servers
3200+ hosts

530+ web servers

2011 9 26

20 Aug 2009

2011 9 26

Aug 2011

15Gbps
(10Gbps + CDN 5Gbps)

2011 9 26

Hadoop in livedoor
• 10 nodes (1+9)
• 36 core, 32TB HDFS
• CDH3b2
• with libhdfs, fuse-hdfs
• Hive 0.6.0 (community package)

2011 9 26

Hadoop in livedoor
data mining
reporting
page views, unique users,
traffic amount per page,
...
2011 9 26

super large scale
'sed | grep | wc'
with
Hadoop Streaming + Hive

2011 9 26

httpd logs

from 96 servers
(apache / nginx)

580GB/day (raw)

2011 9 26

overview

hourly
daily on
hourly
demand

2011 9 26

topics

•log delivery network with scribe
•and 'scribeline'
•hive client web application 'shib'

2011 9 26

overview

hourly
daily on
hourly
demand

2011 9 26

scribe
log delivery daemon
based on Thrift
scalable, reliable
supports HDFS

https://github.com/facebook/scribe

2011 9 26

scribe nodes
scribed

scribed

scribed

2011 9 26

deliver node traffic

2011 9 26

scribe nodes
scribed

scribed

scribed

2011 9 26

what we want
from scribe agent
•easy to deploy
•works w/o any httpd configurations
•delivery target failover/takeback
•lightweight (without JVM)
•stable
2011 9 26

scribe nodes
scribed

scribed

scribeline
scribed

2011 9 26

scribeline
log delivery agent tool
python 2.4, thrift

easy to setup and start/stop
works without any httpd configurations
works with logrotate-ed log files
automatic delivery target failover/takeback

https://github.com/tagomoris/scribe_line
2011 9 26

how to setup scribeline
in livedoor

1. yum install scribeline
(tar xzf && cd && sudo make install)

2. vi /etc/scribeline.conf
blog /var/log/httpd/access_log
blogimg /var/log/nginx/access_log

3. /etc/init.d/scribeline start

2011 9 26

scribe nodes
scribed

scribed

scribed

2011 9 26

overview

hourly
daily on
hourly
demand

2011 9 26

what we want
about hive client
•easy to experiment
•from PC on our desks
•result caching
•protection against data loss
•friendly look & feel
2011 9 26

shib
hive client web application
node.js, thrift, kyoto tycoon

query history browser
query editor, based on copy&paste
result caching & download tsv/csv
filter INSERT/DROP/CREATE ...

https://github.com/tagomoris/shib
2011 9 26

2011 9 26

shib system overview

2011 9 26

what shib cannot
do now
•access control
•graph & chart
•hive 0.7.0+ features support
•database, authentication and ...
•mapreduce status notification
2011 9 26

what we are trying now

•New cluster
•more nodes
•CDH3b2 + Hive 0.6.0 -> CDH3u1
•New tools
•Hoop (instead of fuse-hdfs)
•Any stream processing framework
2011 9 26

thanks!

2011 9 26

Contenu connexe

Tendances

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

ScyllaDB is a distributed database designed to scale horizontally and vertically — in theory. What about in practice? ScyllaDB’s Benny Halevy, Director, Software Engineering, will take you through the process and results of benchmarking our NoSQL database at the petabyte level, showing how you can use advanced features like workload prioritization to control priorities of transactional (read-write) and analytic (read-only) queries on the same cluster with smooth and predictable performance. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2016: Compose on Containing the Database

Scylla Summit 2016: Compose on Containing the Database

Scylla Summit 2016: Compose on Containing the Database

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Benchmarking Redis by itself and versus other NoSQL databases

Benchmarking Redis by itself and versus other NoSQL databases

Benchmarking Redis by itself and versus other NoSQL databases

Scylla Summit 2016: Scylla at Samsung SDS

Scylla Summit 2016: Scylla at Samsung SDS

Scylla Summit 2016: Scylla at Samsung SDS

In this session, we will build a minimum viable Spring Data web service with REST API, add a MySQL backing service as the primary data store, and a Redis Labs backing service for caching. We will demonstrate performance metrics without Redis caching enabled and then with Redis caching enabled. I will also provide an intro-level explanation of the platform capabilities within Pivotal Web Services.

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

There’s a popular misconception about I/O that (modern) SSDs are easy to deal with; they work pretty much like RAM but use a “legacy” submit-complete API. And other than keeping in mind a disk’s possible peak performance and maybe maintaining priorities of different IO streams there’s not much to care about. This is not quite the case – SSDs do show non-linear behavior and understanding the disk’s real abilities is crucial when it comes to squeezing as much performance from it as possible. Diskplorer is an open-source disk latency/bandwidth exploring toolset. By using Linux fio under the hood it runs a battery of measurements to discover performance characteristics for a specific hardware configuration, giving you an at-a-glance view of how server storage I/O will behave under load. ScyllaDB CTO Avi Kivity will share an interesting approach to measuring disk behavior under load, give a walkthrough of Diskplorer and explain how it’s used. With the elaborated model of a disk at hand, it becomes possible to build latency-oriented I/O scheduling that cherry-picks requests from the incoming queue keeping the disk load perfectly Balanced. ScyllaDB engineer Pavel Emelyanov will also present the scheduling algorithm developed for the Seastar framework and share results achieved using it.

Understanding Storage I/O Under Load

Understanding Storage I/O Under Load

Understanding Storage I/O Under Load

Scylla Summit 2018: Scylla 3.0 and Beyond

Scylla Summit 2018: Scylla 3.0 and Beyond

Scylla Summit 2018: Scylla 3.0 and Beyond

People want to have the convenience of deployment through Kubernetes, while still maintaining performance and management control. Moreno first began by getting Scylla working on Docker, and will discuss his in-depth investigation in getting passed performance bottlenecks. After finding how to get most of the performance back, then moved into Kubernetes. StatefulSets are production-ready since Kubernetes 1.9 but there is lot around StatefulSets that is not quite there. What are the tradeoffs of running a stateful application in a stateless environment? How do we minimize those tradeoffs to get the best operational reliability on Kubernetes without losing Scylla performance optimizations? What do you do when you are trying to run as close to the hardware as possible and then you containerize your installation? How do you remain an auto-tuning database when you are running in a containerized world? Learn how to use Docker, Kubernetes and Helm Charts with Scylla. We now invite members of the open source user community for your contributions, testing and feedback. Join our channels for #docker and #kubernetes on our open Slack!

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Customer Data Platforms, commonly called CDPs, form an integral part of the marketing stack powering Zeotap's Adtech and Martech use-cases. The company offers a privacy-compliant CDP platform, and ScyllaDB is an integral part. Zeotap's CDP demands a mix of OLTP, OLAP, and real-time data ingestion, requiring a highly-performant store. In this presentation, Shubham Patil, Lead Software Engineer, and Safal Pandita, Senior Software Engineer at Zeotap will share how ScyllaDB is powering their solution and why it's a great fit. They begin by describing their business use case and the challenges they were facing before moving to ScyllaDB. Then they cover their technical use-cases and requirements for real-time and batch data ingestions. They delve into our data access patterns and describe their data model supporting all use cases simultaneously for ingress/egress. They explain how they are using Scylla Migrator for our migration needs, then describe their multiregional, multi-tenant production setup for onboarding more than 130+ partners. Finally, they finish by sharing some of their learnings, performance benchmarks, and future plans. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Openstack platform -Red Hat Pizza and technology event - Israel

Openstack platform -Red Hat Pizza and technology event - Israel

Openstack platform -Red Hat Pizza and technology event - Israel

Persistent Storage for Containerized Applications

Persistent Storage for Containerized Applications

Persistent Storage for Containerized Applications

Colleen Corrice

Ansible and CloudStack

Ansible and CloudStack

Ansible and CloudStack

Discover the new features and capabilities of Scylla Open Source 5.0 directly from the engineers who developed it. This second block of lightning talks will cover the following topics: - New IO Scheduler and Disk Parallelism - Per-Service-Level Timeouts - Better Workload Estimation for Backpressure and Out-of-Memory Conditions - Large Partition Handling Improvements - Optimizing Reverse Queries To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

How do you handle the continuous transformation and refinement of billions of entities with some sort of reliability and performance? In this talk, Henrik will describe how Scylla enabled him and his team to create a pipelined solution using a series of microservices written in Go communicating with each other using Nats. You’ll hear about the mistakes and learnings they had along the way as they built the services that led to the great performance and stability they are experiencing today.

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Fetching large amount of data in a single query is a longstanding pain for applications. Queries that return a significant amount of data have to be paged, in other words, split into multiple subqueries that return data little by little. In both Scylla and Apache Cassandra, paging is stateless: each subquery is independent of each other and can even be sent to different replicas. Because of that, all the work done in the previous subqueries will not be reused causing a reduction from the maximum expected throughput. In this talk we are going to examine the problems with the previous stateless paging implementation and introduce the new stateful paging implementations that brings vast improvements in the throughput of large partition scans.

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Red Hat Ceph Storage Roadmap: January 2016

Red Hat Ceph Storage Roadmap: January 2016

Red Hat Ceph Storage Roadmap: January 2016

Red_Hat_Storage

Tendances (20)

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

CEPH DAY BERLIN - DEPLOYING CEPH IN KUBERNETES WITH ROOK

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Scylla Summit 2016: Compose on Containing the Database

Scylla Summit 2016: Compose on Containing the Database

Scylla Summit 2016: Compose on Containing the Database

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Benchmarking Redis by itself and versus other NoSQL databases

Benchmarking Redis by itself and versus other NoSQL databases

Benchmarking Redis by itself and versus other NoSQL databases

Scylla Summit 2016: Scylla at Samsung SDS

Scylla Summit 2016: Scylla at Samsung SDS

Scylla Summit 2016: Scylla at Samsung SDS

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal

Understanding Storage I/O Under Load

Understanding Storage I/O Under Load

Understanding Storage I/O Under Load

Scylla Summit 2018: Scylla 3.0 and Beyond

Scylla Summit 2018: Scylla 3.0 and Beyond

Scylla Summit 2018: Scylla 3.0 and Beyond

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Scylla Summit 2022: Building Zeotap's Privacy Compliant Customer Data Platfor...

Openstack platform -Red Hat Pizza and technology event - Israel

Openstack platform -Red Hat Pizza and technology event - Israel

Openstack platform -Red Hat Pizza and technology event - Israel

Persistent Storage for Containerized Applications

Persistent Storage for Containerized Applications

Persistent Storage for Containerized Applications

Ansible and CloudStack

Ansible and CloudStack

Ansible and CloudStack

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: Scylla 5.0 New Features, Part 1

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Scylla Summit 2018: How We Made Large Partition Scans Over Two Times Faster

Red Hat Ceph Storage Roadmap: January 2016

Red Hat Ceph Storage Roadmap: January 2016

Red Hat Ceph Storage Roadmap: January 2016

Similaire à Hadoop and subsystems in livedoor #Hcj11f

Openstack India May Meetup

Openstack India May Meetup

Openstack India May Meetup

Manuel Hurtado. Couchbase paradigma4oct

Manuel Hurtado. Couchbase paradigma4oct

Manuel Hurtado. Couchbase paradigma4oct

Paradigma Digital

Couchbase Day

In this two part presentation we will explore log analysis and log visualization. We will have a look at the history of log analysis; where log analysis stands today, what tools are available to process logs, what is working today, and more importantly, what is not working in log analysis. What will the future bring? Do our current approaches hold up under future requirements? We will discuss a number of issues and will try to figure out how we can address them. By looking at various log analysis challenges, we will explore how visualization can help address a number of them; keeping in mind that log visualization is not just a science, but also an art. We will apply a security lens to look at a number of use-cases in the area of security visualization. From there we will discuss what else is needed in the area of visualization, where the challenges lie, and where we should continue putting our research and development efforts.

Mining Your Logs - Gaining Insight Through Visualization

Mining Your Logs - Gaining Insight Through Visualization

Mining Your Logs - Gaining Insight Through Visualization

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

Cloud Deployments with Apache Hadoop and Apache HBase

Cloud Deployments with Apache Hadoop and Apache HBase

Cloud Deployments with Apache Hadoop and Apache HBase

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

Road to NODES - Handling Neo4j Data with Apache Hop

Road to NODES - Handling Neo4j Data with Apache Hop

Road to NODES - Handling Neo4j Data with Apache Hop

Splunk as a_big_data_platform_for_developers_spring_one2gx

Splunk as a_big_data_platform_for_developers_spring_one2gx

Splunk as a_big_data_platform_for_developers_spring_one2gx

Damien Dallimore

Hadoop is an increasingly popular means of analyzing transaction data from single MySQL or multiple MySQL servers. Up until now mechanisms for moving data between MySQL and Hadoop have been rather limited. The new Continuent Tungsten Replicator 3.0 provides enterprise-quality replication from MySQL to Hadoop. Tungsten Replicator 3.0 is 100% open source, released under a GPL V2 license, and available for download at https://code.google.com/p/tungsten-replicator/. Continuent Tungsten handles MySQL transaction types including INSERT/UPDATE/DELETE operations and can materialize binlogs as well as mirror-image data copies in Hadoop. Continuent Tungsten also has the high performance necessary to load data from busy source MySQL systems into Hadoop clusters with minimal load on source systems while retaining control over how the data is viewed and accessed within Hadoop. We will discuss: - How Hadoop works and why it's useful for processing transaction data from MySQL - Setting up Continuent Tungsten replication from MySQL to Hadoop - Understanding how Hadoop makes data analytics efficient and easy to execute - Tuning replication to maximize performance.

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

The path to a serverless-native era with Kubernetes

The path to a serverless-native era with Kubernetes

The path to a serverless-native era with Kubernetes

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

Join us for a developer workshop where we’ll go hands-on to build DynamoDB-compatible applications that can be deployed wherever you want: on-premises or on any public cloud. You will also learn how to migrate existing DynamoDB workloads. We’ll use Python as well as the ScyllaDB Alternator interface, which is an open source DynamoDB-compatible API. In the process you’ll discover the features and best practices that enable your applications to squeeze maximum performance out of your current database infrastructure – and avoid cloud vendor lock-in. If you’re an application developer who wants more flexible and faster DynamoDB, this workshop is for you!

Build DynamoDB-Compatible Apps with Python

Build DynamoDB-Compatible Apps with Python

Build DynamoDB-Compatible Apps with Python

Couchbase Data Pipeline

Couchbase Data Pipeline

Couchbase Data Pipeline

Justin Michaels

Deploying to the cloud has made it easy to run large numbers of servers, but users may become dissatisfied with their particular cloud platform for reasons such as price, support and performance. There are a number of vendor lock-ins to avoid, this talk discusses how to do so with the open source configuration management and infrastructure automation platform Chef. Chef makes it easy to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. Chef may also be used to deploy cloud infrastructures such as OpenStack, Eucalyptus or CloudStack. By abstracting away the platform, infrastructure becomes portable and you are free to deploy wherever necessary.

Achieving Infrastructure Portability with Chef

Achieving Infrastructure Portability with Chef

Achieving Infrastructure Portability with Chef

How to run a bank on Apache CloudStack

How to run a bank on Apache CloudStack

How to run a bank on Apache CloudStack

AIDevWorld 23 Apache NiFi 101 Introduction and Best Practices https://sched.co/1RoAO Timothy Spann, Cloudera, Principal Developer Advocate In this talk, we will walk step by step through Apache NiFi from the first load to first application. I will include slides, articles and examples to take away as a Quick Start to utilizing Apache NiFi in your real-time dataflows. I will help you get up and running locally on your laptop, Docker or in CDP Public Cloud. Wednesday November 1, 2023 12:00pm - 12:25pm PDT VIRTUAL AI DevWorld -- Main Stage https://app.hopin.com/events/api-world-2023-ai-devworld/stages Retail & E-Commerce AI (Industry AI Conference) Session Type OPEN TALK Track or Conference Retail & E-Commerce AI (Industry AI Conference), Industry AI Conference, VIRTUAL, Tensorflow & PyTorch & Open Source Frameworks (AI/ML Engineering Conference), AI/ML Engineering Conference, AI DevWorld In-Person/Virtual Virtual, Virtual Exclusive apache nifi Timothy Spann Cloudera Principal Developer Advocate for Data in Motion Tim Spann is the Principal Developer Advocate for Data in Motion @ Cloudera where he works with Apache Kafka, Apache Flink, Apache NiFi, Apache Iceberg, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal Field Engineer at Cloudera, a Senior Solutions Architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science. cloudera dataflow

AIDevWorldApacheNiFi101

AIDevWorldApacheNiFi101

AIDevWorldApacheNiFi101

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

dodai grizzly

Chef is an open source configuration management and service integration automation tool that has been integral to a number of large successful OpenStack deployments. This talk will provide a brief introduction to Chef and why it frequently the configuration tool of choice for large deployments and discuss the use of Chef within the OpenStack ecosystem (development, testing, deploying and managing the installation). Chef also provides the ability to manage the instances running on top of Nova through the knife-openstack plugin.

OpenStack Deployments with Chef

OpenStack Deployments with Chef

OpenStack Deployments with Chef

Similaire à Hadoop and subsystems in livedoor #Hcj11f (20)

Openstack India May Meetup

Openstack India May Meetup

Openstack India May Meetup

Manuel Hurtado. Couchbase paradigma4oct

Manuel Hurtado. Couchbase paradigma4oct

Manuel Hurtado. Couchbase paradigma4oct

Couchbase Day

Mining Your Logs - Gaining Insight Through Visualization

Mining Your Logs - Gaining Insight Through Visualization

Mining Your Logs - Gaining Insight Through Visualization

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...

Cloud Deployments with Apache Hadoop and Apache HBase

Cloud Deployments with Apache Hadoop and Apache HBase

Cloud Deployments with Apache Hadoop and Apache HBase

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)

Road to NODES - Handling Neo4j Data with Apache Hop

Road to NODES - Handling Neo4j Data with Apache Hop

Road to NODES - Handling Neo4j Data with Apache Hop

Splunk as a_big_data_platform_for_developers_spring_one2gx

Splunk as a_big_data_platform_for_developers_spring_one2gx

Splunk as a_big_data_platform_for_developers_spring_one2gx

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0

The path to a serverless-native era with Kubernetes

The path to a serverless-native era with Kubernetes

The path to a serverless-native era with Kubernetes

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

Build DynamoDB-Compatible Apps with Python

Build DynamoDB-Compatible Apps with Python

Build DynamoDB-Compatible Apps with Python

Couchbase Data Pipeline

Couchbase Data Pipeline

Couchbase Data Pipeline

Achieving Infrastructure Portability with Chef

Achieving Infrastructure Portability with Chef

Achieving Infrastructure Portability with Chef

How to run a bank on Apache CloudStack

How to run a bank on Apache CloudStack

How to run a bank on Apache CloudStack

AIDevWorldApacheNiFi101

AIDevWorldApacheNiFi101

AIDevWorldApacheNiFi101

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...

dodai grizzly

OpenStack Deployments with Chef

OpenStack Deployments with Chef

OpenStack Deployments with Chef

Plus de SATOSHI TAGOMORI

Ractor's speed is not light-speed

Ractor's speed is not light-speed

Ractor's speed is not light-speed

SATOSHI TAGOMORI

Good Things and Hard Things of SaaS Development/Operations

Good Things and Hard Things of SaaS Development/Operations

Good Things and Hard Things of SaaS Development/Operations

SATOSHI TAGOMORI

Maccro Strikes Back

Maccro Strikes Back

Maccro Strikes Back

SATOSHI TAGOMORI

Invitation to the dark side of Ruby

Invitation to the dark side of Ruby

Invitation to the dark side of Ruby

SATOSHI TAGOMORI

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

SATOSHI TAGOMORI

Make Your Ruby Script Confusing

Make Your Ruby Script Confusing

Make Your Ruby Script Confusing

SATOSHI TAGOMORI

Hijacking Ruby Syntax in Ruby

Hijacking Ruby Syntax in Ruby

Hijacking Ruby Syntax in Ruby

SATOSHI TAGOMORI

Lock, Concurrency and Throughput of Exclusive Operations

Lock, Concurrency and Throughput of Exclusive Operations

Lock, Concurrency and Throughput of Exclusive Operations

SATOSHI TAGOMORI

Data Processing and Ruby in the World

Data Processing and Ruby in the World

Data Processing and Ruby in the World

SATOSHI TAGOMORI

Planet-scale Data Ingestion Pipeline: Bigdam

Planet-scale Data Ingestion Pipeline: Bigdam

Planet-scale Data Ingestion Pipeline: Bigdam

SATOSHI TAGOMORI

Technologies, Data Analytics Service and Enterprise Business

Technologies, Data Analytics Service and Enterprise Business

Technologies, Data Analytics Service and Enterprise Business

SATOSHI TAGOMORI

Ruby and Distributed Storage Systems

Ruby and Distributed Storage Systems

Ruby and Distributed Storage Systems

SATOSHI TAGOMORI

Perfect Norikra 2nd Season

Perfect Norikra 2nd Season

Perfect Norikra 2nd Season

SATOSHI TAGOMORI

Fluentd 101

SATOSHI TAGOMORI

To Have Own Data Analytics Platform, Or NOT To

To Have Own Data Analytics Platform, Or NOT To

To Have Own Data Analytics Platform, Or NOT To

SATOSHI TAGOMORI

The Patterns of Distributed Logging and Containers

The Patterns of Distributed Logging and Containers

The Patterns of Distributed Logging and Containers

SATOSHI TAGOMORI

How To Write Middleware In Ruby

How To Write Middleware In Ruby

How To Write Middleware In Ruby

SATOSHI TAGOMORI

Modern Black Mages Fighting in the Real World

Modern Black Mages Fighting in the Real World

Modern Black Mages Fighting in the Real World

SATOSHI TAGOMORI

Open Source Software, Distributed Systems, Database as a Cloud Service

Open Source Software, Distributed Systems, Database as a Cloud Service

Open Source Software, Distributed Systems, Database as a Cloud Service

SATOSHI TAGOMORI

Fluentd Overview, Now and Then

Fluentd Overview, Now and Then

Fluentd Overview, Now and Then

SATOSHI TAGOMORI

Plus de SATOSHI TAGOMORI (20)

Ractor's speed is not light-speed

Ractor's speed is not light-speed

Ractor's speed is not light-speed

Good Things and Hard Things of SaaS Development/Operations

Good Things and Hard Things of SaaS Development/Operations

Good Things and Hard Things of SaaS Development/Operations

Maccro Strikes Back

Maccro Strikes Back

Maccro Strikes Back

Invitation to the dark side of Ruby

Invitation to the dark side of Ruby

Invitation to the dark side of Ruby

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

Hijacking Ruby Syntax in Ruby (RubyConf 2018)

Make Your Ruby Script Confusing

Make Your Ruby Script Confusing

Make Your Ruby Script Confusing

Hijacking Ruby Syntax in Ruby

Hijacking Ruby Syntax in Ruby

Hijacking Ruby Syntax in Ruby

Lock, Concurrency and Throughput of Exclusive Operations

Lock, Concurrency and Throughput of Exclusive Operations

Lock, Concurrency and Throughput of Exclusive Operations

Data Processing and Ruby in the World

Data Processing and Ruby in the World

Data Processing and Ruby in the World

Planet-scale Data Ingestion Pipeline: Bigdam

Planet-scale Data Ingestion Pipeline: Bigdam

Planet-scale Data Ingestion Pipeline: Bigdam

Technologies, Data Analytics Service and Enterprise Business

Technologies, Data Analytics Service and Enterprise Business

Technologies, Data Analytics Service and Enterprise Business

Ruby and Distributed Storage Systems

Ruby and Distributed Storage Systems

Ruby and Distributed Storage Systems

Perfect Norikra 2nd Season

Perfect Norikra 2nd Season

Perfect Norikra 2nd Season

Fluentd 101

To Have Own Data Analytics Platform, Or NOT To

To Have Own Data Analytics Platform, Or NOT To

To Have Own Data Analytics Platform, Or NOT To

The Patterns of Distributed Logging and Containers

The Patterns of Distributed Logging and Containers

The Patterns of Distributed Logging and Containers

How To Write Middleware In Ruby

How To Write Middleware In Ruby

How To Write Middleware In Ruby

Modern Black Mages Fighting in the Real World

Modern Black Mages Fighting in the Real World

Modern Black Mages Fighting in the Real World

Open Source Software, Distributed Systems, Database as a Cloud Service

Open Source Software, Distributed Systems, Database as a Cloud Service

Open Source Software, Distributed Systems, Database as a Cloud Service

Fluentd Overview, Now and Then

Fluentd Overview, Now and Then

Fluentd Overview, Now and Then

Dernier

CNIC Information System with Pakdata Cf In Pakistan

CNIC Information System with Pakdata Cf In Pakistan

CNIC Information System with Pakdata Cf In Pakistan

Passkeys: Developing APIs to enable passwordless authentication Cody Salas, Sr Developer Advocate | Solutions Architect - Yubico Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Vector Search -An Introduction in Oracle Database 23ai.pptx

Vector Search -An Introduction in Oracle Database 23ai.pptx

Vector Search -An Introduction in Oracle Database 23ai.pptx

Remote DBA Services

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Exploring Multimodal Embeddings with Milvus

Exploring Multimodal Embeddings with Milvus

Exploring Multimodal Embeddings with Milvus

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Rustici Software

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Introduction to use of FHIR Documents in ABDM

Introduction to use of FHIR Documents in ABDM

Introduction to use of FHIR Documents in ABDM

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard-hybrid-app-RailsConf2024.pptx

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Dernier (20)

CNIC Information System with Pakdata Cf In Pakistan

CNIC Information System with Pakdata Cf In Pakistan

CNIC Information System with Pakdata Cf In Pakistan

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Vector Search -An Introduction in Oracle Database 23ai.pptx

Vector Search -An Introduction in Oracle Database 23ai.pptx

Vector Search -An Introduction in Oracle Database 23ai.pptx

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Exploring Multimodal Embeddings with Milvus

Exploring Multimodal Embeddings with Milvus

Exploring Multimodal Embeddings with Milvus

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

AWS Community Day CPH - Three problems of Terraform

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Introduction to use of FHIR Documents in ABDM

Introduction to use of FHIR Documents in ABDM

Introduction to use of FHIR Documents in ABDM

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard-hybrid-app-RailsConf2024.pptx

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Hadoop and subsystems in livedoor #Hcj11f

1. Hadoop and Subsystems in livedoor Hadoop Conference Japan 2011 Fall 2011/09/26 tagomoris 2011 9 26

2. 2011 9 26

3. we are hiring! 2011 9 26

4. what's livedoor? 2011 9 26

5. 2011 9 26

6. large scale web services 2800+ servers 3200+ hosts 530+ web servers 2011 9 26

7. 20 Aug 2009 2011 9 26

8. Aug 2011 15Gbps (10Gbps + CDN 5Gbps) 2011 9 26

9. Hadoop in livedoor • 10 nodes (1+9) • 36 core, 32TB HDFS • CDH3b2 • with libhdfs, fuse-hdfs • Hive 0.6.0 (community package) 2011 9 26

10. Hadoop in livedoor data mining reporting page views, unique users, traffic amount per page, ... 2011 9 26

11. super large scale 'sed | grep | wc' with Hadoop Streaming + Hive 2011 9 26

12. httpd logs from 96 servers (apache / nginx) 580GB/day (raw) 2011 9 26

13. overview hourly daily on hourly demand 2011 9 26

14. topics •log delivery network with scribe •and 'scribeline' •hive client web application 'shib' 2011 9 26

15. overview hourly daily on hourly demand 2011 9 26

16. scribe log delivery daemon based on Thrift scalable, reliable supports HDFS https://github.com/facebook/scribe 2011 9 26

17. scribe nodes scribed scribed scribed 2011 9 26

18. deliver node traffic 2011 9 26

19. scribe nodes scribed scribed scribed 2011 9 26

20. what we want from scribe agent •easy to deploy •works w/o any httpd configurations •delivery target failover/takeback •lightweight (without JVM) •stable 2011 9 26

21. scribe nodes scribed scribed scribeline scribed 2011 9 26

22. scribeline log delivery agent tool python 2.4, thrift easy to setup and start/stop works without any httpd configurations works with logrotate-ed log files automatic delivery target failover/takeback https://github.com/tagomoris/scribe_line 2011 9 26

23. how to setup scribeline in livedoor 1. yum install scribeline (tar xzf && cd && sudo make install) 2. vi /etc/scribeline.conf blog /var/log/httpd/access_log blogimg /var/log/nginx/access_log 3. /etc/init.d/scribeline start 2011 9 26

24. scribe nodes scribed scribed scribed 2011 9 26

25. overview hourly daily on hourly demand 2011 9 26

26. what we want about hive client •easy to experiment •from PC on our desks •result caching •protection against data loss •friendly look & feel 2011 9 26

27. shib hive client web application node.js, thrift, kyoto tycoon query history browser query editor, based on copy&paste result caching & download tsv/csv filter INSERT/DROP/CREATE ... https://github.com/tagomoris/shib 2011 9 26

28. 2011 9 26

29. shib system overview 2011 9 26

30. what shib cannot do now •access control •graph & chart •hive 0.7.0+ features support •database, authentication and ... •mapreduce status notification 2011 9 26

31. what we are trying now •New cluster •more nodes •CDH3b2 + Hive 0.6.0 -> CDH3u1 •New tools •Hoop (instead of fuse-hdfs) •Any stream processing framework 2011 9 26

32. thanks! 2011 9 26