Soumettre la recherche
Mettre en ligne
Cloumon enterprise
•
3 j'aime
•
2,567 vues
Gruter
Suivre
monitoring system for opensource cloud computing, ZooKeeper, Hadoop, Cassandra, HBase....
Lire moins
Lire la suite
Signaler
Partager
Signaler
Partager
1 sur 31
Recommandé
Telecom universal datastatesharingfabric
Telecom universal datastatesharingfabric
Shay Hassidim
Architecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big Data
Richard McDougall
Apachecon Euro 2012: Elastic, Multi-tenant Hadoop on Demand
Apachecon Euro 2012: Elastic, Multi-tenant Hadoop on Demand
Richard McDougall
Inside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworld
Richard McDougall
Hadoop on VMware
Hadoop on VMware
Richard McDougall
Dell web monsters-oct2011-v6-public
Dell web monsters-oct2011-v6-public
Barton George
The Sirocco multi-cloud management framework, OW2con'12, Paris
The Sirocco multi-cloud management framework, OW2con'12, Paris
OW2
Cosbench apac
Cosbench apac
OpenCity Community
Recommandé
Telecom universal datastatesharingfabric
Telecom universal datastatesharingfabric
Shay Hassidim
Architecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big Data
Richard McDougall
Apachecon Euro 2012: Elastic, Multi-tenant Hadoop on Demand
Apachecon Euro 2012: Elastic, Multi-tenant Hadoop on Demand
Richard McDougall
Inside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworld
Richard McDougall
Hadoop on VMware
Hadoop on VMware
Richard McDougall
Dell web monsters-oct2011-v6-public
Dell web monsters-oct2011-v6-public
Barton George
The Sirocco multi-cloud management framework, OW2con'12, Paris
The Sirocco multi-cloud management framework, OW2con'12, Paris
OW2
Cosbench apac
Cosbench apac
OpenCity Community
cosbench-openstack.pdf
cosbench-openstack.pdf
OpenStack Foundation
Hadoop on Virtual Machines
Hadoop on Virtual Machines
Richard McDougall
Ramakrishnan Keynote Ladis2009
Ramakrishnan Keynote Ladis2009
yarapavan
Simplifying Cloud Implementation
Simplifying Cloud Implementation
Morphlabs
Common and unique use cases for Apache Hadoop
Common and unique use cases for Apache Hadoop
Brock Noland
30a accessing your cluster
30a accessing your cluster
mapr-academy
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
cwensel
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
eNovance
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
MapR Technologies Japan
Virtual Hadoop Introduction In Chinese
Virtual Hadoop Introduction In Chinese
天青 王
Zh tw cloud computing era
Zh tw cloud computing era
TrendProgContest13
London hug
London hug
Ted Dunning
50a volumes
50a volumes
mapr-academy
The Value of NetApp with VMware
The Value of NetApp with VMware
Capito Livingstone
Server Virtualization with QNAP® Turbo NAS and VMware®
Server Virtualization with QNAP® Turbo NAS and VMware®
Ali Shoaee
Open repository 2011_duracloud-final
Open repository 2011_duracloud-final
Mark Diggory
Cosbench apac
Cosbench apac
OpenCity Community
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
Alex Gorbachev
Optimization of Resource Provisioning Cost in Cloud Computing
Optimization of Resource Provisioning Cost in Cloud Computing
Sivadon Chaisiri
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
Chris Purrington
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
Gruter
빅데이터와 보안
빅데이터와 보안
Myounghun Kang
Contenu connexe
Tendances
cosbench-openstack.pdf
cosbench-openstack.pdf
OpenStack Foundation
Hadoop on Virtual Machines
Hadoop on Virtual Machines
Richard McDougall
Ramakrishnan Keynote Ladis2009
Ramakrishnan Keynote Ladis2009
yarapavan
Simplifying Cloud Implementation
Simplifying Cloud Implementation
Morphlabs
Common and unique use cases for Apache Hadoop
Common and unique use cases for Apache Hadoop
Brock Noland
30a accessing your cluster
30a accessing your cluster
mapr-academy
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
cwensel
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
eNovance
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
MapR Technologies Japan
Virtual Hadoop Introduction In Chinese
Virtual Hadoop Introduction In Chinese
天青 王
Zh tw cloud computing era
Zh tw cloud computing era
TrendProgContest13
London hug
London hug
Ted Dunning
50a volumes
50a volumes
mapr-academy
The Value of NetApp with VMware
The Value of NetApp with VMware
Capito Livingstone
Server Virtualization with QNAP® Turbo NAS and VMware®
Server Virtualization with QNAP® Turbo NAS and VMware®
Ali Shoaee
Open repository 2011_duracloud-final
Open repository 2011_duracloud-final
Mark Diggory
Cosbench apac
Cosbench apac
OpenCity Community
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
Alex Gorbachev
Optimization of Resource Provisioning Cost in Cloud Computing
Optimization of Resource Provisioning Cost in Cloud Computing
Sivadon Chaisiri
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
Chris Purrington
Tendances
(20)
cosbench-openstack.pdf
cosbench-openstack.pdf
Hadoop on Virtual Machines
Hadoop on Virtual Machines
Ramakrishnan Keynote Ladis2009
Ramakrishnan Keynote Ladis2009
Simplifying Cloud Implementation
Simplifying Cloud Implementation
Common and unique use cases for Apache Hadoop
Common and unique use cases for Apache Hadoop
30a accessing your cluster
30a accessing your cluster
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
Openstack in action2! Automate and accelerate Cloud deployments with Dell Cro...
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
エンタープライズ NoSQL/HBase プラットフォーム – MapR M7 エディション - db tech showcase 大阪 2014 201...
Virtual Hadoop Introduction In Chinese
Virtual Hadoop Introduction In Chinese
Zh tw cloud computing era
Zh tw cloud computing era
London hug
London hug
50a volumes
50a volumes
The Value of NetApp with VMware
The Value of NetApp with VMware
Server Virtualization with QNAP® Turbo NAS and VMware®
Server Virtualization with QNAP® Turbo NAS and VMware®
Open repository 2011_duracloud-final
Open repository 2011_duracloud-final
Cosbench apac
Cosbench apac
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
UTHOC2 - Under The Hood of Oracle Clusterware 2.0 - Grid Infrastructure by Al...
Optimization of Resource Provisioning Cost in Cloud Computing
Optimization of Resource Provisioning Cost in Cloud Computing
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
Ubuntu in the cloud What's Coming - Nick Barcet, Canonical
En vedette
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
Gruter
빅데이터와 보안
빅데이터와 보안
Myounghun Kang
보안사고의 이면
보안사고의 이면
Myounghun Kang
화요일 설찬수유태혁오진주123
화요일 설찬수유태혁오진주123
Jinjoo Oh
자동인식&스마트SCM(MONTHLY AIDC+SMART SCM) 2013년 10월호
자동인식&스마트SCM(MONTHLY AIDC+SMART SCM) 2013년 10월호
고양뉴스
보안과 빅데이터의 올바른 접목
보안과 빅데이터의 올바른 접목
Myounghun Kang
스마트폰 App 기획 사업 전략
스마트폰 App 기획 사업 전략
kimya76
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 인터넷 쇼핑몰의 실시간 분석 플랫폼 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 인터넷 쇼핑몰의 실시간 분석 플랫폼 구축 사례
Gruter
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: GRUTER의 빅데이터 플랫폼 및 전략 소개
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: GRUTER의 빅데이터 플랫폼 및 전략 소개
Gruter
모바일 서비스 기획 시작하기
모바일 서비스 기획 시작하기
Jae-hyung Park
20140915 (주)우노플러스 회사소개서
20140915 (주)우노플러스 회사소개서
rotcbiz
서비스모델별 표준화요소 Final
서비스모델별 표준화요소 Final
KMS School
# Startup building process liftup
# Startup building process liftup
zyro park
En vedette
(13)
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 보안 로그 분석을 위한 빅데이터 시스템 구축 사례
빅데이터와 보안
빅데이터와 보안
보안사고의 이면
보안사고의 이면
화요일 설찬수유태혁오진주123
화요일 설찬수유태혁오진주123
자동인식&스마트SCM(MONTHLY AIDC+SMART SCM) 2013년 10월호
자동인식&스마트SCM(MONTHLY AIDC+SMART SCM) 2013년 10월호
보안과 빅데이터의 올바른 접목
보안과 빅데이터의 올바른 접목
스마트폰 App 기획 사업 전략
스마트폰 App 기획 사업 전략
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 인터넷 쇼핑몰의 실시간 분석 플랫폼 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: 인터넷 쇼핑몰의 실시간 분석 플랫폼 구축 사례
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: GRUTER의 빅데이터 플랫폼 및 전략 소개
GRUTER가 들려주는 Big Data Platform 구축 전략과 적용 사례: GRUTER의 빅데이터 플랫폼 및 전략 소개
모바일 서비스 기획 시작하기
모바일 서비스 기획 시작하기
20140915 (주)우노플러스 회사소개서
20140915 (주)우노플러스 회사소개서
서비스모델별 표준화요소 Final
서비스모델별 표준화요소 Final
# Startup building process liftup
# Startup building process liftup
Similaire à Cloumon enterprise
[OSDC.tw 2011] The Path to Pass into PaaS -- How We Build the Solution
[OSDC.tw 2011] The Path to Pass into PaaS -- How We Build the Solution
Jeff Hung
Sdc2010 scality cloud storage vs object storage for distribution
Sdc2010 scality cloud storage vs object storage for distribution
Jerome Lecat
Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
CLOUDIAN KK
彭—Elastic architecture in cloud foundry and deploy with openstack
彭—Elastic architecture in cloud foundry and deploy with openstack
OpenCity Community
Cloudian at cassandra conference in tokyo
Cloudian at cassandra conference in tokyo
CLOUDIAN KK
Ga cloud scaling 3 30-2012
Ga cloud scaling 3 30-2012
Andy Parsons
VMware, Storage & Kitchen appliances
VMware, Storage & Kitchen appliances
subtitle
Building a Hybrid Cloud
Building a Hybrid Cloud
SVForum Cloud SIG
Aptare Introduction Presentation April 2012
Aptare Introduction Presentation April 2012
nigelhoughton17
Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1
Ram Chinta
CloudStack Architecture Future
CloudStack Architecture Future
Kimihiko Kitase
Cloud computing with AWS
Cloud computing with AWS
ikanow
CA Nimsoft xen desktop monitoring
CA Nimsoft xen desktop monitoring
CA Nimsoft
Danile lee -open stackblocklevelstorage
Danile lee -open stackblocklevelstorage
OpenCity Community
Distributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile lee
Hui Cheng
Hadoop World 2011: How Hadoop Revolutionized Business Intelligence and Advanc...
Hadoop World 2011: How Hadoop Revolutionized Business Intelligence and Advanc...
Cloudera, Inc.
Cloud Computing: Making IT Simple
Cloud Computing: Making IT Simple
Bob Rhubart
Cloud Computing - Making IT Simple
Cloud Computing - Making IT Simple
Bob Rhubart
Track 1, Session 3 - intelligent infrastructure for the virtualized world by ...
Track 1, Session 3 - intelligent infrastructure for the virtualized world by ...
EMC Forum India
Windows Azure Uzerinden Alinabilen Hizmetler
Windows Azure Uzerinden Alinabilen Hizmetler
Mustafa
Similaire à Cloumon enterprise
(20)
[OSDC.tw 2011] The Path to Pass into PaaS -- How We Build the Solution
[OSDC.tw 2011] The Path to Pass into PaaS -- How We Build the Solution
Sdc2010 scality cloud storage vs object storage for distribution
Sdc2010 scality cloud storage vs object storage for distribution
Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
彭—Elastic architecture in cloud foundry and deploy with openstack
彭—Elastic architecture in cloud foundry and deploy with openstack
Cloudian at cassandra conference in tokyo
Cloudian at cassandra conference in tokyo
Ga cloud scaling 3 30-2012
Ga cloud scaling 3 30-2012
VMware, Storage & Kitchen appliances
VMware, Storage & Kitchen appliances
Building a Hybrid Cloud
Building a Hybrid Cloud
Aptare Introduction Presentation April 2012
Aptare Introduction Presentation April 2012
Ram chinta hug-20120922-v1
Ram chinta hug-20120922-v1
CloudStack Architecture Future
CloudStack Architecture Future
Cloud computing with AWS
Cloud computing with AWS
CA Nimsoft xen desktop monitoring
CA Nimsoft xen desktop monitoring
Danile lee -open stackblocklevelstorage
Danile lee -open stackblocklevelstorage
Distributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile lee
Hadoop World 2011: How Hadoop Revolutionized Business Intelligence and Advanc...
Hadoop World 2011: How Hadoop Revolutionized Business Intelligence and Advanc...
Cloud Computing: Making IT Simple
Cloud Computing: Making IT Simple
Cloud Computing - Making IT Simple
Cloud Computing - Making IT Simple
Track 1, Session 3 - intelligent infrastructure for the virtualized world by ...
Track 1, Session 3 - intelligent infrastructure for the virtualized world by ...
Windows Azure Uzerinden Alinabilen Hizmetler
Windows Azure Uzerinden Alinabilen Hizmetler
Plus de Gruter
MelOn 빅데이터 플랫폼과 Tajo 이야기
MelOn 빅데이터 플랫폼과 Tajo 이야기
Gruter
Introduction to Apache Tajo: Future of Data Warehouse
Introduction to Apache Tajo: Future of Data Warehouse
Gruter
Expanding Your Data Warehouse with Tajo
Expanding Your Data Warehouse with Tajo
Gruter
Introduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big Data
Gruter
Introduction to Apache Tajo
Introduction to Apache Tajo
Gruter
스타트업사례로 본 로그 데이터분석 : Tajo on AWS
스타트업사례로 본 로그 데이터분석 : Tajo on AWS
Gruter
What's New Tajo 0.10 and Its Beyond
What's New Tajo 0.10 and Its Beyond
Gruter
Big data analysis with R and Apache Tajo (in Korean)
Big data analysis with R and Apache Tajo (in Korean)
Gruter
Efficient In‐situ Processing of Various Storage Types on Apache Tajo
Efficient In‐situ Processing of Various Storage Types on Apache Tajo
Gruter
Tajo TPC-H Benchmark Test on AWS
Tajo TPC-H Benchmark Test on AWS
Gruter
Data analysis with Tajo
Data analysis with Tajo
Gruter
Gruter TECHDAY 2014 Realtime Processing in Telco
Gruter TECHDAY 2014 Realtime Processing in Telco
Gruter
Gruter TECHDAY 2014 MelOn BigData
Gruter TECHDAY 2014 MelOn BigData
Gruter
Gruter_TECHDAY_2014_04_TajoCloudHandsOn (in Korean)
Gruter_TECHDAY_2014_04_TajoCloudHandsOn (in Korean)
Gruter
Gruter_TECHDAY_2014_03_ApacheTajo (in Korean)
Gruter_TECHDAY_2014_03_ApacheTajo (in Korean)
Gruter
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
Gruter
Apache Tajo - BWC 2014
Apache Tajo - BWC 2014
Gruter
Elastic Search Performance Optimization - Deview 2014
Elastic Search Performance Optimization - Deview 2014
Gruter
Hadoop security DeView 2014
Hadoop security DeView 2014
Gruter
Vectorized processing in_a_nutshell_DeView2014
Vectorized processing in_a_nutshell_DeView2014
Gruter
Plus de Gruter
(20)
MelOn 빅데이터 플랫폼과 Tajo 이야기
MelOn 빅데이터 플랫폼과 Tajo 이야기
Introduction to Apache Tajo: Future of Data Warehouse
Introduction to Apache Tajo: Future of Data Warehouse
Expanding Your Data Warehouse with Tajo
Expanding Your Data Warehouse with Tajo
Introduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo
Introduction to Apache Tajo
스타트업사례로 본 로그 데이터분석 : Tajo on AWS
스타트업사례로 본 로그 데이터분석 : Tajo on AWS
What's New Tajo 0.10 and Its Beyond
What's New Tajo 0.10 and Its Beyond
Big data analysis with R and Apache Tajo (in Korean)
Big data analysis with R and Apache Tajo (in Korean)
Efficient In‐situ Processing of Various Storage Types on Apache Tajo
Efficient In‐situ Processing of Various Storage Types on Apache Tajo
Tajo TPC-H Benchmark Test on AWS
Tajo TPC-H Benchmark Test on AWS
Data analysis with Tajo
Data analysis with Tajo
Gruter TECHDAY 2014 Realtime Processing in Telco
Gruter TECHDAY 2014 Realtime Processing in Telco
Gruter TECHDAY 2014 MelOn BigData
Gruter TECHDAY 2014 MelOn BigData
Gruter_TECHDAY_2014_04_TajoCloudHandsOn (in Korean)
Gruter_TECHDAY_2014_04_TajoCloudHandsOn (in Korean)
Gruter_TECHDAY_2014_03_ApacheTajo (in Korean)
Gruter_TECHDAY_2014_03_ApacheTajo (in Korean)
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
Gruter_TECHDAY_2014_01_SearchEngine (in Korean)
Apache Tajo - BWC 2014
Apache Tajo - BWC 2014
Elastic Search Performance Optimization - Deview 2014
Elastic Search Performance Optimization - Deview 2014
Hadoop security DeView 2014
Hadoop security DeView 2014
Vectorized processing in_a_nutshell_DeView2014
Vectorized processing in_a_nutshell_DeView2014
Cloumon enterprise
1.
Management/Monitoring for Cloud
Computing ! ZooKeeper ! Cassandra ! Hadoop contact@gruter.com 070-8129-2950
2.
Cloumon
? (Cloumon®) ZooKeeper, Cassandra, Hadoop , . Copyright © 2011 Gruter Ltd.. All rights reserved 2
3.
Cloumon
ZooKeeper Cassandra Hadoop • Znode • Cassandra • Hadoop • Znode • Cassandra • Znode • Hadoop • Copyright © 2011 Gruter Ltd.. All rights reserved 3
4.
Why Cloumon?
• – – • – , – • – – Cloumon Copyright © 2011 Gruter Ltd.. All rights reserved 4
5.
Server
Storage ERP SCM Mail Virtualization Tech. Automatic Management Tool , , , IT . IT . Copyright © 2011 Gruter Ltd.. All rights reserved 5
6.
Traditional vs. Cloud
Computing Traditional Cloud(AWS) Request Request Firewall Security Group Load Balancer Elastic AutoScaling (L4) LoadBalancer Physical Server Physical Server EC2 Instance EC2 Instance (Unix/Linux) (Unix/Linux) Cloud Monitoring Watch Database Storage Various DB Cloud Storage (Oracle) (SAN, NAS) (NoSQL, RDBMS) Backup ( Tape, Disk Array) - - Copyright © 2011 Gruter Ltd.. All rights reserved 6
7.
System Architecture for
Cloud Computing Copyright © 2011 Gruter Ltd.. All rights reserved 7
8.
Software Components for
Cloud Computing Infrastructure Service Runtime Environment Cloud Data Storage Interface Protocol (Cassandra, Hbase) (Thrift, SOAP, REST, …) Management API Lightweight Cont Other Framewor Cloud File Storage ainer(Spring) ks (Hadoop) Data Mapper File System API Server Virtualization Application Server (KVM, Xen, VMWare) (Tomcat, .NET, Thrift Server, …) Development Environment Management/Common Service Management Portal IDL Compiler Common Service (Thrift, WSDL) (Billing, Metering, SSO) Service Repository/Framework plug-in Service Repository Development IDE Distributed Coordinator (Eclipse) (ZooKeeper) Copyright © 2011 Gruter Ltd.. All rights reserved 8
9.
ZooKeeper
• – Distributed Lock Coordinator – SPOF(Single Point Of Failure) – : zookeeper.apache.org • – • • n replication – Notification • , , • – – High Availability Master Election – Lock Copyright © 2011 Gruter Ltd.. All rights reserved 9
10.
ZooKeeper Copyright © 2011
Gruter Ltd.. All rights reserved 10
11.
ZooKeeper Usage
1) create() ZooKeeper create(“10.1.1.1”, EPHEMERAL) subA_server1 getChildren(“/service/subA”, watcher) 10.1.1.1 /service/subA subA 10.1.1.1 client create(“10.1.1.2”, event 10.1.1.2 EPHEMERAL) 3) getChildren cache 4) cache subA subA_server2 6) subA / watcher 10.1.1.3 10.1.1.2 cache event 5) (EPHEMERAL) subA_server3 10.1.1.3 Copyright © 2011 Gruter Ltd.. All rights reserved 11
12.
ZooKeeper Usage
High Availability: Master Election 1) lock master 2) master IP setData subA_master_1 ZooKeeper (10.1.1.1) subA_master /subA/master lock release client (10.1.1.1) 3) getData(“/service/subA_master”) 4) cache - acquireLock() 5) watcher release event - register watcher subA_master_2 (10.1.1.2) 6) master lock 7) IP setData Copyright © 2011 Gruter Ltd.. All rights reserved 12
13.
Cloumon ZooKeeper
• – Znode • Action • / / – CloumonManager Acton • – Refresh • ZooKeeper – ZooKeeper – – Historical Notification Listener CloumonManager ZooKeeper Aciton Alarm Node Manager Manager (Mail, SMS) Znode ZooKeeper Cluster Manager Server #1 Action Manager Host Manager ZooKeeper (Mail, SMS) Server #2 Node Metrics ZooKeeper Data Data Server #n Copyright © 2011 Gruter Ltd.. All rights reserved 13
14.
Cloumon ZooKeeper
• Latency min/avg/max: Latency • Received: Packet • Sent: Packet • Outstanding: request • Zxid: ID • Mode zookeeper: (leader, follower) • Node count: znode • Client Connection ( Connection ) – remoteAddress: – Queued: request – Recved: packet – Sent: packet – Sid: session Id – Lop: connection Operation – Est: – to : connection session timeout – Lcxid: Last cxid of this connection – Lzxid: connection Id – Minlat: latency – Avglat: latency – Maxlat: latency Copyright © 2011 Gruter Ltd.. All rights reserved 14
15.
Cloumon ZooKeepper Node
• – Cloumon instance ZooKeeper • – Tree View – / / • ACL • Recursive – • Validation Check Rule – ACL • ACL • ACL – Event Notification • Event Listener • (Node Deleted, Data Changed, Child Changed) Listener • Action Copyright © 2011 Gruter Ltd.. All rights reserved 15
16.
Cassandra
• – Facebook , – RDBMS NoSQL • – Keyspace, ColumnFamily, ColumnKey, Column • – N Linux – Hash(key) – / • – n-replication – • – : Thrift, Avro – : Java, C/C++, PHP, Python – 3rd : Hector Copyright © 2011 Gruter Ltd.. All rights reserved 16
17.
Cassandra
Server1 (token=A) Storage RPC Daemon Server5 Service (Thrift, Avro) Server2 (token=V) (token=D) JMX Server4 Server3 (token=O) (token=K) • (P2P) • – token token – Server2: A → D, Server3: D → K, … Server1: V → A • Partitioner – Random Partitioner: Hash(Key) – OrderPreservingPartitioner: Key Copyright © 2011 Gruter Ltd.. All rights reserved 17
18.
Cloumon Cassandra
• Cassandra Seed – Seed • – – – Historical • Cassandra – , , Notification Data CloumonManager Cassandra Alarm Cassandra Manager Manager Cassandra (Mail, SMS) / Cassandra Cluster Manager Server #1 Action Manager Host Manager Cassandra (Mail, SMS) Server #2 Metrics Cassandra Data Data Server #n Copyright © 2011 Gruter Ltd.. All rights reserved 18
19.
Cloumon Cassandra
• 14 , 190 • Size • db.Caches • db.Caches • Capacity • db.Commitlog • Requests • db.ColumnFailies • Hits • db.IndexColumnFamilies • RecentHitRate • db.CompactionManager • db.DynamicEndpointSnitch • MaxRowSize • db.StorageProxy • MemtableDataSize • db.StorageService • MemtableSwitchCount • net.StreamingService • net.MessagingService • MemtableThroughputInMB • net.FailureDetector • ReadCount • request.MutationStage • TotalDiskSpaceUsed • request.ReadStage • request.RequestResponseStage • WriteCount • request.ReplicateOnWriteStage • RecentRangeLatencyMicros • request.ReadRepairStage • RecentReadLatencyMicros • internal.AntiEntropyStage • RecentWriteLatencyMicros • internal.GossipStage • internal.MigrationStage • Joined • internal.MiscStage • Keyspaces • internal.StreamStage • LeavingNodes • internal.InternalResponseStage • internal.FlushSorter • LiveNodes • internal.FlushWriter • Load • internal.HintedHandoff • RPCServerRunning • internal.FILEUTILS-DELETE-POOL • internal.MemtablePostFlusher • RemovalStatus • Token • Status Copyright © 2011 Gruter Ltd.. All rights reserved 19
20.
Hadoop
• Cloud Computing – / – • Distributed File System – Very Large Scale Distributed File System • 10K nodes, 100 million files, 10 PB – Use Commodity Hardware • self-healing: failover, recovery, backup – Optimized for batch processing • Distributed/Parallel Computing framework – MapReduce , – HDFS • / – • FIFO, Fair, Capacity • – • Job: • Task: n Task(Map or Reduce) – / • • , (streaming) Copyright © 2011 Gruter Ltd.. All rights reserved 20
21.
Hadoop
Secondary NameNode NameNode File Namespace /foo/bar NameNode Request Meta 1 Operation Client Server Status Block Management File Block File Read/Write DataNode #1 DataNode #2 DataNode #3 DataNode #4 DataNode #5 (rack)1 (rack)2 Copyright © 2011 Gruter Ltd.. All rights reserved 21
22.
Cloumon Hadoop
• Hadoop Metrics – Hadoop metrics context CloumonContext • – Hadoop CloumonManager metrics – – – Historical • Hadoop – : , , Notification Data CloumonManager Hadoop Alarm Hadoop Manager Manager (Mail, SMS) Hadoop Cloumon hadoop Cluster Manager Context Server #1 Action Manager Host Manager Cloumon hadoop (Mail, SMS) Context Server #2 Metrics Cloumon hadoop Data Data Context Server #n Copyright © 2011 Gruter Ltd.. All rights reserved 22
23.
Cloumon Hadoop
• NameNode – FSNamesystem • BlocksTotal, CapacityRemain, CapacityTotal, CapacityUse d, FilesTotal, MissingBlocks, TotalLoad – Dfs.NameNode • BlockCorrupted, CreateFileOps, Transactions_avg_time – Jvm.metrics: • gcCount, gcTimeMillis, memHeapUsed, logWarn, logError, threadsNew • DataNode – Dfs.datanode • Blocks_operation, bytes_read, bytes_written, avg_time – Jvm.metrics: • gcCount, gcTimeMillis, memHeapUsed, logWarn, logError, threadsNew Copyright © 2011 Gruter Ltd.. All rights reserved 23
24.
Cloumon
ZooKeeper Copyright © 2011 Gruter Ltd.. All rights reserved 24
25.
Cloumon
ZooKeeper Copyright © 2011 Gruter Ltd.. All rights reserved 25
26.
Cloumon
Cassandra Copyright © 2011 Gruter Ltd.. All rights reserved 26
27.
Cloumon
Hadoop Copyright © 2011 Gruter Ltd.. All rights reserved 27
28.
Cloumon
Hadoop Copyright © 2011 Gruter Ltd.. All rights reserved 28
29.
Cloumon
• 1 instance – – metrics DB • MySQL : 2000 ~ 3000 Transaction/sec – • 60 • ZooKeeper – ZooKeeper ZooKeeper ( clien t 1 record ) – ZooKeeper Client /60 – ZooKeeper • Cassandra – 190 /60 * – Cassandra 600 – 190 • Hadoop – 77 /60 * – Hadoop 800 – 77 • – Historical – 10 12GB Copyright © 2011 Gruter Ltd.. All rights reserved 29
30.
Open Source vs.
Enterprise Open Source Enterprise X Server O ( ) O O O ZooKeeper ACL X O Listener X O Cassandra X O Hadoop O O User Interface Bad Good Copyright © 2011 Gruter Ltd.. All rights reserved 30
31.
Cloumon release milestone
• Cloumon-0.9: 2011.03 – – www.cloumon.org – – ZooKeeper , Hadoop • Cloumon-enterprise-1.0: 2011.05 – ZooKeeper – ZooKeeper ACL, Event Listener – Cassandra – Hadoop – UI • Cloumon-enterprise-2.0: 2011.07 – Cassandra – Cassandra Client(Hector) – (Dashboard) – Cloumon Manager HA – UI ( ) – • Cloumon-enterprise-3.0: 2011.09 – Hadoop MapReduce – Hbase Copyright © 2011 Gruter Ltd.. All rights reserved 31