SlideShare a Scribd company logo
1 of 15
B r i d g i n g t h e G a p 
SwissBigDataUserGroup 23.09.2014 / André Vocat 
Cassandra @ Swiss Telco Operator
•Introduction 
•System environment 
•System overview 
•Non functional requirements 
•System load 
•System architecture 
•Cassandra cluster 
2 
Intersys AG | Zuchwil | www.intersys.ch 
Agenda
•software architect, developer and project lead working for Intersys AG in Zuchwil. 
•holding an engineering degree in electrical engineering 
•more than 20 years experience developing software and designing software architectures 
•specifically for the telecommunications market. 
•lead architect of the system presented today 
–predecessor system (started in late 1999) 
–first version of this critical platform (started in 2002) 
–complete renewal (started in 2012) 
–production use (started mid 2013) 
3 
Intersys AG | Zuchwil | www.intersys.ch 
Bio
Intersys consists of Interfaces and Systems 
Interfaces 
Exchange of data between 
Systems of different Origin 
Systems 
Development of Software 
Systems and system 
components 
4 
Intersys AG | Zuchwil | www.intersys.ch 
Interfaces and Systems 
Main offerings - Software Development (specify, implement, test, run, support) - Test Management (independent software quality certification)
Fast Facts 
Founded 
1999 
Number of Employees 
27 (most of them higher education in IT or electrical engineering) 
Locations 
Zuchwil (HQ), Zürich (Branch) 
Common Stock 
CHF 250’000.-, owned by employees 
DataStax Partner 
First official DataStax Integration Partner in Switzerland 
ISO 9001:2008 certified We educate apprentices 
5 
Intersys AG | Zuchwil | www.intersys.ch
6 
Intersys AG | Zuchwil | www.intersys.ch 
System environment
7 
Intersys AG | Zuchwil | www.intersys.ch 
System overview
•availability 
•high performance 
•flexibility 
•scalability 
•upgradeability (minimal / «zero» downtime) 
•surrounding systems must not affect others 
8 
Intersys AG | Zuchwil | www.intersys.ch 
Non functional requirements
System load per hour 
9 
Intersys AG | Zuchwil | www.intersys.ch 
0 
2'000'000 
4'000'000 
6'000'000 
8'000'000 
00.00 h 
01.00 h 
02.00 h 
03.00 h 
04.00 h 
05.00 h 
06.00 h 
07.00 h 
08.00 h 
09.00 h 
10.00 h 
11.00 h 
12.00 h 
13.00 h 
14.00 h 
15.00 h 
16.00 h 
17.00 h 
18.00 h 
19.00 h 
20.00 h 
21.00 h 
22.00 h 
23.00 h 
request count per hour 
0 
50'000 
100'000 
150'000 
200'000 
250'000 
00.00 h 
01.00 h 
02.00 h 
03.00 h 
04.00 h 
05.00 h 
06.00 h 
07.00 h 
08.00 h 
09.00 h 
10.00 h 
11.00 h 
12.00 h 
13.00 h 
14.00 h 
15.00 h 
16.00 h 
17.00 h 
18.00 h 
19.00 h 
20.00 h 
21.00 h 
22.00 h 
23.00 h 
update count per hour
System load per day 
10 
Intersys AG | Zuchwil | www.intersys.ch 
0 
500'000 
1'000'000 
1'500'000 
SO 
MO 
DI 
MI 
DO 
FR 
SA 
update count per day 
0 
20'000'000 
40'000'000 
60'000'000 
80'000'000 
100'000'000 
120'000'000 
140'000'000 
SO 
MO 
DI 
MI 
DO 
FR 
SA 
request count per day
System load per month 
11 
Intersys AG | Zuchwil | www.intersys.ch 
0 
500'000'000 
1'000'000'000 
1'500'000'000 
2'000'000'000 
2'500'000'000 
nov 
dez 
jan 
feb 
mar 
apr 
mai 
jun 
requests count per month 
0 
5'000'000 
10'000'000 
15'000'000 
20'000'000 
25'000'000 
30'000'000 
mar 
apr 
mai 
jun 
jul 
aug 
update count per month
12 
Intersys AG | Zuchwil | www.intersys.ch 
System architecture
13 
Intersys AG | Zuchwil | www.intersys.ch 
Cassandra Cluster internals
•Lessons learned 
–Indexed queries memory usage 
–write operation only as batch 
–regularly repair data 
–range query only for small data amount 
–do not delete column and reinsert 
–regularly upgrade cassandra version 
•Open learnings 
–Optimize after version upgrade 
–still more to come… 
14 
Intersys AG | Zuchwil | www.intersys.ch 
Lessons learned / Open learnings
•André Vocat 
andre.vocat@intersys.ch 
+41 32 625 76 84 
+41 79 233 33 94 
•Reto Gantenbein 
reto.gantenbein@intersys.ch 
+41 32 625 76 93 
+41 79 701 47 35 
15 
Intersys AG | Zuchwil | www.intersys.ch 
Koordinaten

More Related Content

What's hot

From Data to Action with TV 2
From Data to Action with TV 2From Data to Action with TV 2
From Data to Action with TV 2Elasticsearch
 
Lets Talk Google BigQuery
Lets Talk Google BigQueryLets Talk Google BigQuery
Lets Talk Google BigQueryMatillion
 
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...DataStax Academy
 
Consolidate your data marts for fast, flexible analytics 5.24.18
Consolidate your data marts for fast, flexible analytics 5.24.18Consolidate your data marts for fast, flexible analytics 5.24.18
Consolidate your data marts for fast, flexible analytics 5.24.18Cloudera, Inc.
 
Big data journey to the cloud maz chaudhri 5.30.18
Big data journey to the cloud   maz chaudhri 5.30.18Big data journey to the cloud   maz chaudhri 5.30.18
Big data journey to the cloud maz chaudhri 5.30.18Cloudera, Inc.
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Cathrine Wilhelmsen
 
SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic
 
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive ToolsForecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive ToolsDatabricks
 
Reducing Mean Time to Know
Reducing Mean Time to KnowReducing Mean Time to Know
Reducing Mean Time to KnowSqrrl
 
SugarCreek Digital Manufacturing v1.2
SugarCreek Digital Manufacturing v1.2SugarCreek Digital Manufacturing v1.2
SugarCreek Digital Manufacturing v1.2ramahade
 
Tcod a framework for the total cost of big data - december 6 2013 - winte...
Tcod   a framework for the total cost of big data  - december 6 2013  - winte...Tcod   a framework for the total cost of big data  - december 6 2013  - winte...
Tcod a framework for the total cost of big data - december 6 2013 - winte...Richard Winter
 
Delivering Services Powered by Operational Data - Connected Services
Delivering Services Powered by Operational Data -  Connected ServicesDelivering Services Powered by Operational Data -  Connected Services
Delivering Services Powered by Operational Data - Connected ServicesOSIsoft, LLC
 
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%Cloudera, Inc.
 
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...Tyler Wishnoff
 
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsApache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsTyler Wishnoff
 
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data Management
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data ManagementCloudera Federal Forum 2014: Hadoop's Impact on the Future of Data Management
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data ManagementCloudera, Inc.
 
10 Good Reasons - NetApp OnCommand Insight
10 Good Reasons - NetApp OnCommand Insight 10 Good Reasons - NetApp OnCommand Insight
10 Good Reasons - NetApp OnCommand Insight NetAppUK
 
AWSome Data Visibility with Information Map
AWSome Data Visibility with Information MapAWSome Data Visibility with Information Map
AWSome Data Visibility with Information MapVeritas Technologies LLC
 
Data at the corner of SAP and AWS
Data at the corner of SAP and AWSData at the corner of SAP and AWS
Data at the corner of SAP and AWSOcean9, Inc.
 
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented EngineKyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented EngineSamanthaBerlant
 

What's hot (20)

From Data to Action with TV 2
From Data to Action with TV 2From Data to Action with TV 2
From Data to Action with TV 2
 
Lets Talk Google BigQuery
Lets Talk Google BigQueryLets Talk Google BigQuery
Lets Talk Google BigQuery
 
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
 
Consolidate your data marts for fast, flexible analytics 5.24.18
Consolidate your data marts for fast, flexible analytics 5.24.18Consolidate your data marts for fast, flexible analytics 5.24.18
Consolidate your data marts for fast, flexible analytics 5.24.18
 
Big data journey to the cloud maz chaudhri 5.30.18
Big data journey to the cloud   maz chaudhri 5.30.18Big data journey to the cloud   maz chaudhri 5.30.18
Big data journey to the cloud maz chaudhri 5.30.18
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
 
SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018
 
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive ToolsForecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
 
Reducing Mean Time to Know
Reducing Mean Time to KnowReducing Mean Time to Know
Reducing Mean Time to Know
 
SugarCreek Digital Manufacturing v1.2
SugarCreek Digital Manufacturing v1.2SugarCreek Digital Manufacturing v1.2
SugarCreek Digital Manufacturing v1.2
 
Tcod a framework for the total cost of big data - december 6 2013 - winte...
Tcod   a framework for the total cost of big data  - december 6 2013  - winte...Tcod   a framework for the total cost of big data  - december 6 2013  - winte...
Tcod a framework for the total cost of big data - december 6 2013 - winte...
 
Delivering Services Powered by Operational Data - Connected Services
Delivering Services Powered by Operational Data -  Connected ServicesDelivering Services Powered by Operational Data -  Connected Services
Delivering Services Powered by Operational Data - Connected Services
 
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%
Kelley Blue Book Uses Big Data to Increase User Engagement Over 100%
 
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
 
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsApache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
 
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data Management
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data ManagementCloudera Federal Forum 2014: Hadoop's Impact on the Future of Data Management
Cloudera Federal Forum 2014: Hadoop's Impact on the Future of Data Management
 
10 Good Reasons - NetApp OnCommand Insight
10 Good Reasons - NetApp OnCommand Insight 10 Good Reasons - NetApp OnCommand Insight
10 Good Reasons - NetApp OnCommand Insight
 
AWSome Data Visibility with Information Map
AWSome Data Visibility with Information MapAWSome Data Visibility with Information Map
AWSome Data Visibility with Information Map
 
Data at the corner of SAP and AWS
Data at the corner of SAP and AWSData at the corner of SAP and AWS
Data at the corner of SAP and AWS
 
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented EngineKyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
 

Similar to A real life project using Cassandra at a large Swiss Telco operator

Da Vinci - A scaleable architecture for neural network computing (updated v4)
Da Vinci - A scaleable architecture for neural network computing (updated v4)Da Vinci - A scaleable architecture for neural network computing (updated v4)
Da Vinci - A scaleable architecture for neural network computing (updated v4)Heiko Joerg Schick
 
Service Provider Opportunities in Virtualized Storage and Software Defined St...
Service Provider Opportunities in Virtualized Storage and Software Defined St...Service Provider Opportunities in Virtualized Storage and Software Defined St...
Service Provider Opportunities in Virtualized Storage and Software Defined St...Virtuozzo
 
Is it sensible to use Data Vault at all? Conclusions from a project.
Is it sensible to use Data Vault at all? Conclusions from a project.Is it sensible to use Data Vault at all? Conclusions from a project.
Is it sensible to use Data Vault at all? Conclusions from a project.Capgemini
 
Cisco Impresa40 - Trends, Vision, Solutions
Cisco Impresa40 - Trends, Vision, SolutionsCisco Impresa40 - Trends, Vision, Solutions
Cisco Impresa40 - Trends, Vision, SolutionsMatteo Masi
 
Presentazione IBM System Storage - Evento Torino 19 novembre 2013
Presentazione IBM System Storage - Evento Torino 19 novembre 2013Presentazione IBM System Storage - Evento Torino 19 novembre 2013
Presentazione IBM System Storage - Evento Torino 19 novembre 2013PRAGMA PROGETTI
 
IBM System Storage® : la famiglia si allarga…ultimi annunci
IBM System Storage® : la famiglia si allarga…ultimi annunciIBM System Storage® : la famiglia si allarga…ultimi annunci
IBM System Storage® : la famiglia si allarga…ultimi annunciS.info Srl
 
exoscale at the CloudStack User Group London - June 26th 2014
exoscale at the CloudStack User Group London - June 26th 2014exoscale at the CloudStack User Group London - June 26th 2014
exoscale at the CloudStack User Group London - June 26th 2014Antoine COETSIER
 
AWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data AnalyticsAWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data AnalyticsAmazon Web Services
 
Minikube – get Connections in the smalles possible setup
Minikube – get Connections in the smalles possible setupMinikube – get Connections in the smalles possible setup
Minikube – get Connections in the smalles possible setupMartin Schmidt
 
Augmensys Overview
Augmensys OverviewAugmensys Overview
Augmensys OverviewAugmensys
 
Ceph Day Amsterdam 2015 - Building your own disaster? The safe way to make C...
Ceph Day Amsterdam 2015 - Building your own disaster?  The safe way to make C...Ceph Day Amsterdam 2015 - Building your own disaster?  The safe way to make C...
Ceph Day Amsterdam 2015 - Building your own disaster? The safe way to make C...Ceph Community
 
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation SystemsObeo
 
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frogWebinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frogQualiQuali
 
Oetker Daten und Informationsverarbeitung s
Oetker Daten und Informationsverarbeitung sOetker Daten und Informationsverarbeitung s
Oetker Daten und Informationsverarbeitung sCisco Case Studies
 
Industrie 4.0 Reference Architecture and Plant Service Bus
Industrie 4.0 Reference Architecture and Plant Service Bus Industrie 4.0 Reference Architecture and Plant Service Bus
Industrie 4.0 Reference Architecture and Plant Service Bus Peter Schleinitz
 
IBM i and Your IT Strategy - Trevor Perry
IBM i and Your IT Strategy - Trevor PerryIBM i and Your IT Strategy - Trevor Perry
IBM i and Your IT Strategy - Trevor PerryFresche Solutions
 

Similar to A real life project using Cassandra at a large Swiss Telco operator (20)

Da Vinci - A scaleable architecture for neural network computing (updated v4)
Da Vinci - A scaleable architecture for neural network computing (updated v4)Da Vinci - A scaleable architecture for neural network computing (updated v4)
Da Vinci - A scaleable architecture for neural network computing (updated v4)
 
Service Provider Opportunities in Virtualized Storage and Software Defined St...
Service Provider Opportunities in Virtualized Storage and Software Defined St...Service Provider Opportunities in Virtualized Storage and Software Defined St...
Service Provider Opportunities in Virtualized Storage and Software Defined St...
 
Is it sensible to use Data Vault at all? Conclusions from a project.
Is it sensible to use Data Vault at all? Conclusions from a project.Is it sensible to use Data Vault at all? Conclusions from a project.
Is it sensible to use Data Vault at all? Conclusions from a project.
 
Krones AG case study
Krones AG case studyKrones AG case study
Krones AG case study
 
Cisco Impresa40 - Trends, Vision, Solutions
Cisco Impresa40 - Trends, Vision, SolutionsCisco Impresa40 - Trends, Vision, Solutions
Cisco Impresa40 - Trends, Vision, Solutions
 
Presentazione IBM System Storage - Evento Torino 19 novembre 2013
Presentazione IBM System Storage - Evento Torino 19 novembre 2013Presentazione IBM System Storage - Evento Torino 19 novembre 2013
Presentazione IBM System Storage - Evento Torino 19 novembre 2013
 
IBM System Storage® : la famiglia si allarga…ultimi annunci
IBM System Storage® : la famiglia si allarga…ultimi annunciIBM System Storage® : la famiglia si allarga…ultimi annunci
IBM System Storage® : la famiglia si allarga…ultimi annunci
 
exoscale at the CloudStack User Group London - June 26th 2014
exoscale at the CloudStack User Group London - June 26th 2014exoscale at the CloudStack User Group London - June 26th 2014
exoscale at the CloudStack User Group London - June 26th 2014
 
AWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data AnalyticsAWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data Analytics
 
Minikube – get Connections in the smalles possible setup
Minikube – get Connections in the smalles possible setupMinikube – get Connections in the smalles possible setup
Minikube – get Connections in the smalles possible setup
 
Augmensys Overview
Augmensys OverviewAugmensys Overview
Augmensys Overview
 
Saipem
Saipem Saipem
Saipem
 
Jason Clark, Head of Property Management - UBS
Jason Clark, Head of Property Management - UBSJason Clark, Head of Property Management - UBS
Jason Clark, Head of Property Management - UBS
 
Ceph Day Amsterdam 2015 - Building your own disaster? The safe way to make C...
Ceph Day Amsterdam 2015 - Building your own disaster?  The safe way to make C...Ceph Day Amsterdam 2015 - Building your own disaster?  The safe way to make C...
Ceph Day Amsterdam 2015 - Building your own disaster? The safe way to make C...
 
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems
#SiriusCon - Constraint Graph: Graphical Modeling of Physical Equation Systems
 
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frogWebinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
 
Big Data Analytics, Dave Shuttleworth - 22-9-15
Big Data Analytics, Dave Shuttleworth - 22-9-15Big Data Analytics, Dave Shuttleworth - 22-9-15
Big Data Analytics, Dave Shuttleworth - 22-9-15
 
Oetker Daten und Informationsverarbeitung s
Oetker Daten und Informationsverarbeitung sOetker Daten und Informationsverarbeitung s
Oetker Daten und Informationsverarbeitung s
 
Industrie 4.0 Reference Architecture and Plant Service Bus
Industrie 4.0 Reference Architecture and Plant Service Bus Industrie 4.0 Reference Architecture and Plant Service Bus
Industrie 4.0 Reference Architecture and Plant Service Bus
 
IBM i and Your IT Strategy - Trevor Perry
IBM i and Your IT Strategy - Trevor PerryIBM i and Your IT Strategy - Trevor Perry
IBM i and Your IT Strategy - Trevor Perry
 

More from Swiss Big Data User Group

Building a Hadoop Data Warehouse with Impala
Building a Hadoop Data Warehouse with ImpalaBuilding a Hadoop Data Warehouse with Impala
Building a Hadoop Data Warehouse with ImpalaSwiss Big Data User Group
 
Closing The Loop for Evaluating Big Data Analysis
Closing The Loop for Evaluating Big Data AnalysisClosing The Loop for Evaluating Big Data Analysis
Closing The Loop for Evaluating Big Data AnalysisSwiss Big Data User Group
 
Big Data and Data Science for traditional Swiss companies
Big Data and Data Science for traditional Swiss companiesBig Data and Data Science for traditional Swiss companies
Big Data and Data Science for traditional Swiss companiesSwiss Big Data User Group
 
Design Patterns for Large-Scale Real-Time Learning
Design Patterns for Large-Scale Real-Time LearningDesign Patterns for Large-Scale Real-Time Learning
Design Patterns for Large-Scale Real-Time LearningSwiss Big Data User Group
 
Unleash the power of Big Data in your existing Data Warehouse
Unleash the power of Big Data in your existing Data WarehouseUnleash the power of Big Data in your existing Data Warehouse
Unleash the power of Big Data in your existing Data WarehouseSwiss Big Data User Group
 
Project "Babelfish" - A data warehouse to attack complexity
 Project "Babelfish" - A data warehouse to attack complexity Project "Babelfish" - A data warehouse to attack complexity
Project "Babelfish" - A data warehouse to attack complexitySwiss Big Data User Group
 
Brainserve Datacenter: the High-Density Choice
Brainserve Datacenter: the High-Density ChoiceBrainserve Datacenter: the High-Density Choice
Brainserve Datacenter: the High-Density ChoiceSwiss Big Data User Group
 
Urturn on AWS: scaling infra, cost and time to maket
Urturn on AWS: scaling infra, cost and time to maketUrturn on AWS: scaling infra, cost and time to maket
Urturn on AWS: scaling infra, cost and time to maketSwiss Big Data User Group
 
The World Wide Distributed Computing Architecture of the LHC Datagrid
The World Wide Distributed Computing Architecture of the LHC DatagridThe World Wide Distributed Computing Architecture of the LHC Datagrid
The World Wide Distributed Computing Architecture of the LHC DatagridSwiss Big Data User Group
 
New opportunities for connected data : Neo4j the graph database
New opportunities for connected data : Neo4j the graph databaseNew opportunities for connected data : Neo4j the graph database
New opportunities for connected data : Neo4j the graph databaseSwiss Big Data User Group
 
Technology Outlook - The new Era of computing
Technology Outlook - The new Era of computingTechnology Outlook - The new Era of computing
Technology Outlook - The new Era of computingSwiss Big Data User Group
 

More from Swiss Big Data User Group (20)

Data Analytics – B2B vs. B2C
Data Analytics – B2B vs. B2CData Analytics – B2B vs. B2C
Data Analytics – B2B vs. B2C
 
SQL on Hadoop
SQL on HadoopSQL on Hadoop
SQL on Hadoop
 
Building a Hadoop Data Warehouse with Impala
Building a Hadoop Data Warehouse with ImpalaBuilding a Hadoop Data Warehouse with Impala
Building a Hadoop Data Warehouse with Impala
 
Closing The Loop for Evaluating Big Data Analysis
Closing The Loop for Evaluating Big Data AnalysisClosing The Loop for Evaluating Big Data Analysis
Closing The Loop for Evaluating Big Data Analysis
 
Big Data and Data Science for traditional Swiss companies
Big Data and Data Science for traditional Swiss companiesBig Data and Data Science for traditional Swiss companies
Big Data and Data Science for traditional Swiss companies
 
Design Patterns for Large-Scale Real-Time Learning
Design Patterns for Large-Scale Real-Time LearningDesign Patterns for Large-Scale Real-Time Learning
Design Patterns for Large-Scale Real-Time Learning
 
Educating Data Scientists of the Future
Educating Data Scientists of the FutureEducating Data Scientists of the Future
Educating Data Scientists of the Future
 
Unleash the power of Big Data in your existing Data Warehouse
Unleash the power of Big Data in your existing Data WarehouseUnleash the power of Big Data in your existing Data Warehouse
Unleash the power of Big Data in your existing Data Warehouse
 
Big data for Telco: opportunity or threat?
Big data for Telco: opportunity or threat?Big data for Telco: opportunity or threat?
Big data for Telco: opportunity or threat?
 
Project "Babelfish" - A data warehouse to attack complexity
 Project "Babelfish" - A data warehouse to attack complexity Project "Babelfish" - A data warehouse to attack complexity
Project "Babelfish" - A data warehouse to attack complexity
 
Brainserve Datacenter: the High-Density Choice
Brainserve Datacenter: the High-Density ChoiceBrainserve Datacenter: the High-Density Choice
Brainserve Datacenter: the High-Density Choice
 
Urturn on AWS: scaling infra, cost and time to maket
Urturn on AWS: scaling infra, cost and time to maketUrturn on AWS: scaling infra, cost and time to maket
Urturn on AWS: scaling infra, cost and time to maket
 
The World Wide Distributed Computing Architecture of the LHC Datagrid
The World Wide Distributed Computing Architecture of the LHC DatagridThe World Wide Distributed Computing Architecture of the LHC Datagrid
The World Wide Distributed Computing Architecture of the LHC Datagrid
 
New opportunities for connected data : Neo4j the graph database
New opportunities for connected data : Neo4j the graph databaseNew opportunities for connected data : Neo4j the graph database
New opportunities for connected data : Neo4j the graph database
 
Technology Outlook - The new Era of computing
Technology Outlook - The new Era of computingTechnology Outlook - The new Era of computing
Technology Outlook - The new Era of computing
 
In-Store Analysis with Hadoop
In-Store Analysis with HadoopIn-Store Analysis with Hadoop
In-Store Analysis with Hadoop
 
Big Data Visualization With ParaView
Big Data Visualization With ParaViewBig Data Visualization With ParaView
Big Data Visualization With ParaView
 
Introduction to Apache Drill
Introduction to Apache DrillIntroduction to Apache Drill
Introduction to Apache Drill
 
Oracle's BigData solutions
Oracle's BigData solutionsOracle's BigData solutions
Oracle's BigData solutions
 
Introducing Splunk – The Big Data Engine
Introducing Splunk – The Big Data EngineIntroducing Splunk – The Big Data Engine
Introducing Splunk – The Big Data Engine
 

Recently uploaded

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 

Recently uploaded (20)

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 

A real life project using Cassandra at a large Swiss Telco operator

  • 1. B r i d g i n g t h e G a p SwissBigDataUserGroup 23.09.2014 / André Vocat Cassandra @ Swiss Telco Operator
  • 2. •Introduction •System environment •System overview •Non functional requirements •System load •System architecture •Cassandra cluster 2 Intersys AG | Zuchwil | www.intersys.ch Agenda
  • 3. •software architect, developer and project lead working for Intersys AG in Zuchwil. •holding an engineering degree in electrical engineering •more than 20 years experience developing software and designing software architectures •specifically for the telecommunications market. •lead architect of the system presented today –predecessor system (started in late 1999) –first version of this critical platform (started in 2002) –complete renewal (started in 2012) –production use (started mid 2013) 3 Intersys AG | Zuchwil | www.intersys.ch Bio
  • 4. Intersys consists of Interfaces and Systems Interfaces Exchange of data between Systems of different Origin Systems Development of Software Systems and system components 4 Intersys AG | Zuchwil | www.intersys.ch Interfaces and Systems Main offerings - Software Development (specify, implement, test, run, support) - Test Management (independent software quality certification)
  • 5. Fast Facts Founded 1999 Number of Employees 27 (most of them higher education in IT or electrical engineering) Locations Zuchwil (HQ), Zürich (Branch) Common Stock CHF 250’000.-, owned by employees DataStax Partner First official DataStax Integration Partner in Switzerland ISO 9001:2008 certified We educate apprentices 5 Intersys AG | Zuchwil | www.intersys.ch
  • 6. 6 Intersys AG | Zuchwil | www.intersys.ch System environment
  • 7. 7 Intersys AG | Zuchwil | www.intersys.ch System overview
  • 8. •availability •high performance •flexibility •scalability •upgradeability (minimal / «zero» downtime) •surrounding systems must not affect others 8 Intersys AG | Zuchwil | www.intersys.ch Non functional requirements
  • 9. System load per hour 9 Intersys AG | Zuchwil | www.intersys.ch 0 2'000'000 4'000'000 6'000'000 8'000'000 00.00 h 01.00 h 02.00 h 03.00 h 04.00 h 05.00 h 06.00 h 07.00 h 08.00 h 09.00 h 10.00 h 11.00 h 12.00 h 13.00 h 14.00 h 15.00 h 16.00 h 17.00 h 18.00 h 19.00 h 20.00 h 21.00 h 22.00 h 23.00 h request count per hour 0 50'000 100'000 150'000 200'000 250'000 00.00 h 01.00 h 02.00 h 03.00 h 04.00 h 05.00 h 06.00 h 07.00 h 08.00 h 09.00 h 10.00 h 11.00 h 12.00 h 13.00 h 14.00 h 15.00 h 16.00 h 17.00 h 18.00 h 19.00 h 20.00 h 21.00 h 22.00 h 23.00 h update count per hour
  • 10. System load per day 10 Intersys AG | Zuchwil | www.intersys.ch 0 500'000 1'000'000 1'500'000 SO MO DI MI DO FR SA update count per day 0 20'000'000 40'000'000 60'000'000 80'000'000 100'000'000 120'000'000 140'000'000 SO MO DI MI DO FR SA request count per day
  • 11. System load per month 11 Intersys AG | Zuchwil | www.intersys.ch 0 500'000'000 1'000'000'000 1'500'000'000 2'000'000'000 2'500'000'000 nov dez jan feb mar apr mai jun requests count per month 0 5'000'000 10'000'000 15'000'000 20'000'000 25'000'000 30'000'000 mar apr mai jun jul aug update count per month
  • 12. 12 Intersys AG | Zuchwil | www.intersys.ch System architecture
  • 13. 13 Intersys AG | Zuchwil | www.intersys.ch Cassandra Cluster internals
  • 14. •Lessons learned –Indexed queries memory usage –write operation only as batch –regularly repair data –range query only for small data amount –do not delete column and reinsert –regularly upgrade cassandra version •Open learnings –Optimize after version upgrade –still more to come… 14 Intersys AG | Zuchwil | www.intersys.ch Lessons learned / Open learnings
  • 15. •André Vocat andre.vocat@intersys.ch +41 32 625 76 84 +41 79 233 33 94 •Reto Gantenbein reto.gantenbein@intersys.ch +41 32 625 76 93 +41 79 701 47 35 15 Intersys AG | Zuchwil | www.intersys.ch Koordinaten