SlideShare a Scribd company logo
1 of 17
Download to read offline
Big Data Public Private Forum




BIG DATA
TENDENCIAS TECNOLÓGICAS
First workshop for the construction of a Roadmap for Big
Data in Europe
16/04/2013
Tomás Pariente –      Atos Research and Innovation /
                      BIG Project
WE KNOW WHAT BIG DATA IS,                                                                                       BIG
                                                                                           Big Data Public Private Forum

RIGHT?




                                  Hi, big
                                   Hi, big
                                  brother
                                   brother
                                                                              BIG
                      Small
                       DATA
                                                                         DATA
Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                    2     BIG 318062
BIG DATA IS NOT ONLY ABOUT SIZE:                                                                                  BIG
DATA DIVERSITY MATTERS
                                                                                             Big Data Public Private Forum




              3 Vs: Volume, Velocity, Variety
                                               BIG SIZE

                                               UNSTRUCTURED

                                               MULTIMEDIA



                           +                                            =
                                               REAL TIME


                                                                                           BIG
                                               “EXHAUST”

                                               LINKED/SHARED
         Traditional


                                                                                           DATA
         Structured
            Data                               SOCIAL

                                               OPEN


Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                      3     BIG 318062
THE DATA DELUGE                                                                                                   BIG
                                                                                              Big Data Public Private Forum

 EXPONENTIAL DATA GROWTH
 IBM: “Every day, we create 2.5
 quintillion bytes of data- so much that                           “Data is the new gold”1
 90% of the data in the world today has
 been created in the last two years alone.”       Data
                                                  mgmt
                                                                               Big Data definition
                                                                   When dealing with data becomes The problem

          Data




1 Neelie Kroes Vice-President of the European Commission responsible for the Digital Agenda
Data
   Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                    4     BIG 318062
BIG DATA                                                                                                        BIG
TECHNOLOGIES LANDSCAPE
                                                                                           Big Data Public Private Forum




                                       Batch processing
                                        Batch processing




                                 Real-time processing
                                  Real-time processing




Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                    5     BIG 318062
BIG DATA STORAGE                                                                                                 BIG

NOSQL AND BEYOND
                                                                                            Big Data Public Private Forum




• Distributed File Systems:
  – Hadoop File System (HDFS).
  – Capability to store large amount of unstructured data in a reliable way on
    commodity hardware.
• NoSQL Databases:
  – Use other data models than the relational model known from the SQL world
  – Do not necessarily adhere to transactional properties of atomicity, consistency
    and isolation and durability (ACID).
• NewSQL Databases: Shorthand for new scalable/high-performance SQL DBs.
  – SQL as the primary mechanism for application interaction
  – ACID support for transactions
  – A non locking concurrency control mechanism
  – An architecture providing much higher per-node performances
  – A scale out, shared-nothing architecture, capable of running on a large number
    of nodes without suffering bottlenecks.
  – The expectation is that NewSQL systems are about 50 times faster than
    traditional OLTP RDBMS.


 Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                    6     BIG 318062
BIG DATA                                                                                                                   BIG

THE NOSQL WORLD
                                                                                                      Big Data Public Private Forum




                                   Schema-less
                                   Unstructured
                                   Apache HBase


                                                       Row – Column - Timestamp
                                                       Value = String
                                                       Several columns
                                                       Voldemort

                                                                             Documents
                                                                             Stored in JSON or XML
                                                                             Accessible by Key or content
                                                                             CouchDB, MongoDB

                                                                                                Graphs structures
                                                                                                Highly associative, social networks
                                                                                                Accessible by Key or content
                                                                                                Neo4j




Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                               7     BIG 318062
BIG DATA                                                   BIG
 APACHE BIG DATA TOOLS
                                       Big Data Public Private Forum




                                 Courtesy of Michael Hausenblass

Kick Off meeting 10-11/09/2012                          8     BIG 318062
BIG DATA IS ABOUT CHOOSING THE                                                                                                 BIG
                                                                                                             Big Data Public Private Forum


   RIGHT THING
                                                                                                Distributed massive storage
                                        Map-Reduce (Hadoop)                                     Hadoop File System (HDFS)
                                        Analytical platform                                     NoSQL (Hbase, Cassandra, CouchDB…)
                                        Intelligent parallelization
                                        Reliability                                               Long-lasting analytical algorithms
                                                                                                  Iterative process / might take days
Un-
                                                                                                  Huge volume
structured
                                                                                                  Data curation
                                   Batch / Historical
                                                               Batch processing
                                                                Batch processing

  Social
                  3 V’s
                                                                                                                              Query
                                                                                                                               Query
                   Acquisition
                    Acquisition
 Linked                                                                                                                       facade
                                                                                                                               facade

                Acquisition software                                                                              Performant Queries
                Apache Kafka                                                                                      Cloudera Impala
Corporate                                                                                                         Apache HIVE
                Messaging              Real-time
                Publish-subscribe                           Real-time processing
                                                             Real-time processing                                 Apache Solr (Lucene)
                Hundred of thousands per second                                                                   RDBMS (SQL)
 Events                                                   Processing platforms                                    …
                Apache Flume                              Storm/Apache S4                         Running pipelines
                For events or logs                        Stream processing                       Fast algorithms
                Event pushing                             Intelligent parallelization             High throughput
  Logs                                                    Robust and flexible topologies          No storage or complex storage
   …


     Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                                 9     BIG 318062
TRENDS                                                                                                                       BIG
                                                                                                        Big Data Public Private Forum




Big Data Tendencies
                                                                        New efficient and scalable algorithms
                                                                        Multidisciplinary teams (data scientists)
   Big Data Analytics                                                   Understand technology platforms         Aggregation and
                                                                        correlation algorithms
                                                                        Big Data vs small eyes
   New Visualization and queries techniques                             Take “time” into account
                                                                        Faster queries

                                                                        Stream processing
   Going real-time                                                      Real-time queries
                                                                        Real-time visualization paradigms

                                                                        Performance and scalability
   Data Management                                                      Storage selection and costs
                                                                        Cloud vs. data centers

                                                                        Data selection
   Data curation                                                        Data value and garbage
                                                                        Trust, provenance
                                                                       New business models for selling data
   New business models                                                 Dealing with privacy, ownership
                                                                       Fostering reuse of data
                                                                       “Do it before the competitors do”

 Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                              10      BIG 318062
BUT BE AWARE OF THE RISKS                                                                                             BIG
                                                                                                 Big Data Public Private Forum




          Too many solutions:                                                              Get hold of data
                                                            Policies:
          Blank page blockage                                                              Break Data silos
                                                            Security,
                                                            Privacy, IPR                   Data Quality


             Investment                              Curation
                                                     Trust                             Few Professionals
             Old apps, Storage                                                         Data scientists
             CPDs vs Cloud                           Provenance




Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                        11      BIG 318062
BIG
BIG DATA AND THE PUBLIC SECTOR                                                             Big Data Public Private Forum



FINDINGS FROM THE TECHAMERICA SURVEY
                         Big Data is Here to Stay: 82% Say Real-Time Big Data is the Way of the
                  1      Future

                  2      Real-Time Big Data Could Save Government 10% or More Annually

                  3      Real-Time Big Data Could Save Significant Number of Lives

                  4      Big Data is Helping Improve the Quality of Citizens’ Lives

                  5      State IT Officials Agree Big Data Can Improve Social and Welfare Services

                  6      Big Data Advances in Medicine, Public Safety Seen as Most Important

                  7      Privacy and Policy Concerns Remain a Barrier to Utilizing Big Data

                         Public Sector IT Officials Frustrated With Multiple Data Formats,
                  8      Leadership Changes

                  9      Many Public Sector IT Officials Say Database queries Take Too Much Time

                         Nearly All Government IT Officials Would Opt For Real Time Access to Data
                10       Over Backward Looking Queries


Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                  12      BIG 318062
Big Data in the Public Sector                                                                                   BIG
                                                                                           Big Data Public Private Forum




Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013                  13      BIG 318062
BIG
 PROJECT BIG - SECTOR FORUMS AND TECHNICAL   Big Data Public Private Forum




 WORKING GROUPS




Kick Off meeting 10-11/09/2012                              14      BIG 318062
PROJECT BIG                                                                          BIG
 SECTORS’ ROADMAP
                                                                 Big Data Public Private Forum




                Identification       ▶requirements and objectives from all
                  of Sector’s         Sectors (industry driven working
                  requisites          groups)


                 Applicability of    ▶Introduce technologies and trends to
                Big Data technical    the stakeholders to better understand
                 white papers in      Big Data technologies and its
                   each Sector
                                      capabilities

                                     ▶Sectorial roadmap (elaborate a
                Elaboration of        roadmap per sector).
                    Sector           ▶Contributions towards integrated
                  Roadmap             roadmap (cross-sectorial)


Kick Off meeting 10-11/09/2012                                                  15      BIG 318062
PROJECT BIG - TIMELINE OF THE MOST                                                                      BIG
 IMPORTANT DELIVERABLES
                                                                                    Big Data Public Private Forum




 04/2013                                                                                                       04/2013
D2.2.1-1º version of Technical      D2.2.1             D2.3.1            D2.3.1-1º version of Sector’s requisites
white papers

                                                                                                               06/2013
                                              D4.2.1                   D4.2.1-1ºversion of IPR, Standardization
                                                                                            •recommendations
 10/2013
 D4.3.1-First draft of the
 Big Data Public-Private Forum      D4.3.1             D2.4.1                                                  09/2013
                                                                                                D2.4.1 1ª version of
 01/2014                                                                                         Sector´s Roadmap
 D2.2.2-Final version of                     D2.2.2
 Technical white paper
                                                                                                               04/2014
                                                                                            D2.3.2-Final version of
 04/2014                          D4.2.2               D2.3.2
                                                                                                Sectors requisites
D4.2.2-Final version of IPR,
Standardization recommendations
                                                                                                               10/2014
                                                                D2.5              D2.5-Cross-sectorial roadmap
                                                                                                 consolidation




Kick Off meeting 10-11/09/2012                                                                     16      BIG 318062
Big Data Public Private Forum




THANKS
Tomás Pariente Lobo
Atos Research & Innovation
Atos Spain
tomas.parientelob@atosresearch.eu

More Related Content

What's hot

How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...
How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...
How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...Vladimir Bacvanski, PhD
 
Big Data Fundamentals
Big Data FundamentalsBig Data Fundamentals
Big Data Fundamentalsrjain51
 
Open Data - Where can it take us?
Open Data - Where can it take us? Open Data - Where can it take us?
Open Data - Where can it take us? Derilinx
 
Databases week 8
Databases week 8Databases week 8
Databases week 8Madeinmars
 
Big data data lake and beyond
Big data data lake and beyond Big data data lake and beyond
Big data data lake and beyond Rajesh Kumar
 
Big data - what, why, where, when and how
Big data - what, why, where, when and howBig data - what, why, where, when and how
Big data - what, why, where, when and howbobosenthil
 
Hadoop explained [e book]
Hadoop explained [e book]Hadoop explained [e book]
Hadoop explained [e book]Supratim Ray
 
D.3.1: State of the Art - Linked Data and Digital Preservation
D.3.1: State of the Art - Linked Data and Digital PreservationD.3.1: State of the Art - Linked Data and Digital Preservation
D.3.1: State of the Art - Linked Data and Digital PreservationPRELIDA Project
 
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
Guest Lecture: Introduction to Big Data at Indian Institute of TechnologyGuest Lecture: Introduction to Big Data at Indian Institute of Technology
Guest Lecture: Introduction to Big Data at Indian Institute of TechnologyNishant Gandhi
 
Linked Data Approach for Integration of Human Health & Environmental Data
Linked Data Approach for Integration of Human Health & Environmental DataLinked Data Approach for Integration of Human Health & Environmental Data
Linked Data Approach for Integration of Human Health & Environmental Data3 Round Stones
 

What's hot (12)

How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...
How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...
How to Crunch Petabytes with Hadoop and Big Data using InfoSphere BigInsights...
 
Big Data Fundamentals
Big Data FundamentalsBig Data Fundamentals
Big Data Fundamentals
 
Open Data - Where can it take us?
Open Data - Where can it take us? Open Data - Where can it take us?
Open Data - Where can it take us?
 
Databases week 8
Databases week 8Databases week 8
Databases week 8
 
Big data data lake and beyond
Big data data lake and beyond Big data data lake and beyond
Big data data lake and beyond
 
Big data - what, why, where, when and how
Big data - what, why, where, when and howBig data - what, why, where, when and how
Big data - what, why, where, when and how
 
Hadoop explained [e book]
Hadoop explained [e book]Hadoop explained [e book]
Hadoop explained [e book]
 
D.3.1: State of the Art - Linked Data and Digital Preservation
D.3.1: State of the Art - Linked Data and Digital PreservationD.3.1: State of the Art - Linked Data and Digital Preservation
D.3.1: State of the Art - Linked Data and Digital Preservation
 
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
Guest Lecture: Introduction to Big Data at Indian Institute of TechnologyGuest Lecture: Introduction to Big Data at Indian Institute of Technology
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
 
Intro dm
Intro dmIntro dm
Intro dm
 
Linked Data Approach for Integration of Human Health & Environmental Data
Linked Data Approach for Integration of Human Health & Environmental DataLinked Data Approach for Integration of Human Health & Environmental Data
Linked Data Approach for Integration of Human Health & Environmental Data
 
Big data-ppt
Big data-pptBig data-ppt
Big data-ppt
 

Viewers also liked

Questionnaire charts
Questionnaire chartsQuestionnaire charts
Questionnaire chartsvren88
 
Vacant Land For Sale In Missouri
Vacant Land For Sale In MissouriVacant Land For Sale In Missouri
Vacant Land For Sale In MissouriShanon Sandquist
 
Under graduate records
Under graduate recordsUnder graduate records
Under graduate recordsJohn Nuck
 
CATC Adv Cert Interior Design
CATC Adv Cert Interior DesignCATC Adv Cert Interior Design
CATC Adv Cert Interior DesignKym Camilleri
 
QR Job well done letter 2002
QR Job well done letter 2002QR Job well done letter 2002
QR Job well done letter 2002Kym Camilleri
 
Xornadas micolóxicas
Xornadas micolóxicasXornadas micolóxicas
Xornadas micolóxicasodesvanmarita
 
(Done) double page spread (layout)
(Done) double page spread (layout)(Done) double page spread (layout)
(Done) double page spread (layout)vren88
 
σημειώσεις συνάντηση 6
σημειώσεις συνάντηση 6σημειώσεις συνάντηση 6
σημειώσεις συνάντηση 6natasa08
 
Ops tel offering id lifecm_oct92015
Ops tel offering id lifecm_oct92015Ops tel offering id lifecm_oct92015
Ops tel offering id lifecm_oct92015Tony DeGaetano
 
양방배팅 ox600 ˛ CΘM 양방배팅
양방배팅 ox600 ˛ CΘM 양방배팅양방배팅 ox600 ˛ CΘM 양방배팅
양방배팅 ox600 ˛ CΘM 양방배팅krthrghgfh
 
Tipos de impresoras_gaby_moya
Tipos de impresoras_gaby_moyaTipos de impresoras_gaby_moya
Tipos de impresoras_gaby_moyagabymoyaaguirre
 
YP Local Ads Value Story
YP Local Ads Value StoryYP Local Ads Value Story
YP Local Ads Value StoryKev Carter
 

Viewers also liked (15)

Bill resume
Bill resumeBill resume
Bill resume
 
Questionnaire charts
Questionnaire chartsQuestionnaire charts
Questionnaire charts
 
OHSAS 18001
OHSAS 18001OHSAS 18001
OHSAS 18001
 
Vacant Land For Sale In Missouri
Vacant Land For Sale In MissouriVacant Land For Sale In Missouri
Vacant Land For Sale In Missouri
 
Under graduate records
Under graduate recordsUnder graduate records
Under graduate records
 
CATC Adv Cert Interior Design
CATC Adv Cert Interior DesignCATC Adv Cert Interior Design
CATC Adv Cert Interior Design
 
QR Job well done letter 2002
QR Job well done letter 2002QR Job well done letter 2002
QR Job well done letter 2002
 
Xornadas micolóxicas
Xornadas micolóxicasXornadas micolóxicas
Xornadas micolóxicas
 
(Done) double page spread (layout)
(Done) double page spread (layout)(Done) double page spread (layout)
(Done) double page spread (layout)
 
Flyer maio
Flyer maioFlyer maio
Flyer maio
 
σημειώσεις συνάντηση 6
σημειώσεις συνάντηση 6σημειώσεις συνάντηση 6
σημειώσεις συνάντηση 6
 
Ops tel offering id lifecm_oct92015
Ops tel offering id lifecm_oct92015Ops tel offering id lifecm_oct92015
Ops tel offering id lifecm_oct92015
 
양방배팅 ox600 ˛ CΘM 양방배팅
양방배팅 ox600 ˛ CΘM 양방배팅양방배팅 ox600 ˛ CΘM 양방배팅
양방배팅 ox600 ˛ CΘM 양방배팅
 
Tipos de impresoras_gaby_moya
Tipos de impresoras_gaby_moyaTipos de impresoras_gaby_moya
Tipos de impresoras_gaby_moya
 
YP Local Ads Value Story
YP Local Ads Value StoryYP Local Ads Value Story
YP Local Ads Value Story
 

Similar to Big data-public-private-forum--2013 publioc-sector_meeting_spain_big_data_technological_trends

Introduction to Big Data & Big Data 1.0 System
Introduction to Big Data & Big Data 1.0 SystemIntroduction to Big Data & Big Data 1.0 System
Introduction to Big Data & Big Data 1.0 SystemPetr Novotný
 
big-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptxbig-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptxVaishnavGhadge1
 
Big Data Big Media the new paradigm of multimedia content management with Per...
Big Data Big Media the new paradigm of multimedia content management with Per...Big Data Big Media the new paradigm of multimedia content management with Per...
Big Data Big Media the new paradigm of multimedia content management with Per...ACTUONDA
 
Introduction to Cloud computing and Big Data-Hadoop
Introduction to Cloud computing and  Big Data-HadoopIntroduction to Cloud computing and  Big Data-Hadoop
Introduction to Cloud computing and Big Data-HadoopNagarjuna D.N
 
Introduction to Big Data An analogy between Sugar Cane & Big Data
Introduction to Big Data An analogy  between Sugar Cane & Big DataIntroduction to Big Data An analogy  between Sugar Cane & Big Data
Introduction to Big Data An analogy between Sugar Cane & Big DataJean-Marc Desvaux
 
Big Data Basic Concepts | Presented in 2014
Big Data Basic Concepts  | Presented in 2014Big Data Basic Concepts  | Presented in 2014
Big Data Basic Concepts | Presented in 2014Kenneth Igiri
 
Big data management
Big data managementBig data management
Big data managementzeba khanam
 
Hadoop @ Sara & BiG Grid
Hadoop @ Sara & BiG GridHadoop @ Sara & BiG Grid
Hadoop @ Sara & BiG GridEvert Lammerts
 
An Overview of BigData
An Overview of BigDataAn Overview of BigData
An Overview of BigDataValarmathi V
 
Introduction to big data
Introduction to big dataIntroduction to big data
Introduction to big dataSitaram Kotnis
 
Café da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with HadoopCafé da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with HadoopOCTO Technology
 
Big data présentation
Big data présentationBig data présentation
Big data présentationAbdo Bim
 

Similar to Big data-public-private-forum--2013 publioc-sector_meeting_spain_big_data_technological_trends (20)

Big data
Big dataBig data
Big data
 
Introduction to Big Data & Big Data 1.0 System
Introduction to Big Data & Big Data 1.0 SystemIntroduction to Big Data & Big Data 1.0 System
Introduction to Big Data & Big Data 1.0 System
 
Big Data ppt
Big Data pptBig Data ppt
Big Data ppt
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
big-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptxbig-data-8722-m8RQ3h1.pptx
big-data-8722-m8RQ3h1.pptx
 
Big Data Big Media the new paradigm of multimedia content management with Per...
Big Data Big Media the new paradigm of multimedia content management with Per...Big Data Big Media the new paradigm of multimedia content management with Per...
Big Data Big Media the new paradigm of multimedia content management with Per...
 
Introduction to Cloud computing and Big Data-Hadoop
Introduction to Cloud computing and  Big Data-HadoopIntroduction to Cloud computing and  Big Data-Hadoop
Introduction to Cloud computing and Big Data-Hadoop
 
Big Data: an introduction
Big Data: an introductionBig Data: an introduction
Big Data: an introduction
 
Introduction to Big Data An analogy between Sugar Cane & Big Data
Introduction to Big Data An analogy  between Sugar Cane & Big DataIntroduction to Big Data An analogy  between Sugar Cane & Big Data
Introduction to Big Data An analogy between Sugar Cane & Big Data
 
Big Data Basic Concepts | Presented in 2014
Big Data Basic Concepts  | Presented in 2014Big Data Basic Concepts  | Presented in 2014
Big Data Basic Concepts | Presented in 2014
 
Big data management
Big data managementBig data management
Big data management
 
Hadoop @ Sara & BiG Grid
Hadoop @ Sara & BiG GridHadoop @ Sara & BiG Grid
Hadoop @ Sara & BiG Grid
 
An Overview of BigData
An Overview of BigDataAn Overview of BigData
An Overview of BigData
 
Big data-ppt-
Big data-ppt-Big data-ppt-
Big data-ppt-
 
Introduction to big data
Introduction to big dataIntroduction to big data
Introduction to big data
 
BIG DATA
BIG DATABIG DATA
BIG DATA
 
Café da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with HadoopCafé da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
 
Big data présentation
Big data présentationBig data présentation
Big data présentation
 
Big data
Big dataBig data
Big data
 
1
11
1
 

Recently uploaded

COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 

Recently uploaded (20)

COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 

Big data-public-private-forum--2013 publioc-sector_meeting_spain_big_data_technological_trends

  • 1. Big Data Public Private Forum BIG DATA TENDENCIAS TECNOLÓGICAS First workshop for the construction of a Roadmap for Big Data in Europe 16/04/2013 Tomás Pariente – Atos Research and Innovation / BIG Project
  • 2. WE KNOW WHAT BIG DATA IS, BIG Big Data Public Private Forum RIGHT? Hi, big Hi, big brother brother BIG Small DATA DATA Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 2 BIG 318062
  • 3. BIG DATA IS NOT ONLY ABOUT SIZE: BIG DATA DIVERSITY MATTERS Big Data Public Private Forum 3 Vs: Volume, Velocity, Variety BIG SIZE UNSTRUCTURED MULTIMEDIA + = REAL TIME BIG “EXHAUST” LINKED/SHARED Traditional DATA Structured Data SOCIAL OPEN Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 3 BIG 318062
  • 4. THE DATA DELUGE BIG Big Data Public Private Forum EXPONENTIAL DATA GROWTH IBM: “Every day, we create 2.5 quintillion bytes of data- so much that “Data is the new gold”1 90% of the data in the world today has been created in the last two years alone.” Data mgmt Big Data definition When dealing with data becomes The problem Data 1 Neelie Kroes Vice-President of the European Commission responsible for the Digital Agenda Data Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 4 BIG 318062
  • 5. BIG DATA BIG TECHNOLOGIES LANDSCAPE Big Data Public Private Forum Batch processing Batch processing Real-time processing Real-time processing Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 5 BIG 318062
  • 6. BIG DATA STORAGE BIG NOSQL AND BEYOND Big Data Public Private Forum • Distributed File Systems: – Hadoop File System (HDFS). – Capability to store large amount of unstructured data in a reliable way on commodity hardware. • NoSQL Databases: – Use other data models than the relational model known from the SQL world – Do not necessarily adhere to transactional properties of atomicity, consistency and isolation and durability (ACID). • NewSQL Databases: Shorthand for new scalable/high-performance SQL DBs. – SQL as the primary mechanism for application interaction – ACID support for transactions – A non locking concurrency control mechanism – An architecture providing much higher per-node performances – A scale out, shared-nothing architecture, capable of running on a large number of nodes without suffering bottlenecks. – The expectation is that NewSQL systems are about 50 times faster than traditional OLTP RDBMS. Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 6 BIG 318062
  • 7. BIG DATA BIG THE NOSQL WORLD Big Data Public Private Forum Schema-less Unstructured Apache HBase Row – Column - Timestamp Value = String Several columns Voldemort Documents Stored in JSON or XML Accessible by Key or content CouchDB, MongoDB Graphs structures Highly associative, social networks Accessible by Key or content Neo4j Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 7 BIG 318062
  • 8. BIG DATA BIG APACHE BIG DATA TOOLS Big Data Public Private Forum Courtesy of Michael Hausenblass Kick Off meeting 10-11/09/2012 8 BIG 318062
  • 9. BIG DATA IS ABOUT CHOOSING THE BIG Big Data Public Private Forum RIGHT THING Distributed massive storage Map-Reduce (Hadoop) Hadoop File System (HDFS) Analytical platform NoSQL (Hbase, Cassandra, CouchDB…) Intelligent parallelization Reliability Long-lasting analytical algorithms Iterative process / might take days Un- Huge volume structured Data curation Batch / Historical Batch processing Batch processing Social 3 V’s Query Query Acquisition Acquisition Linked facade facade Acquisition software Performant Queries Apache Kafka Cloudera Impala Corporate Apache HIVE Messaging Real-time Publish-subscribe Real-time processing Real-time processing Apache Solr (Lucene) Hundred of thousands per second RDBMS (SQL) Events Processing platforms … Apache Flume Storm/Apache S4 Running pipelines For events or logs Stream processing Fast algorithms Event pushing Intelligent parallelization High throughput Logs Robust and flexible topologies No storage or complex storage … Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 9 BIG 318062
  • 10. TRENDS BIG Big Data Public Private Forum Big Data Tendencies New efficient and scalable algorithms Multidisciplinary teams (data scientists) Big Data Analytics Understand technology platforms Aggregation and correlation algorithms Big Data vs small eyes New Visualization and queries techniques Take “time” into account Faster queries Stream processing Going real-time Real-time queries Real-time visualization paradigms Performance and scalability Data Management Storage selection and costs Cloud vs. data centers Data selection Data curation Data value and garbage Trust, provenance New business models for selling data New business models Dealing with privacy, ownership Fostering reuse of data “Do it before the competitors do” Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 10 BIG 318062
  • 11. BUT BE AWARE OF THE RISKS BIG Big Data Public Private Forum Too many solutions: Get hold of data Policies: Blank page blockage Break Data silos Security, Privacy, IPR Data Quality Investment Curation Trust Few Professionals Old apps, Storage Data scientists CPDs vs Cloud Provenance Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 11 BIG 318062
  • 12. BIG BIG DATA AND THE PUBLIC SECTOR Big Data Public Private Forum FINDINGS FROM THE TECHAMERICA SURVEY Big Data is Here to Stay: 82% Say Real-Time Big Data is the Way of the 1 Future 2 Real-Time Big Data Could Save Government 10% or More Annually 3 Real-Time Big Data Could Save Significant Number of Lives 4 Big Data is Helping Improve the Quality of Citizens’ Lives 5 State IT Officials Agree Big Data Can Improve Social and Welfare Services 6 Big Data Advances in Medicine, Public Safety Seen as Most Important 7 Privacy and Policy Concerns Remain a Barrier to Utilizing Big Data Public Sector IT Officials Frustrated With Multiple Data Formats, 8 Leadership Changes 9 Many Public Sector IT Officials Say Database queries Take Too Much Time Nearly All Government IT Officials Would Opt For Real Time Access to Data 10 Over Backward Looking Queries Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 12 BIG 318062
  • 13. Big Data in the Public Sector BIG Big Data Public Private Forum Primer taller para la construcción de la hoja de ruta de Big Data para Europa 16/04/2013 13 BIG 318062
  • 14. BIG PROJECT BIG - SECTOR FORUMS AND TECHNICAL Big Data Public Private Forum WORKING GROUPS Kick Off meeting 10-11/09/2012 14 BIG 318062
  • 15. PROJECT BIG BIG SECTORS’ ROADMAP Big Data Public Private Forum Identification ▶requirements and objectives from all of Sector’s Sectors (industry driven working requisites groups) Applicability of ▶Introduce technologies and trends to Big Data technical the stakeholders to better understand white papers in Big Data technologies and its each Sector capabilities ▶Sectorial roadmap (elaborate a Elaboration of roadmap per sector). Sector ▶Contributions towards integrated Roadmap roadmap (cross-sectorial) Kick Off meeting 10-11/09/2012 15 BIG 318062
  • 16. PROJECT BIG - TIMELINE OF THE MOST BIG IMPORTANT DELIVERABLES Big Data Public Private Forum 04/2013 04/2013 D2.2.1-1º version of Technical D2.2.1 D2.3.1 D2.3.1-1º version of Sector’s requisites white papers 06/2013 D4.2.1 D4.2.1-1ºversion of IPR, Standardization •recommendations 10/2013 D4.3.1-First draft of the Big Data Public-Private Forum D4.3.1 D2.4.1 09/2013 D2.4.1 1ª version of 01/2014 Sector´s Roadmap D2.2.2-Final version of D2.2.2 Technical white paper 04/2014 D2.3.2-Final version of 04/2014 D4.2.2 D2.3.2 Sectors requisites D4.2.2-Final version of IPR, Standardization recommendations 10/2014 D2.5 D2.5-Cross-sectorial roadmap consolidation Kick Off meeting 10-11/09/2012 16 BIG 318062
  • 17. Big Data Public Private Forum THANKS Tomás Pariente Lobo Atos Research & Innovation Atos Spain tomas.parientelob@atosresearch.eu

Editor's Notes

  1. STE is an emerging technology where we are involved as one of the main IT companies in Europe.