SlideShare une entreprise Scribd logo
1  sur  55
Télécharger pour lire hors ligne
Insights From Internet Of
Things & Big Data
Insights From Internet Of Things & Big Data
Kostya Goldstein
Sr. Program Manager Microsoft Russia
Business insights through big data
Microsoft’s solution to big data
Intelligent systems service, HDInsight
Features & capabilities
Demo – HDInsight
Office as Big Data visualization platform
Self service BI – Features & capabilities
Demo – Power BI
Hackathon
Tessel
IoT Hands On Task
Agenda
World Today
2003 2010 2015 2020
50 млрд
GROWTH OF DEVICES
World tomorrow
1990
2020
“”
Internet of Things (IoT)
The network of physical
objects that contain
embedded technology to
communicate and interact
with their internal states or
the external environment.
Вопрос
COW IoT ?
Интернет вещей
Аудио /Видео
Журналы операций
Тексты/Изображения
Настроение
высказываний
Обновления витрин
данных
Новости электронного
правительства
Погода
Вики / БлогиПереходы по
ссылкам
Датчики/ RFID / Устройства
Координаты GPS
WEB 2.0Мобильные
устро-ва
Реклама Взаимодействие
Электроннная
коммерция
Цифровой
маркетинг
Поисковый
маркетинг
Протоколы веб-
серверов
Рекомендации
ERP / CRM
Конвейер
продаж
Кредитор
ы
Зарплата
Запасы
Контакты
Отслеживани
е торгов
терабайты
(1012)
гигабайты
(109)
экзабайты
(1018)
петабайты
(1015)
Скорость | разнообразие | изменчивость
Объем
1980
190,000$
2010
0.07$
1990
9,000$
2000
15$
Стоимость хранения за гигабайт, долл
ERP / CRM WEB 2.0 Интернет вещей
What the big data is?
Two main IoT aspects
Start Justin
Customer
Service
ProviderMicrosoft
Consistent
Platform
ONE
Cloud OS Vision
Microsoft’s vision of the unified platform
for modern business
Transform The Datacenter
Unlock Insights On Any Data
Empower People-centric IT
Enable Modern Business Apps
Create The Internet Of Your Things
Intelligent Systems Service
Microsoft Solution For Internet Of Things
Drive InsightsAnalytics ReadyCloud and
infrastructure
Devices and
assets
1010101001100011010101011101001101010101010011011101111011100101010000110101010111010011010
1010111010011101010101011010011010101010101001101100010101111010011101010101011011110100111
1010101001100011010101011101001101010101010011011101111011100101010000110101010111010011010
1010111010011101010101011010011010101010101001101100010101111010011101010101011011110100111
Customer
portal Value
StreamInsights
Power BI
HDInsight
Windows Embedded
Connect new and existing
devices using open-source
agents or gateway
technologies
Azure, HDInsight
Store machine-generated
data with data from other
sources in the cloud
Office 365, Power BI
View data, administer
devices, and configure
rules, alerts, and other
actions using out-of-box
or custom portals
Mine insights from your
data to find gaps and
opportunities to make
better decisions and realize
new business value
User
input
AlertsSensors Gateway
Agent
ADevices
IoT Services Architecture & Platform Components
ISS (Intelligent
Systems Service)
Agent
Gateway
Event Hub &
Azure Service
Bus
Event Processing
&
Rules Engine
Tables
BLOBS
SQL Azure
HDFS
IF {condition}
THEN {action}
Azure Service
Bus
Design &
Engineerin
g
Manufacturin
g & Supply
Chain
Service &
Maintenanc
e
Customer
Relationshi
p
ISS (IntelligentSystems Service)
ID
Industrial
Equipment
Operational Data
Example of modern data storage
How To Generate Value From IoT Data
BIG DATA: Data powered by IoT &
other business systems
BETTER Insights: Transform your
business with better insights.
Unstructured
Structured
Streaming
PB
TB
GB Advanced analytics
Data scientist
Interactivity +
Exploration
Business analyst
Self-service
analysis
BI professional
Decision support
Device operator
Big Data
BIG DATA: Data powered by IoT & other
business systems
BETTER Insights: Transform your
business with better insights.
Unstructured
Structured
Streaming
PB
TB
GB Advanced analytics
Data scientist
Interactivity +
Exploration
Business analyst
Self-service
analysis
BI professional
Decision support
Device operator
Microsoft’s Big Data Solution Stack
Data Management
and Enrichment
Insight
Familiar end user tool
Unstructured and structured data
Sensors Devices Bots Crawlers ERP CRM LOB APPs
Interactive Reports
With Power View
Excel With
Powerpivot
Predictive Analytics
On MS Azure Cloud
Hadoop
HDInsight Machine
learning
Event
Hubs
Stream
Analytics
Data
Factory
Data Management And Enrichment
Data Management
and Enrichment
Insight
Familiar end user tool
Unstructured and structured data
Sensors Devices Bots Crawlers ERP CRM LOB APPs
Interactive Reports
With Power View
Excel With
Powerpivot
Predictive Analytics
On MS Azure Cloud
Hadoop
HDInsight Machine
learning
Event
Hubs
Stream
Analytics
Data
Factory
Hadoop And HDInsight Technology Stack
HDInsight Ecosystem
Metadata (Hcatalog)
Graph
(Pegasus)
Scripting
(PIG)
Query
(Hive)
Machine
learning
(Mahout)
Distributed processing
(Man reduce)
Distributed storage (HDFS)
World’s data (Azure
data marketplace)
Windows Azure
storage
AD, system center
Status
processing
(RHadoop
)
Businessintelligence
(Excel,owerview…)
Dataintegration
ODBCSQOOPREST
NoSQLDatabase
(Hbase)
P
D
W
Pipelineworkflo
w(Oozie)
Logfile
aggregation
(Flume)
Top level
interfaces ETL Tools BI Reporting RDBMS
Top level
abstractions
PIG HIVE Sqoop
Distributed
data
processing
Map-Reduce
HBASE
Database with
real time
access
At the base is a
self healing
clustered
storage system
Hadoop distributed file system
(HDFS)
Hadoop Ecosystem
HDInsight – Feature Set For Data Processing
Data Processing – Map Reduce Framework
Split (Combine) Partition
Read Map ReduceGroup Write
Data Processing – Map Reduce Framework
Костя Дима Миша
Андрей Костя Юра
Сергей Андрей Миша
Костя Дима Миша
Андрей Костя Юра
Сергей Андрей Миша
Костя,1
Дима,1
Миша,1
Андрей,1
Костя ,1
Юра,1
Сергей,1
Андрей ,1
Миша,1
Костя,1
Костя,1
Миша,1
Миша,1
Андрей,1
Андрей,1
Юра,1
Сергей,1
Дима,1
Костя,2
Дима,1
Миша,2
Андрей,2
Сергей,1
Юра,1
K1 ,V1 List(K2 ,V2) K2 ,List(V2)
Split (Combine) Partition
Read Map ReduceGroup Write
Data Preparation Using PIG Language
Data Storage Using HIVE Language
The prototypical MapReduce example counts the appearance of each word in a set of documents
function map(String name, String document):
// name: document name
// document: document contents
for each word w in document:
emit (w, 1)
function reduce(String word, Iterator partialCounts):
// word: a word
// partialCounts: a list of aggregated partial counts
sum = 0
for each pc in partialCounts:
sum += ParseInt(pc)
emit (word, sum)
en.wikipedia.org
PIG vs. HIVE
Sample of solving the same task by PIG &HIVE
PIG - Procedural
Users = load 'users' as (name, age, ipaddr);
Clicks = load 'clicks' as (user, url, value);
ValuableClicks = filter Clicks by value > 0;
UserClicks = join Users by name, ValuableClicks by
user;
Geoinfo = load 'geoinfo' as (ipaddr, dma);
UserGeo = join UserClicks by ipaddr, Geoinfo by
ipaddr;
ByDMA = group UserGeo by dma;
ValuableClicksPerDMA = foreach ByDMA generate
group, COUNT(UserGeo);
store ValuableClicksPerDMA into
'ValuableClicksPerDMA';
HIVE-Declarative
insert into ValuableClicksPerDMA
select dma, count(*)
from geoinfo join (select name, ipaddr
from users join clicks on (users.name = clicks.user)
where value > 0;) using ipaddr
group by dma;
https://developer.yahoo.com/blogs/hadoop/comparing-pig-latin-sql-constructing-data-processing-pipelines-444.html
Demo
Event Hubs
Communication Patterns
Telemetry
Ingest
That‘s easy …
• Ingest rate
• Storage
• Security
• …
Telemetry
Ingest
6
machines
20
sensors /
machine
X 120
sensors
/
production
line
=
Let‘s do the math …
Communication Patterns
Telemetry
Ingest
Communication Patterns
4
production
lines
/
plant
120
sensors /
production
line
X 480
sensors
/
plant
=
Let‘s do the math …
Telemetry
Ingest
Communication Patterns
480
sensors
/
plant
60
telemetry
ingests
/
minute
X 1,728,000
ingests
/
hour
=
Let‘s do the math …
Telemetry
Ingest
Communication Patterns
1,728,000
ingests
/
hour
50
e.g.
customers
X 86,400,000
ingests
/
hour
=
Let‘s do the math …
On a 24/7 basis
Hyper Scale is needed
Services – Service Bus / Event Hub
Overview
Service Bus
Relay
Queue
Topic
Notification
Event
Hub
Interactive Dashboard(s)Production Line(s)
Services – Service Bus / Event Hub
Partitions
Service
Bus
Interactive Dashboard(s)Production Line(s)
* 1 Mio Producers
* 1 MB/sec aggregate
per EventHub
Event Hub
Reader 1
Reader 2
Reader 3
….
Reader 1
Reader 2
Reader 3
….
Consumer
Group
Throughput Units
1 MB/s writes
2 MB/s reads
Stream Analytics
Real-time stream processing in the cloud
Stream millions of events per second
Perform real-time analytics
Correlate across multiple streams of data
Reliable performance and predictable results
No hardware to deploy
Rapid development with familiar SQL-like language
Demo
BIG Data To Better Insights
BIG DATA: Data powered by IoT &
other business systems
BETTER Insights: Transform your
business with better insights.
Unstructured
Structured
Streaming
PB
TB
GB Advanced analytics
Data scientist
Interactivity +
Exploration
Business analyst
Self-service
analysis
BI professional
Decision support
Device operator
Q&A
A Powerful New Way To Work With Data
Self-service business intelligence with familiar Excel and the power of the cloud
Discover And Access Data
Using power query to access data
From Internet From File From Database And More…
Easily Discover And Access Data
Analyzing Data With Excel
Easily discover and access public and
corporate data with Power Query
Model & analyze 100’s of millions of rows
lightning fast with Power Pivot
Explore and visualize data in new ways with
Power View and Power Map
Internet of Things in Tbilisi
Internet of Things in Tbilisi
Modules
▪ Accelerometer
▪ Ambient Light + Sound
▪ Audio
▪ Bluetooth Low Energy
▪ Camera
▪ Climate
▪ GPS
▪ GPRS
▪ Infrared
▪ MicroSD Card
▪ nRF24 Module
▪ Relay
▪ RFID
▪ Servo
What can you do with a Tessel?
▪ Ambient monitoring: monitor temperature, noise… Detect variations
and take action / notify.
– Is the light on at home?Turn on Hue lights automatically at dark.
▪ Accelerometer: game controllers, activity trackers…
▪ Camera: take pictures on event, motion detection…
▪ Infrared: control yourTV
– Clap your hands to turn it on
▪ Lots of projects ideas: https://projects.tessel.io/projects
Node.JS for the Tessel
▪ Node.JS is usually used on the server-side; here we are going to use it
on the client side!
▪ Node.JS is well suited to real-time processing of events, thanks to its
asynchronous nature; this is well adapted to a device whose main job
is to monitor and process events (temperature / noise / light / etc.)
▪ Instead of listening to server-side events (GET, POST, etc.) you will be
listening to module-specific events
▪ Events are handled using callbacks, functions that you pass when
registering for the event
Hello World: tessel run blinky.js
// Import the interface to Tessel hardware
var tessel = require('tessel');
// Set the led pins as outputs with initial states
// Truthy initial state sets the pin high
// Falsy sets it low.
var led1 = tessel.led[0].output(1);
var led2 = tessel.led[1].output(0);
setInterval(function () {
console.log("I'm blinking! (Press CTRL + C to stop)");
// Toggle the led states
led1.toggle();
led2.toggle();
}, 100);
More getting started: Wi-Fi
▪ Connect to localWiFi – ExpoGeorgia
– User:pav#3
– Pass:201567890
▪ OR
▪ Revert to using phone hotspot
▪ tessel wifi -n "iPhone 6" -p "Pass1234“
What can you do with Azure?
▪ In theory, anything you can do in Node.JS
– In practice, some complex modules or projects will cause translation problems
because not all Node constructs are fully supported
– Most notably, the Azure SDKs for Node.JS seem to be causing some problems
– It might be easier to revert to plain old REST APIs when possible
▪ Upload stuff to Azure: Blob Storage
▪ Send monitoring/telemetry to Azure: Service Bus, Event Hubs
– Experiment with different protocols: HTTPS, AMQP, MQTT…
▪ Interact with mobile devices through Mobile Services
– Send notifications
▪ Samples on http://gist.github.com/tomconte and
http://hypernephelist.com
Let’s hack!
▪ Grab your hardware
▪ Pair up
– Might be best to have one person who knows JS/Node per pair
▪ Get something done in 4 hours
– Install Node,Tessel module, plug in board, upgrade firmware
– Use Notepad++ / SublimeText /Visual Studio or whatever
– Do the Hello World thing
– Get connected toWi-Fi
– Plug in a module, test it
– Do the lab https://github.com/Dx-ted-emea/iot-labs
– For advanced
▪ HDInsight
▪ Use in ML
▪ Connect to mobile device
– Present your results/learnings/findings in the last 30 minutes
Internet of Things in Tbilisi
©2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or
trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this
presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee
the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN
THIS PRESENTATION.

Contenu connexe

Tendances

MongoDB company and case studies - john hong
MongoDB company and case studies - john hong MongoDB company and case studies - john hong
MongoDB company and case studies - john hong Ha-Yang(White) Moon
 
Overview of analytics and big data in practice
Overview of analytics and big data in practiceOverview of analytics and big data in practice
Overview of analytics and big data in practiceVivek Murugesan
 
[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics
[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics
[Webinar] Measure Twice, Build Once: Real-Time Predictive AnalyticsInfochimps, a CSC Big Data Business
 
Data Analytics and Artificial Intelligence in the era of Digital Transformation
Data Analytics and Artificial Intelligence in the era of Digital TransformationData Analytics and Artificial Intelligence in the era of Digital Transformation
Data Analytics and Artificial Intelligence in the era of Digital TransformationJan Wiegelmann
 
Big data ibm keynote d advani presentation
Big data ibm keynote d advani presentationBig data ibm keynote d advani presentation
Big data ibm keynote d advani presentationMassTLC
 
Your Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyYour Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyNeo4j
 
Data analysis trend 2015 2016 v071
Data analysis trend 2015 2016 v071Data analysis trend 2015 2016 v071
Data analysis trend 2015 2016 v071Chun Myung Kyu
 
Vital AI: Big Data Modeling
Vital AI: Big Data ModelingVital AI: Big Data Modeling
Vital AI: Big Data ModelingVital.AI
 
Real-time Big Data Analytics: From Deployment to Production
Real-time Big Data Analytics: From Deployment to ProductionReal-time Big Data Analytics: From Deployment to Production
Real-time Big Data Analytics: From Deployment to ProductionRevolution Analytics
 
Cloud Computing for Data Professionals
Cloud Computing for Data ProfessionalsCloud Computing for Data Professionals
Cloud Computing for Data ProfessionalsAnkit Rathi
 
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarial
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarialInteligencia artificial - Quebrando el paradigma de la amnesia empresarial
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarialMarcos Quezada
 
Forecast of Big Data Trends
Forecast of Big Data TrendsForecast of Big Data Trends
Forecast of Big Data TrendsIMC Institute
 
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and SparkVital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and SparkVital.AI
 
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data:  Technical Introduction to BigSheets for InfoSphere BigInsightsBig Data:  Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsightsCynthia Saracco
 
Big data - what, why, where, when and how
Big data - what, why, where, when and howBig data - what, why, where, when and how
Big data - what, why, where, when and howbobosenthil
 
Deep Learning Image Processing Applications in the Enterprise
Deep Learning Image Processing Applications in the EnterpriseDeep Learning Image Processing Applications in the Enterprise
Deep Learning Image Processing Applications in the EnterpriseGanesan Narayanasamy
 

Tendances (20)

MongoDB company and case studies - john hong
MongoDB company and case studies - john hong MongoDB company and case studies - john hong
MongoDB company and case studies - john hong
 
Overview of analytics and big data in practice
Overview of analytics and big data in practiceOverview of analytics and big data in practice
Overview of analytics and big data in practice
 
[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics
[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics
[Webinar] Measure Twice, Build Once: Real-Time Predictive Analytics
 
Data Analytics and Artificial Intelligence in the era of Digital Transformation
Data Analytics and Artificial Intelligence in the era of Digital TransformationData Analytics and Artificial Intelligence in the era of Digital Transformation
Data Analytics and Artificial Intelligence in the era of Digital Transformation
 
Big Data Landscape 2016
Big Data Landscape 2016Big Data Landscape 2016
Big Data Landscape 2016
 
Big data ibm keynote d advani presentation
Big data ibm keynote d advani presentationBig data ibm keynote d advani presentation
Big data ibm keynote d advani presentation
 
Your Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyYour Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph Strategy
 
Data analysis trend 2015 2016 v071
Data analysis trend 2015 2016 v071Data analysis trend 2015 2016 v071
Data analysis trend 2015 2016 v071
 
Vital AI: Big Data Modeling
Vital AI: Big Data ModelingVital AI: Big Data Modeling
Vital AI: Big Data Modeling
 
Hadoop and DynamoDB
Hadoop and DynamoDBHadoop and DynamoDB
Hadoop and DynamoDB
 
Real-time Big Data Analytics: From Deployment to Production
Real-time Big Data Analytics: From Deployment to ProductionReal-time Big Data Analytics: From Deployment to Production
Real-time Big Data Analytics: From Deployment to Production
 
Google and big query
Google and big queryGoogle and big query
Google and big query
 
Cloud Computing for Data Professionals
Cloud Computing for Data ProfessionalsCloud Computing for Data Professionals
Cloud Computing for Data Professionals
 
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarial
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarialInteligencia artificial - Quebrando el paradigma de la amnesia empresarial
Inteligencia artificial - Quebrando el paradigma de la amnesia empresarial
 
Forecast of Big Data Trends
Forecast of Big Data TrendsForecast of Big Data Trends
Forecast of Big Data Trends
 
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and SparkVital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark
Vital AI MetaQL: Queries Across NoSQL, SQL, Sparql, and Spark
 
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data:  Technical Introduction to BigSheets for InfoSphere BigInsightsBig Data:  Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
 
Big data - what, why, where, when and how
Big data - what, why, where, when and howBig data - what, why, where, when and how
Big data - what, why, where, when and how
 
Deep Learning Image Processing Applications in the Enterprise
Deep Learning Image Processing Applications in the EnterpriseDeep Learning Image Processing Applications in the Enterprise
Deep Learning Image Processing Applications in the Enterprise
 
Make your data talk
Make your data talkMake your data talk
Make your data talk
 

Similaire à Internet of Things in Tbilisi

Building IoT and Big Data Solutions on Azure
Building IoT and Big Data Solutions on AzureBuilding IoT and Big Data Solutions on Azure
Building IoT and Big Data Solutions on AzureIdo Flatow
 
Building big data solutions on azure
Building big data solutions on azureBuilding big data solutions on azure
Building big data solutions on azureEyal Ben Ivri
 
Building Large-Scale Applications for the Internet of Things at Bosch
Building Large-Scale Applications for the Internet of Things at BoschBuilding Large-Scale Applications for the Internet of Things at Bosch
Building Large-Scale Applications for the Internet of Things at BoschMongoDB
 
High-performance database technology for rock-solid IoT solutions
High-performance database technology for rock-solid IoT solutionsHigh-performance database technology for rock-solid IoT solutions
High-performance database technology for rock-solid IoT solutionsClusterpoint
 
Empower Your Organization with Microsoft Power Platform
Empower Your Organization with Microsoft Power PlatformEmpower Your Organization with Microsoft Power Platform
Empower Your Organization with Microsoft Power PlatformDavid J Rosenthal
 
Power BI for Big Data and the New Look of Big Data Solutions
Power BI for Big Data and the New Look of Big Data SolutionsPower BI for Big Data and the New Look of Big Data Solutions
Power BI for Big Data and the New Look of Big Data SolutionsJames Serra
 
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...Guido Schmutz
 
Empowering you - Power BI, Power Platform & AI Builder
Empowering you  -  Power BI, Power Platform & AI BuilderEmpowering you  -  Power BI, Power Platform & AI Builder
Empowering you - Power BI, Power Platform & AI BuilderRui Quintino
 
Scaling up your Analytics & Insights
Scaling up your Analytics & InsightsScaling up your Analytics & Insights
Scaling up your Analytics & InsightsLoQutus
 
Modern Business Intelligence and Advanced Analytics
Modern Business Intelligence and Advanced AnalyticsModern Business Intelligence and Advanced Analytics
Modern Business Intelligence and Advanced AnalyticsCollective Intelligence Inc.
 
Cloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch DeckCloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch DeckNicholas Vossburg
 
Microsoft cloud big data strategy
Microsoft cloud big data strategyMicrosoft cloud big data strategy
Microsoft cloud big data strategyJames Serra
 
Big Data Meetup: Analytical Systems Evolution
Big Data Meetup: Analytical Systems EvolutionBig Data Meetup: Analytical Systems Evolution
Big Data Meetup: Analytical Systems EvolutionProvectus
 
Real-time big data analytics based on product recommendations case study
Real-time big data analytics based on product recommendations case studyReal-time big data analytics based on product recommendations case study
Real-time big data analytics based on product recommendations case studydeep.bi
 
Demystify Big Data, Data Science & Signal Extraction Deep Dive
Demystify Big Data, Data Science & Signal Extraction Deep DiveDemystify Big Data, Data Science & Signal Extraction Deep Dive
Demystify Big Data, Data Science & Signal Extraction Deep DiveHyderabad Scalability Meetup
 
Microsoft Business Intelligence
Microsoft Business IntelligenceMicrosoft Business Intelligence
Microsoft Business IntelligenceNic Smith
 
Azure and Predix
Azure and PredixAzure and Predix
Azure and PredixAltoros
 

Similaire à Internet of Things in Tbilisi (20)

Building IoT and Big Data Solutions on Azure
Building IoT and Big Data Solutions on AzureBuilding IoT and Big Data Solutions on Azure
Building IoT and Big Data Solutions on Azure
 
Building big data solutions on azure
Building big data solutions on azureBuilding big data solutions on azure
Building big data solutions on azure
 
Building Large-Scale Applications for the Internet of Things at Bosch
Building Large-Scale Applications for the Internet of Things at BoschBuilding Large-Scale Applications for the Internet of Things at Bosch
Building Large-Scale Applications for the Internet of Things at Bosch
 
High-performance database technology for rock-solid IoT solutions
High-performance database technology for rock-solid IoT solutionsHigh-performance database technology for rock-solid IoT solutions
High-performance database technology for rock-solid IoT solutions
 
Empower Your Organization with Microsoft Power Platform
Empower Your Organization with Microsoft Power PlatformEmpower Your Organization with Microsoft Power Platform
Empower Your Organization with Microsoft Power Platform
 
Power BI for Big Data and the New Look of Big Data Solutions
Power BI for Big Data and the New Look of Big Data SolutionsPower BI for Big Data and the New Look of Big Data Solutions
Power BI for Big Data and the New Look of Big Data Solutions
 
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
 
Empowering you - Power BI, Power Platform & AI Builder
Empowering you  -  Power BI, Power Platform & AI BuilderEmpowering you  -  Power BI, Power Platform & AI Builder
Empowering you - Power BI, Power Platform & AI Builder
 
Scaling up your Analytics & Insights
Scaling up your Analytics & InsightsScaling up your Analytics & Insights
Scaling up your Analytics & Insights
 
Modern Business Intelligence and Advanced Analytics
Modern Business Intelligence and Advanced AnalyticsModern Business Intelligence and Advanced Analytics
Modern Business Intelligence and Advanced Analytics
 
Cloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch DeckCloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch Deck
 
Azure IoT Suite
Azure IoT Suite Azure IoT Suite
Azure IoT Suite
 
Microsoft & IoT
Microsoft & IoTMicrosoft & IoT
Microsoft & IoT
 
Microsoft cloud big data strategy
Microsoft cloud big data strategyMicrosoft cloud big data strategy
Microsoft cloud big data strategy
 
Big Data Meetup: Analytical Systems Evolution
Big Data Meetup: Analytical Systems EvolutionBig Data Meetup: Analytical Systems Evolution
Big Data Meetup: Analytical Systems Evolution
 
Real-time big data analytics based on product recommendations case study
Real-time big data analytics based on product recommendations case studyReal-time big data analytics based on product recommendations case study
Real-time big data analytics based on product recommendations case study
 
Demystify Big Data, Data Science & Signal Extraction Deep Dive
Demystify Big Data, Data Science & Signal Extraction Deep DiveDemystify Big Data, Data Science & Signal Extraction Deep Dive
Demystify Big Data, Data Science & Signal Extraction Deep Dive
 
Internet of Things and Big Data
Internet of Things and Big DataInternet of Things and Big Data
Internet of Things and Big Data
 
Microsoft Business Intelligence
Microsoft Business IntelligenceMicrosoft Business Intelligence
Microsoft Business Intelligence
 
Azure and Predix
Azure and PredixAzure and Predix
Azure and Predix
 

Plus de Alexey Bokov

Product Visions and Strategy - crash course for startups
Product Visions and Strategy - crash course for startupsProduct Visions and Strategy - crash course for startups
Product Visions and Strategy - crash course for startupsAlexey Bokov
 
Windows containers troubleshooting
Windows containers troubleshootingWindows containers troubleshooting
Windows containers troubleshootingAlexey Bokov
 
Monte Carlo modeling in cloud - mc-modeling-sdk
Monte Carlo modeling in cloud - mc-modeling-sdkMonte Carlo modeling in cloud - mc-modeling-sdk
Monte Carlo modeling in cloud - mc-modeling-sdkAlexey Bokov
 
CAP теорема Брюера и ее применения на практике
CAP теорема Брюера и ее применения на практикеCAP теорема Брюера и ее применения на практике
CAP теорема Брюера и ее применения на практикеAlexey Bokov
 
Azure web apps - designing and debugging
Azure web apps  - designing and debuggingAzure web apps  - designing and debugging
Azure web apps - designing and debuggingAlexey Bokov
 
Azure Web App services
Azure Web App servicesAzure Web App services
Azure Web App servicesAlexey Bokov
 
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...Alexey Bokov
 
Creating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceCreating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceAlexey Bokov
 
All about Azure workshop deck
All about Azure workshop deckAll about Azure workshop deck
All about Azure workshop deckAlexey Bokov
 
All about Azure - Kazan
All about Azure - KazanAll about Azure - Kazan
All about Azure - KazanAlexey Bokov
 
Azure and web sites hackaton deck
Azure and web sites hackaton deckAzure and web sites hackaton deck
Azure and web sites hackaton deckAlexey Bokov
 
Tbilisi hackaton intro
Tbilisi hackaton introTbilisi hackaton intro
Tbilisi hackaton introAlexey Bokov
 
Azure for IT pro - TechDays Armenia
Azure for IT pro - TechDays ArmeniaAzure for IT pro - TechDays Armenia
Azure for IT pro - TechDays ArmeniaAlexey Bokov
 
Tech day armenia for developers
Tech day armenia   for developersTech day armenia   for developers
Tech day armenia for developersAlexey Bokov
 
Alexey Bokov key note - TechDays Armenia 2014
Alexey Bokov key note - TechDays Armenia 2014Alexey Bokov key note - TechDays Armenia 2014
Alexey Bokov key note - TechDays Armenia 2014Alexey Bokov
 
Open source technologies in Microsoft cloud - MS SWIT 2014
Open source technologies in Microsoft cloud - MS SWIT 2014Open source technologies in Microsoft cloud - MS SWIT 2014
Open source technologies in Microsoft cloud - MS SWIT 2014Alexey Bokov
 
Windows Azure для стартапов
Windows Azure для стартаповWindows Azure для стартапов
Windows Azure для стартаповAlexey Bokov
 

Plus de Alexey Bokov (20)

Product Visions and Strategy - crash course for startups
Product Visions and Strategy - crash course for startupsProduct Visions and Strategy - crash course for startups
Product Visions and Strategy - crash course for startups
 
Windows containers troubleshooting
Windows containers troubleshootingWindows containers troubleshooting
Windows containers troubleshooting
 
Monte Carlo modeling in cloud - mc-modeling-sdk
Monte Carlo modeling in cloud - mc-modeling-sdkMonte Carlo modeling in cloud - mc-modeling-sdk
Monte Carlo modeling in cloud - mc-modeling-sdk
 
CAP теорема Брюера и ее применения на практике
CAP теорема Брюера и ее применения на практикеCAP теорема Брюера и ее применения на практике
CAP теорема Брюера и ее применения на практике
 
Azure web apps - designing and debugging
Azure web apps  - designing and debuggingAzure web apps  - designing and debugging
Azure web apps - designing and debugging
 
Azure Web App services
Azure Web App servicesAzure Web App services
Azure Web App services
 
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...
Azure: Docker Container orchestration, PaaS ( Service Farbic ) and High avail...
 
Creating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceCreating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplace
 
All about Azure workshop deck
All about Azure workshop deckAll about Azure workshop deck
All about Azure workshop deck
 
All about Azure - Kazan
All about Azure - KazanAll about Azure - Kazan
All about Azure - Kazan
 
Microsoft Azure
Microsoft AzureMicrosoft Azure
Microsoft Azure
 
Azure and web sites hackaton deck
Azure and web sites hackaton deckAzure and web sites hackaton deck
Azure and web sites hackaton deck
 
Asp.net 5 cloud
Asp.net 5 cloudAsp.net 5 cloud
Asp.net 5 cloud
 
Tbilisi hackaton intro
Tbilisi hackaton introTbilisi hackaton intro
Tbilisi hackaton intro
 
Azure for retails
Azure for retailsAzure for retails
Azure for retails
 
Azure for IT pro - TechDays Armenia
Azure for IT pro - TechDays ArmeniaAzure for IT pro - TechDays Armenia
Azure for IT pro - TechDays Armenia
 
Tech day armenia for developers
Tech day armenia   for developersTech day armenia   for developers
Tech day armenia for developers
 
Alexey Bokov key note - TechDays Armenia 2014
Alexey Bokov key note - TechDays Armenia 2014Alexey Bokov key note - TechDays Armenia 2014
Alexey Bokov key note - TechDays Armenia 2014
 
Open source technologies in Microsoft cloud - MS SWIT 2014
Open source technologies in Microsoft cloud - MS SWIT 2014Open source technologies in Microsoft cloud - MS SWIT 2014
Open source technologies in Microsoft cloud - MS SWIT 2014
 
Windows Azure для стартапов
Windows Azure для стартаповWindows Azure для стартапов
Windows Azure для стартапов
 

Dernier

IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Dernier (20)

IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

Internet of Things in Tbilisi

  • 1. Insights From Internet Of Things & Big Data
  • 2. Insights From Internet Of Things & Big Data Kostya Goldstein Sr. Program Manager Microsoft Russia
  • 3. Business insights through big data Microsoft’s solution to big data Intelligent systems service, HDInsight Features & capabilities Demo – HDInsight Office as Big Data visualization platform Self service BI – Features & capabilities Demo – Power BI Hackathon Tessel IoT Hands On Task Agenda
  • 5. 2003 2010 2015 2020 50 млрд GROWTH OF DEVICES
  • 8. “” Internet of Things (IoT) The network of physical objects that contain embedded technology to communicate and interact with their internal states or the external environment.
  • 10. Интернет вещей Аудио /Видео Журналы операций Тексты/Изображения Настроение высказываний Обновления витрин данных Новости электронного правительства Погода Вики / БлогиПереходы по ссылкам Датчики/ RFID / Устройства Координаты GPS WEB 2.0Мобильные устро-ва Реклама Взаимодействие Электроннная коммерция Цифровой маркетинг Поисковый маркетинг Протоколы веб- серверов Рекомендации ERP / CRM Конвейер продаж Кредитор ы Зарплата Запасы Контакты Отслеживани е торгов терабайты (1012) гигабайты (109) экзабайты (1018) петабайты (1015) Скорость | разнообразие | изменчивость Объем 1980 190,000$ 2010 0.07$ 1990 9,000$ 2000 15$ Стоимость хранения за гигабайт, долл ERP / CRM WEB 2.0 Интернет вещей What the big data is?
  • 11. Two main IoT aspects Start Justin
  • 12. Customer Service ProviderMicrosoft Consistent Platform ONE Cloud OS Vision Microsoft’s vision of the unified platform for modern business Transform The Datacenter Unlock Insights On Any Data Empower People-centric IT Enable Modern Business Apps Create The Internet Of Your Things
  • 13. Intelligent Systems Service Microsoft Solution For Internet Of Things Drive InsightsAnalytics ReadyCloud and infrastructure Devices and assets 1010101001100011010101011101001101010101010011011101111011100101010000110101010111010011010 1010111010011101010101011010011010101010101001101100010101111010011101010101011011110100111 1010101001100011010101011101001101010101010011011101111011100101010000110101010111010011010 1010111010011101010101011010011010101010101001101100010101111010011101010101011011110100111 Customer portal Value StreamInsights Power BI HDInsight Windows Embedded Connect new and existing devices using open-source agents or gateway technologies Azure, HDInsight Store machine-generated data with data from other sources in the cloud Office 365, Power BI View data, administer devices, and configure rules, alerts, and other actions using out-of-box or custom portals Mine insights from your data to find gaps and opportunities to make better decisions and realize new business value User input AlertsSensors Gateway Agent ADevices
  • 14. IoT Services Architecture & Platform Components ISS (Intelligent Systems Service) Agent Gateway Event Hub & Azure Service Bus Event Processing & Rules Engine Tables BLOBS SQL Azure HDFS IF {condition} THEN {action} Azure Service Bus Design & Engineerin g Manufacturin g & Supply Chain Service & Maintenanc e Customer Relationshi p ISS (IntelligentSystems Service) ID Industrial Equipment
  • 15. Operational Data Example of modern data storage
  • 16. How To Generate Value From IoT Data BIG DATA: Data powered by IoT & other business systems BETTER Insights: Transform your business with better insights. Unstructured Structured Streaming PB TB GB Advanced analytics Data scientist Interactivity + Exploration Business analyst Self-service analysis BI professional Decision support Device operator
  • 17. Big Data BIG DATA: Data powered by IoT & other business systems BETTER Insights: Transform your business with better insights. Unstructured Structured Streaming PB TB GB Advanced analytics Data scientist Interactivity + Exploration Business analyst Self-service analysis BI professional Decision support Device operator
  • 18. Microsoft’s Big Data Solution Stack Data Management and Enrichment Insight Familiar end user tool Unstructured and structured data Sensors Devices Bots Crawlers ERP CRM LOB APPs Interactive Reports With Power View Excel With Powerpivot Predictive Analytics On MS Azure Cloud Hadoop HDInsight Machine learning Event Hubs Stream Analytics Data Factory
  • 19. Data Management And Enrichment Data Management and Enrichment Insight Familiar end user tool Unstructured and structured data Sensors Devices Bots Crawlers ERP CRM LOB APPs Interactive Reports With Power View Excel With Powerpivot Predictive Analytics On MS Azure Cloud Hadoop HDInsight Machine learning Event Hubs Stream Analytics Data Factory
  • 20. Hadoop And HDInsight Technology Stack HDInsight Ecosystem Metadata (Hcatalog) Graph (Pegasus) Scripting (PIG) Query (Hive) Machine learning (Mahout) Distributed processing (Man reduce) Distributed storage (HDFS) World’s data (Azure data marketplace) Windows Azure storage AD, system center Status processing (RHadoop ) Businessintelligence (Excel,owerview…) Dataintegration ODBCSQOOPREST NoSQLDatabase (Hbase) P D W Pipelineworkflo w(Oozie) Logfile aggregation (Flume) Top level interfaces ETL Tools BI Reporting RDBMS Top level abstractions PIG HIVE Sqoop Distributed data processing Map-Reduce HBASE Database with real time access At the base is a self healing clustered storage system Hadoop distributed file system (HDFS) Hadoop Ecosystem
  • 21. HDInsight – Feature Set For Data Processing
  • 22. Data Processing – Map Reduce Framework Split (Combine) Partition Read Map ReduceGroup Write
  • 23. Data Processing – Map Reduce Framework Костя Дима Миша Андрей Костя Юра Сергей Андрей Миша Костя Дима Миша Андрей Костя Юра Сергей Андрей Миша Костя,1 Дима,1 Миша,1 Андрей,1 Костя ,1 Юра,1 Сергей,1 Андрей ,1 Миша,1 Костя,1 Костя,1 Миша,1 Миша,1 Андрей,1 Андрей,1 Юра,1 Сергей,1 Дима,1 Костя,2 Дима,1 Миша,2 Андрей,2 Сергей,1 Юра,1 K1 ,V1 List(K2 ,V2) K2 ,List(V2) Split (Combine) Partition Read Map ReduceGroup Write
  • 24. Data Preparation Using PIG Language
  • 25. Data Storage Using HIVE Language
  • 26. The prototypical MapReduce example counts the appearance of each word in a set of documents function map(String name, String document): // name: document name // document: document contents for each word w in document: emit (w, 1) function reduce(String word, Iterator partialCounts): // word: a word // partialCounts: a list of aggregated partial counts sum = 0 for each pc in partialCounts: sum += ParseInt(pc) emit (word, sum) en.wikipedia.org
  • 28. Sample of solving the same task by PIG &HIVE PIG - Procedural Users = load 'users' as (name, age, ipaddr); Clicks = load 'clicks' as (user, url, value); ValuableClicks = filter Clicks by value > 0; UserClicks = join Users by name, ValuableClicks by user; Geoinfo = load 'geoinfo' as (ipaddr, dma); UserGeo = join UserClicks by ipaddr, Geoinfo by ipaddr; ByDMA = group UserGeo by dma; ValuableClicksPerDMA = foreach ByDMA generate group, COUNT(UserGeo); store ValuableClicksPerDMA into 'ValuableClicksPerDMA'; HIVE-Declarative insert into ValuableClicksPerDMA select dma, count(*) from geoinfo join (select name, ipaddr from users join clicks on (users.name = clicks.user) where value > 0;) using ipaddr group by dma; https://developer.yahoo.com/blogs/hadoop/comparing-pig-latin-sql-constructing-data-processing-pipelines-444.html
  • 29. Demo
  • 31. Communication Patterns Telemetry Ingest That‘s easy … • Ingest rate • Storage • Security • …
  • 36. Services – Service Bus / Event Hub Overview Service Bus Relay Queue Topic Notification Event Hub Interactive Dashboard(s)Production Line(s)
  • 37. Services – Service Bus / Event Hub Partitions Service Bus Interactive Dashboard(s)Production Line(s) * 1 Mio Producers * 1 MB/sec aggregate per EventHub Event Hub Reader 1 Reader 2 Reader 3 …. Reader 1 Reader 2 Reader 3 …. Consumer Group Throughput Units 1 MB/s writes 2 MB/s reads
  • 38. Stream Analytics Real-time stream processing in the cloud Stream millions of events per second Perform real-time analytics Correlate across multiple streams of data Reliable performance and predictable results No hardware to deploy Rapid development with familiar SQL-like language
  • 39. Demo
  • 40. BIG Data To Better Insights BIG DATA: Data powered by IoT & other business systems BETTER Insights: Transform your business with better insights. Unstructured Structured Streaming PB TB GB Advanced analytics Data scientist Interactivity + Exploration Business analyst Self-service analysis BI professional Decision support Device operator
  • 41. Q&A A Powerful New Way To Work With Data Self-service business intelligence with familiar Excel and the power of the cloud
  • 42. Discover And Access Data Using power query to access data
  • 43. From Internet From File From Database And More… Easily Discover And Access Data
  • 44. Analyzing Data With Excel Easily discover and access public and corporate data with Power Query Model & analyze 100’s of millions of rows lightning fast with Power Pivot Explore and visualize data in new ways with Power View and Power Map
  • 47. Modules ▪ Accelerometer ▪ Ambient Light + Sound ▪ Audio ▪ Bluetooth Low Energy ▪ Camera ▪ Climate ▪ GPS ▪ GPRS ▪ Infrared ▪ MicroSD Card ▪ nRF24 Module ▪ Relay ▪ RFID ▪ Servo
  • 48. What can you do with a Tessel? ▪ Ambient monitoring: monitor temperature, noise… Detect variations and take action / notify. – Is the light on at home?Turn on Hue lights automatically at dark. ▪ Accelerometer: game controllers, activity trackers… ▪ Camera: take pictures on event, motion detection… ▪ Infrared: control yourTV – Clap your hands to turn it on ▪ Lots of projects ideas: https://projects.tessel.io/projects
  • 49. Node.JS for the Tessel ▪ Node.JS is usually used on the server-side; here we are going to use it on the client side! ▪ Node.JS is well suited to real-time processing of events, thanks to its asynchronous nature; this is well adapted to a device whose main job is to monitor and process events (temperature / noise / light / etc.) ▪ Instead of listening to server-side events (GET, POST, etc.) you will be listening to module-specific events ▪ Events are handled using callbacks, functions that you pass when registering for the event
  • 50. Hello World: tessel run blinky.js // Import the interface to Tessel hardware var tessel = require('tessel'); // Set the led pins as outputs with initial states // Truthy initial state sets the pin high // Falsy sets it low. var led1 = tessel.led[0].output(1); var led2 = tessel.led[1].output(0); setInterval(function () { console.log("I'm blinking! (Press CTRL + C to stop)"); // Toggle the led states led1.toggle(); led2.toggle(); }, 100);
  • 51. More getting started: Wi-Fi ▪ Connect to localWiFi – ExpoGeorgia – User:pav#3 – Pass:201567890 ▪ OR ▪ Revert to using phone hotspot ▪ tessel wifi -n "iPhone 6" -p "Pass1234“
  • 52. What can you do with Azure? ▪ In theory, anything you can do in Node.JS – In practice, some complex modules or projects will cause translation problems because not all Node constructs are fully supported – Most notably, the Azure SDKs for Node.JS seem to be causing some problems – It might be easier to revert to plain old REST APIs when possible ▪ Upload stuff to Azure: Blob Storage ▪ Send monitoring/telemetry to Azure: Service Bus, Event Hubs – Experiment with different protocols: HTTPS, AMQP, MQTT… ▪ Interact with mobile devices through Mobile Services – Send notifications ▪ Samples on http://gist.github.com/tomconte and http://hypernephelist.com
  • 53. Let’s hack! ▪ Grab your hardware ▪ Pair up – Might be best to have one person who knows JS/Node per pair ▪ Get something done in 4 hours – Install Node,Tessel module, plug in board, upgrade firmware – Use Notepad++ / SublimeText /Visual Studio or whatever – Do the Hello World thing – Get connected toWi-Fi – Plug in a module, test it – Do the lab https://github.com/Dx-ted-emea/iot-labs – For advanced ▪ HDInsight ▪ Use in ML ▪ Connect to mobile device – Present your results/learnings/findings in the last 30 minutes
  • 55. ©2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.