SlideShare a Scribd company logo
1 of 120
Download to read offline
IOT (M2M) - BIG DATA - ANALYTICS
CHAKER ALLAOUI
EMULATION & DEMONSTRATION
GPS Sensor Philips HUE MQTT
Geo-Location Temperature Commands
Node-Red
Topic - Broker Humidity Data Subscriber
IoT PLATFORM
Publisher
Features Monitoring
Table of Contents
PARTIE II : Emulation
PARTIE III : Demonstration
ARCHITECTURE IoT
PARTIE I : Presentation
Cars Trafics
Data-Transfert
Commands
AnalyticsBig DataM2M
Examples
PARTIE I : PRESENTATION
5
The Internet of Things (IoT) is an emerging global Internet-
based information architecture facilitating the exchange
of goods and services. The IoT has the purpose of
providing an IT-infrastructure facilitating the exchange of
“things” in a secure and reliable manner, i.e. its function is
to overcome the gap between objects in the physical
world and their representation in information systems. The
IoT will serve to increase transparency and enhance the
efficiency of global supply chain networks.
In systems engineering, a 'system monitor (SM) is a process
within a distributed system for collecting and storing state
data. This is a fundamental principle supporting
Application Performance Management. The argument
that system monitoring is just a nice to have, and not
really a core requirement for operational readiness,
dissipates quickly when a critical application goes down
with no warning, configuration data for the monitor,
configuration data for the system being monitored.
IoT Platform
6
Features
The Internet of Things is the network of physical objects or "things" embedded with electronics,
software, sensors, and connectivity to enable objects to exchange data with the manufacturer,
operator and/or other connected devices based on the infrastructure of International
Telecommunication Union's Global Standards Initiative. The Internet of Things allows objects to be
sensed and controlled remotely across existing network infrastructure, creating opportunities for more
direct integration between the physical world and computer-based systems, and resulting in
improved efficiency, accuracy and economic benefit. Each thing is uniquely identifiable through its
embedded computing system but is able to interoperate within the existing Internet infrastructure.
Experts estimate that the IoT will consist of almost 50 billion objects by 2020. The term “Internet of
Things” was coined by British entrepreneur Kevin Ashton in 1999. Typically, IoT is expected to offer
advanced connectivity of devices, systems, and services that goes beyond machine-to-machine
communications (M2M) and covers a variety of protocols, domains, and applications. The
interconnection of these embedded devices (including smart objects), is expected to usher in
automation in nearly all fields, while also enabling advanced applications like a Smart Grid, and
expanding to the areas such as Smart city. Things, in the IoT, can refer to a wide variety of devices
such as heart monitoring implants, biochip transponders on farm animals, electric clams in coastal
waters, automobiles with built-in sensors, or field operation devices that assist fire-fighters in search and
rescue. These devices collect useful data with the help of various existing technologies and then
autonomously flow the data between other devices. Current market examples include smart
thermostat and washer/dryers. https://en.wikipedia.org/wiki/Internet_of_Things
7
IoT History
http://electronicdesign.com/embedded/understanding-protocols-behind-internet-things
8
Monitoring (1)
Protocol: There are many tools for collecting system data from hosts and devices using the SNMP
(Simple Network Management Protocol). Most computers and networked devices will have some
form of SNMP access. Interpretation of the SNMP data from a host or device requires either a
specialized tool (typically extra software from the vendor) or a Management information base (MIB), a
mapping of commands/data references to the various data elements the host or device provides.
The advantage of SNMP for monitoring is its low bandwidth requirements and universal usage in the
industries. Unless an application itself provides a MIB and output via SNMP, then SNMP is not suitable
for collecting application data. Other protocols are suitable for monitoring applications, such as
CORBA (language/OS-independent), JMX (Java-specific management and monitoring protocol), or
proprietary TCP/IP or UDP protocols (language/OS independent for the most part).
Data access: refers to the interface by which the monitor data can be utilized by other processes. For
example, if the system monitor is a CORBA server, clients can connect and make calls on the monitor
for current state of an element, or historical states for an element for some time period. The system
monitor may be writing data directly into a database, allowing other processes to access the
database outside the context of the system monitor. This is dangerous however, as the table design
for the database will dictate the potential for data-sharing. Ideally the system monitor is a wrapper for
whatever persistence mechanism is used, providing a consistent and 'safe' access interface for others
to access the data.
Mode: The data collection mode of the system monitor is critical. The modes are: monitor poll, agent
push, and a hybrid scheme. https://en.wikipedia.org/wiki/System_monitoring
9
Monitoring (2)
Monitor poll: In this mode, one or more processes in the monitoring system actually poll the system
elements in some thread. During the loop, devices are polled via SNMP calls, hosts can be accessed
via Telnet/SSH to execute scripts or dump files or execute other OS-specific commands, applications
can be polled for state data, or their state-output-files can be dumped.
The advantage of this mode is that there is little impact on the host/device being polled. The host's
CPU is loaded only during the poll. The rest of the time the monitoring function plays no part in CPU
loading. The main disadvantage of this mode is that the monitoring process can only do so much in its
time. If polling takes too long, the intended poll-period gets elongated.
Agent push: In agent-push mode, the monitored host is simply pushing data from itself to the system
monitoring application. This can be done periodically, or by request from the system monitor
asynchronously. The advantage of this mode is that the monitoring system's load can be reduced to
simply accepting and storing data. It doesn't have to worry about timeouts for SSH calls, parsing OS-
specific call results, etc.
The disadvantage of this mode is that the logic for the polling cycle/options are not centralized at the
system monitor, but distributed to each remote node. Thus changes to the monitoring logic must be
pushed out to each node. Also, in agent-based monitoring, a host cannot inform that it is completely
"down" or powered off, or if an intermediary system (such as a router) is preventing access to the
system.
Hybrid mode: The median mode between 'monitor-poll' and 'agent-push' is a hybrid approach, where
the system configuration determines where monitoring occurs, either in the system monitor or agent.
10
Node-Red
Internet of Things solutions often require pulling together hardware devices, APIs
and online services in new and interesting ways. Time spent writing the boilerplate
code to talk to these different systems, such as accessing a Serial port or complete
an OAuth flow against Twitter, is not time spent on creating the real value of the
solution. We need tools that make it easier for developers at all levels to bring
together the different streams of events, both physical and digital, that make up
the Internet of Things. Node-RED provides a browser-based UI for creating flows of
events and deploying them to its light-weight runtime. With built in nodeJS, it can
be run at the edge of the network or in the cloud. The node package manager
(npm) ecosystem can be used to easily extend the palette of nodes available,
enabling connections to new devices and services.
Node-Red in its simplest form is an open source visual editor for wiring the internet
of things produced by IBM. What does this mean. Well for someone lazy like me it
means I can spend more time making stuff "talk" to each other than worrying
about all of the interfacing code I will need to write. The system contains "Nodes"
which look simply to be icons that you drag and drop on to the canvas and wire
together. Each Node offers different functionality which can range from a simple
debug node to be able to see what's going on in your flow, through to a
Raspberry Pi node which allows you to read and write to the GPIO pins of your Pi.
http://nodered.org
11
Example 1: Mail Data Collect
12
Example 2: Twitter Data Collect
13
Example 2: Mail and Twitter Node-Red Flow
14
Example 3: MQTT broker data in Node-Red
15
Example 3: MQTT broker Node-Red Flow
16
Example 3: MQTT broker HiveMQ Stats
Node-RED provides a browser-based flow editor that
makes it easy to wire together flows using the wide
range nodes in the palette. Flows can be then
deployed to the runtime in a single-click. JavaScript
functions can be created within the editor using the
a rich text editor. A built-in library allows you to save
useful functions, templates or flows for re-use.
The light-weight runtime is built on nodeJS, taking full
advantage of its event-driven, non-blocking model.
This makes it ideal to run at the edge of the network
on low-cost hardware such as the Raspberry Pi as
well as in the cloud.
With over 120,000 modules in Node's package
repository, it is easy to extend the range of palette
nodes to add new capabilities.
The flows created in Node-RED are stored using JSON
which can be easily imported and exported for
sharing with others.
An online flow library allows you to share your best
flows with the world.
PARTIE II : EMULATION
18
Philips Hue Emulation
Philips hue combines brilliant LED light with intuitive technology. Then puts it in the
palm of your hand.
Together, the bulbs, the bridge and the app will change the way you use light.
Forever. Experiment with shades of white, from invigorating blue/white to cozy
yellow/white. Or play with all the colors in the spectrum.
Hue can wake you up. Help protect your home. Relive your favorite memories.
Improve your mood. Even keep you informed about the weather.
The wireless LED light bulbs bring hue to life. They deliver clear and crisp light. All
shades of white. And every color in the spectrum.
They don’t just look great. They’re practical too. They dim. They flash. They pulse.
They do pretty much anything you want. And they screw straight into your current
light fittings. Naturally.
Hue light bulbs are available in E26, E27, GU10, PAR16 and BR30 formats. Or for
something a bit different, try LivingColors Bloom, Iris and LightStrips.
The heart of the hue system. The bridge is literally a bridge between your app and
the bulbs. Linked to wi-fi via your router, it can connect up to 50 bulbs at a time.
Not to mention hue’s other accessories, like hue tap.
The bridge also connects your system to the wider world. With it you can control
your lights remotely or link them up to the rest of the web, newsfeeds or even your
own inbox. It’s really rather smart. http://www2.meethue.com/en-us
19
Philips Hue Emulation - Features
The key to controlling hue lies in the palm of your hand. Use the app to change colors, brightness,
functions and everything else in between. However you want it.
Control your bulbs one at a time or all together. Find the right intensity and shade of white for the task
at hand. Pick that perfect tone to match the moment. Or select a color from a photo and recreate it
in your room. You don’t have to be at home to control your lights. The Myhue portal is an online
control panel that connects you to hue, from anywhere in the world.
Make it look like someone’s home when you’re relaxing on the beach. Or just switch your lights off if
you’ll be home later than you thought. The portal also saves your personalized scenes and Light
Recipes. And it even keeps your bridge software up-to-date. Lighting is about more than just
illuminating the dark. It doesn’t take an expert to notice bad light. And the right light can transform
entire spaces. Even change the way you feel. With hue, you get full control over the light in your life.
Set the mood. Change the ambience. Feel different. This is what hue can do. With tunable white light
and a full spectrum of color, all at the tap of an app, you can set the perfect tone and enhance a
moment. All in an instant. Pick a photo – from the app or your own library - to use as your palette. Then
simply drag the color picker across the image and your bulbs will match the color instantly. Easy.
So now you can relive that glorious sunset. Or bring the calm of the ocean straight into your living
room. Whatever you feel like. Like your creation. Then save it as a ‘scene’ to use again. Really like it?
Then show off a bit and share it with the hue community.
Scenes can be anything from a single bulb’s color, to a whole house bathed in light. And everything in
between. It’s all up to you.
20
Philips Hue Emulation - Emulator
21
Philips Hue Emulation – GET Status Request
22
Philips Hue Emulation – PUT Request of color settings
23
Philips Hue Emulation – PUT Request Response
24
Philips Hue Emulation – Change color results
25
Sensor Emulation
IBM IoT Foundation
A new service just went live which addresses the key challenge of bringing the
world of sensors and controllers with that of big data and analytics. When
combined with the IBM Bluemix platform, IBM Internet of Things Foundation
provides simple, but powerful, and scalable application access to devices and
their data. You can rapidly build applications, visualization dashboards, mobile
apps, and applications that can feed your back-end enterprise with IoT data.
IBM Internet of Things Foundation is a fully managed, cloud-hosted service that
makes it simple to derive value from Internet of Things (IoT) devices. We start with
your device, be it a sensor, a gateway or something else. Using our recipes you
can get it connected and start sending data securely up to the cloud using the
open, lightweight MQTT messaging protocol. From there, you can setup and
manage your devices using your online dashboard or our secure APIs, so that your
apps can access live and historical data fast.
You are now ready to start creating applications using your device data. You
could do this within our IBM Bluemix platform, another cloud or your own servers. If
you aren’t convinced that these things are sending data via our service, then run
the two on separate machines or mobile devices. To do this all you need to do is
enter the device identifier from one to the other.
http://www-03.ibm.com/software/products/en/internet-of-things-foundation
26
Sensor Emulation - Features
Connect: Register and connect a wide variety of devices to the service.
Uses industry-standard MQTT, MQ Telemetry (MQTT) protocol (OASIS ratified).
Remotely monitor the connectivity of devices.
Securely connect many popular development boards and devices to the cloud using device recipes.
Extend to new devices with open-source device code from Eclipse Paho.
Collect: Collect and manage a time-series view of data.
See what is happening on your devices with near-real time IoT data visualization.
Store your IoT device events in a time-series database for analysis.
Build dashboards and gain access to historical data through APIs.
Control and secure data currently trapped on instrumented devices.
Assemble: Visually assemble events from the IoT into logic flows.
Use Node-Red in Bluemix for easy drag-and-drop flow assembly.
Analyze data and act on insights in near real time, applying them where they matter most.
Optimize business results with near real-time decision making.
Manage: Pay for what you use with purchase options available through Bluemix—pay through the IBM
Cloud marketplace or as a term subscription.
Hosted on IBM SoftLayer®, 24x7 operation, managed and monitored by IBM.
Powered by proven IBM MessageSight and IBM Informix® technologies.
27
Sensor Emulation – IBM Apps and MAC Adresse
28
Sensor Emulation - Device Validation
29
Sensor Emulation – Node Red Apps
30
Sensor Emulation - Collect Data
31
Sensor Emulation - IBM virtualization
32
Sensor Emulation – IBM Interface Commands
33
Sensor Emulation – Logs for HDFS System
34
MQTT Emulation
If you aren’t convinced that these things are sending data via our service, then run
the two on separate machines or mobile devices. To do this all you need to do is
enter the device identifier from one to the other. With the simulator sending data
all is set to now feed the Node-RED flow in your starter application and the
instructions walk you through this change. MQTT (formerly MQ Telemetry Transport)
is a publish-subscribe based "light weight" messaging protocol for use on top of the
TCP/IP protocol. It is designed for connections with remote locations where a
"small code footprint" is required and/or network bandwidth is limited. The Publish-
Subscribe messaging pattern requires a message broker. The broker is responsible
for distributing messages to interested clients based on the topic of a message.
Andy Stanford-Clark and Arlen Nipper of Cirrus Link Solutions authored the first
version of the protocol in 1999.
The specification does not specify the meaning of "small code foot print" or the
meaning of "limited network bandwidth". Thus, the protocol's availability for use
depends on the context. In 2013 IBM submitted MQTT v3.1 to the OASIS
specification body with a charter that ensured only minor changes to the
specification could be accepted. MQTT-SN is a variation of the main protocol
aimed at embedded devices on non-TCP/IP networks, such as ZigBee.
Historically, the 'MQ' in 'MQTT' came from IBM's MQ message queuing product
https://en.wikipedia.org/wiki/MQTT
35
MQTT Emulation – MQTT Definition
line. However, queuing per se is not required to be supported as a standard feature in all situations.
MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight
messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable
networks. The design principles are to minimize network bandwidth and device resource requirements
whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles
also turn out to make the protocol ideal of the emerging “machine-to-machine” (M2M) or “Internet of
Things” world of connected devices, and for mobile applications where bandwidth and battery
power are at a premium.
Alternative protocols include the Advanced Message Queuing Protocol, the IETF Constrained
Application Protocol and XMPP.
There are several MQTT brokers available such as IBM Message Sight. They vary in their feature set and
some of them implement additional features on top of the standard MQTT functionality.
In the real world, there are a number of projects that implement MQTT.
Facebook Messenger. Facebook has used aspects of MQTT in Facebook Messenger. However, it is
unclear how much of MQTT is used or for what; Moreover it is to be noted that this is a phone
application not a sensor application.
IECC Scalable Delta Rail's latest version of their IECC Signaling Control System uses MQTT for
communications within the various parts of the system and other components of the signaling system.
It provides the underlying communications framework for a system that is compliant with the CENELEC
standards for safety-critical communications.
36
MQTT Emulation – Java Publisher Example
37
MQTT Emulation – Java Subscriber Example
38
MQTT Emulation – Stats with MQTT SPY
39
MQTT Emulation – Messages Broker
40
MQTT Emulation - HiveMQ Stats
41
GPS Emulation
Creating software for the Internet of Things seems to get easier every day, as new
development tools make popular programming languages work with more and
more connected devices. One of the latest additions is Cylon.JS, an open-source
JavaScript framework designed for controlling robots, embedded sensors and IoT
devices. JavaScript is already hugely familiar to web developers and other
programmers because it runs in just about every web browser and is used to make
websites interactive (think of the chat boxes on social media sites that let you send
and receive instant messages with friends, without reloading the page). It’s also
fairly easy to learn the basics, even for amateurs. Cylon.JS is built on nodeJS, a
JavaScript framework used for many web apps that provides common ground for
IoT applications—such as the ability to simultaneously send, receive and process
many messages from many sources. Named for the science fiction cyborgs of
Battlestar Galactica, Cylon.JS supports a wide variety of drones, robotic toys,
wearables, smart home devices, IoT prototyping and embedded computing
modules, and cloud-based analytics and messaging platforms. It lets programmers
create powerful interactions between users, devices and services with just a few
lines of code. For instance, you can set up a Pebble smartwatch to act as a
controller for “robots” (connected devices) like a Nest thermostat or Phillips Hue
lightbulb. Or you can build your own projects by wiring physical sensors to
development boards like Arduino or Tessel. http://postscapes.com/javascript-for-robots-a-iot-cylonjs
42
GPS Emulation – Cylon.JS Framework
For example, the cylon-api-http plugin offers a RESTful HTTP interface, along with server-sent events
that can be used to subscribe to device events.
Another API plugin cylon-api-socketio supports an interface for clients based on the popular socket.io
module. All API plugins interface directly with the MCP to fetch robots and information about them.
They are also capable of executing commands on Robots and Devices.
A Cylon robot is a collection of devices and connections, with the necessary glue to allow them to
coordinate together. A User will instantiate one or more robots through the MCP, and either start them
all simultaneously or individually. Robots can also have custom commands (to coordinate multiple
devices/connections) and can communicate with each other through the MCP.
Devices and Connections are an abstraction layer over drivers and adaptors, respectively. These
allow for a layer of indirection between Drivers/Adaptors and the Robot itself, and ensure the
interfaces can remain consistent.
Drivers and Adaptors are the meat of Cylon, and allow Cylon to communicate with devices and
services, and issue commands/receive events from them. These are implemented as part of Cylon
submodules (cylon-sphero, etc), and are platform specific.
Adaptors are in charge of connecting to platforms, no matter the medium. For example, the cylon-
sphero Adaptor class communicates over Bluetooth, while the cylon-leapmotion Adaptor talks to a
WebSocket server.
The Adaptor classes ensure that the Drivers are able to directly communicate with the connected
platform.
43
GPS Emulation – nodeJS Apps
44
GPS Emulation- Node Red Flow
45
GPS Emulation – Node Red Data Transfert and Debug
46
GPS Emulation Process
47
GPS Emulation – Maps Representation
48
Trafic Cars Emulation
IBM IoT Foundation
The Connected Car starter kit for Bluemix allows you to easily model real-time
traffic in a city. The kit comes with a nodeJS vehicle simulator, an HTML5
visualization web app, and a control client that all communicate through the IBM
Internet of Things (IoT) Foundation service for real-time messaging.
The diagram to the right provides an overview of how the kit works. Simulated
vehicles are registered with IoT Foundation, and API keys are generated for each
application in the kit.
The authorization tokens allow for messaging between devices, applications, and
services.
The starter kit can be set up and deployed in Bluemix with the following steps: Set
up your IoT Foundation account: register your simulated vehicles and generate an
API key for your HTML5 applications.
Create a placeholder application and IoT/Geospatial services in Bluemix.
Download and configure the starter kit to match your Bluemix and IoT
environment.
Deploy the starter kit to Bluemix using the Cloud Foundry CLI
Use Node-RED (or build your own application) to extend the starter kit
http://m2m.demos.ibm.com/trafficsimulator.html
49
Trafic Cars Emulation – YAML Config
50
Trafic Cars Emulation – Settings Config
51
Trafic Cars Emulation – nodeJS Apps
52
Trafic Cars Emulation – Cloud-foundry CLI Commands
The Internet of Things already connects billions of
devices, with forecasts predicting steep growth rates
in the coming years. Many of these devices, such as
smartphones and connected vehicles, are mobile.
Awareness of the location of on-the-move devices
opens up exciting new application opportunities.
Support for these new applications requires highly
scalable services that can analyze high volumes of
data in real time. With the Geospatial Analytics service
in IBM Bluemix, you can monitor moving devices from
the Internet of Things. The service tracks device
locations in real time with respect to one or more
geographic regions. Geospatial Analytics can be used
as a building block in applications that support several
use cases. For example, a retail business might want to
monitor for potential customers nearby its stores and
send them promotions via push notifications or tweets.
Geospatial Analytics could also be used to detect
connected motorists entering areas that they might
want to avoid due to an accident, weather…
53
Trafic Cars Emulation - IoT IBM Interface
54
Trafic Cars Emulation – Payload Messages
55
Trafic Cars Emulation – Node Red integration
56
Trafic Cars Emulation – Node Red Data Transfert and Debug
57
Trafic Cars Emulation- Process Stats
PARTIE III : DEMONSTRATION
IOT BIG DATA
ANALYTICS
DEVICES PROTOCOLE HARDWARE
ARCHITECTURE « IOT (M2M) - BIG DATA – ANALYTICS »
M2M
IoT (M2M) – Big Data – Analytics : Architecture
The initial contribution of the Eclipse SmartHome project will be a large
subset of the current code of openHAB (open Home Automation Bus). In
particular: Source code for all mentioned interfaces and services of the
framework. Source code of a couple of extensions (openHAB bindings,
actions and persistence services) Documentation of the architecture
and its extension points. Build scripts and sample files.
To enable others to use code from openHAB in their own solutions and
products, we wanted to put the code base on a solid and trustworthy
foundation. The legal aspects of many open-source projects are often
pretty fuzzy, even if they have a permissive license. But for companies
using such project in their own products, there is always the risk that the
copyright of the code is not clear and that the code might be
contaminated by patents. To reduce such risks, it is very beneficial to
have a rigid intellectual property management and clear contribution
processes - this is one of the things that a foundation such as Eclipse
provides. We therefore decided to contribute the core framework of
openHAB to the Eclipse Foundation, which became the new Eclipse
SmartHome project.
60
Software IOT
http://kaikreuzer.blogspot.com/2014/06/openhab-20-and-eclipse-smarthome.html
61
Software IoT - History
The Eclipse SmartHome project is a proposed open source project under the Eclipse Technology Top-
Level Project. This proposal is in the Project Proposal Phase (as defined in the Eclipse Development
Process) and is written to declare its intent and scope. We solicit additional participation and input
from the Eclipse community. Please send all feedback to the Eclipse Proposals Forum.
Since the emergence of broadband internet connections, smartphones and tablets the smart home
market shows a remarkable upsurge. This has led to a very fragmented market, which makes it difficult
for customers to "bet on the right horse". In fact, there is not one system, protocol or standard that
could possibly fulfill all potential requirements. There is hence a need for platforms that allow the
integration of different systems, protocols or standards and that provide a uniform way of user
interaction and higher level services. The goals of the Eclipse SmartHome project can be summarized
as: Provide a flexible framework for smart home and ambient assisted living (AAL) solutions. This
framework focuses on the use cases of this domain, e.g. on easy automation and visualization
aspects. Specify extension points for integration possibilities and higher-level services. Extending and
thus customizing the solution must be as simple as possible and this requires concise and dedicated
interfaces. Provide implementations of extensions for relevant systems, protocols or standards. Many of
them can be useful to many smart home solutions, so this project will provide a set of extensions that
can be included if desired. They can also be in the shape of a general Java library or an OSGi bundle,
so that these implementations can be used independently of the rest of the project as well.
Provide a development environment and tools to foster implementations of extensions. The right
tooling can support the emergence of further extensions and thus stimulates future contributions to
62
Software IoT - Features
the project. Create a packaging and demo setups. Although the focus is on the framework, it needs
to be shown how to package a real solution from it, which can be used as a starting point and for
demo purposes. The Eclipse SmartHome project is a framework that allows building smart home
solutions that have a strong focus on heterogeneous environments, i.e. solutions that deal with the
integration of different protocols or standards. Its purpose is to provide a uniform access to devices
and information and to facilitate different kinds of interactions with them. This framework consists out
of a set of OSGi bundles that can be deployed on an OSGi runtime and which defines OSGi services
as extension points. The stack is meant to be usable on any kind of system that can run an OSGi stack
- be it a multi-core server, a residential gateway or a Raspberry Pi.
Currently planned initial features and extension points of this framework include:
Type System: A basic but extensible type system for smart home data and commands that provides a
common ground for an abstracted data and device access.
Data Registry: Keeps track of data point instances and their values.
Event Mechanism: Passes data and operations asynchronously in a loosely coupled way between
components. The Infrastructure to exchange data and commands with external systems and devices,
e.g. through regular polling or background threads. Useful binding implementations such as HTTP, NTP
or TCP will be part of the project. An implementation of a flexible rule engine that allows changing
rules during runtime. Provides triggering upon events or on a temporal basis.
Extensible through rule actions that are made available to all rules. Simple but powerful textual
representation of rules Declarative User Interfaces.
63
openHAB Framework – Architecture (1)
The openHAB project is split into two parts:
openhab-runtime: This is the package, which you will actually run on your server and which does the
"real" work.
* penhab-designer:
This is a configuration tool for the openhab-runtime. It comes with user-friendly editors to configure
your runtime, to define your UIs and to implement your rules.
* openHAB Runtime:
The openHAB runtime is a set of OSGi bundles deployed on an OSGi framework (Equinox). It is
therefore a pure Java solution and needs a JVM to run. Being based on OSGi, it provides a highly
modular architecture, which even allows adding and removing functionality during runtime without
stopping the service. Here is an overview of the main bundles and how they depend on each other:
Communication
openHAB has two different internal communication channels:
an asynchronous event bus
a stateful repository, which can be queried
The Event Bus
The event bus is THE base service of openHAB. All bundles that do not require stateful behaviour
should use it to inform other bundles about events and to be updated by other bundles on external
events.
There are mainly two types of events:
64
openHAB Framework – Architecture (2)
Commands which trigger an action or a state change of some item/device.
Status updates which inform about a status change of some item/device (often as a response to a
command). All protocol bindings (which provide the link to the real hardware devices) should
communicate via the Event Bus. This makes sure that there is a very low coupling between the
bundles, which facilitates the dynamic nature of openHAB.
As a technical foundation, the OSGi EventAdmin service is used by openHAB. This is a light-weight
pub-sub implementation, which perfectly meets the requirements.
It is important to note that openHAB is not meant to reside on (or near) actual hardware devices
which would then have to remotely communicate with many other distributed openHAB instances.
Instead, openHAB serves as an integration hub between such devices and as a mediator between
different protocols that are spoken between these devices. In a typical installation there will therefore
be usually just one instance of openHAB running on some central server. Nonetheless, as the OSGi
EventAdmin service can also be used as a remote service, it is possible to connect several distributed
openHAB instances via the Event Bus.
Item Repository
Not all functionality can be covered purely by stateless services. Therefore openHAB also offers the
Item Repository, which is connected to the Event Bus and keeps track of the current status of all items.
The Item Repository can be used whenever needed to access the current state of items. E.g. a user
interface needs to display the current state of items in the moment of the user access. Also the
automation logic execution engine needs to always be informed about the current states.
65
openHAB Framework – Architecture (3)
The Item Repository avoids requiring each bundle to cache states themselves for their internal use. It
also makes sure that the state is in sync for all those bundles and it provides the possibility to persist
states to the file system or a database, so that they are even kept throughout a system restart.
The following diagram shows how these communication channels are used:
Sitemap
openHAB comes with a generic textual configuration for its user interfaces: The Sitemap. The Sitemap
is a tree structure of widgets, which define the different pages of a UI and their content. Widgets can
be associated to items, for which they should show the status and/or control elements.
The definition of the Sitemap is quite abstract by design; it is supposed to be a suitable UI model for
different kinds of user interfaces, so that the user does not have to configure each of them in case he
sets up multiple UIs. If a UI has further requirements on top of the sitemap, it is still possible to introduce
additional configuration options which are then specific for the UI in question.
Item UI Providers
Item UI providers offer a dynamic way to configure the UI, so that not everything must be stored
statically in the Sitemap. An item UI provider can for example define, what widget should be used for
an item, if none is specified and can dynamically define icons and labels (which might depend on
the current state of the item) for the widgets.
An important use case for this feature is the dynamic display of item groups - all that there is stored in
the Sitemap is the information that a certain group should be displayed; the page is then dynamically
assembled at runtime with whatever items are available at that time.
66
openHAB Framework – Commands (1)
67
openHAB Framework – Commands (2)
68
openHAB Framework – Commands (3)
69
openHAB Framework – Commands (4)
70
openHAB Framework – Webservices (1)
71
openHAB Framework – Webservices (2)
72
openHAB Framework – Webservices (3)
73
openHAB Framework – Webservices (4)
OpenHAB is aimed to provide a simple drop-in server where components can be connected and
driven from a user interface, whilst allowing a general mechanism to control all components. Although
many of these devices may have their own mobile applications, controlling each one of them
requires opening a mobile application and they cannot be integrated together. Using OpenHAB as
the main hub, the devices can all be controlled with one mobile application or website, and scripts
can be set up to drive individual components.
74
openHAB Designer
75
openHAB Designer – Geent Template Example
The openHAB Designer is an Eclipse RCP application for configuring the openHAB runtime. It comes
with editors for configuration files like the sitemap. Its advantage over simple text editors is the full IDE
support, like syntax checking, auto completion, highlighting and content assist. It is also meant to
implement and deploy rules for automatic actions.
Rather than reinventing the wheel, SiteWhere depends on many
supporting open source technologies to accomplish its goals. Most of
the technologies used have been tested for years in production
environments and have been continuously upgraded thanks to many
iterations of improvements. The following open source components are
used by SiteWhere:
Tomcat provides the core server in which SiteWhere runs. SiteWhere is
deployed as a web archive (WAR file) that runs when the Tomcat server
is started.
Spring delivers the core configuration framework that bootstraps
SiteWhere and allows it to be configured and extended. Rather than
using a hard-coded set of components, SiteWhere has a core set of
service provider interfaces and default implementations of the
interfaces. By using the same interfaces in custom classes and plugging
them in via Spring, a third parties can extend SiteWhere without touching
the core code.
MongoDB: MongoDB is a NoSQL database that can be used to store
SiteWhere data. It is very quick and easy to install, has great
performance, and does not require the computing resources of HBase.
76
Software M2M
http://docs.sitewhere.org/1.0.4/overview.html#what-is-sitewhere
77
Sitewhere - Software M2M (1)
SiteWhere is an open source M2M integration platform – a system that choreographs data coming
from and going to a web of connected devices. It provides the following functionality:
Supplies a server based on proven technologies that acts as a controller for the processing of device
data. The server can be installed on a local machine or run in the cloud and is built on technologies
that are designed to scale to billions of processed device events per day.
Provides long-term persistence of data sent from devices. Historical device event data is valuable and
SiteWhere offers a platform where data is never deleted, no matter the volume of events.
Offers service provider interfaces (SPIs) that provide a core object model for the platform and allow
third parties to extend and customize the system to work with new technologies.
Delivers an advanced device provisioning system that allows control of the full lifecycle of registering
devices, sending commands based on hardware type, receiving data responses and aggregating
them. The system is based on a core set of interfaces that allow new communications protocols and
encoding schemes to be added and configured easily.
Provides a model for standard types of data generated by devices. For instance, location data is
stored such that it can be geospatially indexed and accessed by queries like ‘which device events
happened near this given location in the given time span’. Device measurement data is stored such
that it can be analyzed, faceted, and graphed. Alerts for exceptional conditions can be captured
and indexed so that an audit trail is available after the fact.
Associates devices with external assets such as people or physical items. For instance, a badge may
be associated with the person wearing it. A tracking device can be associated with the piece of
78
Sitewhere - Software M2M (2)
heavy equipment it is attached to. The asset information is offered via an asset management
framework that allows external systems to drive the information. For instance, the list of people that
badges can be assigned to can come from an existing LDAP data store. SiteWhere tracks the
assignment of devices to assets over time and ties events to the assignment so at a later date you can
query ‘what were the locations of the badge when it was assigned to this person’.
Integrates with third party integration frameworks such as Mule ESB, allowing event data to trigger
complex interactions such as adding data to Salesforce or notifying alert conditions using text
messages generated by Twilio.
Provides an HTML5 administrative application that allows all of the system data to be viewed and
manipulated in a way that makes it easy to understand the data. The admin app uses REST services
provided by the core platform to interact with the data. Third party applications can use the same
REST services to interact with SiteWhere data without having to use the admin application.
Offers a Java client that can interact with most of the provided REST services. For external systems with
existing Java code, interfacing with SiteWhere is as simple as a few lines of code.
Protects information by limiting access to data based on a proven user management system. The
system can be configured to use external sources of identity data such as LDAP data stores.
Generates detailed analytics information from gathered device data and provides factilities for
executing advanced faceted queries on the data.
79
Sitewhere - MongoDB DataBase
80
Sitewhere – WebServices (1)
81
Sitewhere – WebServices (2)
Devices must communicate with each other (D2D). Device data then
must be collected and sent to the server infrastructure (D2S). That server
infrastructure has to share device data (S2S), possibly providing it back to
devices, to analysis programs, or to people. From 30,000 feet, the
protocols can be described in this framework as:
• MQTT • XMPP • DDS • AMQP
Each of these protocols is widely adopted. There are at least 10
implementations of each. Confusion is understandable, because the
high-level positioning is similar. In fact, all four claim to be real-time
publish-subscribe IoT protocols that can connect thousands of devices.
And it’s true, depending on how you define “real time,” “things,” and
“devices.” Nonetheless, they are very different indeed! Today’s Internet
supports hundreds of protocols. The IoT will support hundreds more. It’s
important to understand the class of use that each of these important
protocols addresses.
82
Protocoles & Positioning Systems
83
Protocle MQTT
MQTT (MQ Telemetry Transport) is a publish-subscribe based "light weight" messaging protocol for use
on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small
code footprint" is required and/or network bandwidth is limited. The Publish-Subscribe messaging
pattern requires a message broker. The broker is responsible for distributing messages to interested
clients based on the topic of a message. Andy Stanford-Clark and Arlen Nipper of Cirrus Link Solutions
authored the first version of the protocol in 1999. The specification does not specify the meaning of
"small code foot print" or the meaning of "limited network bandwidth". Thus, the protocol's availability
for use depends on the context. In 2013 IBM submitted MQTT v3.1 to the OASIS specification body with
a charter that ensured only minor changes to the specification could be accepted. MQTT-SN is a
variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as ZigBee.
Historically, the 'MQ' in 'MQTT' came from IBM's MQ message queuing product line. However, queuing
per se is not required to be supported as a standard feature in all situations.
Alternative protocols include the Advanced Message Queuing Protocol, the IETF Constrained
Application Protocol and XMPP.
MQTT Broker: There are several MQTT brokers available such as IBM Message Sight. They vary in their
feature set and some of them implement additional features on top of the standard MQTT
functionality. See MQTT.org for more information. In the real world, there are a number of projects that
implement MQTT. Facebook Messenger. Facebook has used aspects of MQTT in Facebook Messenger.
However, it is unclear how much of MQTT is used or for what; Moreover it is to be noted that this is a
phone application not a sensor application. https://en.wikipedia.org/wiki/MQTT
84
Protocle GSM
GSM (Global System for Mobile Communications, originally Group Special Mobile), is a standard
developed by the European Telecommunications Standards Institute (ETSI) to describe protocols for
second-generation (2G) digital cellular networks used by mobile phones, first deployed in Finland in
July 1992. As of 2014 it has become the default global standard for mobile communications - with over
90% market share, operating in over 219 countries and territories.
2G networks developed as a replacement for first generation (1G) analog cellular networks, and the
GSM standard originally described a digital, circuit-switched network optimized for full duplex voice
telephony. This expanded over time to include data communications, first by circuit-switched
transport, then by packet data transport via GPRS (General Packet Radio Services) and EDGE
(Enhanced Data rates for GSM Evolution or EGPRS).
Subsequently, the 3G++ developed third-generation (3G) UMTS standards followed by fourth-
generation (4G) LTE Advanced standards, which do not form part of the ETSI GSM standard.
"GSM" is a trademark owned by the GSM Association. It may also refer to the (initially) most common
voice codec used, Full Rate. GSM Layer 1: FDMA/TDMA is the air interface(radio), also called Um
interface. For more on this interface refer GSM tutorial in tutorials section. At MS, FDMA/TDMA is used
which is also followed at BTS, BTS takes this format from MS and convert it to 64kbps digital format for
the digital link and interfaces with BSC. BSC communicates with MSC in the same format. GSM Layer 2:
Layer 2 is the data link layer, which does following three main functions.
- Establish, maintain and tear down the link - Flow control - Error detection - Work on the Layer-3 frames
https://en.wikipedia.org/wiki/GSM
85
GPS - Global Positioning System
The Global Positioning System (GPS) is a space-based satellite navigation system that provides
location and time information in all weather conditions, anywhere on or near the earth where there is
an unobstructed line of sight to four or more GPS satellites. The system provides critical capabilities to
military, civil, and commercial users around the world. The United States government created the
system, maintains it, and makes it freely accessible to anyone with a GPS receiver.
The US began the GPS project in 1973 to overcome the limitations of previous navigation systems,
integrating ideas from several predecessors, including a number of classified engineering design
studies from the 1960s. The U.S. Department of Defense (DoD) developed the system, which originally
used 24 satellites. It became fully operational in 1995. Bradford Parkinson, Roger L. Easton, and Ivan A.
Getting are credited with inventing it.
Advances in technology and new demands on the existing system have now led to efforts to
modernize the GPS system and implement the next generation of GPS Block IIIA satellites and Next
Generation Operational Control System (OCX). Announcements from Vice President Al Gore and the
White House in 1998 initiated these changes. In 2000, the U.S. Congress authorized the modernization
effort, GPS III.
In addition to GPS, other systems are in use or under development. The Russian Global Navigation
Satellite System (GLONASS) was developed contemporaneously with GPS, but suffered from
incomplete coverage of the globe until the mid-2000s. There are also the planned European Union
Galileo positioning system, India's Indian Regional Navigation Satellite System, and the Chinese BeiDou
Navigation Satellite System. https://en.wikipedia.org/wiki/Global_Positioning_System
Raspberry PI – Mobile - SIM CARD –
GPS TRACKER – LG SMART TV –
GOOGLE MAPS – LED LIGHTING –
CARS TRACKER – OPENHAB
FRAMEWORK – OSGI MODULE –
SITEWHERE - THERMOMETER
86
Plateforme IoT
87
Application : materials
88
YOUTUBE CHANNEL : INNOVAIOT
Application : Movie Demonstration
INNOVA_IOT
INNOVAIOT
INNOVAIOT
Big data analytics is the process of examining large data sets containing
a variety of data types. Big data, unknown correlations, market trends,
customer preferences and other useful business information. The
analytical findings can lead to more effective marketing, new revenue
opportunities, better customer service, improved operational efficiency,
competitive advantages over rival organizations and other benefits.
To analyze such a large volume of data, big data analytics is typically
performed using specialized software tools and applications for
predictive analytics, data mining, text mining, forecasting and data
optimization. Collectively these processes are separate but highly
integrated functions of high-performance analytics. Using big data tools
and software enables an organization to process extremely large
volumes of data that a business has collected to determine which data
is relevant and can be analyzed to drive better business decisions in the
future.
89
Big Data and Analytics
90
Big Data Analytics – Uses cases
But behavioral analytics can be more than just tracking people. It’s also about understanding the
interactions and dynamics between processes, machines and equipment, even macroeconomic
trends, which may yield new learning into operational risks and opportunities. When combined with
rich insights about consumer behavior, companies gain a more nuanced, detailed and accurate
picture of where markets are headed. And that gives them a huge advantage in directing to
proactive investments to seize the best opportunities. The largest of those opportunities maybe
market-facing or customer-centric, but supply chain improvements or other operational
enhancements first discovered by behavioral analytics may also lead to worthwhile performance
gains. Behavioral analytics has real applications beyond the realm of marketing and customer
intelligence. There is sensor data that can track traffic patterns, for instance, leading to “smarter cities.
It’s possible to know if cold storage chains have been broken or medications have been
compromised during shipments. Such detailed and up-to-the-second views of goods moving across
global supply chains are ultimately another flavor of behavioral analytics. As with big data more
generally, the issue isn’t whether companies have the data they need to do behavioral analytics. It’s
more that they need to determine the optimal strategies to collect, manage and analyze in a
scalable and accessible way the right data to gain better insights into how people – not to mention
processes, equipment and other assets – behave.
It is important to note that big data leaders have long since deployed those strategies and
established strong behavior analytics capabilities. It’s likely not a coincidence while such companies
also tend to be market leaders. http://bigdata.teradata.com/US/Big-Ideas/Behavioral-Analytics
Jaspersoft is a commercial open source software vendor focused on business
intelligence, including data visualization, reporting, and analytics. Jaspersoft
provides commercial and open source software, support, services, and licensing
around the JasperReports, JasperReports Server, Jaspersoft Studio, iReport, and
Jaspersoft ETL products. Jaspersoft's main related product is JasperReports Server,
a Java web application that provides advanced report server capabilities such as
report scheduling, permissions, ad hoc reporting, dashboards, and multi-tenancy.
It is offered in a community open source edition as well as under several
commercial editions with broad support for various databases and data sources,
including NoSQL and other big data datasources.
Jaspersoft was originally called Panscopic, and was founded by Al Campa, CEO,
and Raj Bhargava, VP of Products in 2001. Panscopic raised $23M from Doll
Capital, Discovery Ventures, Morgenthaler Ventures, and Partech. In 2004
Panscopic teamed up with Teodor Danciu, acquired the intellectual property of
JasperReports, and changed the name of the company to Jaspersoft. Brian
Gentile became CEO in 2007. Teodor Danciu began work on JasperReports in
June 2001, the sf.net project was registered in September 2001and JasperReports
0.1.5 was released November 3, 2001. The company released JasperReports
Version 1.0, July 21, 2005. The code was originally licensed under a copyleft
JasperReports License.
91
JasperSoft – Analytics and Reporting
https://en.wikipedia.org/wiki/Jaspersoft
92
JasperSoft – MongoDB Analytics
93
JasperSoft – Data Sources
94
JasperSoft – MongoDB Query Langage
95
Elasticsearch
Elasticsearch is a highly scalable open-source full-text search and analytics
engine. It allows you to store, search, and analyze big volumes of data quickly and
in near real time. It is generally used as the underlying engine/technology that
powers applications that have complex search features and requirements.
Here are a few sample use-cases that Elasticsearch could be used for:
You run an online web store where you allow your customers to search for
products that you sell. In this case, you can use Elasticsearch to store your entire
product catalog and inventory and provide search and autocomplete
suggestions for them. You want to collect log or transaction data and you want to
analyze and mine this data to look for trends, statistics, summarizations, or
anomalies. In this case, you can use Logstash (part of the Elasticsearch, Logstash,
Kibana stack) to collect, aggregate, and parse your data, and then have
Logstash feed this data into Elasticsearch. Once the data is in Elasticsearch, you
can run searches and aggregations to mine any information that is of interest to
you. You run a price alerting platform which allows price-savvy customers to
specify a rule like "I am interested in buying a specific electronic gadget and I
want to be notified if the price of gadget falls below $X from any vendor within the
next month". In this case you can scrape vendor prices, push them into
Elasticsearch and use its reverse-search (Percolator) capability to match price
movements against customer queries. https://www.elastic.co
96
Elasticsearch - Features
Real-Time Data: How long can you wait for insights on your fast-moving data. With Elasticsearch, all
data is immediately made available for search and analytics.
Real-Time Analytics: Combining the speed of search joins with the power of analytics changes your
relationship with your data. Interactively search, discover, and analyze to gain insights that improve
your products or streamline your business.
Distributed: Elasticsearch allows you to start small and scale horizontally as you grow. Simply add more
nodes, and let the cluster automatically take advantage of the extra hardware.
High Availability: Elasticsearch clusters are resilient — they will detect new or failed nodes, and
reorganize and rebalance data automatically, to ensure that your data is safe and accessible.
Multitenancy: A cluster may contain multiple indices that can be queried independently or as a
group. Index aliases allow filtered views of an index, and may be updated transparently to your
application.
Full-Text Search: Elasticsearch builds distributed capabilities on top of Apache Lucene to provide the
most powerful full- text search capabilities available in any open source product. Powerful, developer-
friendly query API supports multilingual search, geolocation, contextual did-you-mean suggestions,
autocomplete, and result snippets.
Document-Oriented: Store complex real world entities in Elasticsearch as structured JSON documents.
All fields are indexed by default, and all the indices can be used in a single query.
Schema-Free, Developer-Friendly, RESTful API, Per-Operation Persistence, Apache 2 Open Source
License, Conflict Management…
97
Elasticsearch – Indexation Example
98
Elasticsearch – MongoDB River Indexation
99
Elasticsearch – MongoDB Replication
100
Elasticsearch – MongoDB Index Status
101
Elasticsearch – MongoDB Collection
102
Elasticsearch –MongoDB Collection Search with index
103
Elasticsearch – index Stats
104
Elasticsearch – Cluster Stats
105
Apache Hadoop
Apache Hadoop is an open-source software framework written in Java for
distributed storage and distributed processing of very large data sets on computer
clusters built from commodity hardware. All the modules in Hadoop are designed
with a fundamental assumption that hardware failures (of individual machines, or
racks of machines) are commonplace and thus should be automatically handled
in software by the framework. The core of Apache Hadoop consists of a storage
part (Hadoop Distributed File System (HDFS)) and a processing part (MapReduce).
Hadoop splits files into large blocks and distributes them amongst the nodes in the
cluster. To process the data, Hadoop MapReduce transfers packaged code for
nodes to process in parallel, based on the data each node needs to process. This
approach takes advantage of data locality nodes manipulating the data that
they have on hand to allow the data to be processed faster and more efficiently
than it would be in a more conventional supercomputer architecture that relies on
a parallel file system where computation and data are connected via high-speed
networking. The base Apache Hadoop framework is composed of the following
modules: Hadoop Common – contains libraries and utilities needed by other
Hadoop modules; Hadoop Distributed File System (HDFS) – a distributed file-system
that stores data on commodity machines, providing very high aggregate
bandwidth across the cluster; Hadoop YARN and Hadoop MapReduce
programming model. https://en.wikipedia.org/wiki/Apache_Hadoop
106
Apache Hadoop – User Interface
107
Apache Hadoop – HDFS Example
The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity
hardware. It has many similarities with existing distributed file systems. However, the differences from
other distributed file systems are significant. HDFS is highly fault-tolerant and is designed to be
deployed on low-cost hardware. HDFS provides high throughput access to application data and is
suitable for applications that have large data sets. HDFS relaxes a few POSIX requirements to enable
streaming access to file system data. HDFS was originally built as infrastructure for the Apache Nutch
web search engine project.
108
Apache Hadoop – Log File example for Querying
109
Apache Hadoop – Log Files in HDFS Format
110
Apache Hadoop – Log Files Status in Hadoop
111
Apache Hadoop – Apache Hive combine
The Apache Hive data warehouse software facilitates querying and managing large datasets residing
in distributed storage. Hive provides a mechanism to project structure onto this data and query the
data using a SQL-like language called HiveQL. At the same time this language also allows traditional
map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or
inefficient to express this logic in HiveQL.
112
Apache Hive – Hive Query Langage
DROP TABLE: delete the table and the file of data, if the table already exists.
CREATE EXTERNAL TABLE: create a new external table in Hive. The external tables store only the
definition of table in Hive. The data stay in their location of origin and in their size of origin.
ROW FORMAT: indicate to Hive the mode of formatting of the data. In this case, the fields of every
newspaper are separated by a space.
STORED AS TEXTFILE LOCATION: indicate to Hive the location of the data (the directory example /
data) and specifies that they are stored under the shape of text. The data can be in a single file or
repartees on several files in the directory.
SELECT: select and count all the lines the column of which t10 contains the value ‘host=chaker’.
113
Apache Hive– Request results (1)
114
Apache Hive– Request results (2)
115
Apache Hive– Request results (3)
116
Hadoop and Hive in Big Data
Apache Hive is a data warehouse system build on top of Hadoop to query Big Data. Hive originated
at Facebook and was open sourced in August 2008. The challenge Facebook had to address is one
faced by many companies since then. Eventually data growth in a company challenges the
capabilities of deployed RDBMS or NoSQL systems. Reports and analytics start to take minutes, then
hours, and eventually overlap with other queries and the whole system grinds to a halt. Another
common scenario companies start processing big data with Hadoop discovers the value of making
the data accessible beyond the development team capable of writing complex map-reduce jobs.
The term Big Data is freely used in this context. A colleague defined Big Data jokingly as anything
beyond one million rows – Microsoft Excels row limit. The underlying point is that Big Data is a point of
view and can be generalized as the point where simple solutions and deployed technology fail.
The subsequent question is if scaling and investing heavily in it is the most economical solution.
Commercial large-scale data warehouse solutions are very expensive. Furthermore, some of the data
collected today, e.g. poorly structured or highly denormalize data, can be impractical to manage
with these systems. The Hadoop ecosystem regularly is utilized to scale data processing in a feasible
manner. Hadoop becomes either a replacement or a batch process addition to the existing
infrastructure for data analysis, extraction, loading, transformation, reporting, and machine learning.
The downside, which Facebook encountered, was that data stored in Hadoop is inaccessible to
business users. There are higher-level languages like Pig, Cascading, Crunch or Scalding. They are,
however, geared towards software developers that want to avoid the verbosity of pure Java map-
reduce development.
Green Internet of Things
The term "Internet of Things" emanated to describe a number of technologies and
research disciplines that enable global connectivity over the world-wide physical
objects. Enabling technologies like RFID, sensor networks, biometrics, and
nanotechnologies are now becoming very common, bringing the Internet of Things
into real implementations addressing varying applications, including smart grid, e-
health, intelligent transportation, etc. They foreshadow an exciting future that closely
interconnects our physical world via green networks. Green networks in IoT will
contribute to reduce emissions and pollutions, exploit environmental conservation and
surveillance, and minimize operational costs and power consumption. The Green
Internet of Things (G-IoT) is predicted to introduce significant changes in our daily life
and would help realizing the vision of “green ambient intelligence”. Within a few years
we will be surrounded by a massive amount of sensors, devices and “things”, which
will be able to communicate via IP, act “intelligently”, and provide green support for
users in managing their tasks. These new smart objects will also be context-aware and
able to perform certain functions autonomously, calling for new forms of green
communication between people and things and between things themselves, where
power consumption is optimized and bandwidth utilization is maximized. This
development would not only be relevant to researchers, but also to corporations and
individuals alike. It is hence the aim of our workshop to focus on both design and
implementation aspects in green networks, or networks that can be utilized in
providing green systems through IoT enabling technologies. At the same time, our
workshop call is open for topics relevant to IoT in general.
http://research.cs.queensu.ca/G-IoT12
Keywords
IoT – M2M Big Data Analytics
Internet ofThings, Machine 2
Machine, MQTT, Broker, Message
Elasticsearch,Apache Hadoop,
Apache Hive, MongoDB
JasperSoft Studio, MongoDB,
Apache Cassandra, MySQL, JSON
Node-Red, nodeJS,Webservices,
Maps, google maps, GPSTracker,
cars trafic, thermostat connecté,
adresse MAC, Device registration
Elasticsearch index, MongoDB river,
Marvel sense, Elasticsearch HQ
Kibana analytic, Logs analytics
HiveMQ,ApacheTomcat, MQTT
Spy, Cylon.JS, Cloud-foundry Cli
commands, openHAB, openHAB
Designer, GreentTemplate,
Sitewhere
MapReduce,Apache Hive, Hive
Query Langage, MongoDB river,
HDFS, MongoDB Query Langage,
Replication, Collection, elasticsearch
index, cluster
Hadoop UI, Monitoring, Protocol,
Data access, Monitor poll, Hybrid
mode
Node-Red Flow,API Keys,APIToken,
Collect Data, Data source,Twitter,
Mail,Webservices, Maps, google
maps, GPSTracker, cars trafic,
thermostat connecté, adresse MAC,
Device registration, MQTT Publisher,
MQTT Subscriber
Mqtt Broker, Publisher, Subscriber,
GET Request, PUT Request, JSON,
YAML
HiveMQ,ApacheTomcat, MQTT
Spy, Cylon.JS, Cloud-foundry Cli
commands, openHAB, openHAB
Designer, GreentTemplate,
Sitewhere
Links
 Apache Haddop : https://hadoop.apache.org
 IBM Bluemix : https://console.ng.bluemix.net
 Elasticsearch : https://www.elastic.co
 openHab : http://www.openhab.org
 Sitewhere : http://www.sitewhere.org
 Jaspersoft : http://www.jaspersoft.com
 Node-red : http://nodered.org
 MongoDB : https://www.mongodb.org
 Apache Tomcat : http://tomcat.apache.org
 Apache Hive : https://hive.apache.org
 nodeJS : https://nodejs.org
Visit my profile in LinkedIn
Visit my website
http://tn.linkedin.com/in/chakerallaoui
http://allaoui-chaker.github.io
Contact
Idea Create Refine

More Related Content

What's hot

Managing your Assets with Big Data Tools
Managing your Assets with Big Data ToolsManaging your Assets with Big Data Tools
Managing your Assets with Big Data ToolsMachinePulse
 
Tec118 Teched2015 IOT use case and examples
Tec118 Teched2015 IOT use case and examplesTec118 Teched2015 IOT use case and examples
Tec118 Teched2015 IOT use case and examplesMarkus Van Kempen
 
The competitive landscape of the Internet of Things
The competitive landscape of the Internet of ThingsThe competitive landscape of the Internet of Things
The competitive landscape of the Internet of ThingsIoTAnalytics
 
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a PalaceInternet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a PalaceDr.-Ing Abdur Rahim Biswas
 
Key Data Management Requirements for the IoT
Key Data Management Requirements for the IoTKey Data Management Requirements for the IoT
Key Data Management Requirements for the IoTMongoDB
 
Proof of concepts and use cases with IoT technologies
Proof of concepts and use cases with IoT technologiesProof of concepts and use cases with IoT technologies
Proof of concepts and use cases with IoT technologiesHeikki Ailisto
 
Internet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M CommunicationInternet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M CommunicationSherin C Abraham
 
Internet of Things Stack - Presentation Version
Internet of Things Stack - Presentation VersionInternet of Things Stack - Presentation Version
Internet of Things Stack - Presentation VersionPostscapes
 
IoT Strategy Pillars
IoT Strategy PillarsIoT Strategy Pillars
IoT Strategy PillarsIBTECAR
 
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術ハイシンク創研 / Laboratory of Hi-Think Corporation
 
Data Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsData Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsCory Andrew Henson
 
Device to Intelligence, IOT and Big Data in Oracle
Device to Intelligence, IOT and Big Data in OracleDevice to Intelligence, IOT and Big Data in Oracle
Device to Intelligence, IOT and Big Data in OracleJunSeok Seo
 
Industrial internet of things (IIOT) - special report-2017
Industrial internet of things (IIOT) - special report-2017Industrial internet of things (IIOT) - special report-2017
Industrial internet of things (IIOT) - special report-2017Ian Beckett
 
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014Ahmed Mahmoud
 
internet of things : 2021 perspective
internet of things : 2021 perspectiveinternet of things : 2021 perspective
internet of things : 2021 perspectiveRoberto Siagri
 

What's hot (20)

Managing your Assets with Big Data Tools
Managing your Assets with Big Data ToolsManaging your Assets with Big Data Tools
Managing your Assets with Big Data Tools
 
Tec118 Teched2015 IOT use case and examples
Tec118 Teched2015 IOT use case and examplesTec118 Teched2015 IOT use case and examples
Tec118 Teched2015 IOT use case and examples
 
The competitive landscape of the Internet of Things
The competitive landscape of the Internet of ThingsThe competitive landscape of the Internet of Things
The competitive landscape of the Internet of Things
 
Making sense of IoT, M2M and Big Data
Making sense of IoT, M2M and Big DataMaking sense of IoT, M2M and Big Data
Making sense of IoT, M2M and Big Data
 
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a PalaceInternet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
Internet of Things (IoT) is a King, Big data is a Queen and Cloud is a Palace
 
Key Data Management Requirements for the IoT
Key Data Management Requirements for the IoTKey Data Management Requirements for the IoT
Key Data Management Requirements for the IoT
 
Proof of concepts and use cases with IoT technologies
Proof of concepts and use cases with IoT technologiesProof of concepts and use cases with IoT technologies
Proof of concepts and use cases with IoT technologies
 
Internet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M CommunicationInternet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M Communication
 
Internet of Things Stack - Presentation Version
Internet of Things Stack - Presentation VersionInternet of Things Stack - Presentation Version
Internet of Things Stack - Presentation Version
 
IoT Strategy Pillars
IoT Strategy PillarsIoT Strategy Pillars
IoT Strategy Pillars
 
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術
社会におけるIoTとセキュリティ、匿名化技術: 産業IoTのサイバーセキュリティ技術
 
Smart City Era
Smart City EraSmart City Era
Smart City Era
 
Iot how it works
Iot   how it worksIot   how it works
Iot how it works
 
Data Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsData Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of Things
 
Wot
WotWot
Wot
 
Device to Intelligence, IOT and Big Data in Oracle
Device to Intelligence, IOT and Big Data in OracleDevice to Intelligence, IOT and Big Data in Oracle
Device to Intelligence, IOT and Big Data in Oracle
 
Industrial internet of things (IIOT) - special report-2017
Industrial internet of things (IIOT) - special report-2017Industrial internet of things (IIOT) - special report-2017
Industrial internet of things (IIOT) - special report-2017
 
IoT - module 4
IoT - module 4IoT - module 4
IoT - module 4
 
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014
Talk on Industrial Internet of Things @ Intelligent systems tech forum 2014
 
internet of things : 2021 perspective
internet of things : 2021 perspectiveinternet of things : 2021 perspective
internet of things : 2021 perspective
 

Viewers also liked

IOT and Big Data - The Perfect Marriage
IOT and Big Data - The Perfect MarriageIOT and Big Data - The Perfect Marriage
IOT and Big Data - The Perfect MarriageDr. Mazlan Abbas
 
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...Alan Quayle
 
Metrics @ App Academy
Metrics @ App AcademyMetrics @ App Academy
Metrics @ App AcademyNiko Vuokko
 
Sensor Data in Business
Sensor Data in BusinessSensor Data in Business
Sensor Data in BusinessNiko Vuokko
 
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...Rainer Sternfeld
 
In-Time Sensor Data Analysis and Pattern Detection
In-Time Sensor Data Analysis and Pattern DetectionIn-Time Sensor Data Analysis and Pattern Detection
In-Time Sensor Data Analysis and Pattern DetectionJordan Barrette
 
The sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of ThingsThe sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of ThingsStephan Reimann
 
Paper6745 presentation tianjian
Paper6745 presentation tianjianPaper6745 presentation tianjian
Paper6745 presentation tianjianTianjian Chen
 
Node-RED Interoperability Test
Node-RED Interoperability TestNode-RED Interoperability Test
Node-RED Interoperability TestBoris Adryan
 
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...Amélie Gyrard
 
Presentation aina2016 seg3.0_methodology_v2
Presentation aina2016 seg3.0_methodology_v2Presentation aina2016 seg3.0_methodology_v2
Presentation aina2016 seg3.0_methodology_v2Amélie Gyrard
 
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014Mario Pastorelli
 
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...Stephan Reimann
 
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...Raffaele Giaffreda
 
7 Hottest Wearable Gadgets
7 Hottest Wearable Gadgets7 Hottest Wearable Gadgets
7 Hottest Wearable GadgetsVolansys
 
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfua
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfuaIEEE Internet of Things (IoT) Initiative in Ukraine #iotconfua
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfuaAndy Shutka
 

Viewers also liked (20)

Internet of Things - Advantech IoT Gateway Starter Kit
Internet of Things - Advantech IoT Gateway Starter KitInternet of Things - Advantech IoT Gateway Starter Kit
Internet of Things - Advantech IoT Gateway Starter Kit
 
IOT and Big Data - The Perfect Marriage
IOT and Big Data - The Perfect MarriageIOT and Big Data - The Perfect Marriage
IOT and Big Data - The Perfect Marriage
 
Big Data use cases in telcos
Big Data use cases in telcosBig Data use cases in telcos
Big Data use cases in telcos
 
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...
Forget Mobile-First … Move your customers to IoT-First, Drew Johnson, Aeris C...
 
Metrics @ App Academy
Metrics @ App AcademyMetrics @ App Academy
Metrics @ App Academy
 
Sensor Data in Business
Sensor Data in BusinessSensor Data in Business
Sensor Data in Business
 
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...
Opportunities in Sensor Networks and Big Data in 2014 (for NIKKEI Big Data Co...
 
In-Time Sensor Data Analysis and Pattern Detection
In-Time Sensor Data Analysis and Pattern DetectionIn-Time Sensor Data Analysis and Pattern Detection
In-Time Sensor Data Analysis and Pattern Detection
 
The sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of ThingsThe sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of Things
 
Paper6745 presentation tianjian
Paper6745 presentation tianjianPaper6745 presentation tianjian
Paper6745 presentation tianjian
 
Node-RED Interoperability Test
Node-RED Interoperability TestNode-RED Interoperability Test
Node-RED Interoperability Test
 
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...
Assisting IoT Projects and Developers in Designing Interoperable Semantic Web...
 
Presentation aina2016 seg3.0_methodology_v2
Presentation aina2016 seg3.0_methodology_v2Presentation aina2016 seg3.0_methodology_v2
Presentation aina2016 seg3.0_methodology_v2
 
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014
"HFSP: Size-based Scheduling for Hadoop" presentation for BigData 2014
 
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...
Marvin, Data Science & Spark – haben wir ohne Mathematik und Technik noch ein...
 
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...
Tutorial at IEEE WF-IOT Dec. 2016 - Five Years of Research and Innovation Exp...
 
IEEE big data 2015
IEEE big data 2015IEEE big data 2015
IEEE big data 2015
 
7 Hottest Wearable Gadgets
7 Hottest Wearable Gadgets7 Hottest Wearable Gadgets
7 Hottest Wearable Gadgets
 
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfua
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfuaIEEE Internet of Things (IoT) Initiative in Ukraine #iotconfua
IEEE Internet of Things (IoT) Initiative in Ukraine #iotconfua
 
TELCO_PORTFOLIO_2015
TELCO_PORTFOLIO_2015TELCO_PORTFOLIO_2015
TELCO_PORTFOLIO_2015
 

Similar to IoT Platform Demo with MQTT, Node-RED and Analytics

Embedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemEmbedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemIOSR Journals
 
Embedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemEmbedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemIOSR Journals
 
Comparative analysis of traditional scada systems and io t implemented scada
Comparative analysis of traditional scada systems and io t implemented scadaComparative analysis of traditional scada systems and io t implemented scada
Comparative analysis of traditional scada systems and io t implemented scadaIJARIIT
 
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...IRJET Journal
 
IOT Network architecture and Design.pptx
IOT Network architecture and Design.pptxIOT Network architecture and Design.pptx
IOT Network architecture and Design.pptxMeghaShree665225
 
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE cscpconf
 
Monitoring in IOT enabled devices
Monitoring in IOT enabled devicesMonitoring in IOT enabled devices
Monitoring in IOT enabled devicesEswar Publications
 
ArtigofinalpublicadoASTESJ_060139.pdf
ArtigofinalpublicadoASTESJ_060139.pdfArtigofinalpublicadoASTESJ_060139.pdf
ArtigofinalpublicadoASTESJ_060139.pdfMeftahMehdawi
 
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...IJTET Journal
 
Industrial Hazard Monitoring using IOT
Industrial Hazard Monitoring using IOTIndustrial Hazard Monitoring using IOT
Industrial Hazard Monitoring using IOTAyush Chhangani
 
An Internet Based Interactive Data Acquisition System
An Internet Based Interactive Data Acquisition System An Internet Based Interactive Data Acquisition System
An Internet Based Interactive Data Acquisition System Saptarshi Nag
 
Development of web-based surveillance system for Internet of Things (IoT) app...
Development of web-based surveillance system for Internet of Things (IoT) app...Development of web-based surveillance system for Internet of Things (IoT) app...
Development of web-based surveillance system for Internet of Things (IoT) app...journalBEEI
 

Similar to IoT Platform Demo with MQTT, Node-RED and Analytics (20)

Embedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemEmbedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control System
 
Embedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control SystemEmbedded Web Server based Interactive data acquisition and Control System
Embedded Web Server based Interactive data acquisition and Control System
 
COMPARATIVE STUDY BETWEEN VARIOUS PROTOCOLS USED IN INTERNET OF THING
COMPARATIVE STUDY BETWEEN VARIOUS  PROTOCOLS USED IN INTERNET OF THINGCOMPARATIVE STUDY BETWEEN VARIOUS  PROTOCOLS USED IN INTERNET OF THING
COMPARATIVE STUDY BETWEEN VARIOUS PROTOCOLS USED IN INTERNET OF THING
 
Comparative analysis of traditional scada systems and io t implemented scada
Comparative analysis of traditional scada systems and io t implemented scadaComparative analysis of traditional scada systems and io t implemented scada
Comparative analysis of traditional scada systems and io t implemented scada
 
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
 
IOT Network architecture and Design.pptx
IOT Network architecture and Design.pptxIOT Network architecture and Design.pptx
IOT Network architecture and Design.pptx
 
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE
UBIQUITOUS NETWORK TECHNICAL ROOM MONITORING SYSTEM MODEL USING WEB SERVICE
 
Designing Internet of things
Designing Internet of thingsDesigning Internet of things
Designing Internet of things
 
Unit 3 IOT.docx
Unit 3 IOT.docxUnit 3 IOT.docx
Unit 3 IOT.docx
 
chapter 3.pdf
chapter 3.pdfchapter 3.pdf
chapter 3.pdf
 
chapter 3.docx
chapter 3.docxchapter 3.docx
chapter 3.docx
 
Monitoring in IOT enabled devices
Monitoring in IOT enabled devicesMonitoring in IOT enabled devices
Monitoring in IOT enabled devices
 
ArtigofinalpublicadoASTESJ_060139.pdf
ArtigofinalpublicadoASTESJ_060139.pdfArtigofinalpublicadoASTESJ_060139.pdf
ArtigofinalpublicadoASTESJ_060139.pdf
 
FIOT_Uni4.pptx
FIOT_Uni4.pptxFIOT_Uni4.pptx
FIOT_Uni4.pptx
 
K010426371
K010426371K010426371
K010426371
 
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...
Arm Based Real Time Monitoring and Controlling of Industrial Parameters Using...
 
Industrial Hazard Monitoring using IOT
Industrial Hazard Monitoring using IOTIndustrial Hazard Monitoring using IOT
Industrial Hazard Monitoring using IOT
 
Ak4103212215
Ak4103212215Ak4103212215
Ak4103212215
 
An Internet Based Interactive Data Acquisition System
An Internet Based Interactive Data Acquisition System An Internet Based Interactive Data Acquisition System
An Internet Based Interactive Data Acquisition System
 
Development of web-based surveillance system for Internet of Things (IoT) app...
Development of web-based surveillance system for Internet of Things (IoT) app...Development of web-based surveillance system for Internet of Things (IoT) app...
Development of web-based surveillance system for Internet of Things (IoT) app...
 

Recently uploaded

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 

Recently uploaded (20)

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 

IoT Platform Demo with MQTT, Node-RED and Analytics

  • 1. IOT (M2M) - BIG DATA - ANALYTICS CHAKER ALLAOUI EMULATION & DEMONSTRATION
  • 2. GPS Sensor Philips HUE MQTT Geo-Location Temperature Commands Node-Red Topic - Broker Humidity Data Subscriber IoT PLATFORM Publisher Features Monitoring Table of Contents PARTIE II : Emulation PARTIE III : Demonstration ARCHITECTURE IoT PARTIE I : Presentation Cars Trafics Data-Transfert Commands AnalyticsBig DataM2M Examples
  • 3.
  • 4. PARTIE I : PRESENTATION
  • 5. 5 The Internet of Things (IoT) is an emerging global Internet- based information architecture facilitating the exchange of goods and services. The IoT has the purpose of providing an IT-infrastructure facilitating the exchange of “things” in a secure and reliable manner, i.e. its function is to overcome the gap between objects in the physical world and their representation in information systems. The IoT will serve to increase transparency and enhance the efficiency of global supply chain networks. In systems engineering, a 'system monitor (SM) is a process within a distributed system for collecting and storing state data. This is a fundamental principle supporting Application Performance Management. The argument that system monitoring is just a nice to have, and not really a core requirement for operational readiness, dissipates quickly when a critical application goes down with no warning, configuration data for the monitor, configuration data for the system being monitored. IoT Platform
  • 6. 6 Features The Internet of Things is the network of physical objects or "things" embedded with electronics, software, sensors, and connectivity to enable objects to exchange data with the manufacturer, operator and/or other connected devices based on the infrastructure of International Telecommunication Union's Global Standards Initiative. The Internet of Things allows objects to be sensed and controlled remotely across existing network infrastructure, creating opportunities for more direct integration between the physical world and computer-based systems, and resulting in improved efficiency, accuracy and economic benefit. Each thing is uniquely identifiable through its embedded computing system but is able to interoperate within the existing Internet infrastructure. Experts estimate that the IoT will consist of almost 50 billion objects by 2020. The term “Internet of Things” was coined by British entrepreneur Kevin Ashton in 1999. Typically, IoT is expected to offer advanced connectivity of devices, systems, and services that goes beyond machine-to-machine communications (M2M) and covers a variety of protocols, domains, and applications. The interconnection of these embedded devices (including smart objects), is expected to usher in automation in nearly all fields, while also enabling advanced applications like a Smart Grid, and expanding to the areas such as Smart city. Things, in the IoT, can refer to a wide variety of devices such as heart monitoring implants, biochip transponders on farm animals, electric clams in coastal waters, automobiles with built-in sensors, or field operation devices that assist fire-fighters in search and rescue. These devices collect useful data with the help of various existing technologies and then autonomously flow the data between other devices. Current market examples include smart thermostat and washer/dryers. https://en.wikipedia.org/wiki/Internet_of_Things
  • 8. 8 Monitoring (1) Protocol: There are many tools for collecting system data from hosts and devices using the SNMP (Simple Network Management Protocol). Most computers and networked devices will have some form of SNMP access. Interpretation of the SNMP data from a host or device requires either a specialized tool (typically extra software from the vendor) or a Management information base (MIB), a mapping of commands/data references to the various data elements the host or device provides. The advantage of SNMP for monitoring is its low bandwidth requirements and universal usage in the industries. Unless an application itself provides a MIB and output via SNMP, then SNMP is not suitable for collecting application data. Other protocols are suitable for monitoring applications, such as CORBA (language/OS-independent), JMX (Java-specific management and monitoring protocol), or proprietary TCP/IP or UDP protocols (language/OS independent for the most part). Data access: refers to the interface by which the monitor data can be utilized by other processes. For example, if the system monitor is a CORBA server, clients can connect and make calls on the monitor for current state of an element, or historical states for an element for some time period. The system monitor may be writing data directly into a database, allowing other processes to access the database outside the context of the system monitor. This is dangerous however, as the table design for the database will dictate the potential for data-sharing. Ideally the system monitor is a wrapper for whatever persistence mechanism is used, providing a consistent and 'safe' access interface for others to access the data. Mode: The data collection mode of the system monitor is critical. The modes are: monitor poll, agent push, and a hybrid scheme. https://en.wikipedia.org/wiki/System_monitoring
  • 9. 9 Monitoring (2) Monitor poll: In this mode, one or more processes in the monitoring system actually poll the system elements in some thread. During the loop, devices are polled via SNMP calls, hosts can be accessed via Telnet/SSH to execute scripts or dump files or execute other OS-specific commands, applications can be polled for state data, or their state-output-files can be dumped. The advantage of this mode is that there is little impact on the host/device being polled. The host's CPU is loaded only during the poll. The rest of the time the monitoring function plays no part in CPU loading. The main disadvantage of this mode is that the monitoring process can only do so much in its time. If polling takes too long, the intended poll-period gets elongated. Agent push: In agent-push mode, the monitored host is simply pushing data from itself to the system monitoring application. This can be done periodically, or by request from the system monitor asynchronously. The advantage of this mode is that the monitoring system's load can be reduced to simply accepting and storing data. It doesn't have to worry about timeouts for SSH calls, parsing OS- specific call results, etc. The disadvantage of this mode is that the logic for the polling cycle/options are not centralized at the system monitor, but distributed to each remote node. Thus changes to the monitoring logic must be pushed out to each node. Also, in agent-based monitoring, a host cannot inform that it is completely "down" or powered off, or if an intermediary system (such as a router) is preventing access to the system. Hybrid mode: The median mode between 'monitor-poll' and 'agent-push' is a hybrid approach, where the system configuration determines where monitoring occurs, either in the system monitor or agent.
  • 10. 10 Node-Red Internet of Things solutions often require pulling together hardware devices, APIs and online services in new and interesting ways. Time spent writing the boilerplate code to talk to these different systems, such as accessing a Serial port or complete an OAuth flow against Twitter, is not time spent on creating the real value of the solution. We need tools that make it easier for developers at all levels to bring together the different streams of events, both physical and digital, that make up the Internet of Things. Node-RED provides a browser-based UI for creating flows of events and deploying them to its light-weight runtime. With built in nodeJS, it can be run at the edge of the network or in the cloud. The node package manager (npm) ecosystem can be used to easily extend the palette of nodes available, enabling connections to new devices and services. Node-Red in its simplest form is an open source visual editor for wiring the internet of things produced by IBM. What does this mean. Well for someone lazy like me it means I can spend more time making stuff "talk" to each other than worrying about all of the interfacing code I will need to write. The system contains "Nodes" which look simply to be icons that you drag and drop on to the canvas and wire together. Each Node offers different functionality which can range from a simple debug node to be able to see what's going on in your flow, through to a Raspberry Pi node which allows you to read and write to the GPIO pins of your Pi. http://nodered.org
  • 11. 11 Example 1: Mail Data Collect
  • 12. 12 Example 2: Twitter Data Collect
  • 13. 13 Example 2: Mail and Twitter Node-Red Flow
  • 14. 14 Example 3: MQTT broker data in Node-Red
  • 15. 15 Example 3: MQTT broker Node-Red Flow
  • 16. 16 Example 3: MQTT broker HiveMQ Stats Node-RED provides a browser-based flow editor that makes it easy to wire together flows using the wide range nodes in the palette. Flows can be then deployed to the runtime in a single-click. JavaScript functions can be created within the editor using the a rich text editor. A built-in library allows you to save useful functions, templates or flows for re-use. The light-weight runtime is built on nodeJS, taking full advantage of its event-driven, non-blocking model. This makes it ideal to run at the edge of the network on low-cost hardware such as the Raspberry Pi as well as in the cloud. With over 120,000 modules in Node's package repository, it is easy to extend the range of palette nodes to add new capabilities. The flows created in Node-RED are stored using JSON which can be easily imported and exported for sharing with others. An online flow library allows you to share your best flows with the world.
  • 17. PARTIE II : EMULATION
  • 18. 18 Philips Hue Emulation Philips hue combines brilliant LED light with intuitive technology. Then puts it in the palm of your hand. Together, the bulbs, the bridge and the app will change the way you use light. Forever. Experiment with shades of white, from invigorating blue/white to cozy yellow/white. Or play with all the colors in the spectrum. Hue can wake you up. Help protect your home. Relive your favorite memories. Improve your mood. Even keep you informed about the weather. The wireless LED light bulbs bring hue to life. They deliver clear and crisp light. All shades of white. And every color in the spectrum. They don’t just look great. They’re practical too. They dim. They flash. They pulse. They do pretty much anything you want. And they screw straight into your current light fittings. Naturally. Hue light bulbs are available in E26, E27, GU10, PAR16 and BR30 formats. Or for something a bit different, try LivingColors Bloom, Iris and LightStrips. The heart of the hue system. The bridge is literally a bridge between your app and the bulbs. Linked to wi-fi via your router, it can connect up to 50 bulbs at a time. Not to mention hue’s other accessories, like hue tap. The bridge also connects your system to the wider world. With it you can control your lights remotely or link them up to the rest of the web, newsfeeds or even your own inbox. It’s really rather smart. http://www2.meethue.com/en-us
  • 19. 19 Philips Hue Emulation - Features The key to controlling hue lies in the palm of your hand. Use the app to change colors, brightness, functions and everything else in between. However you want it. Control your bulbs one at a time or all together. Find the right intensity and shade of white for the task at hand. Pick that perfect tone to match the moment. Or select a color from a photo and recreate it in your room. You don’t have to be at home to control your lights. The Myhue portal is an online control panel that connects you to hue, from anywhere in the world. Make it look like someone’s home when you’re relaxing on the beach. Or just switch your lights off if you’ll be home later than you thought. The portal also saves your personalized scenes and Light Recipes. And it even keeps your bridge software up-to-date. Lighting is about more than just illuminating the dark. It doesn’t take an expert to notice bad light. And the right light can transform entire spaces. Even change the way you feel. With hue, you get full control over the light in your life. Set the mood. Change the ambience. Feel different. This is what hue can do. With tunable white light and a full spectrum of color, all at the tap of an app, you can set the perfect tone and enhance a moment. All in an instant. Pick a photo – from the app or your own library - to use as your palette. Then simply drag the color picker across the image and your bulbs will match the color instantly. Easy. So now you can relive that glorious sunset. Or bring the calm of the ocean straight into your living room. Whatever you feel like. Like your creation. Then save it as a ‘scene’ to use again. Really like it? Then show off a bit and share it with the hue community. Scenes can be anything from a single bulb’s color, to a whole house bathed in light. And everything in between. It’s all up to you.
  • 21. 21 Philips Hue Emulation – GET Status Request
  • 22. 22 Philips Hue Emulation – PUT Request of color settings
  • 23. 23 Philips Hue Emulation – PUT Request Response
  • 24. 24 Philips Hue Emulation – Change color results
  • 25. 25 Sensor Emulation IBM IoT Foundation A new service just went live which addresses the key challenge of bringing the world of sensors and controllers with that of big data and analytics. When combined with the IBM Bluemix platform, IBM Internet of Things Foundation provides simple, but powerful, and scalable application access to devices and their data. You can rapidly build applications, visualization dashboards, mobile apps, and applications that can feed your back-end enterprise with IoT data. IBM Internet of Things Foundation is a fully managed, cloud-hosted service that makes it simple to derive value from Internet of Things (IoT) devices. We start with your device, be it a sensor, a gateway or something else. Using our recipes you can get it connected and start sending data securely up to the cloud using the open, lightweight MQTT messaging protocol. From there, you can setup and manage your devices using your online dashboard or our secure APIs, so that your apps can access live and historical data fast. You are now ready to start creating applications using your device data. You could do this within our IBM Bluemix platform, another cloud or your own servers. If you aren’t convinced that these things are sending data via our service, then run the two on separate machines or mobile devices. To do this all you need to do is enter the device identifier from one to the other. http://www-03.ibm.com/software/products/en/internet-of-things-foundation
  • 26. 26 Sensor Emulation - Features Connect: Register and connect a wide variety of devices to the service. Uses industry-standard MQTT, MQ Telemetry (MQTT) protocol (OASIS ratified). Remotely monitor the connectivity of devices. Securely connect many popular development boards and devices to the cloud using device recipes. Extend to new devices with open-source device code from Eclipse Paho. Collect: Collect and manage a time-series view of data. See what is happening on your devices with near-real time IoT data visualization. Store your IoT device events in a time-series database for analysis. Build dashboards and gain access to historical data through APIs. Control and secure data currently trapped on instrumented devices. Assemble: Visually assemble events from the IoT into logic flows. Use Node-Red in Bluemix for easy drag-and-drop flow assembly. Analyze data and act on insights in near real time, applying them where they matter most. Optimize business results with near real-time decision making. Manage: Pay for what you use with purchase options available through Bluemix—pay through the IBM Cloud marketplace or as a term subscription. Hosted on IBM SoftLayer®, 24x7 operation, managed and monitored by IBM. Powered by proven IBM MessageSight and IBM Informix® technologies.
  • 27. 27 Sensor Emulation – IBM Apps and MAC Adresse
  • 28. 28 Sensor Emulation - Device Validation
  • 29. 29 Sensor Emulation – Node Red Apps
  • 30. 30 Sensor Emulation - Collect Data
  • 31. 31 Sensor Emulation - IBM virtualization
  • 32. 32 Sensor Emulation – IBM Interface Commands
  • 33. 33 Sensor Emulation – Logs for HDFS System
  • 34. 34 MQTT Emulation If you aren’t convinced that these things are sending data via our service, then run the two on separate machines or mobile devices. To do this all you need to do is enter the device identifier from one to the other. With the simulator sending data all is set to now feed the Node-RED flow in your starter application and the instructions walk you through this change. MQTT (formerly MQ Telemetry Transport) is a publish-subscribe based "light weight" messaging protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small code footprint" is required and/or network bandwidth is limited. The Publish- Subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to interested clients based on the topic of a message. Andy Stanford-Clark and Arlen Nipper of Cirrus Link Solutions authored the first version of the protocol in 1999. The specification does not specify the meaning of "small code foot print" or the meaning of "limited network bandwidth". Thus, the protocol's availability for use depends on the context. In 2013 IBM submitted MQTT v3.1 to the OASIS specification body with a charter that ensured only minor changes to the specification could be accepted. MQTT-SN is a variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as ZigBee. Historically, the 'MQ' in 'MQTT' came from IBM's MQ message queuing product https://en.wikipedia.org/wiki/MQTT
  • 35. 35 MQTT Emulation – MQTT Definition line. However, queuing per se is not required to be supported as a standard feature in all situations. MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The design principles are to minimize network bandwidth and device resource requirements whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles also turn out to make the protocol ideal of the emerging “machine-to-machine” (M2M) or “Internet of Things” world of connected devices, and for mobile applications where bandwidth and battery power are at a premium. Alternative protocols include the Advanced Message Queuing Protocol, the IETF Constrained Application Protocol and XMPP. There are several MQTT brokers available such as IBM Message Sight. They vary in their feature set and some of them implement additional features on top of the standard MQTT functionality. In the real world, there are a number of projects that implement MQTT. Facebook Messenger. Facebook has used aspects of MQTT in Facebook Messenger. However, it is unclear how much of MQTT is used or for what; Moreover it is to be noted that this is a phone application not a sensor application. IECC Scalable Delta Rail's latest version of their IECC Signaling Control System uses MQTT for communications within the various parts of the system and other components of the signaling system. It provides the underlying communications framework for a system that is compliant with the CENELEC standards for safety-critical communications.
  • 36. 36 MQTT Emulation – Java Publisher Example
  • 37. 37 MQTT Emulation – Java Subscriber Example
  • 38. 38 MQTT Emulation – Stats with MQTT SPY
  • 39. 39 MQTT Emulation – Messages Broker
  • 40. 40 MQTT Emulation - HiveMQ Stats
  • 41. 41 GPS Emulation Creating software for the Internet of Things seems to get easier every day, as new development tools make popular programming languages work with more and more connected devices. One of the latest additions is Cylon.JS, an open-source JavaScript framework designed for controlling robots, embedded sensors and IoT devices. JavaScript is already hugely familiar to web developers and other programmers because it runs in just about every web browser and is used to make websites interactive (think of the chat boxes on social media sites that let you send and receive instant messages with friends, without reloading the page). It’s also fairly easy to learn the basics, even for amateurs. Cylon.JS is built on nodeJS, a JavaScript framework used for many web apps that provides common ground for IoT applications—such as the ability to simultaneously send, receive and process many messages from many sources. Named for the science fiction cyborgs of Battlestar Galactica, Cylon.JS supports a wide variety of drones, robotic toys, wearables, smart home devices, IoT prototyping and embedded computing modules, and cloud-based analytics and messaging platforms. It lets programmers create powerful interactions between users, devices and services with just a few lines of code. For instance, you can set up a Pebble smartwatch to act as a controller for “robots” (connected devices) like a Nest thermostat or Phillips Hue lightbulb. Or you can build your own projects by wiring physical sensors to development boards like Arduino or Tessel. http://postscapes.com/javascript-for-robots-a-iot-cylonjs
  • 42. 42 GPS Emulation – Cylon.JS Framework For example, the cylon-api-http plugin offers a RESTful HTTP interface, along with server-sent events that can be used to subscribe to device events. Another API plugin cylon-api-socketio supports an interface for clients based on the popular socket.io module. All API plugins interface directly with the MCP to fetch robots and information about them. They are also capable of executing commands on Robots and Devices. A Cylon robot is a collection of devices and connections, with the necessary glue to allow them to coordinate together. A User will instantiate one or more robots through the MCP, and either start them all simultaneously or individually. Robots can also have custom commands (to coordinate multiple devices/connections) and can communicate with each other through the MCP. Devices and Connections are an abstraction layer over drivers and adaptors, respectively. These allow for a layer of indirection between Drivers/Adaptors and the Robot itself, and ensure the interfaces can remain consistent. Drivers and Adaptors are the meat of Cylon, and allow Cylon to communicate with devices and services, and issue commands/receive events from them. These are implemented as part of Cylon submodules (cylon-sphero, etc), and are platform specific. Adaptors are in charge of connecting to platforms, no matter the medium. For example, the cylon- sphero Adaptor class communicates over Bluetooth, while the cylon-leapmotion Adaptor talks to a WebSocket server. The Adaptor classes ensure that the Drivers are able to directly communicate with the connected platform.
  • 43. 43 GPS Emulation – nodeJS Apps
  • 45. 45 GPS Emulation – Node Red Data Transfert and Debug
  • 47. 47 GPS Emulation – Maps Representation
  • 48. 48 Trafic Cars Emulation IBM IoT Foundation The Connected Car starter kit for Bluemix allows you to easily model real-time traffic in a city. The kit comes with a nodeJS vehicle simulator, an HTML5 visualization web app, and a control client that all communicate through the IBM Internet of Things (IoT) Foundation service for real-time messaging. The diagram to the right provides an overview of how the kit works. Simulated vehicles are registered with IoT Foundation, and API keys are generated for each application in the kit. The authorization tokens allow for messaging between devices, applications, and services. The starter kit can be set up and deployed in Bluemix with the following steps: Set up your IoT Foundation account: register your simulated vehicles and generate an API key for your HTML5 applications. Create a placeholder application and IoT/Geospatial services in Bluemix. Download and configure the starter kit to match your Bluemix and IoT environment. Deploy the starter kit to Bluemix using the Cloud Foundry CLI Use Node-RED (or build your own application) to extend the starter kit http://m2m.demos.ibm.com/trafficsimulator.html
  • 49. 49 Trafic Cars Emulation – YAML Config
  • 50. 50 Trafic Cars Emulation – Settings Config
  • 51. 51 Trafic Cars Emulation – nodeJS Apps
  • 52. 52 Trafic Cars Emulation – Cloud-foundry CLI Commands The Internet of Things already connects billions of devices, with forecasts predicting steep growth rates in the coming years. Many of these devices, such as smartphones and connected vehicles, are mobile. Awareness of the location of on-the-move devices opens up exciting new application opportunities. Support for these new applications requires highly scalable services that can analyze high volumes of data in real time. With the Geospatial Analytics service in IBM Bluemix, you can monitor moving devices from the Internet of Things. The service tracks device locations in real time with respect to one or more geographic regions. Geospatial Analytics can be used as a building block in applications that support several use cases. For example, a retail business might want to monitor for potential customers nearby its stores and send them promotions via push notifications or tweets. Geospatial Analytics could also be used to detect connected motorists entering areas that they might want to avoid due to an accident, weather…
  • 53. 53 Trafic Cars Emulation - IoT IBM Interface
  • 54. 54 Trafic Cars Emulation – Payload Messages
  • 55. 55 Trafic Cars Emulation – Node Red integration
  • 56. 56 Trafic Cars Emulation – Node Red Data Transfert and Debug
  • 57. 57 Trafic Cars Emulation- Process Stats
  • 58. PARTIE III : DEMONSTRATION
  • 59. IOT BIG DATA ANALYTICS DEVICES PROTOCOLE HARDWARE ARCHITECTURE « IOT (M2M) - BIG DATA – ANALYTICS » M2M IoT (M2M) – Big Data – Analytics : Architecture
  • 60. The initial contribution of the Eclipse SmartHome project will be a large subset of the current code of openHAB (open Home Automation Bus). In particular: Source code for all mentioned interfaces and services of the framework. Source code of a couple of extensions (openHAB bindings, actions and persistence services) Documentation of the architecture and its extension points. Build scripts and sample files. To enable others to use code from openHAB in their own solutions and products, we wanted to put the code base on a solid and trustworthy foundation. The legal aspects of many open-source projects are often pretty fuzzy, even if they have a permissive license. But for companies using such project in their own products, there is always the risk that the copyright of the code is not clear and that the code might be contaminated by patents. To reduce such risks, it is very beneficial to have a rigid intellectual property management and clear contribution processes - this is one of the things that a foundation such as Eclipse provides. We therefore decided to contribute the core framework of openHAB to the Eclipse Foundation, which became the new Eclipse SmartHome project. 60 Software IOT http://kaikreuzer.blogspot.com/2014/06/openhab-20-and-eclipse-smarthome.html
  • 61. 61 Software IoT - History The Eclipse SmartHome project is a proposed open source project under the Eclipse Technology Top- Level Project. This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. We solicit additional participation and input from the Eclipse community. Please send all feedback to the Eclipse Proposals Forum. Since the emergence of broadband internet connections, smartphones and tablets the smart home market shows a remarkable upsurge. This has led to a very fragmented market, which makes it difficult for customers to "bet on the right horse". In fact, there is not one system, protocol or standard that could possibly fulfill all potential requirements. There is hence a need for platforms that allow the integration of different systems, protocols or standards and that provide a uniform way of user interaction and higher level services. The goals of the Eclipse SmartHome project can be summarized as: Provide a flexible framework for smart home and ambient assisted living (AAL) solutions. This framework focuses on the use cases of this domain, e.g. on easy automation and visualization aspects. Specify extension points for integration possibilities and higher-level services. Extending and thus customizing the solution must be as simple as possible and this requires concise and dedicated interfaces. Provide implementations of extensions for relevant systems, protocols or standards. Many of them can be useful to many smart home solutions, so this project will provide a set of extensions that can be included if desired. They can also be in the shape of a general Java library or an OSGi bundle, so that these implementations can be used independently of the rest of the project as well. Provide a development environment and tools to foster implementations of extensions. The right tooling can support the emergence of further extensions and thus stimulates future contributions to
  • 62. 62 Software IoT - Features the project. Create a packaging and demo setups. Although the focus is on the framework, it needs to be shown how to package a real solution from it, which can be used as a starting point and for demo purposes. The Eclipse SmartHome project is a framework that allows building smart home solutions that have a strong focus on heterogeneous environments, i.e. solutions that deal with the integration of different protocols or standards. Its purpose is to provide a uniform access to devices and information and to facilitate different kinds of interactions with them. This framework consists out of a set of OSGi bundles that can be deployed on an OSGi runtime and which defines OSGi services as extension points. The stack is meant to be usable on any kind of system that can run an OSGi stack - be it a multi-core server, a residential gateway or a Raspberry Pi. Currently planned initial features and extension points of this framework include: Type System: A basic but extensible type system for smart home data and commands that provides a common ground for an abstracted data and device access. Data Registry: Keeps track of data point instances and their values. Event Mechanism: Passes data and operations asynchronously in a loosely coupled way between components. The Infrastructure to exchange data and commands with external systems and devices, e.g. through regular polling or background threads. Useful binding implementations such as HTTP, NTP or TCP will be part of the project. An implementation of a flexible rule engine that allows changing rules during runtime. Provides triggering upon events or on a temporal basis. Extensible through rule actions that are made available to all rules. Simple but powerful textual representation of rules Declarative User Interfaces.
  • 63. 63 openHAB Framework – Architecture (1) The openHAB project is split into two parts: openhab-runtime: This is the package, which you will actually run on your server and which does the "real" work. * penhab-designer: This is a configuration tool for the openhab-runtime. It comes with user-friendly editors to configure your runtime, to define your UIs and to implement your rules. * openHAB Runtime: The openHAB runtime is a set of OSGi bundles deployed on an OSGi framework (Equinox). It is therefore a pure Java solution and needs a JVM to run. Being based on OSGi, it provides a highly modular architecture, which even allows adding and removing functionality during runtime without stopping the service. Here is an overview of the main bundles and how they depend on each other: Communication openHAB has two different internal communication channels: an asynchronous event bus a stateful repository, which can be queried The Event Bus The event bus is THE base service of openHAB. All bundles that do not require stateful behaviour should use it to inform other bundles about events and to be updated by other bundles on external events. There are mainly two types of events:
  • 64. 64 openHAB Framework – Architecture (2) Commands which trigger an action or a state change of some item/device. Status updates which inform about a status change of some item/device (often as a response to a command). All protocol bindings (which provide the link to the real hardware devices) should communicate via the Event Bus. This makes sure that there is a very low coupling between the bundles, which facilitates the dynamic nature of openHAB. As a technical foundation, the OSGi EventAdmin service is used by openHAB. This is a light-weight pub-sub implementation, which perfectly meets the requirements. It is important to note that openHAB is not meant to reside on (or near) actual hardware devices which would then have to remotely communicate with many other distributed openHAB instances. Instead, openHAB serves as an integration hub between such devices and as a mediator between different protocols that are spoken between these devices. In a typical installation there will therefore be usually just one instance of openHAB running on some central server. Nonetheless, as the OSGi EventAdmin service can also be used as a remote service, it is possible to connect several distributed openHAB instances via the Event Bus. Item Repository Not all functionality can be covered purely by stateless services. Therefore openHAB also offers the Item Repository, which is connected to the Event Bus and keeps track of the current status of all items. The Item Repository can be used whenever needed to access the current state of items. E.g. a user interface needs to display the current state of items in the moment of the user access. Also the automation logic execution engine needs to always be informed about the current states.
  • 65. 65 openHAB Framework – Architecture (3) The Item Repository avoids requiring each bundle to cache states themselves for their internal use. It also makes sure that the state is in sync for all those bundles and it provides the possibility to persist states to the file system or a database, so that they are even kept throughout a system restart. The following diagram shows how these communication channels are used: Sitemap openHAB comes with a generic textual configuration for its user interfaces: The Sitemap. The Sitemap is a tree structure of widgets, which define the different pages of a UI and their content. Widgets can be associated to items, for which they should show the status and/or control elements. The definition of the Sitemap is quite abstract by design; it is supposed to be a suitable UI model for different kinds of user interfaces, so that the user does not have to configure each of them in case he sets up multiple UIs. If a UI has further requirements on top of the sitemap, it is still possible to introduce additional configuration options which are then specific for the UI in question. Item UI Providers Item UI providers offer a dynamic way to configure the UI, so that not everything must be stored statically in the Sitemap. An item UI provider can for example define, what widget should be used for an item, if none is specified and can dynamically define icons and labels (which might depend on the current state of the item) for the widgets. An important use case for this feature is the dynamic display of item groups - all that there is stored in the Sitemap is the information that a certain group should be displayed; the page is then dynamically assembled at runtime with whatever items are available at that time.
  • 66. 66 openHAB Framework – Commands (1)
  • 67. 67 openHAB Framework – Commands (2)
  • 68. 68 openHAB Framework – Commands (3)
  • 69. 69 openHAB Framework – Commands (4)
  • 70. 70 openHAB Framework – Webservices (1)
  • 71. 71 openHAB Framework – Webservices (2)
  • 72. 72 openHAB Framework – Webservices (3)
  • 73. 73 openHAB Framework – Webservices (4) OpenHAB is aimed to provide a simple drop-in server where components can be connected and driven from a user interface, whilst allowing a general mechanism to control all components. Although many of these devices may have their own mobile applications, controlling each one of them requires opening a mobile application and they cannot be integrated together. Using OpenHAB as the main hub, the devices can all be controlled with one mobile application or website, and scripts can be set up to drive individual components.
  • 75. 75 openHAB Designer – Geent Template Example The openHAB Designer is an Eclipse RCP application for configuring the openHAB runtime. It comes with editors for configuration files like the sitemap. Its advantage over simple text editors is the full IDE support, like syntax checking, auto completion, highlighting and content assist. It is also meant to implement and deploy rules for automatic actions.
  • 76. Rather than reinventing the wheel, SiteWhere depends on many supporting open source technologies to accomplish its goals. Most of the technologies used have been tested for years in production environments and have been continuously upgraded thanks to many iterations of improvements. The following open source components are used by SiteWhere: Tomcat provides the core server in which SiteWhere runs. SiteWhere is deployed as a web archive (WAR file) that runs when the Tomcat server is started. Spring delivers the core configuration framework that bootstraps SiteWhere and allows it to be configured and extended. Rather than using a hard-coded set of components, SiteWhere has a core set of service provider interfaces and default implementations of the interfaces. By using the same interfaces in custom classes and plugging them in via Spring, a third parties can extend SiteWhere without touching the core code. MongoDB: MongoDB is a NoSQL database that can be used to store SiteWhere data. It is very quick and easy to install, has great performance, and does not require the computing resources of HBase. 76 Software M2M http://docs.sitewhere.org/1.0.4/overview.html#what-is-sitewhere
  • 77. 77 Sitewhere - Software M2M (1) SiteWhere is an open source M2M integration platform – a system that choreographs data coming from and going to a web of connected devices. It provides the following functionality: Supplies a server based on proven technologies that acts as a controller for the processing of device data. The server can be installed on a local machine or run in the cloud and is built on technologies that are designed to scale to billions of processed device events per day. Provides long-term persistence of data sent from devices. Historical device event data is valuable and SiteWhere offers a platform where data is never deleted, no matter the volume of events. Offers service provider interfaces (SPIs) that provide a core object model for the platform and allow third parties to extend and customize the system to work with new technologies. Delivers an advanced device provisioning system that allows control of the full lifecycle of registering devices, sending commands based on hardware type, receiving data responses and aggregating them. The system is based on a core set of interfaces that allow new communications protocols and encoding schemes to be added and configured easily. Provides a model for standard types of data generated by devices. For instance, location data is stored such that it can be geospatially indexed and accessed by queries like ‘which device events happened near this given location in the given time span’. Device measurement data is stored such that it can be analyzed, faceted, and graphed. Alerts for exceptional conditions can be captured and indexed so that an audit trail is available after the fact. Associates devices with external assets such as people or physical items. For instance, a badge may be associated with the person wearing it. A tracking device can be associated with the piece of
  • 78. 78 Sitewhere - Software M2M (2) heavy equipment it is attached to. The asset information is offered via an asset management framework that allows external systems to drive the information. For instance, the list of people that badges can be assigned to can come from an existing LDAP data store. SiteWhere tracks the assignment of devices to assets over time and ties events to the assignment so at a later date you can query ‘what were the locations of the badge when it was assigned to this person’. Integrates with third party integration frameworks such as Mule ESB, allowing event data to trigger complex interactions such as adding data to Salesforce or notifying alert conditions using text messages generated by Twilio. Provides an HTML5 administrative application that allows all of the system data to be viewed and manipulated in a way that makes it easy to understand the data. The admin app uses REST services provided by the core platform to interact with the data. Third party applications can use the same REST services to interact with SiteWhere data without having to use the admin application. Offers a Java client that can interact with most of the provided REST services. For external systems with existing Java code, interfacing with SiteWhere is as simple as a few lines of code. Protects information by limiting access to data based on a proven user management system. The system can be configured to use external sources of identity data such as LDAP data stores. Generates detailed analytics information from gathered device data and provides factilities for executing advanced faceted queries on the data.
  • 82. Devices must communicate with each other (D2D). Device data then must be collected and sent to the server infrastructure (D2S). That server infrastructure has to share device data (S2S), possibly providing it back to devices, to analysis programs, or to people. From 30,000 feet, the protocols can be described in this framework as: • MQTT • XMPP • DDS • AMQP Each of these protocols is widely adopted. There are at least 10 implementations of each. Confusion is understandable, because the high-level positioning is similar. In fact, all four claim to be real-time publish-subscribe IoT protocols that can connect thousands of devices. And it’s true, depending on how you define “real time,” “things,” and “devices.” Nonetheless, they are very different indeed! Today’s Internet supports hundreds of protocols. The IoT will support hundreds more. It’s important to understand the class of use that each of these important protocols addresses. 82 Protocoles & Positioning Systems
  • 83. 83 Protocle MQTT MQTT (MQ Telemetry Transport) is a publish-subscribe based "light weight" messaging protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small code footprint" is required and/or network bandwidth is limited. The Publish-Subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to interested clients based on the topic of a message. Andy Stanford-Clark and Arlen Nipper of Cirrus Link Solutions authored the first version of the protocol in 1999. The specification does not specify the meaning of "small code foot print" or the meaning of "limited network bandwidth". Thus, the protocol's availability for use depends on the context. In 2013 IBM submitted MQTT v3.1 to the OASIS specification body with a charter that ensured only minor changes to the specification could be accepted. MQTT-SN is a variation of the main protocol aimed at embedded devices on non-TCP/IP networks, such as ZigBee. Historically, the 'MQ' in 'MQTT' came from IBM's MQ message queuing product line. However, queuing per se is not required to be supported as a standard feature in all situations. Alternative protocols include the Advanced Message Queuing Protocol, the IETF Constrained Application Protocol and XMPP. MQTT Broker: There are several MQTT brokers available such as IBM Message Sight. They vary in their feature set and some of them implement additional features on top of the standard MQTT functionality. See MQTT.org for more information. In the real world, there are a number of projects that implement MQTT. Facebook Messenger. Facebook has used aspects of MQTT in Facebook Messenger. However, it is unclear how much of MQTT is used or for what; Moreover it is to be noted that this is a phone application not a sensor application. https://en.wikipedia.org/wiki/MQTT
  • 84. 84 Protocle GSM GSM (Global System for Mobile Communications, originally Group Special Mobile), is a standard developed by the European Telecommunications Standards Institute (ETSI) to describe protocols for second-generation (2G) digital cellular networks used by mobile phones, first deployed in Finland in July 1992. As of 2014 it has become the default global standard for mobile communications - with over 90% market share, operating in over 219 countries and territories. 2G networks developed as a replacement for first generation (1G) analog cellular networks, and the GSM standard originally described a digital, circuit-switched network optimized for full duplex voice telephony. This expanded over time to include data communications, first by circuit-switched transport, then by packet data transport via GPRS (General Packet Radio Services) and EDGE (Enhanced Data rates for GSM Evolution or EGPRS). Subsequently, the 3G++ developed third-generation (3G) UMTS standards followed by fourth- generation (4G) LTE Advanced standards, which do not form part of the ETSI GSM standard. "GSM" is a trademark owned by the GSM Association. It may also refer to the (initially) most common voice codec used, Full Rate. GSM Layer 1: FDMA/TDMA is the air interface(radio), also called Um interface. For more on this interface refer GSM tutorial in tutorials section. At MS, FDMA/TDMA is used which is also followed at BTS, BTS takes this format from MS and convert it to 64kbps digital format for the digital link and interfaces with BSC. BSC communicates with MSC in the same format. GSM Layer 2: Layer 2 is the data link layer, which does following three main functions. - Establish, maintain and tear down the link - Flow control - Error detection - Work on the Layer-3 frames https://en.wikipedia.org/wiki/GSM
  • 85. 85 GPS - Global Positioning System The Global Positioning System (GPS) is a space-based satellite navigation system that provides location and time information in all weather conditions, anywhere on or near the earth where there is an unobstructed line of sight to four or more GPS satellites. The system provides critical capabilities to military, civil, and commercial users around the world. The United States government created the system, maintains it, and makes it freely accessible to anyone with a GPS receiver. The US began the GPS project in 1973 to overcome the limitations of previous navigation systems, integrating ideas from several predecessors, including a number of classified engineering design studies from the 1960s. The U.S. Department of Defense (DoD) developed the system, which originally used 24 satellites. It became fully operational in 1995. Bradford Parkinson, Roger L. Easton, and Ivan A. Getting are credited with inventing it. Advances in technology and new demands on the existing system have now led to efforts to modernize the GPS system and implement the next generation of GPS Block IIIA satellites and Next Generation Operational Control System (OCX). Announcements from Vice President Al Gore and the White House in 1998 initiated these changes. In 2000, the U.S. Congress authorized the modernization effort, GPS III. In addition to GPS, other systems are in use or under development. The Russian Global Navigation Satellite System (GLONASS) was developed contemporaneously with GPS, but suffered from incomplete coverage of the globe until the mid-2000s. There are also the planned European Union Galileo positioning system, India's Indian Regional Navigation Satellite System, and the Chinese BeiDou Navigation Satellite System. https://en.wikipedia.org/wiki/Global_Positioning_System
  • 86. Raspberry PI – Mobile - SIM CARD – GPS TRACKER – LG SMART TV – GOOGLE MAPS – LED LIGHTING – CARS TRACKER – OPENHAB FRAMEWORK – OSGI MODULE – SITEWHERE - THERMOMETER 86 Plateforme IoT
  • 88. 88 YOUTUBE CHANNEL : INNOVAIOT Application : Movie Demonstration INNOVA_IOT INNOVAIOT INNOVAIOT
  • 89. Big data analytics is the process of examining large data sets containing a variety of data types. Big data, unknown correlations, market trends, customer preferences and other useful business information. The analytical findings can lead to more effective marketing, new revenue opportunities, better customer service, improved operational efficiency, competitive advantages over rival organizations and other benefits. To analyze such a large volume of data, big data analytics is typically performed using specialized software tools and applications for predictive analytics, data mining, text mining, forecasting and data optimization. Collectively these processes are separate but highly integrated functions of high-performance analytics. Using big data tools and software enables an organization to process extremely large volumes of data that a business has collected to determine which data is relevant and can be analyzed to drive better business decisions in the future. 89 Big Data and Analytics
  • 90. 90 Big Data Analytics – Uses cases But behavioral analytics can be more than just tracking people. It’s also about understanding the interactions and dynamics between processes, machines and equipment, even macroeconomic trends, which may yield new learning into operational risks and opportunities. When combined with rich insights about consumer behavior, companies gain a more nuanced, detailed and accurate picture of where markets are headed. And that gives them a huge advantage in directing to proactive investments to seize the best opportunities. The largest of those opportunities maybe market-facing or customer-centric, but supply chain improvements or other operational enhancements first discovered by behavioral analytics may also lead to worthwhile performance gains. Behavioral analytics has real applications beyond the realm of marketing and customer intelligence. There is sensor data that can track traffic patterns, for instance, leading to “smarter cities. It’s possible to know if cold storage chains have been broken or medications have been compromised during shipments. Such detailed and up-to-the-second views of goods moving across global supply chains are ultimately another flavor of behavioral analytics. As with big data more generally, the issue isn’t whether companies have the data they need to do behavioral analytics. It’s more that they need to determine the optimal strategies to collect, manage and analyze in a scalable and accessible way the right data to gain better insights into how people – not to mention processes, equipment and other assets – behave. It is important to note that big data leaders have long since deployed those strategies and established strong behavior analytics capabilities. It’s likely not a coincidence while such companies also tend to be market leaders. http://bigdata.teradata.com/US/Big-Ideas/Behavioral-Analytics
  • 91. Jaspersoft is a commercial open source software vendor focused on business intelligence, including data visualization, reporting, and analytics. Jaspersoft provides commercial and open source software, support, services, and licensing around the JasperReports, JasperReports Server, Jaspersoft Studio, iReport, and Jaspersoft ETL products. Jaspersoft's main related product is JasperReports Server, a Java web application that provides advanced report server capabilities such as report scheduling, permissions, ad hoc reporting, dashboards, and multi-tenancy. It is offered in a community open source edition as well as under several commercial editions with broad support for various databases and data sources, including NoSQL and other big data datasources. Jaspersoft was originally called Panscopic, and was founded by Al Campa, CEO, and Raj Bhargava, VP of Products in 2001. Panscopic raised $23M from Doll Capital, Discovery Ventures, Morgenthaler Ventures, and Partech. In 2004 Panscopic teamed up with Teodor Danciu, acquired the intellectual property of JasperReports, and changed the name of the company to Jaspersoft. Brian Gentile became CEO in 2007. Teodor Danciu began work on JasperReports in June 2001, the sf.net project was registered in September 2001and JasperReports 0.1.5 was released November 3, 2001. The company released JasperReports Version 1.0, July 21, 2005. The code was originally licensed under a copyleft JasperReports License. 91 JasperSoft – Analytics and Reporting https://en.wikipedia.org/wiki/Jaspersoft
  • 94. 94 JasperSoft – MongoDB Query Langage
  • 95. 95 Elasticsearch Elasticsearch is a highly scalable open-source full-text search and analytics engine. It allows you to store, search, and analyze big volumes of data quickly and in near real time. It is generally used as the underlying engine/technology that powers applications that have complex search features and requirements. Here are a few sample use-cases that Elasticsearch could be used for: You run an online web store where you allow your customers to search for products that you sell. In this case, you can use Elasticsearch to store your entire product catalog and inventory and provide search and autocomplete suggestions for them. You want to collect log or transaction data and you want to analyze and mine this data to look for trends, statistics, summarizations, or anomalies. In this case, you can use Logstash (part of the Elasticsearch, Logstash, Kibana stack) to collect, aggregate, and parse your data, and then have Logstash feed this data into Elasticsearch. Once the data is in Elasticsearch, you can run searches and aggregations to mine any information that is of interest to you. You run a price alerting platform which allows price-savvy customers to specify a rule like "I am interested in buying a specific electronic gadget and I want to be notified if the price of gadget falls below $X from any vendor within the next month". In this case you can scrape vendor prices, push them into Elasticsearch and use its reverse-search (Percolator) capability to match price movements against customer queries. https://www.elastic.co
  • 96. 96 Elasticsearch - Features Real-Time Data: How long can you wait for insights on your fast-moving data. With Elasticsearch, all data is immediately made available for search and analytics. Real-Time Analytics: Combining the speed of search joins with the power of analytics changes your relationship with your data. Interactively search, discover, and analyze to gain insights that improve your products or streamline your business. Distributed: Elasticsearch allows you to start small and scale horizontally as you grow. Simply add more nodes, and let the cluster automatically take advantage of the extra hardware. High Availability: Elasticsearch clusters are resilient — they will detect new or failed nodes, and reorganize and rebalance data automatically, to ensure that your data is safe and accessible. Multitenancy: A cluster may contain multiple indices that can be queried independently or as a group. Index aliases allow filtered views of an index, and may be updated transparently to your application. Full-Text Search: Elasticsearch builds distributed capabilities on top of Apache Lucene to provide the most powerful full- text search capabilities available in any open source product. Powerful, developer- friendly query API supports multilingual search, geolocation, contextual did-you-mean suggestions, autocomplete, and result snippets. Document-Oriented: Store complex real world entities in Elasticsearch as structured JSON documents. All fields are indexed by default, and all the indices can be used in a single query. Schema-Free, Developer-Friendly, RESTful API, Per-Operation Persistence, Apache 2 Open Source License, Conflict Management…
  • 98. 98 Elasticsearch – MongoDB River Indexation
  • 105. 105 Apache Hadoop Apache Hadoop is an open-source software framework written in Java for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware. All the modules in Hadoop are designed with a fundamental assumption that hardware failures (of individual machines, or racks of machines) are commonplace and thus should be automatically handled in software by the framework. The core of Apache Hadoop consists of a storage part (Hadoop Distributed File System (HDFS)) and a processing part (MapReduce). Hadoop splits files into large blocks and distributes them amongst the nodes in the cluster. To process the data, Hadoop MapReduce transfers packaged code for nodes to process in parallel, based on the data each node needs to process. This approach takes advantage of data locality nodes manipulating the data that they have on hand to allow the data to be processed faster and more efficiently than it would be in a more conventional supercomputer architecture that relies on a parallel file system where computation and data are connected via high-speed networking. The base Apache Hadoop framework is composed of the following modules: Hadoop Common – contains libraries and utilities needed by other Hadoop modules; Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster; Hadoop YARN and Hadoop MapReduce programming model. https://en.wikipedia.org/wiki/Apache_Hadoop
  • 106. 106 Apache Hadoop – User Interface
  • 107. 107 Apache Hadoop – HDFS Example The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS provides high throughput access to application data and is suitable for applications that have large data sets. HDFS relaxes a few POSIX requirements to enable streaming access to file system data. HDFS was originally built as infrastructure for the Apache Nutch web search engine project.
  • 108. 108 Apache Hadoop – Log File example for Querying
  • 109. 109 Apache Hadoop – Log Files in HDFS Format
  • 110. 110 Apache Hadoop – Log Files Status in Hadoop
  • 111. 111 Apache Hadoop – Apache Hive combine The Apache Hive data warehouse software facilitates querying and managing large datasets residing in distributed storage. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL.
  • 112. 112 Apache Hive – Hive Query Langage DROP TABLE: delete the table and the file of data, if the table already exists. CREATE EXTERNAL TABLE: create a new external table in Hive. The external tables store only the definition of table in Hive. The data stay in their location of origin and in their size of origin. ROW FORMAT: indicate to Hive the mode of formatting of the data. In this case, the fields of every newspaper are separated by a space. STORED AS TEXTFILE LOCATION: indicate to Hive the location of the data (the directory example / data) and specifies that they are stored under the shape of text. The data can be in a single file or repartees on several files in the directory. SELECT: select and count all the lines the column of which t10 contains the value ‘host=chaker’.
  • 116. 116 Hadoop and Hive in Big Data Apache Hive is a data warehouse system build on top of Hadoop to query Big Data. Hive originated at Facebook and was open sourced in August 2008. The challenge Facebook had to address is one faced by many companies since then. Eventually data growth in a company challenges the capabilities of deployed RDBMS or NoSQL systems. Reports and analytics start to take minutes, then hours, and eventually overlap with other queries and the whole system grinds to a halt. Another common scenario companies start processing big data with Hadoop discovers the value of making the data accessible beyond the development team capable of writing complex map-reduce jobs. The term Big Data is freely used in this context. A colleague defined Big Data jokingly as anything beyond one million rows – Microsoft Excels row limit. The underlying point is that Big Data is a point of view and can be generalized as the point where simple solutions and deployed technology fail. The subsequent question is if scaling and investing heavily in it is the most economical solution. Commercial large-scale data warehouse solutions are very expensive. Furthermore, some of the data collected today, e.g. poorly structured or highly denormalize data, can be impractical to manage with these systems. The Hadoop ecosystem regularly is utilized to scale data processing in a feasible manner. Hadoop becomes either a replacement or a batch process addition to the existing infrastructure for data analysis, extraction, loading, transformation, reporting, and machine learning. The downside, which Facebook encountered, was that data stored in Hadoop is inaccessible to business users. There are higher-level languages like Pig, Cascading, Crunch or Scalding. They are, however, geared towards software developers that want to avoid the verbosity of pure Java map- reduce development.
  • 117. Green Internet of Things The term "Internet of Things" emanated to describe a number of technologies and research disciplines that enable global connectivity over the world-wide physical objects. Enabling technologies like RFID, sensor networks, biometrics, and nanotechnologies are now becoming very common, bringing the Internet of Things into real implementations addressing varying applications, including smart grid, e- health, intelligent transportation, etc. They foreshadow an exciting future that closely interconnects our physical world via green networks. Green networks in IoT will contribute to reduce emissions and pollutions, exploit environmental conservation and surveillance, and minimize operational costs and power consumption. The Green Internet of Things (G-IoT) is predicted to introduce significant changes in our daily life and would help realizing the vision of “green ambient intelligence”. Within a few years we will be surrounded by a massive amount of sensors, devices and “things”, which will be able to communicate via IP, act “intelligently”, and provide green support for users in managing their tasks. These new smart objects will also be context-aware and able to perform certain functions autonomously, calling for new forms of green communication between people and things and between things themselves, where power consumption is optimized and bandwidth utilization is maximized. This development would not only be relevant to researchers, but also to corporations and individuals alike. It is hence the aim of our workshop to focus on both design and implementation aspects in green networks, or networks that can be utilized in providing green systems through IoT enabling technologies. At the same time, our workshop call is open for topics relevant to IoT in general. http://research.cs.queensu.ca/G-IoT12
  • 118. Keywords IoT – M2M Big Data Analytics Internet ofThings, Machine 2 Machine, MQTT, Broker, Message Elasticsearch,Apache Hadoop, Apache Hive, MongoDB JasperSoft Studio, MongoDB, Apache Cassandra, MySQL, JSON Node-Red, nodeJS,Webservices, Maps, google maps, GPSTracker, cars trafic, thermostat connecté, adresse MAC, Device registration Elasticsearch index, MongoDB river, Marvel sense, Elasticsearch HQ Kibana analytic, Logs analytics HiveMQ,ApacheTomcat, MQTT Spy, Cylon.JS, Cloud-foundry Cli commands, openHAB, openHAB Designer, GreentTemplate, Sitewhere MapReduce,Apache Hive, Hive Query Langage, MongoDB river, HDFS, MongoDB Query Langage, Replication, Collection, elasticsearch index, cluster Hadoop UI, Monitoring, Protocol, Data access, Monitor poll, Hybrid mode Node-Red Flow,API Keys,APIToken, Collect Data, Data source,Twitter, Mail,Webservices, Maps, google maps, GPSTracker, cars trafic, thermostat connecté, adresse MAC, Device registration, MQTT Publisher, MQTT Subscriber Mqtt Broker, Publisher, Subscriber, GET Request, PUT Request, JSON, YAML HiveMQ,ApacheTomcat, MQTT Spy, Cylon.JS, Cloud-foundry Cli commands, openHAB, openHAB Designer, GreentTemplate, Sitewhere
  • 119. Links  Apache Haddop : https://hadoop.apache.org  IBM Bluemix : https://console.ng.bluemix.net  Elasticsearch : https://www.elastic.co  openHab : http://www.openhab.org  Sitewhere : http://www.sitewhere.org  Jaspersoft : http://www.jaspersoft.com  Node-red : http://nodered.org  MongoDB : https://www.mongodb.org  Apache Tomcat : http://tomcat.apache.org  Apache Hive : https://hive.apache.org  nodeJS : https://nodejs.org
  • 120. Visit my profile in LinkedIn Visit my website http://tn.linkedin.com/in/chakerallaoui http://allaoui-chaker.github.io Contact Idea Create Refine