Utilising Cloud Resources to Gain Knowledge from Social Media
1. Utilising Cloud Resources to Gain
Knowledge from Social Media
Presentation to
Social Media Research Workshop
Wednesday, Feb 5 - Feb 6, 2014
IIIT Bangalore, Electronic City, Bangalore
Professor Gerard Parr
Chair in Telecommunications Engineering
Computer Science Research Institute
University of Ulster
Northern Ireland, UK
gp.parr@ulster.ac.uk
1
2. My Background
• Professor of Telecommunications Engineering at University of Ulster in
Northern Ireland, UK which is the largest University on the island of Ireland
• PhD in Self Stablizing Protocols- Aspects under Prof Jon Postel (USC-ISI)
DARPA USA
• Invited Member of EPSRC ICT Strategic Advisory Team- UK Gov’t
• Member of Technical Advisory Board at EBTIC , Abu Dhabi, UAE
• Visiting Professor to Trinity College Dublin
• Contributor to NSF in USA on US-UK-India Links
• PhD External Examiner for IIT Delhi , IIT Mumbai and IIT Madras
• Invited Member of EPSRC Centre for Doctoral Training in Communications
Engineering, University of Bristol
• Research Interests include Cloud Computing for Health, Virtualisation,
Smart Cities, Internet Technologies, High-speed Network Management,
Wireless Sensor Networks and ICT for Bridging the Urban Rural Divide
• Lead UK-PI on India-UK Advanced Technology Centre (IU-ATC)
• Co-PI for ESRC Administrative Data Research Centre (ADRC)
2
• Founding Member of IEEE Intercloud Testbed Initiative Oct 8th 2013
3. So where does BIG data
come from?
This data comes from everywhere including:•
•
•
•
•
•
•
•
•
•
•
Sensors used to gather climate information
Posts to social media sites
Digital pictures and videos
Experiments for drug discovery
Tracking mobile phones
Streaming NETFLIX Movies
Manufacturers monitoring equipment
Financial Services
Medical data and patient records
e.gov having services and public data available on the Internet
Social media ‘noise’
4. The Growth of Global Internet
Traffic
Worldwide mobile data traffic will increase 18-fold over the next five years.
Mobile Internet will be the main driver - connected devices which will exceed the
number of people on earth- (2016 world population estimate of 7.3 billion: source:
United Nations).
Google Project Glass:
Internet enabled!
During 2011−2016 Cisco anticipates that global mobile data
traffic will outgrow global fixed data traffic by three times.
In context this is equivalent to:33 billion DVDs.
4.3 quadrillion MP3 files (music/audio).
813 quadrillion text messages.
Ref: Cisco® Visual Networking Index Global Mobile Data Traffic Forecast for 2011 to 2016,
5. A few helpful terms…..
Terabyte: A Terabyte is approximately one trillion bytes, or 1,000 Gigabytes. To put it in some
perspective, a Terabyte could hold about 3.6 million 300 Kilobyte images or maybe about 300
hours of good quality video. A Terabyte could hold 1,000 copies of the Encyclopaedia
Britannica. Ten Terabytes could hold the printed collection of the Library of Congress.
Petabyte: A Petabyte is approximately 1,000 Terabytes or one million Gigabytes.
1 Petabyte could hold approximately 20 million 4-door filing cabinets full of text.
It could hold 500 billion pages of standard printed text.
Exabyte: An Exabyte is approximately 1,000 Petabytes.
An Exabyte is approximately one quintillion bytes
or one billion Gigabytes.
There is not much to compare an Exabyte to.
It has been said that 5 Exabytes would be
equal to all of the words ever spoken by mankind.
Zettabyte:
A Zettabyte is approximately 1,000 Exabytes.
9. BIG DATA is not just HADOOP
(Courtesy of Martin Pavlík- IBM)
Understand and navigate
federated big data sources
Federated Discovery and Navigation
Manage & store huge
volume of any data
Hadoop File System
MapReduce
Structure and control data
Data Warehousing
Manage streaming data
Stream Computing
Analyze unstructured data
Text Analytics Engine
Integrate and govern all
data sources
Integration, Data Quality, Security,
Lifecycle Management, MDM
9
10. Supported by Research Councils UK(EPSRC) and
Department of Science and Technology(DST), Government of
India
Professor Gerard Parr
UK- PI of the IU-ATC
Chair in Telecommunications Engineering
University of Ulster, Coleraine, Northern Ireland, UK
gp.parr@ulster.ac.uk
http://www.iu-atc.com
10
11. What is the IU-ATC?
The IU-ATC (India-UK Advanced Technology Centre) of Excellence in Next
Generation Networks, Systems and Services is a joint multi-million pound
research initiative funded by the British and Indian Governments which
will put in place a focused agenda to support collaborative PhD, Post
Doctorate projects and joint fundamental research programmes and
technology transfer between the UK and India.
Phase One of the Centre officially started in June 2009 with total funding
of £9.2 million and funding for Phase two was announced at the UK-India
Science and Innovation Council meeting in a joint statement during April
2012 with total funding of just over £10 million: It is an internationally
leading resource for UK and Indian Research Institutions and ICT
Companies to collaborate in support of the future Global Information
Economies.
11
12. •
The IU-ATC is structured on three Integrated
Research Groups comprising of thirteen
Cross-layer Research Work packages .
•
These inter-related work packages were
identified after detailed technical
discussions at various consortium
workshops to ensure our programme of joint
research between UK and Indian scientists is
internationally leading and competitive.
•
We also wish to contribute to the further
development and deployment of Next
Generation Converged Networks in support
of affordable applications.
•
These Work packages build on the strength
of our consortium members and the
successful outcomes of IU-ATC Phase One.
•
They provide the greatest opportunities for
the scoping of technology demonstrators
that will underpin the development of
policies and initiatives for both the rural and
urban Digital Information Economy in India
12
and the UK.
15. Group 1: Services and Applications
This group aims to develop a number of service demonstrators, with focus on
novel applications to meet the requirements in rural India and remote areas of
the UK. The services must be scalable and have the necessary user interfaces to
cater for different user needs.
WP 1: e-Agriculture
WP 3:
Collaborative
Learning
WP 2: e-Health ICT Towards Managing
Healthcare Delivery
Services and
Applications
WP4: Internet of
Things (IoT) - e-Health
WP 5: Provisioning
and Delivery of
Emergency Services
15
16. Exemplar project- Cloud Based Crop
Disease Detection
I. Pest Disease Image Upload
application (PDIU)
Mobile application allows
farmers to take photos of
affected crops and upload
them,
making
them
available to the expert
II. Advisories related to pest and
disease
Using existing call centre set-up, personalized advisories
regarding pest and diseases were given to farmers
III. Automatic colour correction
(University of Bristol/University of Ulster)
Demonstrated simple disease classification/recognition in plant images
captured in controlled environments. The research produced a number
of approaches for colour correction/normalisation and leaf segmentation
to aid recognition.
16
18. Group 1: Services and Applications
– Provisioning and Delivery of Emergency Services
WP5 focuses on developing a scalable and robust Early Warning System for natural and manmade
disasters, leveraging existing communication infrastructure and taking into account the vast
differences in end users’ device capabilities.
The Early Warning System uses a diverse set of delivery mechanisms including SMS, CBS and IP
messaging over the available access networks to disseminate warnings, with context-aware
adaptation of content and user interface.
18
19. Group 1: Services and Applications
ADAPTIVE MULTIMODAL INTERFACES
We follow a user centred inclusive design approach that involves end-users in all stages of
development ensuring that the final product is usable and enjoyable for users with a wide range of
abilities. We have developed an Inclusive User Model that simulates users’ interaction patterns
and we deploy this to personalize electronic interfaces for both health, physical and situational
impairments
We have conducted an extensive user survey in both UK
and India to understand the objective range of abilities
and the subjective attitudes towards technology of
elderly users.
19
20. Group 1: Services and Applications
ADAPTIVE MULTIMODAL INTERFACES
We interviewed key First Responder and governmental stake-holders for the design of a resilient
wireless disaster management system, that is contributing towards an inclusive disaster early
warning and rescue management system to be trialled in UK and India.
20
21. Deriving Real-Time Value..
A new proposal under development
Global First Responder System for
Man-made or Natural Disasters
Integrating, Fusing and Extracting
Useful Intelligence from
Social Media
Real-time Feeds
21
22. Group 2: Core Network Systems
Core Fixed and Wireless Infrastructure with Intelligent Monitoring and
Dynamic Resource Allocation
22
23. Group 2: Core Network Systems
WP1: Resource Allocation for Wireless Access to the Cloud
With recent developments in cloud services, Group 2 WP1 will aim to develop a framework that will dynamically
expose and scale cloud resources used in delivering mobile services. Our work will propose and develop a “Cloud
Resource Broker” that interfaces the mobile environment to existing cloud management platforms for dynamic
resource allocation via programmable interfaces. This will enable the deployment of cloud resources at the edge
of the mobile network. Why is the support for mobile environments important? IoT services are introducing more
intelligent mobile devices with critical requirements – i.e. healthcare and security services. Efficient use of mobile
resources such as spectrum, battery power and location services. Optimisation of applications by using
techniques such as edge caching and opportunistic networks. The ability to offload cloud resources to the edge of
the network.
23
24. Group 2: Core Network Systems
WP2: Cloud Aware Transport Protocols and Adaptation
App1
Users consume a range of
applications and data over WLAN
using a variety of devices. The
demand for bandwidth varies
according to the user’s needs.
New research focuses on assigning
the appropriate 802.11 protocol to
a connecting device, according to
its requirements. This dynamic
and intelligent protocol selection
should increase energy efficiency
while assuring adequate QoS for
users.
This research is built upon a testbed consisting of a range of fixed
and wireless technologies with
provision
for
inter-platform
operability.
App0
App2
802.11g
Wireless
Devices
802.11g
802.11n
Internet
To Central Data
Centre
Cloud Edge Node
24
25. Group 2: Core Network Systems
WP3: WiCAP – Wireless Cloud Application Proxy
Mobile networks currently play a key
role in the evolution of the Internet
due to exponential increase in
demand for Internet-enabled mobile
devices and applications. This has led
to various demands to re-think basic
designs of the current Internet
architecture, investigating new and
innovative ways in which key
functionalities such as end-to-end
connectivity, mobility, security, cloud
services and future requirements can
be added to its foundational core
design.
A data centre scaling out resources to
the Wireless Application Cloud Proxy
In this work package, we investigate, propose and design a functional element, known as the mobile cloud proxy,
that enables the seamless integration and extension of core cloud services on the public Internet into mobile
networks.
The mobile cloud proxy function addresses current limitations in the deployment of cloud services in mobile
networks tackling limitations such as dynamic resource allocation, transport protocols, application caching and
security. This is achieved by leveraging advances in software-defined radios (SDRs) and networks (SDNs) to
dynamically interface key functions within the mobile and Internet domains.
25
26. Group 3: Heterogeneous Wireless Access Networks
Energy Efficient Heterogeneous and Cognitive Access Networks
This work area aims to investigate and develop effective techniques to provide low-cost and
ubiquitous wireless access networks, particularly, to rural communities as the enabling technologies
for green and sustainable economies in both UK and India. The technical approach in this project
focuses on exploitation of self-organised heterogeneous and cognitive techniques in designing both
physical layer and upper layers of wireless communications systems.
WP 1: PHY layer
technologies for
4G+
WP 2: Radio Resource
Management (RRM) in
Future HETNETS
Heterogeneous
Wireless Access
Networks
WP 3: Cognitive
Radio and Future
Spectrum
WP4: Enhancement of
the Wireless Testbed
26
27. New UK International Connectivity and
Research Infrastructure
Supporting BIG Data and Cloud Services
Globally
“PROJECT KELVIN”
Professor Gerard Parr
Scientific Advisor to UK and Irish Governments
Department of Enterprise Trade and Investment (DETI) in Northern
Ireland and the Minister for Communications, Energy and Natural
Resources (MCENR) in Ireland
29. Landing Station Locations
Operating in 125 POPs across 50 Markets in North America, Europe and
Asia
26,000 km of owned terrestrial and submarine fibre assets
with leases on an additional 80,000 km worldwide
Sales come from high growth segments, specifically from three
primary vertical solutions
Wholesale Capacity
Financial Network Services
Media Network Services
Hibernia’s network is:
– Utilised by all of the top 20 largest carriers in the world
– Connected to all of the top 10 largest financial exchanges
– Transporting live video programming regularly viewed
by an estimated 20M people per day
29
30. The Cable Landing Station at
Coleraine…. “our TARDIS”
Time and Relative Dimension in Space (and Data)!
£millions of equipment carrying
£billions of data
32. USPs for KELVIN Link in Northern Ireland
• 1.92Tbit/s Northern Ireland to US & Canada Coast- currently under
upgrade
• 7.68Tbit/s Northern Ireland to European Coast
• 13 x Carrier Neutral Points of Presence (POPs) on Island of Ireland
• 52ms from Coleraine to Halifax (Nova Scotia)- lowest latency!
33. To put in context…
Using the 100Gigabit/sec capabilities the KELVIN
Links could carry :
979,584 simultaneous HD videos
600,000,000 simultaneous SKYPE calls
….and the link is getting faster!!!
Ref: Hibernia Networks
27th March 2013
35. International Connectivity
KELVIN provides the fastest link across the Atlantic Ocean:
Excellent for High frequency Real-time Trading, large-scale multi-player
Cloud Gaming, real-time TV streaming or analysing oil/gas exploration data.
37. The Northern Ireland
ESRC Administrative Data Research Centre
•
•
•
•
Linkage service
Safe setting
Safe-setting support
Data acquisition
• Public engagement
• Training, capacity building
• Research
37
38. Providing the tools and
techniques to give the
wider public
(citizens, agencies and
companies) access to
the data they paid
government to
Collate in the Cloud!!!
38
39. IT as a Utility
funded under Research Council UK's Digital Economy theme
The Digital Economy vision is of the
transformational impact of digital
technologies on all aspects of life.
39
40. • IT as a Utility Network+
– 1 of 4 networks supported by RCUK Digital
Economy Theme
• Aim: better understanding the benefits
and opportunities afforded by the digital
economy
• Objective: foster collaboration between
academia, business and policy-making
bodies
40
41. Partners & collaborators
• PI and Co-Is:
Jeremy Frey ( Uni Southampton)
Gerard Parr (Ulster)
Mark Sandler (QMUL)
Richard Mortier ( Uni Nottingham)
Mike Surridge (Uni Southampton)
Steve Brewer – Network Coordinator
• Advisory group:
–
–
–
–
–
–
–
–
–
–
ustwo
IBM
BT
Thales
Microsoft
BBC Reseaerch
Cabinet Office
DSTL
Zenotech
Cambridge &
Newcastle
Universities
41
42. The Digital Economy and ITaaU
3D printing
The Cloud
cloud computing
food security
traditional
utilities
“Network+
smart spaces/
smart cities”
“apps are the
new taps…”
sensors &
actuators
libraries of
the future
“tangible
Interfaces”
data-driven
science
electronic lab
telecommunications notebooks
42
44. First two successful pilot projects have now been running for a
while:
• Trusted Tiny Things – led from University of Aberdeen
• Using Wireless Networks to Support First Responders
and Resilience in Upland Areas – led from QMUL &
Cambridge
44
45. Pilot Projects about to start
• Typology of Loss in Vaccine Supply (TOLIVS)
– Deploy pilot cloud service/mobile sensing in the field for vaccine
tracking. Particularly typology of drug loss in the cold supply chain
• CloudMaker
– A utility to support social creativity between children
• BluPoint
– Provision of digital content as a utility in low-resourced off-grid
communities
• Sun & Sky
– A sun and sky environmental monitoring system for crowd sourcing
• Communities in the Cloud
– Technology to support high-density/high/rise communities
45
46. Secondment Projects being arranged
• Exploring social knowledge integration with
digital mapping technologies to benefit
communities
• Integration of IoT and M2M technologies for the
weightless network. Investigation of challenging
propagation environments. Workshop planned.
• Social pedestrian modelling of Clapham Junction
and London Bridge Station.
• Story as Utility: how can HDI unleash ITaaU
research in the wild?
46
47. Where does this lead us?
• Data lies at the heart of the digital economy
• This has many implications for the future
• Big data
– huge quantities of homogenous, heterogeneous
and disparate – new mathematics needed
• Security and trust issues will pervade
• Design matters – democratization of data and
its by products: information and knowledge
47
49. Thanks for your attention
The RCUK DE “IT as a Utility”
Project would like to explore
how we might help
with the follow-up
to this Bangalore workshop.
Please let me know
what we can do..!
Professor Gerard Parr
UK Lead PI for India-UK CoE
University of Ulster, UK
gp.parr@ulster.ac.uk