Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Rich feeds policy, the cloud, and CAP
1. Rich Feeds
Policy, the Cloud, and CAP
Barry Demchak
California Institute for Telecommunications and Information Technology (Calit2)
June 18, 2008
2. Overview
• Problems (RESCUE)
– Research data feeds accessible over time
– Needs for particular feeds cannot be
predicted
– Future restrictions and constraints can’t be
anticipated
• Objectives (RESCUE)
– Capture Research Data Feeds
– Expose Datasets
– Remain Flexible and Extensible
3. User View and Status
• Today’s Data Feeds
– Traffic
– Trackable Objects
– UCSD Police Cameras
– CalIT2 Cameras
– Situation Aware
• Today’s Visualizations
– Google Maps
4. Current Goals
• Empower Additional Providers
• Enable Selective Information Dissemination
• Scale to Dynamic Need
5. Current Goals
• Empower Additional Providers
– Allow Owners to
– Define dataset
– Define provider and privileges
– Define consumer and privileges
– Allow Providers to
– Contribute data
– Define consumer and privileges
• Enable Selective Information Dissemination
• Scale to Dynamic Need
6. Current Goals
• Empower Additional Providers
• Enable Selective Information Dissemination
– Allow Consumers to
– Fetch data
– Restrict data based on data or provider attributes (e.g.,
reputation)
• Scale to Dynamic Need
7. Current Goals
• Empower Additional Providers
• Enable Selective Information Dissemination
• Scale to Dynamic Need
– Data capacity
– Incoming data bandwidth
– Query bandwidth
Amazon Web
Services (AWS)
S3 – Simple
Storage Service
EC2 – Elastic
Compute Cloud
9. Logical Architecture w/Policy
• Policies – the key to empowerment
• Owner – Who can store data, who can read data
• Producer – Who can read data
• Consumer – Which providers to view
10. Physical Deployment
• Currently
– Windows Server 2003
– WinXP VM w/Mule, MySQL
• Cloud
– Amazon Servers
– Linux VM w/Mule,
MySQL,Apache/Tomcat
RESCUE Virtual Machine
Traffic
Server
Tracked
Object
Server
Browser,
Javascript,
Google Maps
Internet
Physical Machine
Other VMs
Non-
RESCUE
clients
11. Current Goals
• Empower Additional Providers
• Enable Selective Information Dissemination
• Scale to Dynamic Need
• Common Alert Protocol (CAP)
– Standard disaster alert format based on XML
– Currently used as interchange format
• CAP Server
– Always on
– Quickly scalable
– Provider-driven
– Opportunity for consumer-grade viewers
(e.g., Google Maps, Twitter, SMS, etc)