2. Tech Talk - Frontend
Stephen Boak (steve@boundary.com)
Mark Mahoney (mark@boundary.com)
Matt King (m@boundary.com)
3. When we say real-time we mean it: JSON being
pushed to the browser once a second and
visualizations built to constantly move and adapt
to streaming data
5. Meters !
Lightweight and highly scalable, these sit on the cloud as well as
private data centers across virtual or physical servers.
6. Intercept Meter data via Transport
Collectors Layer Security (TLS) Authentication.
7. Collect all of the data at high resolution to gain rich insight into complex
Data Store environments & problems before they can impact critical business services.
9. Stream data to your dashboard with sub-second latency,
Streaming UI providing you with intuitive, powerful dashboard.
10. How We Get Data
• CometD server (streaker)!
• Pulls in data from multiple backend services and
streams it out to clients!
• Can also aggregate and filter data on demand
11. Subscriptions
1 2 3 4 5
Make a Get back a Subscribe to Receive Receive
subscription request subscription ID new query state-dump add/remove
(query and filters) (Query ID) (unique channel) (inserts & schema) messages
!!
!
!
!
!
12. Data Structure
• Schema and key
(which fields are concatenated for each record)
• State dump with schema for requested time window
(up to 100kb state dumps)
• Save bandwidth with N-tuples indexed by the schema
• Adds/removes update the state with keys (subset of schema)
14. DataSource.js
• Negotiates between multiple data
sources and subscribers on a page Subscriptions
• One data source for N subscribers Data
Sources
• Abstracts CometD subscription
process for JS development
Subscribers
• Smart enough to resubscribe and
notifies subscribers
(so that state dumps aren’t doubled up)
17. Data and Subcription Problems
• Monolithic, Multi-Purpose Queries
• Large State Dumps cause CometD timeout
(5-10MBs per second)
• No Resubscription
• Aggregated all data on front-end
18. Data and Subcription Solutions
• Stratified, Filterable Queries and server-side aggregation
• Top-N Limitations
(constrained output)
• Resubscription & filtering
• Web workers haven’t helped much
• Time smoothing (1-second ticks)
20. What’s Next
• WebSockets
• Historical Data and long term data storage
• HTML5 local storage (store data)
• Machine-learning for real-time network visualization