Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2xFRA6n. Diptanu Choudhury talks about the platform they are developing at NASA for running computations as functions which would make it easier for researchers to program their applications & algorithms without any boilerplate details about the underlying infrastructure. He goes into the problems they are trying to solve and provides a high level overview of the platform. Filmed at qconnewyork.com. Diptanu Choudhury is an infrastructure engineer, and works on large-scale distributed systems, cluster schedulers, service discovery and highly available and high throughput systems on the public cloud. He currently works at Facebook.

1. Serverless Platforms
Diptanu Choudhury
@diptanu
1
QCon New York

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
serverless-scientific-computations

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. 2

5. The GOES Mission
A lineage of geostationary satellites
Current mission is called GOES-R
ABI views earth with 16 different spectral bands

6. GOES-R Products
• Atmospheric events
• Fire control
• Vegetation Monitoring
• NASA makes most of the data available to
research community

7. 5Product Development Workflow

8. 6Product Development Workflow
Iterative Experimentation
based Model Development
Deployment of Models
in Production and
Image Processing
Data serving
and
Collaboration APIs

9. 7The organization
Science Engineering
• Plans mission objectives
• Develop models and products
• Conducts research and drives collaboration
• Operates infrastructure to run models
• Design systems to scale to petabytes of data and 10s
of 1000s of cores.
• Design real time services for services to consume data
• Dev-Infra to improve productivity of researchers

10. Infrastructure
• Cluster Schedulers that can scale to 10s of 1000s of cores
• Storage services that can scale to petabytes of data
• Interactive batch processing systems
• Services to stream real time information
• APIs to access data from various transformation jobs

11. 9
Job Management
System
Authorization/
Authentication
Service
Identity
Management
Telemetry System
Compute
Node
Compute
Node
Object Store
Jobs

12. 10

13. Infrastructure Stack of Circa 2017
Locking and Co-ordination Management
Cluster Schedulers
Block Storage
Service
Discovery
Telemetry
Services
Object Store
Microservices
Batch
Processing
LBaaS
Identity
Management
Tracing Services
Applications
The Platform

14. Platforms as a Service
Locking and Co-ordination Management
Cluster Schedulers
Block Storage
Service
Discovery
Telemetry
Services
Object Store
Microservices
Batch
Processing
LBaaS
Identity
Management
Tracing Services
Applications
Release Management
Build Infrastructure
Configuration Management

15. 13
Serverless Architecture

16. Serverless Architecture
Function
ControllerRequests
Application
Logic
Application Container
Payload

17. Serverless Architecture
• Incremental evolution over PAAS
• Serverless from the perspective of users
• Clear separation of concerns between
infrastructure engineering and application
developers

18. Serverless Platform
• Optimistically concurrent Cluster Scheduler
• Application Container
• Function Controller
• RPC Middleware
• Function Invocation shims in data sources
• Packaging and distribution service

19. Cluster Scheduler
• Low latency and high throughput scheduling
• No head of line blocking
• Optimistically concurrent
• Scale up to a large number of containers

20. 18

21. 19
THE PATH TO PRODUCTION FOR A SCHEDULER IS FILLED WITH PAIN
CHOOSE AN EXISTING SCHEDULER IF YOU CAN

22. Event Driven Scheduler
Evaluations ~= State Change Event

23. Data Model

24. 22
External Event
Evaluation Creation
Evaluation Queuing
Evaluation Processing
Optimistic Coordination
State Updates

25. Templated Jobs
• Can be used for low throughput use cases
• Expensive to start new processes
• Bounded by network operations of scheduler

26. Variability
• Sharing resources
• Daemons
• Global resources
• Maintainance activities
• Queuing
• Power Limits

27. 25
Source: Tale at Scale

28. Reduce Variability
• Differentiate service classes
• Break down long running functions into small ones.
• Minimize background activity

29. Application Containers
• Application containers range from language sandboxes
to full blown containers
• Reduced surface area helps in making the UX better
• Application containers have function invocation
middleware which are packaged during build

30. Function Controller
• Maintains the network topology of functions
• Optimizes for low latency and not cost
• Works with the scheduler to scale containers
• Works with telemetry system to scale underlying
clusters

31. Software Load Balancer
• Service discovery aware LB
• Reactive Socket protocol for delivery of request payloads to
function containers
• Doesn't effect container life cycles
• Drops requests which doesn't have any containers associated

32. Reactive Sockets
• RSocket is a means for asynchronous communication between functions and
data sources
• Messages are multiplexed over a single connection
• Flexible interaction models
• Provides mechanisms for back pressure and request cancellation

33. RPC Middleware
• RPC implementation moved into the platform
• Hedged requests to reduce tail latency
• Tied requests to reduce mean and tail latency
• Heavy use of back pressure

34. Cluster Topology
• Scheduler chooses the cluster topology for containers
• Function Controller can change the topology based on
real time latency
• Functions which are chained needs to be placed as
close as possible

35. Caching
• Various forms of caching
• Depends on life cycle management of functions
• External caches like Redis, Memcached works best
• Hard to use in-memory clustered caches like
Groupcache

36. External Data Sources
• Functions can read from external data sources just like
traditional applications
• Avoid polling external data sources
• Data Sources like Kafka could co-operate with functions by a
shim which calls functions with data mutations

37. Build and Distribution
• Packaging is part of the server less experience
• Packaging pipeline should transform a function to an
application container.
• Unit Testing should be much easier with Functions than
they were for traditional applications
• Performance of the platform can be improved by caching
containers on compute nodes

38. Operations in the world of Serverless
• Reduced amount of operations for developers
• Metrics related to various concerns of the platform should
be very targetted.

39. Application metrics
• Latency at the API Gateway level
• Throughput of events processed
• Latency distribution and planning.

40. Platform Metrics
• Latency of function invocation
• Throughput of events dispatched to the
platform
• Number of active functions

41. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
serverless-scientific-computations