Monitoring the Dynamic Nature of Cloud Computing

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lee Atchison,Principal Cloud Architectand Advocate atNew Relic,Inc.
Life’s Too Short…
... for Cloud without Analytics Monitoring the Dynamic Nature of Cloud Computing

2. Safe Harbor
This document and the information herein (including any information that may be incorporated by reference) is provided for
informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
(“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is
proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import.
Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof,
and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting
business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success
of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs
and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results,
performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to
time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at http://ir.newrelic.com or the
SEC’s website at www.sec.gov.
New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

3. Who am I?
Specialize in:
Cloud computing
Services & Microservices
Scalability, Availability
28 years in industry
7 in Amazon Retail & AWS
(BuiltSW/VG AppStore,AWS Elastic Beanstalk)
4 in New Relic
(Architecture Lead,Cloud,Service Migration)
@leeatchison leeatchison

4. We want better apps faster
Dynamic
Environment
Better
Data Center
How do we use the cloud to accomplish this?

5. Cloud as a Better Data Center

6. Cloud as Better Data Center
Resources
are allocated to
uses, just like in
a data center

7. Provisioning
process is faster
Cloud as Better Data Center
Resources
are allocated to
uses, just like in
a data center

8. Cloud as Better Data Center
Resources
are allocated to
uses, just like in
a data center
Lifetime of
components is
relatively long
Provisioning
process is faster

9. Cloud as Better Data Center
Lifetime of
components is
relatively long
Provisioning
process is faster
Capacity planning
is still important
and still applies
Resources
are allocated to
uses, just like in
a data center

10. Why use a “Better Data Center”?

11. Why use a “Better Data Center”?
Add new
Capacity
(faster)

12. Why use a “Better Data Center”?
Improve
Application
Availability
(redundancy)
Add new
Capacity
(faster)

13. Why use a “Better Data Center”?
ComplianceImprove
Application
Availability
(redundancy)
Add new
Capacity
(faster)

14. Who is impacted?
Better Data
Center
Operations Development

15. Who is impacted?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations Development

16. Who is impacted?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development

17. Who is impacted?
Better Data Center
Faster Application
Launch / Deploy=
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development

18. How do I monitor it?
Similar to monitoring any other data center…

19. Monitoring an application
Typical Server / EC2 Instance
• Application &
Application Microservices
• Server OS
• Hardware (virtual)
EC2 Instance
Application &
Application
Microservices
Server OS
Server (Virtual)
Hardware

20. AWS Monitoring
CloudWatch
• Monitors:
– EC2 instance
– Virtualization
– Hardware
– [CPU/Disk/Networking]
• Doesn’t know about:
– Server OS
– Memory/Filesystem
– Processes
– Application
• Latency
• Error rates
• Internal insights
EC2 Instance
Application &
Application
Microservices
Server OS
Server (Virtual)
Hardware
CloudWatch
AWS CONSOLE

21. New Relic Monitoring
New Relic
• Monitors (Server):
– How O.S. is performing
– Processes
– Hardware
• Monitors (Application):
– App health
– App performance
– Microservices
• Doesn’t know
– Virtualization
EC2 Instance
Application &
Application
Microservices
New Relic
Application
Monitoring
Server OS
Server (Virtual)
Hardware
New Relic
Server
Monitoring
CloudWatch
AWS CONSOLE
DASHBOARDS

22. AWS çè New Relic work together
EC2 Instance
Application &
Application
Microservices
New Relic
Application
Monitoring
Server OS
Server (Virtual)
Hardware
New Relic
Server
Monitoring
CloudWatch
AWS CONSOLE
DASHBOARDS
AWS / CloudWatch
• Visibility into virtualization
• CPU / Disk / Networking
New Relic
• CPU / Disk / Networking
• Memory / Filesystem
• Processes
• Infrastructure components
• Application / Microservices:
– Latency
– Error rates
– App insights CloudWatch
monitors
New Relic
monitors

23. Cloud as a Dynamic Tool

24. Cloud as a “Dynamic Tool for Dynamic Apps”
Use only
the resources
you need

25. Cloud as a “Dynamic Tool for Dynamic Apps”
Use only
the resources
you need
Allocate /
de-allocate
resources on
the fly

26. Cloud as a “Dynamic Tool for Dynamic Apps”
Use only
the resources
you need
Resource
allocation is an
integral part of
your application
architecture
Allocate /
de-allocate
resources on
the fly

27. Dynamic Cloud
Resources are:
Application is aware of
and is controlling
traditional OPs resourcesDe-allocatedConsumedAllocated

28. Dynamic Cloud
Docker containers:
• Quicker / cheaper startup / shutdown
than EC2 instances
• Easy to consume Docker containers
much like you do “processes”
• Perform jobs:
– In a script
– Off of some queue of job requests

29. Dynamic Cloud
100
10K
1M
3.7 M
83 days 333 days
Docker Container Age
(Count vs. Hours)

30. Dynamic Cloud
950,000
11% under one minute
Docker Container Age
(by Minute Under and Hour)

31. EC2 Auto Scaling
Dynamic Cloud Technologies
Mobile / IoT
Dynamic routing
Load balancing
Data transport
Queues and notifications
Docker
Dynamic Cloud is about scaling

32. How do I monitor the Dynamic Cloud?
Dynamic Cloud has unique monitoring requirements…

33. Responsibility (Dynamic Cloud)
Server OS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
ProvisioningServer (Virtual)
Hardware
• Multiple Applications &
Application Microservices &
Dynamic services
• Servers &
Virtual Hardware &
Infrastructure

34. AWS Infrastructure
Server OS
Server (Virtual)
Hardware
CloudWatch
AWS CONSOLE
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning

35. AWS Infrastructure çè New Relic work together
Server OS
Server (Virtual)
Hardware
CloudWatch
AWS CONSOLE
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning

36. AWS Infrastructure çè New Relic work together
Server OS
Server (Virtual)
Hardware
CloudWatch
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
CloudWatch
& AWS
monitoring
New Relic
monitors
AWS
CONSOLE

37. How do you monitor this?
Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
?
How do you monitor this?

38. Where did it go? It was just here!!!
? The thing you monitored 10 minutes ago…
...doesn’t exist anymore!

39. Dynamic Cloud
950,000
11% under one minute
Docker Container Age
(by Minute Under and Hour)

40. Monitoring the Dynamic Cloud
Very different than monitoring traditional Data Center components
Monitor the
Cloud Components
themselves
Monitor the
lifecycle of the
Cloud Components

41. Who Uses It?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development
Dynamic
Cloud

42. Who Uses It?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development
Dynamic
Cloud
What is a container?
What is a Lambda?
It was just here, where did it go?

43. Who Uses It?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development
Dynamic
Cloud
What is a container?
What is a Lambda?
It was just here, where did it go?
Cloud architecture is integral
to the application architecture
Developers deeply involved
in cloud activities

44. Change is speeding up
Traditional
Data Center
Cloud
Data Center
Dynamic
Cloud
Dynamic Cloud enables better applications faster.
The rate of change is increasing…
Good Better Best

45. Dynamic Cloud
EC2
Server running
application / processes
Docker container
Process running
a command
This is
hard

46. Dynamic Cloud
EC2
Server running
application / processes
Docker container
Process running
a command
Lambda
Function performing a
task or operation

47. The Future with Lambda

48. Microcomputing & AWS Lambda
• Newest entrance to the “dynamic cloud”
• Provides event driven compute capabilities

49. Microcomputing & AWS Lambda
• No infrastructure to provision
• Massively shared infrastructure

50. Why use Lambda?
Lambda scripts
Run in response to
a state change or
action in the cloud
Stateless, “filters”
Perform quick actions
Virtually no
startup/shutdown
cost

51. AWS Lambda
S3
Bucket
Dynamo
DB
API
Gateway
SQS
RESOURCESSOME
• Takes an event from an
AWS resource (A Trigger)

52. AWS Lambda
S3
Bucket
Dynamo
DB
API
Gateway
SQS
RESOURCESSOME
Lambda
Script
• Takes an event from an
AWS resource (A Trigger)
• Creates an instance
to execute
Lambda
Instance

53. AWS Lambda
S3
Bucket
Dynamo
DB
API
Gateway
SQS
RESOURCESSOME
S3
Bucket
API
Gateway SQS
RESOURCESSOME
Lambda
Script
• Takes an event from an
AWS resource (A Trigger)
• Creates an instance
to execute
• Can impact original or
different AWS Resource
Lambda
Instance

54. AWS Lambda
S3
Bucket
Dynamo
DB
API
Gateway
SQS
RESOURCESSOME
S3
Bucket
API
Gateway SQS
RESOURCESSOME
Lambda
Script
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
Lambda
Instance
• Takes an event from an
AWS resource (A Trigger)
• Creates an instance
to execute
• Can impact original or
different AWS Resource
• Any number of instances
can run at a time

55. Photo Management App
Lambda example #1

56. Photo management application
User
S3 Bucket
Image
Import
• Photos uploaded to S3 Upload
File

57. Photo management application
User
Upload
File
S3 Bucket
Image
Import
S3 Bucket
Image
Thumbnails
Lambda
Script
• Photos uploaded to S3
• Lambda script creates
thumbnails
View
Thumbnail

58. Photo management application
User
Image
Database
S3 Bucket
Image
Import
S3 Bucket
Image
Thumbnails
Lambda
Script
Lambda
Script
• Photos uploaded to S3
• Lambda script creates
thumbnails
• Lambda script updates
metadata in database
Upload
File
View
Thumbnail

59. Photo management application
User
Image
Database
Application
S3 Bucket
Image
Import
S3 Bucket
Image
Thumbnails
Lambda
Script
Lambda
Script
• Photos uploaded to S3
• Lambda script creates
thumbnails
• Lambda script updates
metadata in database
• Application only has
to deal with metadata
editing, not photo /
file management
App
Interactions
Upload
File
View
Thumbnail

60. Mobile Game App
Lambda example #2

61. Mobile game platform
API
Gateway
Lambda
Script
Lambda
Script
Lambda
Script
Mobile Phone
Application Users
Database
• Cloud platform hosts an
API for mobile app
– API Gateway
• Lambda scripts implement
the API
• Lambda scripts
manipulate database
• Extremely high
scale possible
– No infrastructure

62. Monitoring
Lambda Scripts
Less like server monitoring /
More like web application monitoring
We
Care About
We Don’t
Care About
• Run time
(average, extremes –
TP90/TP99)
• Statistical metrics
• Error rates and other
deviations from norm
• “Drill down” into
individual “runs”
• Details about
all ”runs”
• Server /
infrastructure
metrics

63. Monitoring Lambda
More like application performance monitoring than
infrastructure monitoring

64. Monitoring Lambda
More like application performance monitoring than
infrastructure monitoring

65. Change is speeding up
Traditional
Data Center
Cloud
Data Center
Dynamic
Cloud
Dynamic Cloud enables better applications faster.
The rate of change is increasing…
Good Better Best
The way you’ve done things in the past won’t work in the future.

66. Monitoring just the server
EC2 Instance
Application &
Application
Microservices
Server OS
Server (Virtual)
Hardware
CloudWatch
AWS CONSOLE
Worked when rate of change was low…

67. Monitoring just the server
Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
Insufficient in the cloud:
• Rate of change is faster
• Problems come up quicker
• “Server” isn’t a server anymore
• “Provisioning” isn’t provisioning anymore
You need top to bottom monitoring…
You need full stack accountability...

68. New Relic enables accountability between
your code & AWS
Customers
Mobile / Apps / Browser
On-Premises
On Premises
Relational Data
EC2
AWS
Container Service
AWS RDSSynthetic
Customers
S3
NoSQL
AWS Storage
Services
Your Responsibilities
• Application code & 3rd
party services
• App architecture
• Customer experience
• On-Premises components
AWS Responsibility:
• Service Availability & Performance
• Provisioning & operation of DC
• Isolation & security of resources
• Service maintenance & innovation
NoSQL

69. Want to Learn More?

70. Architecting for Scale
By: Lee Atchison
Published by: O’Reilly Media, Available: June 2016
www.architectingforscale.com
Want to Learn More?

71. Fabien Ruffin
Technology Director for Group Services
Domain Group

72. Fabien Ruffin
Technology Director for Group Services
Domain Group
Fabien will be speaking at:
NDC Sydney
August 1-5, 2016
ndcsydney.com

73. Thank You!
Lee Atchison, Principal Cloud Architect and Advocate at New Relic, Inc.