This document discusses strategies for scaling cloud services across the enterprise. It recommends adopting a cloud factory approach using DevOps processes to provide on-demand application development infrastructure. This enables cost-effective development, collaboration, and deployment while establishing best practices and process repeatability. Quick start use cases are proposed around DevOps tooling, automated governance, elastic scaling, and consumption-based pricing to demonstrate value. Key metrics include time to provision resources, scale frequency, and cost per user.
Scaling API-first – The story of a global engineering organization
Scale Cloud across Enterprise
1. Scale Cloud
Across the Enterprise
Chris Haddad
Vice President, Technology Evangelism
Follow me on Twitter @cobiacomm
Read architecture guidance at http://blog.cobia.net/cobiacomm
2. Skate towards the puck
Business Value
Best Practice Adoption
Process Repeatability
Fast time to value
3. Cloud Business Value For Development Teams
• Lower development barriers
• Lower adoption barriers
3
4. Best Practice Adoption and
Process Repeatability
• Cost-effective, development, collaboration, and
deployment infrastructure enabling a long tail of
application development
• A shared environment for cross-organization
application development and delivery
• Infrastructure enabling user experience composition
across multiple disparate application providers
5. Fast Time to Value – Increase Agility
• Rapidly adapt and fulfill new market demand
• Reduce time to introduce new services,
applications, and products into long tail market(s)
6. Enterprise Architecture
Business Model Challenges
• Unable to cost-effectively support long tail of
application delivery
• Lack of metrics and baseline measurements
• Limited ability to influence adoption
• High friction and interaction cost when building
cross-team business processes
• Inability to capitalize on business opportunities
due to long IT solution lead times
12/9/2012 6
7. Cloud Business Value Blockers
• Silo owners
• Adoption curve
• Trust barrier
• Unknown baseline
• Funding model
Source: http://blog.industryweapon.com/2010/01/beware-of-the-business-blocker/
8. To succeed and scale Cloud –
Build a Digital Ecosystem Platform
Your Core Business
Capability
Your Digital
3rd Party
Business
Partners
Ecosystem
Digital
Ecosystem
Personalized SaaS DevOps Forge
Solution and Governance
App Store Cloud Deployment
8
9. New Enterprise Architecture Approach
• Self-organization
• Crowdsourcing – source apps and capabilities from
crowd
• Dynamic distribution channels and fluid connections
• Adaptation and evolution
• Local autonomy leading to unforeseen ecosystem partner
extensions
• Spontaneous selection and aggregation across services
and solutions
• Low interaction cost
• Frictionless information flow
• Enables cost effectively serving a smaller, niche audience
12/9/2012 9
10. Recommendations from the Coach
Business Value
Best Practice Adoption
Process Repeatability
Fast time to value
11. Cloud Business Value For Development Teams
• Lower development barriers
• Provide on-demand Application Development project
infrastructure and run-time environments
• Catalogue of re-usable open APIs, cloud services,
and domain frameworks
• Lower adoption barriers
• On-demand web application and Cloud API
subscriptions via a self-service provisioning portal
• Establish searchable registry of app, service, api, and
data descriptors
• Reliable, available, and scalable solutions
11
12. Best Practice Adoption and
Process Repeatability
• Cost-effective, development, collaboration, and
deployment infrastructure enabling a long tail of
application development
• Architecture templates and application platform services
• A shared environment for cross-organization
application development and delivery
• Governed, iterative lifecycle management across hybrid
clouds and composite applications
• IT Business performance metrics and analytics
• Infrastructure enabling user experience composition
across multiple disparate application providers
13. Fast Time to Value with
On-demand Contextual Personalization
• Increase agility
• Rapidly adapt and fulfill new market demand
• Reduce time to introduce new services, applications,
and products into long tail market(s)
• On-demand Contextual Personalization
• Information access and social network access privileges
• Information aggregation and composition
• Business processes and rules
• Service levels, Quality of Service, and monetization rates
• Security policies
14. Fast path to goals!
Cloud DevOps Factory
Platform as a Service
API Management
Business Activity Dashboards
16. Scale best practices with
Cloud Factories
Cloud
Factories
Development
Forges
Application
Lifecycle
Management
Integrated (ALM)
Development
Environments
(IDE)
Command
Line
Compilers
and
Linkers Time 2012+
12/9/2012 16
17. What is a Cloud Factory?
A Cloud-based, automated, DevOps approach to agile application
design, development, delivery, and visibility
• Simplifies the developer experience
• On-demand self service and Cloud provisioning
• Continuous build, continuous integration, continuous test, and
continuous delivery
• Governed, iterative lifecycle management across hybrid
clouds and composite applications
• Architecture templates and application platform services
• IT Business performance metrics and analytics
• Builds a Digital Business Ecosystem
• Enable a long tail of application development
• Fosters cross-organization development collaboration
18. Open Source DevOps, Agile, and Cloud Tooling
Project and
Team
Management Software
Continuous
development
Deployment
workflow
Governance
Test
and
Automation
Compliance
Continuous Development
Integration Dashboards
Continuous Develop
Build Code
12/9/2012 Source Issue 18
Control Tracking
19. Platform as a Service Capabilities
• Self service • Service catalogue
configuration • Service tiers
• Automated provisioning • Demand and capacity
• Policy configuration management
• Process automation • Lifecycle management
• Infrastructure Authority
integration
Dev Ops Automated
Tooling Governance
Consumption
Service Level
based
Management
Funding
• Metering • Resource monitoring
• Reporting • Resource management
• Billing • Performance
management
• Traffic orchestration
20. Cloud-aware Reference Architecture
Cloud Platform
Platform as a Service Run-time Framework (i.e. WSO2
Cloud Stratos, Cloud Foundry, RedHat OpenShift)
Management Service-aware, tenant-aware
Elastic Load Balancer
PaaS
Manager Stratos Controller Cloud Native Container(s)
Cloud Asset/Code Deployer Tenant
Governance Tenant (n)
1
Identity Asset/Code Synchronizer Application Platform
Management Services (web server,
Metering and Billing database, ESB)
Asset Repositories and
Registries(tenant code, Infrastructure as a Service
service endpoints, meta-data, (AWS, Eucalyptus,
configuration, policies) OpenStack, CloudStack)
21. Platform as a Service Reference Architecture
aPaaS
iPaaS
Carbon Any
Carbon ESB Other Carbon PHP
App Server Pluggable
Cartridge Cartridges Cartridge
Cartridge Cartridge
Relational Column File Task
Message Logging Security Registry Billing
Data Storage Storage Mgmt
Service Service Service Service Service
Foundation
Service Service Service Service
PaaS
Stratos Controller
Elastic Cloud Artefact
Deployment Management Load
Load Controller/ Distribution
Synchroniser
Autoscaler Controller Console Monitor
Balancer
IaaS
Infrastructure Cloud (EC2, vmWare, Rackspace, OpenStack, Eucalyptus, etc)
27. Cloud Strategy Goals
1. Decrease solution provisioning time
2. Right-size infrastructure, improve resiliency, and
enhance Quality of Service (QoS)
3. Accelerate software development iterations
4. Share across multiple users and groups
5. Integrate and bridge heterogeneous Cloud
environments
6. Optimize Cloud footprint and lower application
cost
28. Cloud Characteristic Roadmap
Cloud
On-demand Consumption
Elastic Scalability Resource Pooling
Self-service based Funding
Multi-tenancy Provisioning Usage Policies Portfolio Models
Service Activity Metering and
Federation
Catalogue Monitoring Billing
29. 7 +/- 2 Objectives
1. Engage stakeholders in a collaborative
development workspace
2. Promote best practice workflow,
architecture, and governance practices
3. Deploy applications into a Cloud run-
time environment
4. On-demand application subscriptions
via a self-service provisioning portal
5. Share applications across multiple
tenants (e.g. departments,
workgroups, employees, partners)
6. Scale run-time to meet usage
7. Deploy Open APIs
8. Encourage API adoption via API Store
9. Track business activity and analyze
Cloud service usage, performance,
and cost
29
30. Sharing Business Capabilities at Scale
Business
Functional Proces
Presentation Business
and Mashups Role Process
Presentation Functional Business
Role Business
Role Functional Process
Presentation Process and
and Mashups Role Business
Presentation Functional
code Rules
and Mashups
Client Tier
Resource
Integration Services Tier
Resource
Services
Resource
Services
Integration Services
Public Cloud Services
Private
Applications
31. Scaling through API composition
Composite Mobile
Infrastructure as a
Application Application Service (IaaS)
SaaS Application
Cloud API
Web
Endpoint application
Endpoint
Integration Services
Business Service Portfolio Application Portfolio
Presentation Hybrid
Services Platform
Business
Process and Functional Services
Rules Services
Services
Resources Services
Integration Services
32. Scaling across Regions and Zones
Multiple IaaS (hybrid) Deployment
HIGHEST
Zone 1
Private cloud (data center) Zone 2
Zone 1
Zone 2
Amazon EC2
Zone 1
Availability
Cost
Zone 2
LOWEST
Rackspace Cloud
32
33. Single Node
Primary-Secondary, single LB
Primary-Secondary,
with multiple LBs
at a cost
Multi-node active
cluster
- Single zone
Multi-zone
Multi-region
Multi-IaaS
Cloud Scale = High Availability
33
34. Scaling Best Practices
New-Normal Cloud architecture practices
• Distributed and federated interactions
• Event based, heterogeneous systems, network latency
• Configurable containers and engines
• Declarative data, rules, and process definitions
• De-normalized and simplified data models
• Hadoop/BigTable, Hypertext media, simple NoSQL entities
• Expect failure
• Systems span transactional control
• Applications decomposed into distinct services
• Federated environment drives autonomy, statelessness, and
composition
35. Mission-critical Cloud
requires new architecture components
Availability Scalability Management
State replication Tenant partitioning Management nodes
Private jet mode
Session replication Ghost deployment Logging infrastructure
Multiple load BAM 2.0 architecture Deployment
balancers with synchronization
keepalived or DNS RR
Auto-scaling
Elastic Load Balancer
37. Tenant Scaling Method Impacts Value
Proposition
Single Application Tenant per Multiple Application Tenants per
Shared Application Platform
Dedicated Application Server Service Container
Container
39. Cloud Topology and Value
Three (3) Container Instances, Three Tenants
Tenant
Application Solution Traffic
Web
Service Partition ESB
Application
Container ESB-1 ESB-2 AS-1
Tenant Context Tenant-1 Tenant-3 Tenant-2 Tenant-1 Tenant-2
40. Scaling a Cloud Business
Measuring Revenue versus Cost
Source: http://giffconstable.com/2009/11/freemium-business-model-template/
41. Scale Strategy and Tactics
• Gain Credibility
• Demonstrate improved resiliency and QoS
• Converge teams
• Offer standard shared application platform services
• Establish and promote the service catalogue
• Be inclusive
• Support multiple development languages,
environments, and use cases
• Lower barriers to entry (e.g. financial, tooling, skills)
42. 7 +/- 2 Objectives
1. Engage stakeholders in a collaborative
development workspace
2. Promote best practice workflow,
architecture, and governance practices
3. Deploy applications into a Cloud run-
time environment
4. On-demand application subscriptions
via a self-service provisioning portal
5. Share applications across multiple
tenants (e.g. departments,
workgroups, employees, partners)
6. Scale run-time to meet usage
7. Deploy Open APIs
8. Encourage API adoption via API Store
9. Track business activity and analyze
Cloud service usage, performance,
and cost
42
45. Key Metrics
Foundation
Time to create new application environment
Time to redeploy application
• Optimize
Minimum and maximum scale
Scale frequency (i.e. time to scale up/down)
• Transformation
Time and effort required integrating business process, event
processor – creating a complex app.
Time and effort required to apply policy across tenant(s)
Cost to operate application per user or transaction
47. Quick Start Use Cases
1. DevOps Tooling and On-demand self-
service
2. Automated Governance
3. Service level management and elastic scale
4. Consumption based pricing and billing
48. Quick Start Use Cases
DevOps Tooling and On-demand self-service
• Use Cases
• Rapidly provision application environment
• Rapidly provision application tenant
• Allocate, provision, monitor, manage, and administer
resources across multiple tenants, nodes, and locations
• Develop complex, composite integrated applications
• Key metrics
• Time to create new application environment
• Time to redeploy application
49. Quick Start Use Cases
Automated Governance
• Use Cases
• Create users and configure rights
• Deploy on preferred topology that meets deterministic
performance requirements (e.g., replication, utilization,
latency, bandwidth, and coherency)
• Create service throttling and security governance
(XACML) policies
• Key Metrics
• Time and effort required integrating business process,
event processor – creating a complex app.
• Time and effort required to apply policy across tenant(s)
50. Quick Start Use Cases
Service level management and elastic scale
• Use cases
• Ensure application satisfies consumer demand while
maximizing resource utilization
• Scale workload processing and increase performance while
minimizing infrastructure spend
• Load test application service
• Demonstrate multi-tenant web application
• Key Metrics
• Minimum and maximum scale
• Scale frequency (i.e. time to scale up/down)
51. Quick Start Use Cases
Consumption based pricing and billing
• Use cases
• View service logs
• View bill by business value
• Key Metric
• Cost to operate application per user or transaction
Notes de l'éditeur
In this session, Chris Haddad will describe where your Cloud application strategy may encounter challenges, when to migrate applications to the Cloud, and how to scale cloud strategy and cloud tactics across the organization.45 minute version
eTrade: API, consumers must submit application for code review* compliance focus * *automation* enables long tail *Cloud is a game changing enabler use cases
What is It that customers want and how might it construct a new business model or even an ecosystem of partnerships and external relationships to provide that new value to them?Value creation via certain business processes and activities ( also refered to as value network, value chain, etc.) and a revenue model for value captureCreate value: processes/activities , resources/assetsCapture value: cost, financial aspect, profitValue network: suppliers, customer information, customer relationships, information flows, product/service flowsStrategic choices: customer (target market / scope), value proposition, capabilities/competencies, revenue/pricing, competitors, output (offering), strategy, branding, differentiation, missionSOURCE: Shafer et al, 2005, p.202Infrastructure: Partners, key activities, key resourcesOffering: value propositionCustomer interface: relationships, customers, distribution and acquisition channelsFinance: cost/structure, revenue stream / pricingSource: adapted from Osterwalder & Pigneur 2009
eTrade: API, consumers must submit application for code review* compliance focus * *automation* enables long tail *Cloud is a game changing enabler use cases
Apply DevOps activities across software development life-cycle phases (i.e. design, develop, test, build, deploy, manage). Teams use DevOps tools and processes to manage application construction and maintenance tasks. Iterative, incremental, continuous, automated, self-service
Message: win for developers - simplifies the developer experience during build processeTrade: API, consumers must submit application for code review* compliance focus * *automation* enables long tail *Cloud is a game changing enabler use cases
Workflow – bps and brsGovernance and compliance – wso2 registryDevelopment dashboard – BAM, Jaggery
Some solution architects find Cloud characteristics and supporting PaaS attributes too abstract and infrastructure focused. Architects may be more interested in delivering measurable business value, shielding IT personnel from complex dependencies, and deliver a productive development and operations (i.e. DevOps) environment. The following PaaS capabilities (See Figure 4) are used to achieve these objectives:DevOps ToolingAutomated GovernanceService Level ManagementConsumption based pricing Figure 4: Platform as a Service (PaaS) Capabilities and supporting practices DevOps ToolingDevOps tooling creates an environment fostering collaboration between development and operations team members. Practice and tooling enable teams to implement self-service configuration, automated provisioning, policy configuration, and process automation practices which bridge the design, build, deploy, and manage phases within the software development life-cycle. By integrating DevOps tooling with on-demand resource instances, teams can reduce time to market and increase agility. Automated GovernanceGovernance is a practice, which defines policies, people, and processes. Effective governance mitigates risks, improves performance, and facilitates correct actions. Automated governance enables application and infrastructure services to efficiently scale across numerous consumers and providers while effectively monetizing, maintaining, and securing assets and consumer-provider interactions. By publishing a service catalogue offering tiered levels of service, teams can promote standard offerings that meet customer requirements. By streamlining access and approval, automated governance encourages customers to choose standard offerings and reduce cost. Scaling a Cloud environment while right-sizing available capacity is non-trivial, and the infrastructure must support demand management and capacity management activities. When organizations move beyond their first Cloud service release, automated lifecycle management becomes a predominant concern. To effectively manage the service lifecycle, the infrastructure must report on service versions, subscribed consumers, and usage trends. In the run-time environment, an infrastructure authority component makes resource allocation decisions, which are enforced by service level management components. Service Level ManagementService level management enforces governance policies. PaaS infrastructure should integrate service level management activities throughout the solution stack (i.e. network, processing, storage, managed code container, application platform engines, and application logic). Resource monitoring, resource management, performance management, and traffic orchestration must monitor, manage, and optimize machine node instances, message routing, application service location, tenant security, and session state. Intelligent service level management on Cloud native PaaS infrastructure has the ability to raise infrastructure utilization while maintaining quality of service. Consumption Based PricingToday, cloud consumption based pricing reflects IT asset monetization (e.g. machine instance per hour, network I/O, storage bytes). However, business users don’t really care how many instances are running in the Cloud. Business users care about business entities, business activity performance, and associated cost. Table 1 below illustrates various pricing units. For example, the number of market leads generated by a marketing piece, or cost to process an insurance policy. Decoupling metering and billing from IT assets and shifting the reporting model to focus on business activity and holistic IT cost will positively change the IT investment conversation. Coupling multi-tenant metering and billing with business activity monitoring and reporting will facilitate the shift. “Cloud consumers of PaaS can employ the tools and execution resources provided by cloud providers to develop, test, deploy and manage the applications hosted in a cloud environment. PaaS consumers can be application developers who design and implement application software, application testers who run and test applications in cloud-based environments, application deployers who publish applications into the cloud, and application administrators who configure and monitor application performance on a platform. PaaS consumers can be billed according to, processing, database storage and network resources consumed by the PaaS application, and the duration of the platform usage.”Source: http://collaborate.nist.gov/twiki-cloud-computing/pub/CloudComputing/ReferenceArchitectureTaxonomy/NIST_SP_500-292_-_090611.pdf“For PaaS, the Cloud Provider manages the computing infrastructure for the platform and runs the cloud software that provides the components of the platform, such as runtime software execution stack, databases, and other middleware components. The PaaS Cloud Provider typically also supports the development, deployment and management process of the PaaS Cloud Consumer by providing tools such as integrated development environments (IDEs), development version of cloud software, software development kits (SDKs), deployment and management tools. The PaaS Cloud Consumer has control over the applications and possibly some the hosting environment settings, but has no or limited access to the infrastructure underlying the platform such as network, servers, operating systems (OS), or storage.”Source: http://collaborate.nist.gov/twiki-cloud-computing/pub/CloudComputing/ReferenceArchitectureTaxonomy/NIST_SP_500-292_-_090611.pdf
The Asset/Code deployer will provision tenant specific applications, services, and business process The artifacts are deployed into one or more containersContainers are partitioned based on ability to isolate and share resourcesSingle tenant containers dedicate one or more containers for exclusive use by an application tenant (during a period of time)Cloud Native, Multi-tenant containers share containers across multiple simultaneous application tenants
A Platform as a Service offering should promote deploying applications onto a flexible, distributed topology. To maximize Cloud characteristics, a PaaS should facilitate scaling way out (e.g. across cloud zones, data centers) and automatically distribute fine-grained service component resources. Figure 6 presents a logical view of a cloud application executing across a distributed topology. The Integration Services PaaS service component is used to connect application service components and external cloud services by message passing, not function invocation. Integration services commonly include a Enterprise Service Bus (ESB), service governance registry, service gateways, and message brokers. Figure 6: Cloudy Topology
Tenant = application
Service-level partitions are more efficientAll solutions deployed within a single partitionAll tenants deployed within a single partition
To perform an activityTo build and deploy solution
Agreeing on a strategic goal and gaining stakeholder buy-in is only the first step. Teams should define key metrics, baseline current performance, and create a quick start project plan to prove PaaS benefits. Metrics can be divided into foundational, optimal, and transformational categories. A few example metrics to consider are: FoundationTime to create new application environmentTime to redeploy applicationOptimizeMinimum and maximum scale Scale frequency (i.e. time to scale up/down)TransformationTime and effort required integrating business process, event processor – creating a complex app.Time and effort required to apply policy across tenant(s)Cost to operate application per user or transaction
In today’s IT environment, demonstrating tangible improvement is often illusive. Rather than simply stating a goal, randomly selecting a PaaS provider, installing an application on the PaaS, and declaring success, your team has an opportunity to demonstrate and prove how PaaS capabilities can improve agility, efficiency, or platform economics. Your team should correlate quick start demonstration use cases with PaaS capabilities and key metrics. The following listing details a sample plan:
DevOps Tooling and On-demand self-serviceUse CasesRapidly provision application environmentRapidly provision application tenantAllocate, provision, monitor, manage, and administer resources across multiple tenants, nodes, and locationsDevelop complex, composite integrated applicationsKey metricsTime to create new application environmentTime to redeploy application
Automated GovernanceUse CasesCreate users and configure rightsDeploy on preferred topology that meets deterministic performance requirements (e.g., replication, utilization, latency, bandwidth, and coherency)Create service throttling and security governance (XACML) policiesKey MetricsTime and effort required integrating business process, event processor – creating a complex app.Time and effort required to apply policy across tenant(s)
Service level management and elastic scaleUse casesEnsure application satisfies consumer demand while maximizing resource utilizationScale workload processing and increase performance while minimizing infrastructure spendLoad test application serviceDemonstrate multi-tenant web applicationKey MetricsMinimum and maximum scale Scale frequency (i.e. time to scale up/down)