A 100-200 level introduction to building hybrid ops and apps on the AWS cloud. Including DirectConnect, VPC, IAM, hybrid Monitoring and Management tools (Packer, Terraform, New Relic, Splunk etc.) and application architectures
12. • Secure, flexible networking between
cloud and on-premise
• Secure, federated access
management
• Management tools for hybrid
environments
• Integrated monitoring tools
HYBRID OPS - REQUIREMENTS
13. • Secure, flexible networking between
cloud and on-premise
• Secure, federated access
management
• Management tools for hybrid
environments
• Integrated monitoring tools
HYBRID OPS - REQUIREMENTS
14. Secure, flexible connectivity
OPS | NETWORKING
AWS DirectConnect
• Extend your data center network to the
AWS cloud using a leased-line/circuit
• Secure, consistent performance on a
private network - avoid internet traversal
• Lower data transfer costs (vs VPN)
• 1Mbps to multiple 10Gbps
• Simpler management of multi-VPC
environments
• IPSEC VPNs can also be used for small
deployments, POCs and extra redundancy
15. Secure, flexible networking
OPS | NETWORKING
AWS Virtual Private Cloud
• Create a software-defined network
topology for your cloud including private
and public subnets (RFC1918), routing,
firewall policies and NAT
• Connect VPCs together using peering, or
directly to your data center and offices
Implement network isolation at any level, e.g.
• App environment, tier, business unit, team,
application / project and data classification
16. OPS | NETWORKING
Your Data Center
IPSEC VPN
Tunnels(x2)
AWS DirectConnect
Peering Location
Circuit(s), e.g
Metro Ethernet
AWS
Fibre cross connect
Terminated on an AWS
or customer managed gateway
(Internet)
Network Extension
17. (Optional) Bring your favorite security tools
Unified Threat
Management & WAF
VPN / Routing,
Application Delivery,
Key Management
AVAILABLE NOW
20. Federated Access Management
OPS | SECURE ACCESS MANAGEMENT
AWS Directory Service – AD Connector
• Easily federate your corporate Active
Directory environment to AWS and enable
single sign-on – no need for SAML
infrastructure
• Proxy only – does not store credentials
• Supports RADIUS-based MFA
• Connects to Domain Controllers in your
VPC or on-premise Domain Controllers
Customers can also use ADFS or partner
solutions
21. AWS Identity & Access Management
OPS | SECURE ACCESS MANAGEMENT
AWS Identity and Access Management
• Securely control access to AWS services
and resources
• Combine IAM and AD Connector to
develop role based security policies for
AWS resources using your existing AD
identities
• Fine grained control of permissions with
auditing via CloudTrail
22. OPS | SECURE ACCESS MANAGEMENT
AWS Management
Console
Your Identity Provider
E.g. Active Directory
AWS IAM
(Federated users)
Policies AWS Services &
Resources
AD Connector – (Proxy only)
AWS Directory Service
Forward Authentication
Access per IAM
policies
Authentication
Authorization
Allow / deny
23. OPS | SECURE ACCESS MANAGEMENT
AWS Management
Console
Your Identity Provider
E.g. Active Directory
AWS IAM
(Federated users)
Policies AWS Services &
Resources
AD Connector – (Proxy only)
AWS Directory Service
Forward Authentication
Access per IAM
policies
Authentication
Authorization
Allow / deny
Ready in
15 minutes!
34. Importing existing VM images
AWS Management Portal for
VMWARE vCenter
AWS VM Import
Point and click
migration for VMware
Migrate VMWare, Hyper-V
and Citrix Xen images
OPS | MANAGEMENT
35. AWS Import / Export
Snowball
• 50 TB Amazon-owned appliance design
to help move petabytes of data per week
• 256-bit data encryption (KMS)
• Tamper resistant, durable and rugged
enclosure
• 10 GB network – takes ~13 hours to load
a 50TB Snowball
Use Snowball to move data centers, large
data sets or individual VMs
OPS | MANAGEMENT
36. • Secure, flexible networking
between cloud and on-premise
• Secure, federated access
management
• Management tools for hybrid
environments
• Integrated monitoring tools
HYBRID OPS - REQUIREMENTS
VPC & DirectConnect IAM, Directory Service
Packer, Terraform, Ansible and VM Import
37. • Secure, flexible networking
between cloud and on-premise
• Secure, federated access
management
• Management tools for hybrid
environments
• Integrated monitoring tools
HYBRID OPS - REQUIREMENTS
VPC & DirectConnect IAM, Directory Service
Packer, Terraform, Ansible and VM Import
44. The first, and the most important, reason we go with AWS is for the self-service flexibility. The fact
that we can be up and running with a test environment in minutes … in the classic model, I’d
probably still be negotiating data center contracts.
– Lorin Kobashigawa-Bates
Director of Technology, Expedia APAC & AirAsia Go
”
“
45. Dev & Test on AWS
Performant Lower costAutomated Available
Automate
environment
builds and release
processes
(CI/CD)
Deploy and test
code at full scale
and in replica
environments
Pay as you go –
but turn it off
when idle
On demand
resources in 11
regions globally –
no need to wait
for hardware
46. On-Demand Development Environments
Amazon Workspaces
• Secure and isolated virtual desktop
environments
• Windows 7 desktop experience
• Fully customizable image and apps
• Active Directory integration & MFA
• Pay as you go
Amazon Workspaces
On-demand
virtual desktops
Dev Environments
Great for company contractors or
companies that outsource development
47. AWS Device Farm
AWS CodeCommit
Managed GIT Repository
AWS CodePipeline
Continous Delivery
Real mobile device
testing
AWS CodeDeploy
Automated Deployments
Develop, test and release
49. ..[AWS] significantly improved our business
continuity capabilities, including seamless
failovers
- Kevin Quinlivan
Chief Information Officer, Delaware North
”
“
Hybrid and Disaster Recovery
Architecture
50. Disaster Recovery on AWS
Performant No secondary
site expense
Highly
Secure
Geo DR
AWS data
centers are
compliant to
15+
international
security
standards
Provision DR
environments
to production
scale
Turn it on when
you need it
and run DR
tests frequently
without financial
penalties
Backup your
systems to 11
AWS regions
globally
Eliminate
Tape
Use more
durable disk
based storage
for backup,
archive and
compliance
workloads
51. Store backups data on AWS
Amazon S3 / Glacier Connector &
AWS Storage Gateway VTL
AWS Storage Gateway (VM)
Mirror data to AWS
Pay as you grow storage
Infinitely scalable
On-prem storage
52. Live on-prem to AWS VM/App replication
Network
constrained?
Use Snowball to
transfer VM’s or
data to AWS for
disaster recovery
54. Production on AWS
Highly
Secure
AWS data centers
are compliant to
15+ international
security standards
Digital
Ready
Extensive set of
services for big
data, predictive
analytics, IoT and
mobile apps
Global
Footprint
11 Regions and
over 50 availability
zones and POPs
Open &
Flexible
Language and
operating system
agnostic
55. Adobe Experience Manager (formely CQ)
The rich API set that AWS offers us allows us to
deeply integrate our automation systems into AWS
and provide an efficient operating environment for our
customers
– Mitch Nelson
Director of Managed Services Products, Adobe
”
“
56. Digital Applications
You code. AWS builds and deploys
Amazon Elastic Beanstalk (PaaS)
Mobile and Web
Supports: Supports Java, .NET, PHP,
Node.js, Python, Ruby, Go, and Docker
Mobile only
AWS Mobile Hub
A complete mobile platform
Features: User sign-in and data storage
(Cognito), cloud logic (Lamda), Push
notifications (SNS), analytics, content delivery
& app testing (Device Farm)
60. • Secure, flexible networking between cloud and on-premise
• Secure, federated access management
• Management tools for hybrid environments
• Integrated monitoring tools
Development and Test
Backup & Disaster Recovery
Mobile & Web
Streaming / Analytics
Questions and Feedback? karimh@amazon.com
Duration: ~60 mins.
This session will provide a 100-200 level introduction into hybrid on AWS.
We’ll talk about the AWS hybrid story
And we’ll split the body of the session into two key parts
Implementing hybrid ops
Implementing apps
Then we’ll wrap it up
…NEXT
Hybrid Ops provide the foundations, including networking, security and management tools
Hybrid Apps sit on top of these foundations. Later in the session we wil talk about the common patterns for hybrid apps
At AWS is not an all or nothing proposition.
There are four dimensions to this.
Security, is our #1 priority at AWS and we have invested heavily into improving security for all customers
Scale, 10x more than the others,
Breadth of services, more than 50 at last count
Depth of features, we are adding more than 1 new feature every single day.
Security certifications apply globally to all of our DCs
Security is also not a static craft at AWS. We are constantly evolvong and just recently we announce we have achieved ISO27018 certification which relates to data privacy in the cloud
We also announced a number of new security products, including a security assessment service Amazon Inpsect, a WAF and a cloud governance solution called AWS config rules
In 2015 we have already surpassed this number
And at our annual confernece “re:invent” just a few weeks ago we launched a number of entirely new products
Complete technology and enterprise services platform
Compute Server Storage Networking Security
Enterprise Apps for email, virtual desktops, directory services
Mobile development tools, scalable Analytics infrastructure and now reporting tools like Quicksight,
Marketplace offers easy software and infrastructure procurement
Enterprise Support, professional services and training
So how do you capitalise on these products and sevrices.
In this section we will talk about how companies start by laying the foundations for a hybrid operating model.
This is the type of plumbing that enterprises have told us they need to implement
The first thing that is critical is secure, reliable networking
Next, we need to make it easy for companies to connect their access management systems like Active Directory to the cloud
Then, enterprises need tools that work with both AWS and their on-premise infrastructure
Finally, monitoring tools that can keep your services running smoothly
As a best practice we don’t disclose locations of our data centers, and so we use peering partners like Equinix who provide demarcation point and cross connect into our cloud services.
DX is a Layer 3 service between AWS and your co-lo.
BGP and HSRP for failover etc.
Customers can use multiple DX and VPNs for redundancy and BGP/HSRP to manage failover
If you have additional security products in your on-premise envioronment you can bring them along with you.
AWS Marketplace makes procuring and implementing these solutions a one-click experience. Licensing is handled by marketplace and fees are automatically charged to your AWS account. Once purchased Marketplace install the solution into the VPC of your choice.
There are thousands of products across multiple categories like security, business intelligence and management in AWS Marketplcaae
So I’ve shown you how easy it is to setup a high performance, secure and scalable connectivity and networking between AWS and your existing environment using DirectConnect and VPC. Let’s move onto the next step …. NEXT
Secure access management is the next step…NEXT
AWS Directory Service is a full managed directory service that operates in two modes. One is as a standadlone SAMBA4 compatible directory.
The other is as a Connector (or Proxy) to your Active Directory environment.
With a few clicks you can have your corporate AD environment federated with AWS to for secure access management and single-signon. This means your AD identities can be associated with permissions to AWS platform services. Such as the ability to create and terminate EC2 instances. In CloudTrail, which keeps a log of activities – down to the API level – you will be able to sit individual AD user and what action they attempted to perform.
IAM is our policy driven access management system. It is extremely fine grained.
You can use IAMs built in identities, or federate, to link IAM to your existing AD environment
Using AWS Directory Service and AD Connector it takes just 15 minutes to setup!
For those who want to use their existing Identity Management solutions, we have partnered with all the leading vendors, including those which you can see here
Almost every company we work with that is implementing a hybrid strategy is actually choosing just one cloud to partner with. Customers tell us they prefer a cloud partner with a large breadth and depth of services.
Multi-cloud is being talked about as a potential long term option, but they are all starting with one knowing that the beginning of a long journey starts with a few humble footsteps.
Cloud brokers, or systems that abstract customers from cloud providers APIs, products and services, negate just about all of the benefits of the cloud. Broker platforms are slow to respond to or never support new services, features and API. They can become bottlenecks and introduce reliability issues. In fact, many brokers stop at “VM” integration which is simply not enough.
Open source has well and truly arrived and it is an extremely viable, and in many cases better alternative to commercial solutions. OS is free, often field tested at scale and is updated frequently. Some of the worlds largest startups like Netflix use AWS and are building ecosystems of management tools to add additional features and capabilities.
Enterprises are using tools like these to fully automate the deployment of their applications
HASHICORP Packer: solves the problem of building multi-platform (VMWare, AWS) machine images
Terraform: allows you to describe an application stack (e.g. 3 tier) as a template and then use that template to deploy app environments
Ansible: a popular configuration manegment tooll for scripting changes to virtual machines, EC2 instances and applications
Describe the basic machine type (specifications, running services tc. and types of images you want to produce “builders”) and packer automatically generates those images (AMI, VMX etc.)
The config follows the industry standard JSON format
If you were using VMWare you would create another builder config with the desired specification
Terraform uses a similar JSON based format. This script for an application running on AWS creates 5 amazon EC2 instances of “m1.small” type and puts them behind a AWS manage load balancer on port 80 and routes through to internal port of 8000.
This is a common workflow in enterprise environments. Let’s walk through it
If you have existing VM images you can easily migrate them to AWS using these tools
What if you wanted to move lots of virtual machines or evne an entire data center
That’s where Snowball comes in.
Keys are not stored on the appliance
It can withstand a 6 G jolt
So we have talked about how packer, terraform, ansible can help you implement automated builds across your on-prem and AWS environments
Now let’s talk about monitoring tools
AWS comes with a built-in monitoring platform called CloudWatch. CloudWatch provides useful dashboards and metrics and simple alerts for our platform services and features. Many of these are at the infrastructure level. But you WILL need more detail than that, so you want to invest or textend in your existing tools.
New Relic and App dynamics provide application and infrastructure performance management. And data dog provides really detailed infrastructure.
These vendors all have integration with AWS and are mutli platform supporting AWS cloud and on-prem infrastructure
Splunk App covers cloudtrail, AWS config, EC2 ++
Now that you are paying as you go, it’s important to track your spend.
AWS provides native tools to do so, and its easy to setup billing alerts.
Many of our enterprise customers also use a produc called Cloudability” which gives them detailed spending reports, usage analytics and tools to govern and regulate spending
Finally, we added application performance monitoring and integrated splunk for operational analytics
And you’re done.
Enterprises start with dev and test because it’s a low risk and high return prospect
More performant: Using AWS, we decreased average network latency from 700 milliseconds to less than 50 milliseconds.” Figure 1 demonstrates the ESS typeahead suggestion service running on AWS. They push servers from 30% utilization up to 70% utilization.
Scale your testing: Blue green deployments – multiple live production environments
Create application templates, implement CI, CD and make DevOps a reality.
Test full scale environments.
One thing I hear over and over again from developerss that there simply arnet enough development environments to go around
Developers want more, and they want their own discrete environments. And they want environments that replicate the performance dynamics of the production environment. What about a fully built and managed, data center clustered Oracle 12 database in 15 mins? Or SQL, Mysql, Postgres or even MariaDB?
And if you’re outsourcing development you also need to provide a secure workspace. With Workspaces, our virrtual desktop as a service, its easy to do this. Plus with the power of VPC, you can provide network isolated environments.
J&J is in the process of rollut out 25,000 Amazon Workspaces for temporary employees
AWS have developer focused services for enterprises and startups.
Even if you’re not doing continuous integration and delivery, you can still harvest the benefits of our developer tools.
CodeCommit, Git compatible repository fully managed (we handle storage scalability, security and availability)
CodeDeploy to managed deployments. This is based on a tool named Apollo that Amazon has been buulding over the last 10 years as part of our journey towards micro-services.
The next step some customers take is to move Disaster Recovery environments to the cloud. Because enterprises are paying a lot of money for secondary sites that they seldom use. With AWS, you pay for what you use when you’re using it.
Not all customers move DR, in fact many skip it and move straight into production workloads.
https://aws.amazon.com/solutions/case-studies/delaware-north/
x months into its cloud migration, Delaware North was realizing benefits in addition to its data center consolidation, including cost-effective security compliance, enhanced disaster recovery, and faster deployment times for new services.
“Robust security in a retail environment is critical for us because of our many retail operations, and AWS is enormously helpful for that,” says Brian Mercer, the senior software architect for the project. “By leveraging the security best practices of AWS, we’ve been able to eliminate a lot of compliance tasks that in the past took up valuable time and money.”
He adds that the company also has increased its disaster recovery capabilities at a lower cost than what was available in its previous data center deployment. “It significantly improved our business continuity capabilities, including seamless failovers,” he says.
The solution is also helping Delaware North operate with greater speed and agility. For example, it can bring in new businesses—either through contracts or acquisitions—and get them online much faster than in the past by eliminating the need for traditional IT procurement and provisioning. It used to take between two and three weeks to provision new business units; now it takes one day. The Delaware North IT team is also using AWS to overhaul its operations by eliminating outdated and cumbersome processes, cleaning up documentation, and leveraging the benefits of running test and development tasks in combination with rapid deployment of services through the cloud.
“Our DevOps team can now spin up the resources to push out a service in just minutes, compared to the weeks it used to take,” says Scott Mercer. “With AWS, we can respond much faster to business needs. And we can start repurposing time and resources to deliver more value and services to our internal teams and to our customers.”
If you decomission tape altogether, at the very least send your daily, weekly backups to AWS.
Companies like CommVault and Symantec have made that easy with NATIVE integration for AWS S3 as a storage provider.
Even if you doint use those products you can use the AWS Storage Gateway, a virtual appliance, that asynchronously replicates data to Amazon S3. Likewise, ordinary server volumes mounted across iSCSI can be replicated into the AWS cloud enabling you to protect specific systems or datasets attached to servers (like databases)
What about live replicating virtual machines and apps to AWS for DR? That’s possible too
Racemi and CLoudVelox both provide solutions to do this and to failover in the event of an issue.
Or consider this as a simple and cost effective insurance policy. Use VM Export to make a monthly copy of all of your VMs to a Snowball device and then import them back into the cloud as on-standby virtual machine instances.
So you got through dev and test and DR… and an amazing new digital project has come up.
First off, digital projects are the BEST place to start with AWS … so why is that
AWS is digital. In fact we have a lot of digital services, like IoT, machine learning, data analytics – but we wont have time to cover all of them in this session – so I’ll just focus on a few
Adobe launched their cloud computing offering on AWS and one of the primary reasons for that is the level of automation the AWS platform provides.
We have a large variety of services. If you’re building a new mobile app from scratch then check out Mobile Hub. It This is designed to help you ship your product faster by providing you with all the basic mobile app functionality including identity management and session storage, analytics, content delivery, and app testing.
If you have an existing app, or a project already underway, you should definitely check out Elastic Beanstalk. With EB, just upload the code and AWS does the rest, buidling the infrastructure (LB app and database) and scaling to adapt to peaks in demand. Apps can take as little as 2 minutes to deploy and it supports all of the major languages and container types.
If you want to implement an app architecture that uses both on-prem and cloud, then you can – using DirectConnect. Just remember to keep the app and database as together for a better customer experience.
For streaming analytics we added two products recently – Amazon Kinesis Firehose which make it easier to get data into Redshift and Amazon S3 and Kinesis Analytics which enables analysts to query streaming data using standard SQL queries.
So today we’ve talked about the 4 steps to enable Hybrid Operations and some of the really common initial use cases for enterprises.
Finally, lets stay in touch…please connect with me on LinkedIn
Thank you