Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Taking Docker to Dance: Continuous Delivery on AWS

52 vues

Publié le

Have you ever tried to deploy an application in AWS? …trying to fit continuous delivery in the picture? In this talk, we will explore best practices to take the most out of your CI/CD pipelines without breaking the bank. We will use a sample application to discuss how to design Docker applications so that they can dance gracefully across all the environments.

Publié dans : Technologie
  • Soyez le premier à commenter

Taking Docker to Dance: Continuous Delivery on AWS

  1. 1. Jessie Wei Seek Taking Docker to Dance: Continuous Delivery on AWSTwitter: @jessieweiyi
  2. 2. Twitter: @jessieweiyi Why Dance? Continuous delivery requires a lot of moving parts to work with in a coordinated and smooth manner. In the container world, this translates into juggling with several containers at once… Therefore, let’s take Docker to dance!
  3. 3. Twitter: @jessieweiyi Objective Show how application containerization and current cloud offerings can greatly streamline the delivery of new features from inception to production. Tools & Tech Focus on Docker and AWS (Some knowledge required). Demo App Demonstrate these concepts with a demo application.
  4. 4. Twitter: @jessieweiyi How to setup a AWS-like local development environment? How to optimize the CI process to build better docker images more quickly? How to seamlessly migrate changes into production without service interruption? More specifically…. we will answer these questions:
  5. 5. Twitter: @jessieweiyi application Source code: https://github.com/jessieweiyi/aws-demo-app-2019-infra A simple image processing service…
  6. 6. UI API Worker Twitter: @jessieweiyi S3 SQS DynamoDB S3: Simple Storage Service SQS: Simple Queue Service
  7. 7. localstack Twitter: @jessieweiyi demo-ui demo-api demo-worker
  8. 8. LocalStack: https://github.com/localstack/localstack Twitter: @jessieweiyi AWS service compliant interface:  from cli: -endpoint-url http://localhost:4572  compatible with AWS SDK, no additional tooling Web console for user-friendly managementThe good …. No capabilities for services configuration on startupthe bad …. Workaround: local nc ping for service ready, and use AWS cli to setup the services needed. and the ugly! Local implementation of a wide range of AWS Services (e.g. S3, DynamoDB, SQS, SNS, Lambda, API Gateway, SES…)
  9. 9. continuous integration deploy to staging deploy to prod Twitter: @jessieweiyi source Install Dependencies Unit Test Code Quality Testing Coverage Build Docker Image Security Test Publish Sanity CheckSmoke Test
  10. 10. Twitter: @jessieweiyi Optimize Builds
  11. 11. Docker Image Size Twitter: @jessieweiyi Docker image size has a considerable impact on the build (and the deployment) process, as it influences download and upload times. Keeping docker image size small has several benefits:  reduces download/upload times  reduces the attack surface  reduces the potential of incompatibilities Tips  use slim base images (-alpine, -slim, or scratch)  install only essential dependencies/packages  remove build dependencies
  12. 12. Docker Build Context Twitter: @jessieweiyi With ..dockerignore file Clean checkout and without ..dockerignore file Local Dev and without ..dockerignore file docker build -t $(APP_IMAGE) . .dockerignore
  13. 13. Multi-Stage Build Twitter: @jessieweiyi FROM node:10.15.3-alpine ENV usernode WORKDIR /opt/app COPY ./package.json/opt/app/package.json COPY ./yarn.lock /opt/app/yarn.lock RUN chown $user–R /opt/app USER $user RUN yarn COPY ./ /opt/app RUN yarnbuild ENTRYPOINT ["yarn"] CMD["start"] Without Multi-Stage Builds With Multi-Stage Builds 378MB 161MB
  14. 14. Speed-up with Caching Twitter: @jessieweiyi Docker images are built of layers, which are incremental file-system portions, and the main contributors to the overall image size. Layer caching reduces the need to download layers during the build process, thus speeding up the build. Layer caching is the default behavior on your local environment, while it may need to be explicitly configured in your CI/CD environment.
  15. 15. Enable Layer Caching in AWS Twitter: @jessieweiyi AWS CodeBuild supports caching in Local and S3, and this can be easily configured by accessing the configuration options.
  16. 16. Use Cached Docker Image version: 0.2 phases: pre_build: commands: - `aws ecr get-login --region $(AWS_REGION) --no-include-email` - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7) - VERSION=${COMMIT_HASH:=latest} - docker pull $(REGISTRY)/aws-demo-app-2019-api-build:latest || exit 0 build: commands: - docker build --cache-from $REGISTRY/aws-demo-app-2019-api-build:latest -t aws-demo-app-2019-api:$VERSION . post_build: commands: - docker tag aws-demo-app-2019-api:$VERSION $REGISTRY/aws-demo-app-2019-api:$VERSION - docker push $REGISTRY/aws-demo-app-2019-api:$VERSION - docker tag aws-demo-app-2019-api:$VERSION $REGISTRY/aws-demo-app-2019-api:latest - docker push $(REGISTRY)/aws-demo-app-2019-api:latest - docker build --target dependencies --cache-from $REGISTRY/aws-demo-app-2019-api-build:latest -t $REGISTRY/aws-demo-app-2019-api-build:latest . - docker push $REGISTRY/aws-demo-app-2019-api-build:latest artifacts: files: imagedefinitions.json Twitter: @jessieweiyi
  17. 17. Docker Hosting Twitter: @jessieweiyi Docker and Docker Compose are a good solution for a local environment, but in production we need something more…. … in short, we need a “much bigger boat”! rolling updates for uninterrupted service container orchestration elastic scalability, to serve varying demand
  18. 18. AWS Offerings Twitter: @jessieweiyi “AWS ECS is for you if you like using Docker!” “AWS EKS is for you if you love Kubernetes!” “AWS Fargate is for you if do not want the grunt work of managing either Docker or Kubernetes!” AWS Fargate - AWS ECS Fargate Launch Mode AWS EC2 - AWS ECS EC2 Launch Mode AWS EKS - Amazon Elastic Container Service for Kubernetes ECS Vs. EKS Vs. Fargate: The Good, the Bad, the Ugly http://blog.totalcloud.io/ecs-vs-eks-vs-fargate-good-bad-ugly/
  19. 19. AWS ECS Fargate Twitter: @jessieweiyi Run containers without having to manage EC2 instances Serverless Pay-as-you-go (vCPUs and memory used): ☹ more expensive than running on ECS EC2 � cheaper considering maintenance costs
  20. 20. App Environment on AWS
  21. 21. Environment Provisioning Network VPC, Subnets, Security Groups, Internet Gateway AWS Services SQS, S3, DynamoDB Table ECS Cluster ECS, Load Balancer Export-Name Fn::ImportValue ECS Service Service, Task Definition, LoadBalancerListener, Rout53RecordSet , TaskIAMRole Tip: by structuring your deployment in layers, you can make reusable components of infrastructure for other deployments! AWS CloudFormation “Infrastructure as Code” approach for your deployment topology
  22. 22. Deployment Process Objective: Zero Downtime Deployments! AWS Fargate Deployment Types  Rolling updates  Blue-Green deployments
  23. 23. Rolling Update  Default ECS Deployment Type  Does not require a load balancer (health check can be shell command)  Great for background tasks (e.g. demo-worker) T0 T1 Both versions exist, until connections to old version are completed. T2 After a while, only new versions exist Update!
  24. 24. Blue-Green Deployments  Reduces risk  Adds complexities at scale  Great for public APIs (e.g. demo-api) T0 T1 A twin deployment with the new version is added to the load balancer, and the previous one removed Listener (port 80) Listener (port 80) Listener (port 8080) T2 Listener (port 80)
  25. 25. Rolling Updates vs Blue-Green Deployments GET /hello_world { "message" :"hello worldv1" } { "message" :"hello worldv2" } from: to: Rolling Updates { "message": "helloworldv1"} { "message": "helloworldv1"} { "message": "helloworldv2"} { "message": "helloworldv1"} { "message": "helloworldv2"} { "message": "helloworldv1"} { "message": "helloworldv2"} { "message": "helloworldv2"} { "message": "helloworldv2"} { "message": "helloworldv1"} { "message": "helloworldv1"} { "message": "helloworldv1" }{ "message": "helloworldv1" } { "message": "helloworldv2"} { "message": "helloworldv2"} { "message": "helloworldv2"} { "message": "helloworldv2"} { "message": "helloworldv2"} Blue-Green DeploymentsTime
  26. 26. Service Auto Scaling Twitter: @jessieweiyi  Scale up to deal with high demand during peak times  Scale down to reduce costs during periods of low utilizations  Choose the right auto-scaling metrics  Service Utilization (% of CPU, Memory)  # active connections  # jobs in the SQS queue
  27. 27. Demo Twitter: @jessieweiyi
  28. 28. Twitter: @jessieweiyi What did we cover? Setup of a local dev environment that mimics AWS services Techniques for optimizing the build process of containerized applications Overview of AWS offerings for running Docker containers Deployment updates techniques for uninterrupted service operation
  29. 29. Twitter: @jessieweiyi