K8 Meetup Utkarsh
•Télécharger en tant que PPTX, PDF•
0 j'aime•50 vues
Automating application deployment on GKE using Spinnaker
Recommandé
Contenu connexe
Similaire à K8 Meetup Utkarsh
Similaire à K8 Meetup Utkarsh (20)
Dernier
Dernier (20)
K8 Meetup Utkarsh
- 1. KuberNauts!
- 2. Who am I? Utkarsh Sagar
- 3. Automating application deployment on GKE using Spinnaker
- 4. Deploying software is a big Challenge
- 5. Why continuous delivery? ● Decrease risks of deployment ● Decrease cost of deployment ● Decrease delay between feature development and availability
- 6. Deployments should be: Safe Automated Flexible
- 7. How?
- 8. Spinnaker, what is it?
- 9. “The most powerful tool we have as devops is automation.”
- 10. Multi cloud support… WOW!!!
- 11. Spinnaker Features ● CI Integrations ● Monitoring Integrations ● CLI for Setup and Admin ● Deployment Strategies ● VM Bakery ● Notifications ● Manual Judgments ● Chaos Monkey Integration
- 12. What is Spinnaker made of? Deck ● Browser based UI Orca ● Orchestration engine ● It handles all ad-hoc operations and pipelines. Rosco ● Bakery. Machine images(AMI) ● Currently wraps Packer Echo ● Eventing bus. ● Slack, email, Hipchat, SMS Fiat ● Authorization service ● used to query a user’s access permissions
- 13. What is Spinnaker made of? Gate ● API Gateway ● The Spinnaker UI and all API callers communicate with Spinnaker via Gate. CloudDriver ● responsible for all mutating calls to the cloud providers and for indexing/caching all deployed resources. Igor ● Trigger pipelines via continuous integration jobs ● Allows Jenkins/Travis stages in pipelines. Halyard ● configuration service. ● manages the lifecycle of each services Front50 ● Persist the metadata of applications, pipelines, projects and notifications.
- 15. Spinnaker - K8 Concepts ● Application ● Instance == Pod ● Cluster == Deployment ● Server Group == Replica Set ● Load Balancer == Service ● Firewall
- 16. Provider ● App Engine ● Amazon Web Services ● Amazon Web Services - ECS ● Azure ● Docker Registry ● Google Compute Engine ● Kubernetes ● Openstack ● Oracle
- 17. Storage ● Azure Storage ● Google Cloud Storage ● Minio ● S3 ● Oracle Object Storage
- 18. Artifacts ● Google Cloud Storage / AWS S3 ● Docker Image ● Amazon Machine Image (AMI) ● Github / Gitlab / BitBucket ● HTTP file
- 19. Notifications ● Slack ● Email ● HipChat ● SMS via Twilio
- 22. Demo
- 23. “Everything fails, all the time”
- 25. Questions?
Notes de l'éditeur
- Welcome all the Kubernauts. This is cloudcovers first meetups - Special thanks to vishal biyani from Kubernetes & Cloud Native Computing Pune for allowing us to host this. Before we start, quick questions? How many of you here are are ? Developers Devops Have used kubernet
- Before we start, we would like to introduce ourself Utkarsh Sagar Cloudcover He is sagar patil and we have named him named rubix in CC as he can solve 3x3 cube in 60 secs, he’s expert in cloud and will be doing docker certification next week, he likes to travel onsite and has been to singapore and jakarta to google’s office last month just to enjoy food and desserts DevOps Our DevOps team builds templates, playbooks and scripts to automate cloud infrastructure using Kubernetes, Helm, Terraform, Ansible, Bash, etc. They also build CI/CD pipelines using tools like Jenkins, CircleCI, GitLab, Spinnaker, etc. and conduct automated load tests to double check scalability.
- Automating application deployment on GKE using Spinnaker
- Deploying software is a big Challenge- Organizations are moving faster than ever to release applications, relying on DevOps, agile development and continuous integration/continuous delivery (CI/CD) practices - However, most enterprises still have a manual - creates a bottleneck for business innovation Hence continous delivery is need to ovecome this challenges - A Research surveyed over 300 application development, application release where devops shared their real life experience 90% face pressure to release applications more quickly. 93% have application delivery challenges because of manual database deployment processes. 71% report more than half of application changes also require database changes. 91% admit the need to rework application changes multiple times to make sure they are production-ready. 60% face downtime while deployments
- Why continuous delivery? Decrease risks of deployment Decrease cost of deployment Decrease delay between feature development and availability Deliver software with fewer bugs and lower risk. Release new features to market more frequently Respond to marketing conditions more quickly. Life is saner for everyone: IT operations, software development, QA, product owners and business line owners. Benefits to all stakeholders Developers – More Efficiency Operations – Less Firefighting, More Innovation QA – Never Shipping the Broken Code Into Production Business – Features Reach the Market Faster
- Make sure your are breaking the production environment while deploying, ensure no business impact while deployment Make sure you don’t use ftp to deploy code, yeah many people do that Your deployment mechanism should adjust in any env and be it on prem or cloud How many of you have done this type of deployment in past??? Like copying code manually on server ??
- Hey sagar what tools do you currently use to deploy application on Kubernetes?
- Spinnaker is an open source, multi-cloud continuous delivery platform Created at Netflix, it has been battle-tested in production by hundreds of teams over millions of deployments. It combines a powerful and flexible pipeline management system with integrations to the major cloud providers. Spinnaker creates a powerful abstraction for engineers to easily understand which code is running in which environment and seamlessly move code between environments.
- Spinnaker is an open source, multi-cloud continuous delivery platform Created at Netflix, it has been battle-tested in production by hundreds of teams over millions of deployments. It combines a powerful and flexible pipeline management system with integrations to the major cloud providers.
- listen to events, collect artifacts, and trigger pipelines from Jenkins or Travis CI. Triggers via git, cron, or a new image in a docker registry are also supported. Datadog, Prometheus, Stackdriver, or SignalFx, Install, configure, and update your Spinnaker instance with halyard, Spinnaker’s CLI tool. Configure pipelines with built-in deployment strategies such as highlander and red/black, with rolling red/black and canary in active development, or define your own custom strategy. Highlander: when the new ASG is up and healthy, all old ASGs are destroyed automatically. Red/Black: A new ASG is launched, some manual (or more complicated than in Highlander) verification steps are done, and only after those steps are completed is the old ASG manually deleted. Bake VM images via Packer, and offers support for Chef and Puppet templates Set up event notifications for email, Slack, HipChat, or SMS (via Twilio). Require a manual approval prior to releasing an update with a manual judgement stage. Test that your application can survive instance failures by terminating them on purpose.
- Deck is the browser-based UI. Orca is the orchestration engine. It handles all ad-hoc operations and pipelines. Rosco is the bakery. It produces immutable VM images (or image templates) for various cloud providers. It is used to produce machine images (for example GCE images, AWS AMIs, Azure VM images). It currently wraps packer, but will be expanded to support additional mechanisms for producing images. Echo is Spinnaker’s eventing bus. It supports sending notifications (e.g. Slack, email, Hipchat, SMS), and acts on incoming webhooks from services like Github. Fiat is Spinnaker’s authorization service. It is used to query a user’s access permissions for accounts, applications and service accounts.
- Gate is the API gateway. The Spinnaker UI and all api callers communicate with Spinnaker via Gate. Clouddriver is responsible for all mutating calls to the cloud providers and for indexing/caching all deployed resources. Igor - trigger pipelines via continuous integration jobs Allows Jenkins/Travis stages in pipeline Halyard is Spinnaker’s configuration service. Halyard manages the lifecycle of each of the above services. It only interacts with these services during Spinnaker startup, updates, and rollbacks. Front50 is used to persist the metadata of applications, pipelines, projects and notifications. Kayenta provides automated canary analysis for Spinnaker.
- Deck is the browser-based UI. Gate is the API gateway. The Spinnaker UI and all api callers communicate with Spinnaker via Gate. Orca is the orchestration engine. It handles all ad-hoc operations and pipelines. Clouddriver is responsible for all mutating calls to the cloud providers and for indexing/caching all deployed resources. Front50 is used to persist the metadata of applications, pipelines, projects and notifications. Rosco is the bakery. It produces immutable VM images (or image templates) for various cloud providers. It is used to produce machine images (for example GCE images, AWS AMIs, Azure VM images). It currently wraps packer, but will be expanded to support additional mechanisms for producing images. Igor is used to trigger pipelines via continuous integration jobs in systems like Jenkins and Travis CI, and it allows Jenkins/Travis stages to be used in pipelines. Echo is Spinnaker’s eventing bus. It supports sending notifications (e.g. Slack, email, Hipchat, SMS), and acts on incoming webhooks from services like Github. Fiat is Spinnaker’s authorization service. It is used to query a user’s access permissions for accounts, applications and service accounts. Kayenta provides automated canary analysis for Spinnaker. Halyard is Spinnaker’s configuration service. Halyard manages the lifecycle of each of the above services. It only interacts with these services during Spinnaker startup, updates, and rollbacks.
- An application in Spinnaker is a collection of clusters, which in turn are collections of server groups. An application represents the service which you are going to deploy using Spinnaker. A Spinnaker Instance maps to a Kubernetes Pod. What differentiates this from other Cloud Providers is the ability for Pods to run multiple containers at once. Clusters are logical groupings of Server Groups in Spinnaker. The base resource, the Server Group, identifies the deployable artifact (VM image, Docker image, source location) and basic configuration settings such as number of instances, autoscaling policies, metadata, etc. Load Balancer is associated with an ingress protocol and port range. It balances traffic among instances in its Server Groups. Firewall defines network traffic access. It is effectively a set of firewall rules defined by an IP range (CIDR) along with a communication protocol (e.g., TCP) and port range.
- A Docker image is a snapshot of a container, to be run locally, or in the cloud. Docker image artifacts are used as references to images in registries, such as GCR, or Docker Hub. The artifacts can be deployed to Kubernetes or App Engine, and generally trigger pipelines from notifications sent by their registry. HTTP file artifacts are references to files stored in plaintext reachable via HTTP. They are generally consumed by stages that read configuration from text files, such as a Deploy Manifest stage.
- The sequence of events is: Developer promotes code by pushing to the CloudSource repository Cloud Build automatically builds and pushes your app as a Docker image to Container Registry. Spinnaker detects that the new image tag and triggers a pipeline to deploy the image to canaries, run tests, and roll out the same image to all pods in the deployment. User validates the deployment in the staging environment Spinnaker promotes the container to the production Kubernetes environment GKE (Google Kubernetes Engine) Google Compute Google APIs Cloud Source Repository Container Builder Cloud Storage Cloud Load Balancing
- What are We Going to Do? Set up your environment by creating a Kubernetes Engine cluster, and configuring your identity and user management scheme. Download a application, create a Git repository, and upload it to a Cloud Source Repository. Deploy Spinnaker to Kubernetes Engine using Helm. Build a Docker image from the source code. Create triggers to create Docker images when the source code for application changes. Configure a Spinnaker pipeline to reliably and continuously deploy your application to Kubernetes Engine. Deploy a code change, triggering the pipeline, and watch it roll out to production.
- Very complex to understand Hard to troubleshoot issues Alternatives: Keel.sh Jenkins pipelines Slack bots Harness - enterprise tool Pipeline level access authorisation is not there. Spinnaker on its own has 10 underlying micro services. Managing Spinnaker needs a focussed platform approach. User authentication is easy but authorisation management is not straight forward. Understanding the Spinnaker architecture allows us to quickly shed light on an event such as clouddriver — the cloud-interface abstraction — losing connectivity to a Kubernetes cluster or a Redis pod failure which is used as the Spinnaker state store. Developing this understanding, designing a deployment strategy, and deciding on how engineers will use Spinnaker requires a multi-pronged approach. On a platform such as Kubernetes this can prove to be problematic when Spinnaker’s Redis pod fails and get redeployed. We came to this realization when older builds were randomly triggering from Jenkins and several builds were running concurrently on multiple environments. We switched to a persistent Redis cluster via ElasticCache after discovering the effects of what occurs when you lose Redis connectivity and the circuit breaker kicks in. Any HA solution requires multiple replicas to exist for mission critical components. Spinnaker is no exception. Critical spinnaker components include CloudDriver, which handles caching of your cloud agent data and determines the state of your deployments. Orca handles orchestration of tasks across Spinnaker. It’s essential to adjust the number of replicas based upon your workload across clusters. If too few resources are allocated you will get task queueing which will delay critical operations like rollbacks. We must ensure unintended behaviors do not bubble up to our production environment. Like any other software, Spinnaker has bugs. Some are caused by the ever changing Kubernetes platform, others by Spinnaker itself. Finding bugs and possible workarounds to them built up our confidence up in using Spinnaker.
- Thank you so much for coming. Your visit was a compliment for us in our office. I wish we would have had more time to talk, but perhaps that will happen another day. Thanks for your kindness and generosity. - Special thanks to vishal biyani from Kubernetes & Cloud Native Computing Pune for allowing us to host this. My collegue sagar is not on social media except twitter, hence please tweet