Christian Kniep has over 10 years of experience in HPC and automotive industries in Germany. He co-founded a container and cloud workshop at an HPC conference when told HPC could not learn from cloud companies. Since then, he has led DevOps and containerization efforts. He joined Docker Inc in 2017 to help push adoption forward. Walid Shaari is passionate about open source, DevOps, and security. He is a Red Hat Certified Architect and Certified Kubernetes Administrator. He organizes Docker and Ansible meetups as a community leader. The workshop aims to get participants up and running with the containers ecosystem through an informal, interactive format.
This 2nd version of the last year workshop will shed light on a modern solution to solve application portability, building, delivery, packaging, and system dependency issues. Containers especially Docker have seen accelerated adoption in the web, cloud and recently the enterprise. HPC environments are seeing something similar to the introduction of HPC containers Singularity and Shifter. They provide a good use case for solving software portability, not to mention ensure repeatability of results. Not to mention their ECO system provides for the better development, delivery, testing workflows that were alien to most of HPC environments. This workshop will cover the Theory and hands-on of containers and Its ecosystem. Introducing Docker and singularity containers; Docker as a general-purpose container for almost any app, Singularity as the particular container technology for HPC. The workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Tsunami of Technologies. Are we prepared?
Slide from workshop with open source community in Malaysia.
"Bengkel Bersama Komuniti Sumber Terbuka Bilangan 1 Tahun 2020" in De Baron Resort, Langkawi, Kedah, Malaysia
This document discusses containers and related technologies:
1. Containers provide isolated, portable environments for running applications and their dependencies. Docker is a popular container platform that packages applications into containers using Linux kernel features like namespaces and cgroups.
2. The Open Container Initiative (OCI) aims to develop standards around container formats and runtime. Technologies like Docker, rkt, and AppC implement the OCI specifications.
3. Container orchestration systems like Kubernetes and Mesos manage the deployment and lifecycles of containers at scale across clusters of hosts.
This document provides an overview of containers and Docker for automating DevOps processes. It begins with an introduction to containers and Docker, explaining how containers help break down silos between development and operations teams. It then covers Docker concepts like images, containers, and registries. The document discusses advantages of containers like low overhead, environment isolation, quick deployment, and reusability. It explains how containers leverage kernel features like namespaces and cgroups to provide lightweight isolation compared to virtual machines. Finally, it briefly mentions Docker ecosystem tools that integrate with DevOps processes like configuration management and continuous integration/delivery.
The document discusses DevOps Indonesia, a DevOps community event happening on March 22, 2018 in Jakarta. It focuses on containers and Docker. It provides information about ticket prices, speakers including Yusuf Hadiwinata Sutandar. The document discusses the Open Container Initiative (OCI) and its role in driving innovation. It also discusses concerns with Docker and the container ecosystem including competing technologies like Kubernetes, rkt, and CRI-O. The conclusion emphasizes developing container-agnostic microservices.
This document discusses Docker and the Docker ecosystem. It provides descriptions of various tools related to Docker including orchestration, service discovery, networking, data management, and monitoring tools. It also discusses some companies and projects that are part of the Docker ecosystem like Docker itself, CoreOS, Kubernetes, Marathon, Consul, etcd, and others.
TryStack.cn is a non-profit OpenStack testbed and community project in China that aims to promote OpenStack adoption. It operates the largest OpenStack testbed in China with hardware from various vendors. TryStack.cn provides reference architectures, best practices, and contributes code back to the community. It also organizes OpenStack meetups and training to help grow the OpenStack ecosystem in China.
DockerCon 2019 took place in San Francisco, from April 29th to May 2nd.
Open Source @ Dockercon Summit took place Thursday, May 2nd.
Dockercon 2019 was a success with 5000+ participants. We are planning a recap Meetup to highlight overall announcements, new features & news from the event:,
- new CLI plugins announcement (docker app, docker buildx, docker pipeline etc);
- features of Docker Enterprise 3.0 ( assemble, template etc)
- takeaways; useful links, demos, tips and tricks and of course all videos from all the sessions
- cool stuff from the Open summit, like the powerful buildkit
- Demo: Multi-arch Docker Builds
Under this Meetup, we'll discuss news / new feature announcements during Dockercon and their implications for the ecosystem and end user. In addition to the DockerCon recap, we'll have the usual opportunities for networking and Q&A. We will look to answer any questions you have about Dockercon at this meetup.
We invite all of our members to come -- whether you're a beginner or an experienced user of containers. Don't forget to RSVP for this event so we can make sure we have plenty of place for everyone. Save the date for Docker Timisoara Meetup on May 23th @ CoWork The Garden!
This 2nd version of the last year workshop will shed light on a modern solution to solve application portability, building, delivery, packaging, and system dependency issues. Containers especially Docker have seen accelerated adoption in the web, cloud and recently the enterprise. HPC environments are seeing something similar to the introduction of HPC containers Singularity and Shifter. They provide a good use case for solving software portability, not to mention ensure repeatability of results. Not to mention their ECO system provides for the better development, delivery, testing workflows that were alien to most of HPC environments. This workshop will cover the Theory and hands-on of containers and Its ecosystem. Introducing Docker and singularity containers; Docker as a general-purpose container for almost any app, Singularity as the particular container technology for HPC. The workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Tsunami of Technologies. Are we prepared?
Slide from workshop with open source community in Malaysia.
"Bengkel Bersama Komuniti Sumber Terbuka Bilangan 1 Tahun 2020" in De Baron Resort, Langkawi, Kedah, Malaysia
This document discusses containers and related technologies:
1. Containers provide isolated, portable environments for running applications and their dependencies. Docker is a popular container platform that packages applications into containers using Linux kernel features like namespaces and cgroups.
2. The Open Container Initiative (OCI) aims to develop standards around container formats and runtime. Technologies like Docker, rkt, and AppC implement the OCI specifications.
3. Container orchestration systems like Kubernetes and Mesos manage the deployment and lifecycles of containers at scale across clusters of hosts.
This document provides an overview of containers and Docker for automating DevOps processes. It begins with an introduction to containers and Docker, explaining how containers help break down silos between development and operations teams. It then covers Docker concepts like images, containers, and registries. The document discusses advantages of containers like low overhead, environment isolation, quick deployment, and reusability. It explains how containers leverage kernel features like namespaces and cgroups to provide lightweight isolation compared to virtual machines. Finally, it briefly mentions Docker ecosystem tools that integrate with DevOps processes like configuration management and continuous integration/delivery.
The document discusses DevOps Indonesia, a DevOps community event happening on March 22, 2018 in Jakarta. It focuses on containers and Docker. It provides information about ticket prices, speakers including Yusuf Hadiwinata Sutandar. The document discusses the Open Container Initiative (OCI) and its role in driving innovation. It also discusses concerns with Docker and the container ecosystem including competing technologies like Kubernetes, rkt, and CRI-O. The conclusion emphasizes developing container-agnostic microservices.
This document discusses Docker and the Docker ecosystem. It provides descriptions of various tools related to Docker including orchestration, service discovery, networking, data management, and monitoring tools. It also discusses some companies and projects that are part of the Docker ecosystem like Docker itself, CoreOS, Kubernetes, Marathon, Consul, etcd, and others.
TryStack.cn is a non-profit OpenStack testbed and community project in China that aims to promote OpenStack adoption. It operates the largest OpenStack testbed in China with hardware from various vendors. TryStack.cn provides reference architectures, best practices, and contributes code back to the community. It also organizes OpenStack meetups and training to help grow the OpenStack ecosystem in China.
DockerCon 2019 took place in San Francisco, from April 29th to May 2nd.
Open Source @ Dockercon Summit took place Thursday, May 2nd.
Dockercon 2019 was a success with 5000+ participants. We are planning a recap Meetup to highlight overall announcements, new features & news from the event:,
- new CLI plugins announcement (docker app, docker buildx, docker pipeline etc);
- features of Docker Enterprise 3.0 ( assemble, template etc)
- takeaways; useful links, demos, tips and tricks and of course all videos from all the sessions
- cool stuff from the Open summit, like the powerful buildkit
- Demo: Multi-arch Docker Builds
Under this Meetup, we'll discuss news / new feature announcements during Dockercon and their implications for the ecosystem and end user. In addition to the DockerCon recap, we'll have the usual opportunities for networking and Q&A. We will look to answer any questions you have about Dockercon at this meetup.
We invite all of our members to come -- whether you're a beginner or an experienced user of containers. Don't forget to RSVP for this event so we can make sure we have plenty of place for everyone. Save the date for Docker Timisoara Meetup on May 23th @ CoWork The Garden!
The document discusses Android containerization using Linux container (LXC) technology. It describes how LXC can be used to run multiple Android instances within containers on a single device. Key points include using Linux namespaces to isolate containers, virtualizing Android devices and binder IPC using namespaces, and challenges around scheduling and resource management with multiple containerized Androids. The document also summarizes some open source projects from ITRI related to containerization and virtualization technologies.
The lessons I learned is that Open source quickly becomes the natural choice wherever commoditization is happening in the software stack. Thus we expect business-to-business open source, which is already a significant trend in recent history, to become an increasingly common form of open source collaboration. Companies who understand the ground rules of business-to-business open source will be better positioned to identify and take advantage of open source opportunities in the competitive spaces that they share with other companies.
So I will share why open strategy is import for the enterprise. And how to do contributions for the open source projects n today’s topic.
OpenShift and next generation application developmentSyed Shaaf
OpenShift is a Platform as a Service (PaaS) cloud application platform built on Red Hat technologies that allows developers to easily deploy and scale applications in a cloud environment. It provides developers flexibility to work how they want through options like a web console, command line tools, and IDE integrations while choosing from various programming languages, frameworks, and middleware. OpenShift handles automated application builds, testing, deployment and scaling across its infrastructure which includes nodes managed by brokers that run on instances of Red Hat Enterprise Linux.
OpenShift is a DevOps platform that provides a container application platform for deploying and managing containerized applications and microservices. It uses Kubernetes for orchestration and Docker containers. OpenShift provides features for the complete application lifecycle including continuous integration/delivery (CI/CD), automated image builds, deployments, networking, authentication, and integration with external services and registries. Developers can create and deploy applications from source code, templates, or Docker images to OpenShift without needing deep knowledge of Docker or Kubernetes.
This document discusses Docker and OpenStack integration. It begins with introductions to OpenStack and Docker, explaining that OpenStack is an open source cloud operating system and Docker is a container-based virtualization framework. It then discusses how Docker can be used with OpenStack, with Nova supporting Docker as a hypervisor starting in Havana. It concludes with mentioning a demo of Docker + OpenStack integration and inviting questions.
An introduction to the open source project that empowers modern workflows to build, deploy and manage the lifecycle of containers. You will learn what OpenShift is, what are its use cases, and more about all the fuss around Cloud computing, microservices, DevOps and whatnot.
Docker, containers, rkt, kubernetes, the Open Container Project, CoreOS, and RancherOS are some of the new buzzwords in cloud. If you've read any articles on them you may have been left thinking this is the new hot technology space but you're unsure of how to leverage it in your own environments. You may even still be wondering how this, in a practical sense, is different from working with virtual machines or the other platforms you've been using.
In this session we'll start with an introduction into containers. We'll look where we are going in computing and how containers can be useful in ways virtual machines can't. From there we'll look at ways you can use containers and Docker in your environments today. We'll round out our time by looking at what's being built with container technology that can help you.
Build and run applications in a dockerless kubernetes worldJorge Morales
Talk at Dev Days Riga 2018:
Kubernetes has rapidly grown to support many container runtime formats. In this talk, I'm presenting all the alternatives you have to run your applications in kubernetes, and will present CRI-O which is steadily becoming a replacement to run your Docker containers on production. And since you will no longer have Docker, how will you build now your Docker containers? Buildah is a project that facilitates building Docker containers in a Dockerless world.
The document summarizes an OpenShift Taiwan Meetup for vol. 1 on October 3, 2018. It discusses key concepts of OpenShift including self-service, multi-tenant, standards-based, and enterprise-grade capabilities. It also covers OpenShift features like any app, any infrastructure, application lifecycle management, and container orchestration using Kubernetes. Lastly, it provides details on the meetup including the location, WiFi access, and topics to be discussed around OpenShift and container technologies.
Deploying & Scaling OpenShift on OpenStack using Heat - OpenStack Seattle Mee...Diane Mueller
OpenShift Origin is an open-source Platform-as-a-Service project sponsored by Red Hat. In this session, Diane will be discussingOpenShift's use of Heat to deploy OpenShift on OpenStack showcase a number of aspects of configuring and managing a complex application on OpenStack’s Diskimage-builder and OpenStack’s Heat, both tools are bundled with RHOS 4.
Diane will walk thru the basic architecture of the application being deployed (OpenShift), then discuss how to configure OpenStack Neutron networking for OpenShift, register images with Glance, monitor Heat, and then show how to point OpenShift command line client to the broker's public ip address and begin using OpenShift.
All the heat templates used are available here:https://github.com/openstack/heat-templates and this is an awesome way to learn about Heat and contribute to both the OpenShift & OpenStack communities.
Speaker: Diane Mueller, OpenShift Origin Community Manager
OpenShift pour le developpement cloud native - 20171214Laurent Broudoux
Talk donné au Cloud Workshop Azure - Red Hat & Microsoft, le 14/12/2017. Découvrez comment la plateforme OpenShift de Red Hat permet de faciliter le développement, le déploiement et le monitoring d'applications Cloud Native !
This document provides an overview of Docker and containers. It begins with a brief introduction to 12 Factor Applications methodology and then defines what Docker is, explaining that containers utilize Linux namespaces and cgroups to isolate processes. It describes the Docker software and ecosystem, including images, registries, Docker CLI, Docker Compose, building images with Dockerfile, and orchestrating with tools like Kubernetes. It concludes with a live demo and links to additional resources.
Accelerate your software development with DockerAndrey Hristov
Docker is in all the news and this talk presents you the technology and shows you how to leverage it to build your applications according to the 12 factor application model.
Docker: Redistributing DevOps cards, on the way to PaaSAdrien Blind
This talk first presents Docker through its key characteristics: being Portable, Disposable, Live, Social. It then discusses a new type of cloud, the CaaS (Container as a Service), and it potential benefits for PaaS (Platform as a Service).
Building specialized container-based systems with Moby: a few use cases
This talk will explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud or bare metal scenarios. We will cover Moby itself, the framework, and tooling around the project, as well as many of it’s components: LinuxKit, InfraKit, containerd, SwarmKit, Notary. Then we will present a few use cases and demos of how different companies have leveraged Moby and some of the Moby components to create their own container-based systems.
Containers provide isolation at the operating system level through mechanisms like namespaces and cgroups. While containers isolate applications from each other better than traditional virtualization, some experts argue that full virtualization using hypervisors provides stronger security due to stronger isolation between virtual machines. However, container security has improved significantly over time and many argue containers can provide adequate security for many use cases. There is an ongoing debate in the industry around the relative security of containers versus virtual machines.
Podman, Buildah, and Quarkus - The Latest in Linux Containers Technologies Daniel Oh
Podman, Buildah, and CRI-O are new open source projects that address early concerns with Docker like requiring a daemon, secret handling issues, and root/privileged concerns. Podman is a daemonless container engine that provides a Docker-compatible command line. Buildah is a tool for building OCI container images without needing a daemon. CRI-O implements the Kubernetes Container Runtime Interface for running containers securely in production clusters without daemons.
Back in 2015, as part of a DevOps presenation, I put together a slide showing the container landscape. For fun I've created an updated version showing the environment today. This slide set incluces both landscapes along with the orginal talk track that went with the 2015 landscape.
Bahrain ch9 introduction to docker 5th birthday Walid Shaari
A hands-on workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
Build your own container-based system
with the Moby project
Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.
The document discusses Android containerization using Linux container (LXC) technology. It describes how LXC can be used to run multiple Android instances within containers on a single device. Key points include using Linux namespaces to isolate containers, virtualizing Android devices and binder IPC using namespaces, and challenges around scheduling and resource management with multiple containerized Androids. The document also summarizes some open source projects from ITRI related to containerization and virtualization technologies.
The lessons I learned is that Open source quickly becomes the natural choice wherever commoditization is happening in the software stack. Thus we expect business-to-business open source, which is already a significant trend in recent history, to become an increasingly common form of open source collaboration. Companies who understand the ground rules of business-to-business open source will be better positioned to identify and take advantage of open source opportunities in the competitive spaces that they share with other companies.
So I will share why open strategy is import for the enterprise. And how to do contributions for the open source projects n today’s topic.
OpenShift and next generation application developmentSyed Shaaf
OpenShift is a Platform as a Service (PaaS) cloud application platform built on Red Hat technologies that allows developers to easily deploy and scale applications in a cloud environment. It provides developers flexibility to work how they want through options like a web console, command line tools, and IDE integrations while choosing from various programming languages, frameworks, and middleware. OpenShift handles automated application builds, testing, deployment and scaling across its infrastructure which includes nodes managed by brokers that run on instances of Red Hat Enterprise Linux.
OpenShift is a DevOps platform that provides a container application platform for deploying and managing containerized applications and microservices. It uses Kubernetes for orchestration and Docker containers. OpenShift provides features for the complete application lifecycle including continuous integration/delivery (CI/CD), automated image builds, deployments, networking, authentication, and integration with external services and registries. Developers can create and deploy applications from source code, templates, or Docker images to OpenShift without needing deep knowledge of Docker or Kubernetes.
This document discusses Docker and OpenStack integration. It begins with introductions to OpenStack and Docker, explaining that OpenStack is an open source cloud operating system and Docker is a container-based virtualization framework. It then discusses how Docker can be used with OpenStack, with Nova supporting Docker as a hypervisor starting in Havana. It concludes with mentioning a demo of Docker + OpenStack integration and inviting questions.
An introduction to the open source project that empowers modern workflows to build, deploy and manage the lifecycle of containers. You will learn what OpenShift is, what are its use cases, and more about all the fuss around Cloud computing, microservices, DevOps and whatnot.
Docker, containers, rkt, kubernetes, the Open Container Project, CoreOS, and RancherOS are some of the new buzzwords in cloud. If you've read any articles on them you may have been left thinking this is the new hot technology space but you're unsure of how to leverage it in your own environments. You may even still be wondering how this, in a practical sense, is different from working with virtual machines or the other platforms you've been using.
In this session we'll start with an introduction into containers. We'll look where we are going in computing and how containers can be useful in ways virtual machines can't. From there we'll look at ways you can use containers and Docker in your environments today. We'll round out our time by looking at what's being built with container technology that can help you.
Build and run applications in a dockerless kubernetes worldJorge Morales
Talk at Dev Days Riga 2018:
Kubernetes has rapidly grown to support many container runtime formats. In this talk, I'm presenting all the alternatives you have to run your applications in kubernetes, and will present CRI-O which is steadily becoming a replacement to run your Docker containers on production. And since you will no longer have Docker, how will you build now your Docker containers? Buildah is a project that facilitates building Docker containers in a Dockerless world.
The document summarizes an OpenShift Taiwan Meetup for vol. 1 on October 3, 2018. It discusses key concepts of OpenShift including self-service, multi-tenant, standards-based, and enterprise-grade capabilities. It also covers OpenShift features like any app, any infrastructure, application lifecycle management, and container orchestration using Kubernetes. Lastly, it provides details on the meetup including the location, WiFi access, and topics to be discussed around OpenShift and container technologies.
Deploying & Scaling OpenShift on OpenStack using Heat - OpenStack Seattle Mee...Diane Mueller
OpenShift Origin is an open-source Platform-as-a-Service project sponsored by Red Hat. In this session, Diane will be discussingOpenShift's use of Heat to deploy OpenShift on OpenStack showcase a number of aspects of configuring and managing a complex application on OpenStack’s Diskimage-builder and OpenStack’s Heat, both tools are bundled with RHOS 4.
Diane will walk thru the basic architecture of the application being deployed (OpenShift), then discuss how to configure OpenStack Neutron networking for OpenShift, register images with Glance, monitor Heat, and then show how to point OpenShift command line client to the broker's public ip address and begin using OpenShift.
All the heat templates used are available here:https://github.com/openstack/heat-templates and this is an awesome way to learn about Heat and contribute to both the OpenShift & OpenStack communities.
Speaker: Diane Mueller, OpenShift Origin Community Manager
OpenShift pour le developpement cloud native - 20171214Laurent Broudoux
Talk donné au Cloud Workshop Azure - Red Hat & Microsoft, le 14/12/2017. Découvrez comment la plateforme OpenShift de Red Hat permet de faciliter le développement, le déploiement et le monitoring d'applications Cloud Native !
This document provides an overview of Docker and containers. It begins with a brief introduction to 12 Factor Applications methodology and then defines what Docker is, explaining that containers utilize Linux namespaces and cgroups to isolate processes. It describes the Docker software and ecosystem, including images, registries, Docker CLI, Docker Compose, building images with Dockerfile, and orchestrating with tools like Kubernetes. It concludes with a live demo and links to additional resources.
Accelerate your software development with DockerAndrey Hristov
Docker is in all the news and this talk presents you the technology and shows you how to leverage it to build your applications according to the 12 factor application model.
Docker: Redistributing DevOps cards, on the way to PaaSAdrien Blind
This talk first presents Docker through its key characteristics: being Portable, Disposable, Live, Social. It then discusses a new type of cloud, the CaaS (Container as a Service), and it potential benefits for PaaS (Platform as a Service).
Building specialized container-based systems with Moby: a few use cases
This talk will explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud or bare metal scenarios. We will cover Moby itself, the framework, and tooling around the project, as well as many of it’s components: LinuxKit, InfraKit, containerd, SwarmKit, Notary. Then we will present a few use cases and demos of how different companies have leveraged Moby and some of the Moby components to create their own container-based systems.
Containers provide isolation at the operating system level through mechanisms like namespaces and cgroups. While containers isolate applications from each other better than traditional virtualization, some experts argue that full virtualization using hypervisors provides stronger security due to stronger isolation between virtual machines. However, container security has improved significantly over time and many argue containers can provide adequate security for many use cases. There is an ongoing debate in the industry around the relative security of containers versus virtual machines.
Podman, Buildah, and Quarkus - The Latest in Linux Containers Technologies Daniel Oh
Podman, Buildah, and CRI-O are new open source projects that address early concerns with Docker like requiring a daemon, secret handling issues, and root/privileged concerns. Podman is a daemonless container engine that provides a Docker-compatible command line. Buildah is a tool for building OCI container images without needing a daemon. CRI-O implements the Kubernetes Container Runtime Interface for running containers securely in production clusters without daemons.
Back in 2015, as part of a DevOps presenation, I put together a slide showing the container landscape. For fun I've created an updated version showing the environment today. This slide set incluces both landscapes along with the orginal talk track that went with the 2015 landscape.
Bahrain ch9 introduction to docker 5th birthday Walid Shaari
A hands-on workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
Build your own container-based system
with the Moby project
Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.
This document discusses Docker Inc. developer relations manager Patrick Chanezon's work programming the world with Docker. The key points discussed are:
- Patrick Chanezon works at Docker Inc. in developer relations and aims to program the world with Docker.
- Docker allows for platforms and networks to be programmed through containers and orchestration, enabling tools for mass innovation across industries.
- Docker 1.12 introduces built-in orchestration through Swarm mode and the Docker Service API, allowing for self-organizing and self-healing container orchestration without external dependencies.
A hitchhiker‘s guide to the cloud native stackQAware GmbH
Container Days 2017, Hamburg: Vortrag von Mario-Leander Reimer (@LeanderReimer, Cheftechnologe bei QAware).
Abstract: Cloud-Größen wie Google, Twitter und Netflix haben die Kernbausteine ihrer Infrastruktur quelloffen verfügbar gemacht. Das Resultat aus vielen Jahren Cloud-Erfahrung ist nun frei zugänglich, und jeder kann seine eigenen Cloud-nativen Anwendungen entwickeln – Anwendungen, die in der Cloud zuverlässig laufen und fast beliebig skalieren. Die einzelnen Bausteine wachsen zu einem großen Ganzen zusammen, dem Cloud Native Stack.
In dieser Session stellen wir die wichtigsten Konzepte und Schlüsseltechnologien vor und bringen dann eine Spring-Cloud-basierte Beispielanwendung schrittweise auf Kubernetes und DC/OS zum Laufen. Dabei diskutieren wir verschiedene praktikable Architekturalternativen.
This document provides an overview of cloud native applications and the cloud native stack. It discusses key concepts like microservices, containerization, composition using Docker and Docker Compose, and orchestration using Kubernetes. It provides examples of building a simple microservices application with these technologies and deploying it on Kubernetes. Overall it serves as a guide to developing and deploying cloud native applications.
The document discusses Docker's platform and ecosystem. It describes Docker's mission to build tools for mass innovation by providing a software layer to program the internet. It outlines key components of Docker including Docker Engine, Swarm for clustering multiple Docker hosts, Compose for defining and running multi-container apps, and Docker Hub for sharing apps. It also summarizes new features in Docker Engine 1.9 like network and volume management and previews upcoming features in Engine 1.10.
Docker Bday #5, SF Edition: Introduction to DockerDocker, Inc.
In celebration of Docker's 5th birthday in March, user groups all around the world hosted birthday events with an introduction to Docker presentation and hands-on-labs. We invited Docker users to recognize where they were on their Docker journey and the goal was to help them take the next step of their journey with the help of mentors. This presentation was done at the beginning of the events (this one is from the San Francisco event in HQ) and gives a run down of the birthday event series, Docker's momentum, a basic explanation of containers, the benefits of using the Docker platform, Docker + Kubernetes and more.
Neo4j works very well in cloud environments. However, with such variance in compute, network, and storage options, the job of configuring a production database environment is getting complex. In this demo-oriented session, Patrick and David Makogon will introducing straightforward ways to configure and deploy Neo4j with Docker containers, as well as showing how to use automated cloud resource configuration with the new Azure Resource Manager.
Containers and Nutanix - Acropolis Container ServicesNEXTtour
This presentation was given at the London Nutanix user group (NUG) on Oct 26 by Denis Guyadeen. If you would like to join a NUG, you can find more information here http://bit.ly/NTNXUG - Hope to see you at a community meeting!
'Package Once/Run Anywhere' Big Data and HPC workloadsGreenQloud
GreenQloud provides a hybrid private/public cloud infrastructure. They advocate using Docker containers to package applications in a portable way so they can run anywhere, from local machines to public clouds to HPC clusters. Containers provide advantages over virtual machines like simplicity, low overhead, and portability. As container technologies develop further, they enable a more distributed cloud model where workloads can run across multiple cloud environments rather than being centralized. This improves flexibility, speed of deployment, and collaboration for HPC developers and administrators.
Docker Container As A Service
X11 Linux apps on mac in a container.
In container Java development with STS or Eclipse in a container.
Docker UCP and swarm load balancing with Interlock.
0507 057 01 98 * Adana Klima Tamir Servisi, Adana Klima Tamir Servisi, Adana Klima Tamir Servisleri, Arçelik Klima Tamir Servisi Adana, Beko Klima Tamir Servisi Adana, Demirdöküm Klima Tamir Servisi Adana, Vestel Klima Tamir Servisi Adana, Aeg Klima Tamir Servisi Adana, Bosch Klima Tamir Servisi Adana, Ariston Klima Tamir Servisi Adana, Samsung Klima Tamir Servisi Adana, Siemens Klima Tamir Servisi Adana, Profilo Klima Tamir Servisi Adana, Fujitsu Klima Tamir Servisi Adana, Baymak Klima Tamir Servisi Adana, Sharp Klima Tamir Servisi Adana, Mitsubishi Klima Tamir Servisi Adana, Alaska Klima Tamir Servisi Adana, Aura Klima Tamir Servisi Adana, Adana Çukurova Klima Servisleri, Adana Yüreğir Klima Servisleri, Adana Seyhan Klima Servisleri
This document provides an overview of Docker and Michel Courtine's role at Docker Inc. It discusses Docker's mission to build tools for mass innovation and how Docker aims to enable a software layer to program the internet. It also highlights key Docker technologies like containers, images, and orchestration tools and how Docker is being used across various industries and platforms.
Docker moves very fast, with an edge channel released every month and a stable release every 3 months. Patrick will talk about how Docker introduced Docker EE and a certification program for containers and plugins with Docker CE and EE 17.03 (from March), the announcements from DockerCon (April), and the many new features planned for Docker CE 17.05 in May.
This talk will be about what's new in Docker and what's next on the roadmap
This document contains a summary of a presentation given by Patrick Chanezon of Docker Inc. about Docker and the container ecosystem. The presentation covered Docker's history and growth, key products like Docker Engine, Docker Hub, Docker Compose and Docker Machine. It discussed how Docker enables developers and operations teams through containerization. The presentation also looked at related projects and companies in the container space, as well as Docker's open governance model and efforts to contribute container plumbing projects to open standards.
Docker Overview - Rise of the ContainersRyan Hodgin
Containers allow for applications to become more portable, organized more efficiently, and configured to make better use of system resources. This presentation will explain Docker's container technology, DevOps approach, partner ecosystem, popularity, performance, challenges, and roadmap. We'll review how containers are changing application and operating system designs.
DockerCon SF 2015: Docker Community in ChinaDocker, Inc.
1) The document discusses the Docker community in China, noting that early adopters like Baidu helped drive adoption.
2) Meetups and content contributed to scaling the community from 1 to over 19 cities with thousands of attendees. Chinese contributors are also among the top for the Docker project.
3) The market for Docker in China is driven by the "Internet Plus" strategy and sectors like e-commerce, social media, and IoT. This is creating opportunities for startups and traditional businesses to embrace mobile and cloud technologies.
4) The ecosystem involves startups building tools for CI/CD, container services, and management, and projects like Hyper focusing on running containers on any hypervisor. Developers are also using
Tampere Docker meetup - Happy 5th Birthday DockerSakari Hoisko
Part of official docker meetup events by Docker Inc.
https://events.docker.com/events/docker-bday-5/
Meetup event:
https://www.meetup.com/Docker-Tampere/events/248566945/
Docker Orchestration: Welcome to the Jungle! Devoxx & Docker Meetup Tour Nov ...Patrick Chanezon
In two years, Docker hit the sweet spot for devs and ops, with tools for building, shipping, and running distributed apps architected as a set of collaborating microservices packaged as Linux containers. One area of the Docker ecosystem that saw a lot of innovation in the past year is container orchestration systems. This session compares and contrasts various Docker orchestration systems (Swarm, Machine, and Compose), the batteries included with Docker itself, Mesos, Kubernetes, CoreOS/Fleet, Deis, Cloud Foundry, and Tutum. It includes a demo of how to deploy a Java 8 app with MongoDB on several of these systems. The goal of the session is to give you a framework to help evaluate how these systems can meet your particular requirements.
Demo code at https://github.com/chanezon/docker-tips/blob/master/orchestration-networking/README.md
Docker Orchestration: Welcome to the Jungle! JavaOne 2015Patrick Chanezon
In two years, Docker hit the sweet spot for devs and ops, with tools for building, shipping, and running distributed apps architected as a set of collaborating microservices packaged as Linux containers. One area of the Docker ecosystem that saw a lot of innovation in the past year is container orchestration systems. This session compares and contrasts various Docker orchestration systems (Swarm, Machine, and Compose), the batteries included with Docker itself, Mesos, Kubernetes, CoreOS/Fleet, Deis, Cloud Foundry, and Tutum. It includes a demo of how to deploy a Java 8 app with MongoDB on several of these systems. The goal of the session is to give you a framework to help evaluate how these systems can meet your particular requirements.
Inspired by the cloud native community and CNCF Research end-users such as CERN, University of Michigan and many others. With our small contribution, Nora Alwadah and I extended the bridge to the Saudi HPC community.
Key takeaway: Follow and join the new Kubernetes Batch Working Group. Help them nourish and evolve.
AWS user group meetup container series DXB Dubai
n this session, we will explore the popular workload manager and scheduler Kubernetes. Amazon managed kubernetes service, Elastic Container Service for Kubernetes (Amazon EKS) takes care of the heavy-lifting and lets one focus on managing the containerized workloads. EKS, however, still gives you the flexibility and choice where to run, and how to efficiently run your data-plane that hosts your workloads. In this session, we cover what you need to know to get your application up and running with Kubernetes on AWS. We show how Amazon EKS makes deploying Kubernetes on AWS simple and scalable.
⏳ Agenda
1- Review the general Kubernetes architecture and relate to EKS
2- How to set up and provision your Kubernetes cluster using console and eksctl.
3- Discuss the important abstractions that developers use to map their traditional application into any kubernetes platform.
4- How to deploy software efficiently, while sustaining reliable and scalable applications.
5- Deploy your first microservices on EKS
6- EKS possible development deployment workflow
Presenting a line of thought were OKD the parallel opensource project of Openshift could be a valuble multicloud Kubernetes distribution coupled with the open soruce Red Hat Advanced cluster manager. it is not yet fully open sourced yet, however it is just a matter of time. for now the hub needs to be openshift and spokes can be any kubernted including OKD
Okd wg kubecon marathon azure & vsphereWalid Shaari
this was part of open shift commons events, fringe to the Kubecon 2020 Europe/Amsterdam (virtual) . along with my good online friend and mentor Josef we presented our experience for installing OKD in Azure and VMware.
- Docker started as an internal project at dotcloud and was later open sourced in 2013. It allows for standardized packaging of software and isolates applications from each other while sharing the same OS kernel.
- Containers provide benefits over traditional virtual machines by providing an application-level rather than infrastructure-level construct, resulting in better performance and efficiency.
- Kubernetes is an open source container orchestration platform originally developed by Google that provides self-healing and automated scaling of containerized applications. It abstracts away underlying infrastructure to provide a uniform interface for workloads.
This document provides an agenda and materials for a presentation on containerizing applications. The agenda includes check-in and updates, a presentation on why to containerize apps, demos and container labs, and a discussion of next steps. The presentation materials explain what containers are, their advantages over virtual machines like improved resource utilization and portability. Open source tools for containers like Docker and Kubernetes are also discussed. There are demos of building Docker images and running containers, as well as how orchestration tools can automate deployment and management of containerized apps.
Network Automation Journey, A systems engineer NetOps perspectiveWalid Shaari
Network devices play a crucial role; they are not just in the Data Center. It's the Wifi, VOIP, WAN and recently underlays and overlays. Network teams are essential for operations. It's about time we highlight to the configuration management community the importance of Network teams and include them in our discussions. This talk describes the personal experience of systems engineer on how to kickstart a network team into automation. Most importantly, how and where to start, challenges faced, and progress made. The network team in question uses multi-vendor network devices in a large traditional enterprise.
NetDevOps, we do not hear that term as frequent as we should. Every time we hear about automation, or configuration management, it is usually the application, if not, it is the systems that host the applications. How about the network systems and devices that interconnect and protects our services? This talk aims to describe the journey a systems engineer had as part of an automation assignment with the network management team. Building from lessons learned and challenges faced with system automation, how one can kickstart an automation project and gain small wins quickly. Where and how to start the journey? What to avoid? What to prioritise? How to overcome the lack of network skills for the automation engineer and lack of automation and Linux/Unix skills for network engineers. What challenges were faced and how to overcome them? What fights to give up? Where do I see network automation and configuration management as a systems engineer? What are the status quo and future expectations?
This talk aims to describe the journey a systems engineer had as part of an automation assignment with the network management team. building from lessons learned and challenges faced with system automation for the last three years. Where and how to start the journey? what to avoid? what to prioritize? how to overcome the lack of network skills for the automation engineer and lack of automation and Linux/Unix skills for network engineers. what challenges were faced and ho w to overcome them? what fights to win, and which to give up? where do I see network automation and configuration management as a systems engineer? what are the status quo and future expectations?
first practical introduction to Kubernetes and Openshift. container orchestration systems. the slides are meant as a quick introduction to Kubernetes/Openshift to prepare for the hands-on demo.
his workshop will shed light on a modern solution to solve application portability, building, delivery, packaging, and system dependency issues. Containers especially Docker have seen accelerated adoption in the web, cloud and recently the enterprise. HPC environments are seeing something similar to the introduction of HPC containers Singularity and Shifter. They provide a good use case for solving software portability, not to mention ensure repeatability of results. Not to mention their ECO system provides for the better development, delivery, testing workflows that were alien to most of HPC environments. This workshop will cover the Theory and hands-on of containers and Its ecosystem. Introducing Docker and singularity containers; Docker as a general-purpose container for almost any app, Singularity as the particular container technology for HPC. The workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.
Docker "Global Mentor Week" is your opportunity to #learndocker. to learn how to build, ship, and run modern distributed applications with ease. thanks to the Docker platform.
Right now, Docker has developed out a series of self-paced online labs that will be available during the meetup. Docker’s meetup groups worldwide are hosting a series of complimentary events to help newcomers and intermediate users learn Docker.
We'll have hands-on labs for both beginners and intermediate users, labs targeting both developers and operations. There is something for everyone. Docker mentor will be on hand at this event to help you prepare. and work through the self-paced materials. Bring your laptop, have fun and learn Docker!
DevOps and HPC: Saudi Aramco HPC use case discusses how DevOps practices like infrastructure as code and configuration management tools like Puppet can help optimize HPC clusters. Benefits include speeding up cluster deployments from days to hours, continuous deployment, drift control, and team collaboration through version control. Containers are also discussed as a potential way to improve portability, scalability and software delivery for HPC workloads. However, challenges include changing processes, kernel requirements, security, and keeping pace with the fast-moving container ecosystem.
CONTAINERS WORKSHOP DURING SAUDI HPC 2016 : DOCKER 101, DOCKER, AND ITS ECO SYSTEM FOR DISTRIBUTED SYSTEMS by Walid Shaari
This workshop will cover the Theory and hands-on of Docker containers, and Its eco system. The foundations of the Docker platform, including an overview of the platform system components, images, containers and repositories, installation , using Docker containers from repositories e.g. dockerhub, how to create a container using Dockerfile, containers development life cycle. The strategy is to demonstrate through "live demo, and shared exercise" the reuse and customization of components to build a distributed system case service gradually
http://www.hpcsaudi.com/
Home security is of paramount importance in today's world, where we rely more on technology, home
security is crucial. Using technology to make homes safer and easier to control from anywhere is
important. Home security is important for the occupant’s safety. In this paper, we came up with a low cost,
AI based model home security system. The system has a user-friendly interface, allowing users to start
model training and face detection with simple keyboard commands. Our goal is to introduce an innovative
home security system using facial recognition technology. Unlike traditional systems, this system trains
and saves images of friends and family members. The system scans this folder to recognize familiar faces
and provides real-time monitoring. If an unfamiliar face is detected, it promptly sends an email alert,
ensuring a proactive response to potential security threats.
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Sinan KOZAK
Sinan from the Delivery Hero mobile infrastructure engineering team shares a deep dive into performance acceleration with Gradle build cache optimizations. Sinan shares their journey into solving complex build-cache problems that affect Gradle builds. By understanding the challenges and solutions found in our journey, we aim to demonstrate the possibilities for faster builds. The case study reveals how overlapping outputs and cache misconfigurations led to significant increases in build times, especially as the project scaled up with numerous modules using Paparazzi tests. The journey from diagnosing to defeating cache issues offers invaluable lessons on maintaining cache integrity without sacrificing functionality.
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...Transcat
Join us for this solutions-based webinar on the tools and techniques for commissioning and maintaining PV Systems. In this session, we'll review the process of building and maintaining a solar array, starting with installation and commissioning, then reviewing operations and maintenance of the system. This course will review insulation resistance testing, I-V curve testing, earth-bond continuity, ground resistance testing, performance tests, visual inspections, ground and arc fault testing procedures, and power quality analysis.
Fluke Solar Application Specialist Will White is presenting on this engaging topic:
Will has worked in the renewable energy industry since 2005, first as an installer for a small east coast solar integrator before adding sales, design, and project management to his skillset. In 2022, Will joined Fluke as a solar application specialist, where he supports their renewable energy testing equipment like IV-curve tracers, electrical meters, and thermal imaging cameras. Experienced in wind power, solar thermal, energy storage, and all scales of PV, Will has primarily focused on residential and small commercial systems. He is passionate about implementing high-quality, code-compliant installation techniques.
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODELijaia
As digital technology becomes more deeply embedded in power systems, protecting the communication
networks of Smart Grids (SG) has emerged as a critical concern. Distributed Network Protocol 3 (DNP3)
represents a multi-tiered application layer protocol extensively utilized in Supervisory Control and Data
Acquisition (SCADA)-based smart grids to facilitate real-time data gathering and control functionalities.
Robust Intrusion Detection Systems (IDS) are necessary for early threat detection and mitigation because
of the interconnection of these networks, which makes them vulnerable to a variety of cyberattacks. To
solve this issue, this paper develops a hybrid Deep Learning (DL) model specifically designed for intrusion
detection in smart grids. The proposed approach is a combination of the Convolutional Neural Network
(CNN) and the Long-Short-Term Memory algorithms (LSTM). We employed a recent intrusion detection
dataset (DNP3), which focuses on unauthorized commands and Denial of Service (DoS) cyberattacks, to
train and test our model. The results of our experiments show that our CNN-LSTM method is much better
at finding smart grid intrusions than other deep learning algorithms used for classification. In addition,
our proposed approach improves accuracy, precision, recall, and F1 score, achieving a high detection
accuracy rate of 99.50%.
Supermarket Management System Project Report.pdfKamal Acharya
Supermarket management is a stand-alone J2EE using Eclipse Juno program.
This project contains all the necessary required information about maintaining
the supermarket billing system.
The core idea of this project to minimize the paper work and centralize the
data. Here all the communication is taken in secure manner. That is, in this
application the information will be stored in client itself. For further security the
data base is stored in the back-end oracle and so no intruders can access it.
Discover the latest insights on Data Driven Maintenance with our comprehensive webinar presentation. Learn about traditional maintenance challenges, the right approach to utilizing data, and the benefits of adopting a Data Driven Maintenance strategy. Explore real-world examples, industry best practices, and innovative solutions like FMECA and the D3M model. This presentation, led by expert Jules Oudmans, is essential for asset owners looking to optimize their maintenance processes and leverage digital technologies for improved efficiency and performance. Download now to stay ahead in the evolving maintenance landscape.
Software Engineering and Project Management - Introduction, Modeling Concepts...Prakhyath Rai
Introduction, Modeling Concepts and Class Modeling: What is Object orientation? What is OO development? OO Themes; Evidence for usefulness of OO development; OO modeling history. Modeling
as Design technique: Modeling, abstraction, The Three models. Class Modeling: Object and Class Concept, Link and associations concepts, Generalization and Inheritance, A sample class model, Navigation of class models, and UML diagrams
Building the Analysis Models: Requirement Analysis, Analysis Model Approaches, Data modeling Concepts, Object Oriented Analysis, Scenario-Based Modeling, Flow-Oriented Modeling, class Based Modeling, Creating a Behavioral Model.
Call For Paper -3rd International Conference on Artificial Intelligence Advan...
IAU workshop 2018 day one
1. Containers: Portable, repeatable user-oriented application delivery II
HPC Saudi 2018
#dockerbday
Christian Kniep @CQnib
Walid Shaari @walidshaari
2. AGENDA : Good Morning Containers
https://events.docker.com/events/details/docker-az-zahran-presents-docker-birthday-5-celebration-dammam-edition-day-1#/
3. $id Christian
Over ten-year journey rooted in the industrial, automotive HPC in Germany, Christian started his career in Bull R&D supporting CAE
applications and VR installations, then later Dyna.
Co-founded the container and cloud workshop in ISC HPC conference when told at a meeting that HPC can not learn anything from
the emerging Cloud and BigData companies.
Since then, he is curious and leading DevOps and containerization effort wherever he goes.
Just before Docker, he worked on the cloud-stack team at Sony PlayStation.
Christian joined Docker Inc in 2017 to help push the adoption forward and be part of the innovation instead of an external bystander.
During the day he helps Docker customers in the EMEA region to fully utilise the power of containers; at night he likes to explore new
emerging trends by containerising them first and seek application in the nebulous world of DevOps.
@kniepbert
christian.kniep@docker.com
https://www.linkedin.com/in/christian-kniep-3004b053/
4. $id walid
Passionate about Openness, Open Source, DevOps, Infosec
Team member of the Expec Computer Center systems division
Red Hat Certified Architect RHCA V
Certified Kubernetes Administrator CKA
SANS GIAC Incident handler, Forensics and Web security certified.
Dhahran Docker & Ansible meetup organizer “Community Leader”
@walidshaari
walid.shaari@linux.com
https://www.linkedin.com/in/walidshaari/
https://github.com/walidshaari
5. Join the Docker Student
Community! Sign up here:
http://dockr.ly/students (with your school email) for
access to our free Docker Student Developer Kit and
more!
Become a Docker
Campus Ambassador!
For leaders on campus who want to help their
peers learn Docker! Learn more and apply here:
http://dockr.ly/campus-ambassador
Are you a student?
7. Let's get to know each other
▪ Assuming everyone knows a bit of
▪ Linux
▪ Unix
▪ Mac OSX CLI ?
▪ Development, Operations, Security, Research, Business, Others?
▪ DevOps
▪ Containers
▪ Schedulers
▪ Containers ecosystem
▪ Clusters, Load balancers, Orchestration
8. Goal
Up and running with containers ecosystem
informal interactive workshop format
11. Docker Momentum
Thank You for 5 Amazing Years!
Docker EE
commercial
customers
450+
Job listings on
LinkedIn
15K
Container
downloads
37B 3.5M 200+
Active Docker
user groups
Dockerized
apps
12. Containers are the “Fastest Growing Cloud Enabling Technology”
By 2020, more than 50% of global
organizations will be running
containers in production.
-Gartner
Title source: 451 Research
2017
24B
PULLS
13. Lab Instructions
STEP 1: Visit
https://training.play-with-docker.com/alacart/
Or https://goo.gl/hLA1VN
Create Docker hub/store account: https://hub.docker.com/
Join the Docker Community - dockr.ly/community
Join the slack channel: #5th-bday #dockerbday
16. HPC or Scientific Computing?
▪HPC workloads mostly
▪ Runs on Linux
▪ Preferably on bare-metal for maximum performance, lower overhead
▪HPC Application
▪ Broken into smaller parallel distributed problems across a cluster of nodes.
▪ Utilizes interprocess communications heavily via shared memory, or across the
network.
17. HPC Status Quo
▪ HPC dominated by Academics research and discovery
▪ Business HPC by the industry in the last 5-10 years seen an increase in HPC
interest (Automotive, Finance, O&E)
▪ Possible constraints:
▪ Snowflake deployments, each HPC cluster/supercomputer is build in mind with
specific use cases
▪ Long lived nodes.
▪ Bloated/drift/unclean maybe diskless reboots
▪ Reboot time, or launching app could be long due to system/memory checks, bootstrapping
▪ Old Linux distribution
▪ Fixed installation based on single enterprise distro (Scientific, RHEL)
▪ Old kernel features
19. Which workloads and frameworks are running on OpenStack?
Source: https://www.openstack.org/assets/survey/Public-User-Survey-Report.pdf
> 38%
scientific/technical
computing already
happening on
Openstack
27. Namespaces
Processes Isolation
● host sees all processes with real PID from the Kernels perspective
● first process within PID namespace gets PID=1
Host
cnt0
ps -ef
cnt1
start.sh
java -jar ..
cnt2
start.sh
java -jar ..
health.sh
28. Resource Isolation of Process Groups
7 as of Kernel 4.10
1. MNT: Controls mount points
2. PID: Individual process table
3. NET: Network resources (IPs, routing,...)
4. IPC: Prevents the use of shared memory between processes
5. UTS: Individual host- and domain name
6. USR: Maps container UID to a different UID of the host
7. CGRP: Hides system cgroup hierarchy from container
Other (incomplete list):
● RDMA
● Syslog
● Time
29. Container Namespaces
A starting container gets his own namespaces.
PIDMNT IPCNET USR
Host
UTS CGRP
cnt0 cnt1 cnt2
But can share namespaces with other containers or even the host
30. Host
All In
When using all host namespaces - we are on the host (almost like ssh).
PIDMNT IPCNET USRUTS CGRP
cnt0
$ docker run -ti --rm
--privileged
--security-opt=seccomp=unconfined
--pid=host
--uts=host
--ipc=host
--net=host
-v /:/host
ubuntu bash
root@linuxkit-025000000001:/# chroot /host
/ # ash
/ #
33. Overlay Filesystem
Compose a FS from multiple pieces
ubuntu:16.04
openjre:9-b114
appA.jar:1.1 appB.jar
ARG FROM openjre:9-b114
COPY appB.jar /usr/local/bin/
CMD [“java”, “-jar”, “/usr/local/bin/appB.jar”]
ARG FROM openjre:9-b114
COPY appA.jar /usr/local/bin/
CMD [“java”, “-jar”, “/usr/local/bin/appA.jar”]
FROM ubuntu:16.04
ARG JRE_VER=9~b114-0ubuntu1
RUN apt-get update
&& apt-get install -y openjdk-9-jre-headless=${JRE_VER}
&& java -version
openjre:9-b117
34. First Step, toward a container definition?
• What matters most? The application or data
• The application can be a process or a set of processes
• The use case might be not a running app
• Set of tools to develop an app
• Set of scripts "apps" that are part of a pipeline
• complete appliance
• Isolated contained environment "Encapsulation"
• Technical synonyms
• chroot
• jail
• partition
• namespace
• zone
35. chroot/jail
A chroot on Unix operating systems is an operation that
changes the apparent root directory for the current running
process and its children. A program that is run in such a
modified environment cannot name (and therefore normally
cannot access) files outside the designated directory tree.
The term "chroot" may refer to the chroot(2) system call or
the chroot(8) wrapper program. The modified environment
is called a chroot jail.
https://en.wikipedia.org/wiki/Chroot
36. Scott McCarty Twitter: @fatherlinux Blog: bit.ly/fatherlinux
THE HISTORY OF CONTAINERS
2008:
KERNEL & USER
NAMESPACES
2008:
LINUX
CONTAINER
PROJECT (LXC)
2013:
DOTCLOUD
BECOMES
DOCKER
2013:
RED HAT
ENTERPRISE
LINUX
2000
2010
2005
2015
2000:
JAILS ADDED
TO FREEBSD
2006:
PROCESS
CONFINEMENT
2007:
GPC RENAMED
CGROUPS
2014:
GOOGLE
KUBERNETES
2015:
RED HAT
CONTAINER
PLATFORM
2001:
LINUX -VSERVER
PROJECT
2015:
STANDARDS VIA
OCI AND CNCF
2003:
SELINUX
ADDED TO LINUX
MAINLINE
2005:
FULL RELEASE
OF SOLARIS
ZONES
2013:
DOTCLOUD PYCON
LIGHTNING TALK
Docker
provides
simple user
tools and
images.
Containers go
mainstream
38. CONTAINERS?
WHAT ARE THEY REALLY?
Linux features?
Namespace
cgroupsLXC
Union file systems
Configuration management?
Virtualization technology?
npm
jar
Packaging ?
rpm
deb
tar.gz
Virtual/environment management ?
Sandboxing?
chroot
BSD jail Solaris zones
IBM VM/370 (1972)
seccomp
39. IT DEPENDS
Manual
Configuration
Traditional VMs
Less Portable
Minimal overhead
Most Portable
Lots of overhead
Configuration
Management tools
Containers
Docker
Intel Clear ContainersSingularity
LXC/LXD
Non-Repeatable Repeatable
rkt
40. Container
Containment, isolation, or encapsulation of an environment.
Machine container:
Encapsulates a complete system image. e.g. Ubuntu, RHEL, Scientific Linux.
Application container:
Encapsulates a service/software. e.g. Django, ROR, Gitlab, redis, Openfoam, kafka, spark.
what is the smallest application container?
41. Possible HPC Caveats/Constraints
1. Memory/storage deduplication
2. Code Optimization for specific architecture
3. Limited take on HPC specific orchestration and scheduling
4. Hardware topology assumptions (e.g. GPU brand, interconnect)
5. Chroot based containers have none/limited tooling (e.g. introspection )
6. Chroot based containers might be hard to scan for security vulnerabilities,
hardening, and composition.
43. KUBERNETES SEEING THE MOST DEVELOPER
TRACTION
43https://www.slideshare.net/dberkholz/cloud-native-in-the-enterprise-realworld-data-on-container-and-microservice-ado
ption
48. Use Cases: Packaging
Agnostic packaging
Captures
○ Dependencies
○ Environment
○ Configurations
○ Executables
○ How about data?
○ What Else?
■ hint: m*
Pack once, Run everywhere
http://hpcbios.readthedocs.io/en/latest/HPCBIOS_2012-92.html
#EasyBuild #lmod #GUIX #NYU-Environment
49. Use Case: Portability
Portable/Scalable across
● platforms
● Distributions
● Environments
Separation of concerns, e.g. development pack and ship, operations scale and deploy.
development ensures app is resilient, operations enure infra is HA resilient and scalable
51. Use Case: Reproducible
Paolo Di Tommaso from the Center for Genomic Regulation presented : Manage Reproducibility of Computational Workflows with Docker Containers
and Nextflow.
https://www.slideshare.net/insideHPC/reproducible-computational-pipelines-with-docker-and-nextflow
https://youtu.be/Doo9H2-gBAk
52. Cloud use Case
- Transport
- Security CIA
- at rest encrypted signed image
- at runtime:
- platform specific
- scalability issues
- PMIx to the rescue?!
53. Data Center current state
SchedulerScheduler
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Scheduler
Jobs
Jobs
Jobs
Cluster Management A
Cluster Management B
Cluster Management C
54. Data Center
Secure Allocation of Resources
VC3
BigData
VC1
Infra
VC2
HPC
SchedulerSchedulerScheduler
DataCenter
Scheduler
jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
2nd
Generation Cluster Management
55. Mesos
▪ Mature, Open Source Apache Project
▪ Cluster Resource Manager
▪ Scalable to over 10,000s of nodes
▪ Fault tolerant, no single point of failure
▪ Multi-tenancy with strong resource isolation
▪ Improved resource utilization
58. MPI batch jobs
● use ssh inside container
● dssh
● Capitalize on openmpi
○ Openmpi/pbs/TORQUE
○ Process Management Interfaces PMIx
● Singularity examples uses Openmpi/Slurm
● mesos
● Commercial Univa support
● Research, and contribute ideas, pull requests to swarm,
kubernetes, slurm, pbs pro
● Joing the HPC-SIG
59. DISCLAIMER
@kelseyhightower :
The problem with most blog posts attempting to compare two different systems is
the author not having the sufficient experience to do so.
https://twitter.com/kelseyhightower/status/826974374536187905
61. What is Docker?
The leading open source platform to pack, ship and run
apps as lightweight containers.
Developers: use Docker to eliminate “works on my machine” problems when
collaborating on code with co-workers.
Operators: use Docker to run and manage apps side-by-side in isolated
containers to get better compute density.
Enterprises: use Docker to build agile software delivery pipelines to ship new
features faster, more securely and with confidence for both
Linux and Windows Server apps.
#dockerbday
62. • Standardized packaging for
software and dependencies
• Isolate apps from each other
• Share the same OS kernel
• Works for all major Linux
distributions
• Containers native to Windows
Server 2016
What are Docker containers?
63. Containers and VMs together
Containers and VMs together provide a tremendous amount of
flexibility for IT to optimally deploy and manage apps.
66. Architecture on Linux
Operating System
Control Groups
(cgroups)
Namespaces
(mnt,pid,ipc,...)
Layer Capabilities
AUFS,overlay,...
Other OS
Functionality
Docker Engine
REST interface
libcontainerd libnetwork storage plugins
containerd + runc
Docker Client Docker Compose Docker Registry Docker Swarm/K8s
67. Runtime
runc + containerd
●
● containerd
An industry-standard container runtime with an emphasis on simplicity, robustness and portability.
● runc
CLI tool for spawning and running containers according to the OCI specification
rootfs
config.json
runc executed container
68. libnetwork
Provide IP connectivity
The goal of libnetwork is to deliver a robust Container Network
Model that provides a consistent programming interface and the
required network abstractions for applications.
70. Plugins
Extend Functionality of the Engine
Framework to ‘intercept’ certain API calls and act on them.
Current supported drivers:
- VolumeDriver
- NetworkDriver
- IPAMDriver
- LogDriver
- MetricsCollector
- Authentication (authz)
// VolumeDriver
type Driver interface {
Create(Request) Response
List(Request) Response
Get(Request) Response
Path(Request) Response
Mount(Request) Response
Unmount(Request) Response
Capabilities(Request) Response
}
71. Architecture on Windows
Operating System
Other OS
Functionality
Docker Engine
REST interface
libcontainer libnetwork storage plugins
Docker Client Docker Compose Docker Registry Docker Swarm/K8s
Host Compute Service
Control Groups Namespaces Layer Capabilities
Object Namespace,
Process Table,
Networking
Job Objects Registry, Union like
filesystem extension
73. Docker is the only Containers-as-a-Service platform for IT that manages and secures
diverse applications across disparate infrastructure, both on-premises and in the cloud
Multi-Architecture
Operations
Infrastructure Independence
Secure Software
Supply Chain
COST SAVINGS
Linux Mainframe AWS Azure Other Public
Clouds
Windows
ENGINE FOR INNOVATION
DOCKER ENTERPRISE EDITION
74. Docker Enterprise Edition Capabilities
Enterprise Edition
Optimized Container Engine
Integrated App and Cluster
Management
Certification and Support
Policy Management
Image Scanning and
Monitoring
Secure Access and
User Management
Content Trust and
Verification
Application and
Cluster Management
Image Management
Security
Distributed State
Network
Container Runtime
Volumes
Orchestration
Application Composition, Deployment and Reliability
Certified Containers Certified Plugins
Certified Infrastructure
84. What is rkt?
From the rkt GitHub page, "rkt (pronounced "rock-it") is a CLI for running app
containers on Linux. rkt is designed to be secure, composable, and
standards-based.
#ACI
85. Why rkt?
● Don’t want to run dockerd daemon.
● Don’t require the Docker’s rich feature set/ecosystem.
● Can’t trust Docker security yet.
86. rkt
# rkt run --interactive docker://ubuntu --insecure-options=image
89. DOCKER HISTORY
▪ Started as internal project @ dotcloud
▪ Open Sourced in 2013
▪ Developed in the open
http://www.taos.com/from-dotcloud-to-docker/
90. Forces and Motivations behind containers
90
Loosely
Coupled
Services
Many Small
Servers
~2000 Today
Monolithic
Big Servers
Slow
changing
Rapidly
updated