An introduction to the Docker concept. Experiences with ASP.NET Core and Docker, How Docker can help produce modular deployments for ASP.NET web applications. Presented at Vermont Code Camp #8, UVM, Burlington VT, September 17, 2016
1. An Industry Leader In Human
Capital Management Technology
From big and complex
to small and simple:
ASP.NET Core and Docker
Chuck Megivern
cmegivern@evolutionhcm.com
iSystems, LLC
2. Copyright 2016
Outline
• The need and the possibilities
• Industry trends
• Example: an iSystems production application in
ASP.NET Core
• Containerized Microsoft
2
3. Copyright 2016
Our company obtained a project that had Solr as a component … this is what we
received.
Extract the jetty-7.2.zip on the desired server
Copy folder jetty-7.2 to C:
Open command prompt as administrator and run the following commands :
cd "C:jetty-7.2"
nssm install SOLR "C:jetty-7.2startsolr.bat"
sc.exe config "SOLR" obj= "isystemsuser" password= ""
(replace user with the good one and change password if needed)
Add the same user with read/write rights to folder "C:jetty-7.2"
This way SOLR is going to run as a service on the Windows server
To be able to communicate between SOLR and the SQL database there's a Windows service (ThePortSearchIndexFeeder) that needs to be installed. As a
recommandation it's preferable to install this service on the same server as SOLR so we keep everything together.
Extract ThePortSearchIndexFeeder.zip and copy folder ThePortSearchIndexFeeder to C:componentsservices
You need to edit the "TemplateLocation" key under the config file of the service
"C:componentsservicesThePortSearchIndexFeederThePort.SearchIndexFeeder.exe.config" to the indexfeed folder
Install the service, open command prompt as administrator :
cd "C:componentsservicesThePortSearchIndexFeeder"
C:WindowsMicrosoft.NETFrameworkv4.0.30319installutil.exe ThePort.SearchIndexFeeder.exe
sc.exe config "ThePortSearchIndexFeeder" obj= "isystemsUser" password= ""
(replace user with the good one and change password if needed)
You need to edit the "SearchPartition" table under "Portbase" SQL database. You need to update all of these 3 values to the name of the server where the solr
instance is installed
ClusterName, NodeAName, NodeBName
UPDATE SearchPartition SET ClusterName = 'desired_server_name', NodeAName = 'desired_server_name', NodeBName = 'desired_server_name' WHERE
ID = '00000000-0000-0000-0000-000000000001'
Edit both reg files included in theport_reg_files.zip go through each line of the file and replace value for database name, connection string, share name etc.
Add them to the registry of the server.
Run those commands to start both services installed earlier :
net start SOLR
net start ThePortSearchIndexFeeder
4. Copyright 2016
Our company obtained a project that had Solr as a component … this is what we
received.
Extract the jetty-7.2.zip on the desired server
Copy folder jetty-7.2 to C:
Open command prompt as administrator and run the following commands :
cd "C:jetty-7.2"
nssm install SOLR "C:jetty-7.2startsolr.bat"
sc.exe config "SOLR" obj= "isystemsuser" password= ""
(replace user with the good one and change password if needed)
Add the same user with read/write rights to folder "C:jetty-7.2"
This way SOLR is going to run as a service on the Windows server
To be able to communicate between SOLR and the SQL database there's a Windows service (ThePortSearchIndexFeeder) that needs to be installed. As a
recommandation it's preferable to install this service on the same server as SOLR so we keep everything together.
Extract ThePortSearchIndexFeeder.zip and copy folder ThePortSearchIndexFeeder to C:componentsservices
You need to edit the "TemplateLocation" key under the config file of the service
"C:componentsservicesThePortSearchIndexFeederThePort.SearchIndexFeeder.exe.config" to the indexfeed folder
Install the service, open command prompt as administrator :
cd "C:componentsservicesThePortSearchIndexFeeder"
C:WindowsMicrosoft.NETFrameworkv4.0.30319installutil.exe ThePort.SearchIndexFeeder.exe
sc.exe config "ThePortSearchIndexFeeder" obj= "isystemsUser" password= ""
(replace user with the good one and change password if needed)
You need to edit the "SearchPartition" table under "Portbase" SQL database. You need to update all of these 3 values to the name of the server where the solr
instance is installed
ClusterName, NodeAName, NodeBName
UPDATE SearchPartition SET ClusterName = 'desired_server_name', NodeAName = 'desired_server_name', NodeBName = 'desired_server_name' WHERE
ID = '00000000-0000-0000-0000-000000000001'
Edit both reg files included in theport_reg_files.zip go through each line of the file and replace value for database name, connection string, share name etc.
Add them to the registry of the server.
Run those commands to start both services installed earlier :
net start SOLR
net start ThePortSearchIndexFeeder
5. Copyright 2016
My comment on the company wiki (which was soon deleted)
How to run an official Solr image, configured by the
professionals:
From: https://hub.docker.com/_/solr/
To run a single Solr server:
$ docker run --name my_solr -d -p 8983:8983 -t solr
Then with a web browser go to http://localhost:8983/ to see the
Admin Console (adjust the hostname for your docker host).
6. Copyright 2016
Software gets more complex over the years
New applications arise all the time.
Existing applications get new functions and more features
New architectures and frameworks arise all the time
Existing architectures and frameworks get new functionality to respond to
marketplace forces
We know how this goes!
With a traditional environment:
Physical server + operating system + application server + application
Lots of configuration points, lots of dependencies between these layers
The price of complexity
7. Copyright 2016
Just look at control panels for application server software!
Example:
Microsoft IIS,
IBM WebSphere,
Open Source / Red Hat
Jboss control panels.
A hands on lab for IBM Websphere install runs 37 pages.
(just the install)
8. Copyright 2016
The problem, continued
•… and because apps and servers are getting increasingly complicated
•This affects our ability to rapidly deploy new features and more servers.
•A typical environment has grown from small numbers of servers
(each server has a name)
lLaunch in days or weeks. Lifetimes of many months
•… to many more of these kinds of servers.
lServer lifetime can be hours or even minutes.
•And as hardware keeps getting cheaper and cheaper
lThe cost of run time implementations keeps going down.
9. Copyright 2016
Virtualization and server scripting helps
lBut each virtual server is still a server
lAnd, in a cloud environment: servers are getting really cheap
lApi to launch a virtual server
lNow servers have to have numbers (not necessarily names).
lLaunched in a number of minutes. Lifetime of hours or days.
lWe can have a script to launch the server, instead of wading through menus and the
hands on server launching labs.
Server Launching Frameworks:
Chef
Puppet
10. Copyright 2016
Can we remove complexity?
From an architecture? … it's happening
Example: J2EE specification:
Web Container
EJB Container (persistence framework)
Messaging beans
Most recent applications:
Web Container only
Selectively add frameworks
But the application server still must support old features
Launching and configuring a server still the same.
l“Why not take a fully configured application and just run it?”
11. Copyright 2016
Software in a container
Deployment:
• Instead of application
• Deploy an application plus all its dependencies together. “A container”
• And then: Come up with frameworks to run these containers.
Not really that new:
VMWare has Virtual Appliance
We can “image” a virtual machine.
lBut instead of running on a proprietary platform (VMWare)
how about running a container anywhere.
each VM (or container) could have its own operating system maybe we could ..
13. Copyright 2016
Images and Containers
Containers are made from images, which are a complete file system
Built layer upon layer
Are immutable once built
There are many standard official images, like Solr.
= Solr plus all Solr dependencies, including the OS. And a
default configuration.
There are images like Debian, Ubuntu, CentOS, MongoDB, MySQL, Postgres, nginx,
apache httpd, wordpress.
You can use them as a starting point, add your own configuration. Now you have
your image.
Under the covers, Docker can share layers, which helps make Docker images and
containers lightweight, portable and efficient.
14. Copyright 2016
Images and Containers, cont.d
Terminology:
Image = complete file system. Immutable.
Container = started from an image. It's running and has PID's and networks inside.
Registry = images can be published to (pushed) and pulled from. Used to share
images across servers. Others beside Docker host registries also. You can host your
own.
Containers run anywhere Docker runs.
All containers start / stop the same way.
No overall control panel for a container needed.
Test anywhere. Images are immutable.
To share an image, they are published in a registry.
Docker has a “docker hub” registry
15. Copyright 2016
What goes inside a container?
Your application or configured framework
You might call this your “microservice”.
All the applications dependencies
Perhaps even a server which is a dependency to run your application
Nothing more is needed.
For just running my single application, I can use a simple server.
• Microsoft ASP.NET Core app:
• instead of Windows Server and IIS, the Kestrel server by Microsoft is
designed to run a single application. Kestrel is much simpler and faster
than IIS.
• Java JVM based app:
• Instead of a full J2EE server, just run Tomcat, or Jetty, or Undertow or
Netty. (listed in order of decreasing complexity.
• Maybe Node or Python based server are suitable for you.
You could run all the above on a single host using Docker.
20. Copyright 2016
Benefits to shipping industry
Standard containers: standard infrastructure
Ship cargo payload: 16 X bigger
Load and unload time: days to hours
Container contents: sealed
Some new benefits, including:
Automated tracking
21. Copyright 2016
Docker is container shipping system for code.
Benefits are similar to shipping industry
Standard containers: standard deployments
Cargo payload : Much more effective use of physical server resources than Virtual Machines
A container is not a full VM
Load and unload time: encourages microservice architecture, build quickly
Container contents: sealed ie: immutable.
o Debug on your workstation, works the same in all environments.
o A dependency can change, but existing images don’t. (Positive impact on quality). If you like
your container’s function, it will stay that way, even if you move it to a different physical host.
Some new benefits, including:
Rapid development: based on adding to and enhancing open source contributions.
Many official containers available.
22. Copyright 2016
iSystems API Gateway
Docker ideal for an API Gateway: OAuth2 authentication, multi-tenant, multi-client,
in front of API implementation code.
The elements to be deployed are diverse, lightweight and simple components. Think
modular.
Component: Nginx
Reverse proxy
Load balancer
SSL termination
URL rewriting
Static file serving.
Production stability: configuration is inside the container
Can have api endpoints dynamically update
23. Copyright 2016
Docker usage example: our nginx container
Dockerfile: controls image build:
FROM nginx ← says use official image
COPY ./src /etc/nginx <- this line says: copy my configuration files and
certificates into the image
Docker build # builds the image
Docker push # puts it to the registry
Docker run # runs it in a server
24. Copyright 2016
Other components
•Component: API Client Manager
•Manages OAuth2 Clients
•ASP.NET Core Web application
•Running in a Docker container
•Interacts with the nginx “upstream” server definitions
•Access to client manager UI, used by billing, IT, support.
ASP.NET Identity framework
ASP.NET “Razor” view engine is in ASP.NET Core
Persistence of the gateway data using Entity Framework Core
Writes nginx endpoint configuration info.
•Component: Health Checking
•ASP.NET Core Web Application
•Component: Redis
•Component: HAProxy
•Our usage: to be placed in the dmz and connect to servers inside the firewall.
25. Copyright 2016
Design and deployment considerations
Can add features and functionality by replacing the
ASP.NET Client Manager Container
Its a microservice
Looking at extending our identity and single signon capabilities.
Other components can be also be changed or upgraded independently
Production:
No single point of fail deployment by wiring together containers.
Originally: used custom systemd unit files for each container.
Now: using docker compose. It's production ready.
A YAML file describes containers and how they interconnect
26. Copyright 2016
ASP.NET Core and Docker
ASP.NET Core
Core common runtime
Official Microsoft Docker images: microsoft/dotnet
Two versions: Core, and a full ASP.NET is available. Full version is Windows
only.
ASP.NET Core open source
Source on GitHub
Dockerfiles are in Microsoft ASP.NET source code commits
27. Copyright 2016
ASP.NET Core vs 4.5
ASP.NET core runs on Windows, Linux, MacOs, Docker.
Command line: dotnet new, dotnet restore, dotnet run, ..
project.json: defines NuGet dependencies, tools (like EF
tools),
Q: how do you use ASP.NET Identity and EF Core?
A: they are NuGet dependencies
If you are an open source developer, project.json seems to
have similaritities to a maven pom.xml.
28. Copyright 2016
ASP.NET Core observations
NuGet ASP.NET Core dependencies are very modular. Just
bring in the dependencies you need. Even MVC.
Dependency injection is a first class citizen.
You don’t need to add Autofac or Unity.
However, you can add in your own dependency injection
framework if you want.
Logging has been rewritten, and there is basic console
logging built in. You can add a logging framework, and log
to Splunk or Seq with configuration changes only.
Simplified, refactored, and improved configuration
framework. Use json, xml, ini. Start with a base
“appconfig.json” and overlay a “appconfig.qa.json” based on
environment. Overlay with runtime environment variables
also.
29. Copyright 2016
ASP.NET Core experience
Our development environment:
Visual Studio 2015 templates, then develop controllers, restful APIs,
ASP.NET MVC “Razor” service side based web pages, launching
AngularJs pages.. all work well.
Visual Studio Debug (can run either IIS express or Kestrel)
Use “docker-machine” on workstation to build and debug containers.
Containers use .NET Core images, target a Linux underlying host.
Deployment experience:
VMWare -> CentOS VM -> Docker -> (inside the container, Microsoft
starts with Debian and then adds .NET Core CLR. Out application is
added on top of that).
Found this to be extremely stable, will stay up for months unattended. No
IIS application pool restarting required.