1.
An Introduction to Using
PostgreSQL with
Docker & Kubernetes
Jonathan S. Katz
DC PostgreSQL User Group
April 2, 2018
1

2.
About Crunchy Data
• Leading provider of trusted
open source PostgreSQL and
PostgreSQL related
technologies, support and
training to enterprises.
• We're Hiring!
• crunchydata.com
2

3.
About Me
• Director of Communications &
Customer Success, Crunchy Data
• Longtime PostgreSQL community
contributor
• Director, PgUS
• Co-Organizer, NYCPUG
• @postgresql + .org content
• Conference organization +
speaking galore!
• @jkatz05
!3

4.
Outline
• Containers: A Brief History
• Containers + PostgreSQL
• Setting up PostgreSQL with Containers
• Deploying! - Container Orchestration
• Look Ahead: Trends in the Container World
!4

5.
What are Containers?
• Containers are processes that
contain all the requirements to
execute an application
• Similar to virtual machines,
Sandbox for applications
similar to a virtual machine but
with increased density on a
single host
5
Source: Docker

6.
Container Glossary
• Image
• The base filesystem that defines what to load in a
container runtime. This is static.
• Container
• The runtime loads an image and instructions on how to
set up the run time environment. File changes can be
persisted to external mounts
https://docs.docker.com/glossary/
!6

7.
Why Containers?
• Lightweight
• Compared to VMs, use less disk, RAM, CPU
• Ability to sandbox
• Runtime is in an isolated environment
• Portability
• Can run containers on different platforms
!7

8.
Container Runtimes
• Open Containers Initiative (OCI)
• Developing standards for how containers should run
and interface with platforms
• containerd, Docker, runC, rkt
!8

9.
Container Example:
Web Application
Create Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir -p /path/to/app/code/
WORKDIR /path/to/app/code/
ADD requirements.txt /path/to/app/code/
RUN pip install -r requirements.txt
ADD . /path/to/app/code/
!9
1.11.11
7.4

10.
Container Example:
Web Application
Define and Execute the Container Runtime
!10
/path/to/app/code/
8000
1.11.11
8000
7.4

11.
Container Example:
Web Application
Create multiple runtimes
!11
/path/to/app/code/
8001
1.11.11
8000
7.4
8000
1.11.11
8000
7.4
8002
1.11.11
8000
7.4

12.
Container Example:
Web Application
Update Django Versions
!12

13.
Containers & PostgreSQL
• Containers provide several advantages to running
PostgreSQL:
• Setup & distribution for developer environments
• Ease of packaging extensions & minor upgrades
• Separate out secondary applications (monitoring,
administration)
• Automation and scale for provisioning and creating
replicas, backups
Images sourced: docker.com & postgresql.org!13

14.
Containers & PostgreSQL
• Containers also introduce several challenges:
• Administrator needs to understand and select appropriate
storage options
• Configuration for individual database specifications and
user access
• Managing 100s - 1000s of containers requires appropriate
orchestration (more on that later)
• However, these are challenges you will find in most database
environments
Images sourced: docker.com & postgresql.org!14

15.
• We will use the Crunchy Container Suite
• PostgreSQL (+ PostGIS): our favorite database; option to add our favorite geospatial extension
• pgpool + pgbouncer: connection pooling, load balancing
• pgbackrest: terabyte-scale backup management
• Monitoring: Prometheus + export
• Failover: "crunchy-watch"
• pgadmin4: UX-driven management
• Open source!
• Apache 2.0 license
• Support for Docker 1.12+, Kubernetes 1.5+
• Actively maintained and updated
Getting Started With
Containers + PostgreSQL
https://github.com/CrunchyData/crunchy-containers!15

16.
Getting Started With
Containers + PostgreSQL
https://github.com/CrunchyData/crunchy-containers
!16

17.
Getting Started with
Containers & PostgreSQL
mkdir postgres
cd postgres
docker volume create --driver local --name=pgvolume
docker network create --driver bridge pgnetwork
cat << EOF > pg-env.list
PG_MODE=primary
PG_PRIMARY_USER=postgres
PG_PRIMARY_PASSWORD=password
PG_DATABASE=whales
PG_USER=jkatz
PG_PASSWORD=password
PG_ROOT_PASSWORD=password
PG_PRIMARY_PORT=5432
EOF
docker run --publish 5432:5432
--volume=pgvolume:/pgdata
--env-file=pg-env.list
--name="postgres"
--hostname="postgres"
--network="pgnetwork"
--detach
crunchydata/crunchy-postgres:centos7-10.3-1.8.2
Load the crunchy-postgres image
!17

18.
Demo
!18

19.
Adding pgadmin4
• pgadmin4 is an open source
user interface for working with
PostgreSQL
• Desktop + Web support
19

20.
Adding pgadmin4
Load the crunchy-pgadmin4 image
docker volume create --driver local --name=pga4volume
cat << EOF > pgadmin4-env.list
PGADMIN_SETUP_EMAIL=jonathan.katz@crunchydata.com
PGADMIN_SETUP_PASSWORD=securepassword
SERVER_PORT=5050
EOF
docker run --publish 5050:5050
--volume=pga4volume:/var/lib/pgadmin
--env-file=pgadmin4-env.list
--name="pgadmin4"
--hostname="pgadmin4"
--network="pgnetwork"
--detach
crunchydata/crunchy-pgadmin4:centos7-10.3-1.8.2
!20

21.
Demo
!21

22.
Recap So Far
• Explored what / why / how of containers
• Set up a PostgreSQL 10 instance that can be managed
with pgadmin4 in a development environment
!22

23.
How do I manage
these things at scale?
!23

24.
Kubernetes:
Container Orchestration
• "Open-source system for
automating deployment,
scaling, and management of
containerized applications."
• Manage the full lifecycle of a
container
• Pods => Services
• Nodes => Cluster
• Volumes
24
Source: https://kubernetes.io

25.
Kubernetes is
incredibly powerful
!25

26.
Kubernetes is
*not* easy to set up
!26

27.
When to Use Kubernetes
• Managing a few
containers is very different
than managing 100s, if
not 1000s
• Advanced “orchestration”
required to manage this
many containers
27

28.
PostgreSQL in a
Kubernetes World
• Advantages for using Kubernetes with PostgreSQL
• Your own "database-as-a-service"
• Mass apply databases commands:
• Updates
• Backups / Restores
• ACL rule changes
• Scale up / down replicas, though...
!28

29.
PostgreSQL in a
Kubernetes World
• Kubernetes is "turnkey" for stateless applications
• e.g. web servers
• Databases do maintain state: permanent storage
• Persistent Volumes (PV)
• Persistent Volume Claims (PVC)
!29

30.
The "postgres operator"
• (aka "Crunchy PostgreSQL for Kubernetes")
• Allows an administrator to run PostgreSQL-specific
commands to manage database clusters, including:
• Creating / Deleting a cluster (your own DBaaS)
• Scaling up / down replicas
• Apply user policies to PostgreSQL instances
• REST API
!30
https://github.com/CrunchyData/postgres-operator

31.
How The Operator Works
!31

32.
Example Architecture
!32

33.
Looking Ahead
• Containers are no longer "new" - orchestration technologies have
matured
• Big debate with containers + databases: storage & management
• No different than virtual machines + databases
• Databases are still databases: need expertise to manage
• Database deployment automation flexibility
• Deploy your architecture to any number of clouds
• "Monitoring at scale"
!33

34.
Conclusions
• Containers + PostgreSQL gives you:
• Easy-to-setup development environments
• Your own production database-as-a-service
• Tools to automate management of over 1000s of
instances in short-order
!34

35.
Questions?
!35
jonathan.katz@crunchydata.com
@jkatz05