Deutsche Telekom plans to dramatically improve the processes and the Time to Market of Fiber To The Home Infrastructure deployments in Germany. Working with legacy processes and software doesn’t meet their needs. A brand-new world is needed to sustain these goals. Success key items are automated geospatial data collection and analysis, end-to-end digital pipelines, online collaboration platforms sustaining the workflows and validation systems. Implementing agile workflows for the project requires solid foundations based on DevOps principles of automation and embodied in the role of SRE (Site Reliability Engineer). The deployment of an Open Source Spatial Data Infrastructure on an OpenShift cluster, which enables processes to scale and provides atomic architecture patterns for the deployment of modular components, is a central part of that project. In this talk, we’ll present the key challenges and how Deutsche Telekom and Camptocamp successfully implemented agile workflows for the project.

1. FTTH Factory
An illustration of the “Everything as Code” paradigm
Raphaël Pinson @raphink

2. www.camptocamp.com
Who am I?
■ Raphaël Pinson
aka Raphink
■ Infrastructure Developer &
Trainer
■ Automation
(Puppet, Augeas, Docker,
Kubernetes, Terraform)
■ Lausanne, CH

3. www.camptocamp.com
Camptocamp

4. www.camptocamp.com
Camptocamp Infrastructure Department
■ Involved in Open Source communities
(Puppet, Terraform, Rancher)
■ 14 Systems Engineers
■ Manages ~800 servers
■ Partnerships
SWITZERLAND
GERMANY

5. www.camptocamp.com
Measurements inside FTTH-factory
Leverages
Measurements
2D & 3D data
collections
Identification of
objects and surfaces
1
Data
conflation
Shift to UTM
spatial corrections
2
Automated
planning
optimized planning
based on detected
surfaces
3
Simplified
purchase
processes
All inclusive prices
for fixed offers
4
Digital plan
corrections
APP based mapping
of trenches
5

6. www.camptocamp.com
Why Automate?
■ Repetitive task error prone⇒
■ Dev / Int / prod
■ Define once, reuse many times

7. www.camptocamp.com
3 Ways of Managing Cloud Resources
■ Web Interface
○ easy, not flexible, not reproducible
■ CLI
○ flexible, not easily reproducible
■ As Code
○ flexible, reproducible, + collaboration
7

8. www.camptocamp.com
Infrastructure Technology Stack

9. www.camptocamp.com
PROVISIONING AS CODE
Cloud Infrastructure Automation
■ TOOLS: OPENSTACK & TERRAFORM
○ Provision networks, security groups, etc. from code
○ Provision VMs and orchestrate their deployment from code
○ Support for multiple cloud providers and third party tools
■ BENEFITS
○ Full control of all cloud resources
○ Reproductibility of environments
○ Easy decommissioning

10. www.camptocamp.com
PROVISIONING AS CODE
Case Study: Provision a Development Environment
■ OBJECTIVE
Deploy Development and Production Environments
in a fully automated way
■ IMPLEMENTATION
Terraform is used to:
○ provision networks and nodes, integrate with Puppet/Ansible
■ BENEFITS
○ Fully automated, extremely fast deployment of deployments
○ All resources managed in one location

11. www.camptocamp.com
CONFIGURATION MANAGEMENT
Maintaining Machine State
■ TOOLS: TWO APPROACHES
○ Pull approach (Cfgmgmt): Cfengine, Puppet, Chef
○ Push approach (deploy): Ansible, Salt
■ BENEFITS
○ No manual management of nodes
○ Reproductibility of nodes
○ Inventory
○ More confidence in environments and deployments

12. www.camptocamp.com
CONFIGURATION MANAGEMENT
Case Study: Manage Server Fleet with Puppet
■ OBJECTIVE
Manage configuration for ~50 servers in two environments
■ IMPLEMENTATION
○ Puppet Open Source (Camptocamp’s Dockerized Stack)
○ DBs managed with Puppet
○ Identity Management
■ BENEFITS
○ Very fast setup of new nodes
○ Dynamic, up-to-date inventory
○ Nodes are kept up-to-date with their target configuration

13. www.camptocamp.com
DEPLOYMENT AS CODE
Deploying Complex Software
■ TOOLS: ANSIBLE, BOLT
○ Orchestration of multi-node deployments with dependencies
■ BENEFITS
○ Reproductibility of deployments
○ More confidence in deployments

14. www.camptocamp.com
DEPLOYMENT AS CODE
Case Study: OpenShift Deployment
■ OBJECTIVE
Deploy an OpenShift cluster, with various roles
and inter-nodes dependencies.
■ IMPLEMENTATION
Red Hat provides Ansible playbooks to deploy an OpenShift cluster.
■ BENEFITS
○ Fully automated OpenShift deployment
○ Much easier/faster to deploy new nodes in the cluster

15. www.camptocamp.com
CONTAINER ORCHESTRATION
Containerized Application Deployment as Code
■ TOOLS: OPENSHIFT & HELM
○ Gather all Kubernetes objects in one package
■ BENEFITS
○ Distribute applications as catalog entries
○ Fully customizable packages

16. www.camptocamp.com
CONTAINER ORCHESTRATION
Case Study: Puppet Stack Deployment with Helm
■ OBJECTIVE
Deploy GIS applications (geOrchestra) on OpenShift
■ IMPLEMENTATION
○ Helm charts for each component
○ Helmfile for full stacks
■ BENEFITS
○ Easily deploy a complex application on OpenShift
○ Parameters and secrets passed to templates

17. www.camptocamp.com
CONTINUOUS INTEGRATION
CI/CD Pipelines as Code
■ TOOLS: JENKINS
○ Continuous Integration platform
○ Integrates with multiple Version Control Systems
○ Allows to build pipelines
○ CI/CD as Code with Jenkinsfile
○ Extensible with Groovy code
■ BENEFITS
○ Flexible way to program CI/CD tasks
○ Portable to other Jenkins instances
○ Pipeline approach

18. www.camptocamp.com
CONTINUOUS INTEGRATION
Case Study: CI/CD with OpenShift
■ OBJECTIVE
○ Automatically deploy a GIS multi-tiered application
from code with OpenShift
■ IMPLEMENTATION
○ Use ImageStreams with OpenShift builds (using Jenkins)
and Jenkinsfile, along with triggers for deployment.
■ BENEFITS
○ Fully automated pipeline workflow for developers,
with automated checks and triggers.

19. www.camptocamp.com
OBSERVABILITY AS CODE
Logs, Metrics & Monitoring
■ TOOLS: ELK, PROMETHEUS, GRAFANA
○ Gather logs & metrics
○ Provide code for data processing and dashboards
■ BENEFITS
○ Prometheus time series provide
a flexible query language
○ JSON dashboards provide portability

20. www.camptocamp.com
OBSERVABILITY AS CODE
Case Study: Monitor Puppet resources and performance
■ OBJECTIVE
Gather performance metrics for the Puppet stack,
both on the server and agents.
■ IMPLEMENTATION
○ The Puppet prometheus_reporter module
provides a report plugin to collect agent data
○ JMX for internal Puppet Enterprise server metrics,
gathered in Prometheus
■ BENEFITS
Easily deployed dashboards with tuned metrics and monitoring.

21. www.camptocamp.com
BACKUPS AS CODE
Automate Backups of Container Volumes
■ TOOLS: KUBERNETES AND BIVAC
○ Automatic backups of PVCs
○ Tune backup parameters with Labels
■ BENEFITS
○ Allow users of container platforms
○ to choose with Data Volumes to backup
○ Allow users to tune how backups are performed

22. www.camptocamp.com
BACKUPS AS CODE
Case Study: Backup Data Volumes in OpenShift
■ OBJECTIVE
○ Provide an API for developers to backup their Data Volumes.
■ IMPLEMENTATION
○ The Bivac stack is deployed on OpenShift projects
to detect and backup Physical Volume Claims automatically.
■ BENEFITS
○ Backup container data where it is stored and let users
tune how to perform the backup.

23. www.camptocamp.com
CONCLUSION
Everything as Code
■ TOOLS
○ Many open-source tools to automate:
■ Systems Administration
■ Software Testing and Deployment
■ BENEFITS
○ Increased flexibility
○ Increased collaboration between teams
○ DevOps everything