Setting up a local development environment is an integral part of the start of any web-project. In the report, I will share with you the challenges our team encountered during the existence of the project and the ways in which they are solved. We will go from local installation to the workstation through VirtualBox, Vagrant + Chef and Docker-compose. Join, it will be interesting! Presentation given by Sid at Wise TechTalks

1. Local development
environment
evolution
Serhiy, DevOps Engineer
Title
Title
Project review

2. Project highlights:
● Intellectual system for jobs searching
● Over 50 production servers (scalable)
● Millions of users
● 6Tb+ of processing data
● Over 1 billion emails sent yearly
Title
Title
Project review
Env definition

3. Project review
Env definition
Environment components:
1. Hardware
2. Operating system
3. Compiler/Interpreter/Translator
4. Modules, libraries
5. Database
6. etc.
Env requirements

4. Title
Env definition
Env requirements
Stage 0 schema
Requirements:
1. Cross-platform
2. Maximum likeness to production
3. Optimal resource usage
4. Fast and easy implementation
5. Convenient usage
6. All production environment changes should be
fetched to local environment automatically
7. Space for experiments

5. Title
Env requirements
Stage 0 schema
Stage 0 review

6. Title
Stage 0 schema
Stage 0 review
Stage 1 schema
Procs:
● StackOverflow guides
enough to start
● No need to deal with
virtualization
Cons:
● Host system become
messy
● Software versions
issues
● Difficulties with
backup/reproducing
● Different systems on
Production and
Workstation
Stage 0: Self management

7. Title
Stage 0 review
Stage 1 schema
Virtualbox

8. Title
Stage 1 schema
Virtualbox
Stage 1 review
● Cross platform
● Import/export VM appliances
● Shared folders
● Port forwarding
● Shared Clipboard

9. Title
Virtualbox
Stage 1 review
Stage 2 schema
Procs:
● Less time to start with
development
● Possibility to keep
applications isolated
● Similar environment for
all devs
● Easy to reset
environment
Cons:
● Manual software updates
● State dataset with manual updating
● Environments desynchronization
with a time
● Time-cost process of centralised
image update
Stage 1: Prepared VirtualBox images

10. Title
Stage 1 review
Stage 2 schema
Chef

11. Title
Stage 2 schema
Chef
Vagrant
● Source control
● Cross-platform
● Possibility to keep infrastructure
up to date permanently
● Ruby-powered
● Awesome Chef Supermarket
● TestKitchen
● Inspec Tests
● Attributes overriding
● Secrets management

12. Title
Chef
Vagrant
Vagrant config
● Source control
● Cross-platform
● Different VMs with same
tool(VirtualBox/VMware/EC2...)
● Ruby-powered
● Different provisioners
● MultiVM and internal networking
management

13. Title
Vagrant
Vagrant config
Vagrantfile
name: fetch
box_url: "https://s3.amazonaws.com/~/virtualbox.box"
hostname: serhii-local.project.com
cpu: 2
memory: 1024
network:
ports:
guest: 8082
host: 8081
ip: 192.168.0.228
chef:
version: 12.19.36
server_url:"https://chef.project.com/organizations/project"
key_path: "vagrant.pem"
client_name: vagrant
environment: development
role: ['basic', 'local']
synced_dirs:
- name: 'Api_1'
host_path: "PUT YOUR LOCAL PATH HERE"
guest_path: "/home/api_1/app"
owner: api_1
group: api
mount: false

14. Title
Vagrant config
Vagrantfile
Stage 2 review
require 'yaml'
cfg = YAML.load_file(File.join(File.dirname(__FILE__), 'config.yaml'))
Vagrant.configure("2") do |config|
config.vm.box = cfg['name']
config.vm.box_url = cfg['box_url']
config.vm.hostname = cfg['hostname']
config.vm.network(:private_network, {:ip=>cfg['network']['ip']})
config.vm.provider "virtualbox" do |v|
v.memory = cfg['memory']
v.cpus = cfg['cpu']
end
mount_synced_dirs(config.vm,cfg['synced_dirs'])
config.vm.provision :chef_client do |chef|
chef.version = cfg['chef']['version']
chef.chef_server_url = cfg['chef']['server_url']
chef.validation_key_path = cfg['chef']['key_path']
chef.validation_client_name = cfg['chef']['client_name']
chef.environment = cfg['chef']['environment']
chef.encrypted_data_bag_secret_key_path = 'encrypted_data_bag_secret'
cfg['chef']['role'].each do |role|
chef.add_role role
end
end
end

15. Title
Vagrantfile
Stage 2 review
Stage 3 schema
Procs:
● Set up environment in 1
command
● Keeping local env in synch
with production
● Environment versioning
● Easy to reproduce
Cons:
● Long initial provisioning
● High workstation
resource usage
Stage 2: Vagrant + Chef

16. Title
Stage 2 review
Stage 3 schema
Docker

17. Title
Stage 3 schema
Docker
Solution example
● Lightweight system-level
virtualization
● Keeping build configuration
alongside the code
● Layers
● Docker images as artifacts
● Docker-compose
● Different orchestration
services
● Docker-hub
● Alpine-based images for
production

18. Title
Docker
Solution example
Stage 3 review
Dockerfile example:
FROM node:8.6
WORKDIR /code
COPY package.json /code/package.json
RUN npm install
COPY . /code
RUN npm test
CMD ["npm", "start"]
docker-compose.yml example:
version: "3"
services:
cache:
image: redis:4
networks:
- project
ports:
- "6379:6379"
web:
build: .
command: node --inspect-brk=0.0.0.0:5858 worker.js
volumes:
- .:/code
networks:
- project
ports:
- "5858:5858"
networks:
project:
driver: bridge

19. Title
Solution example
Stage 3 review
Thanks
Procs:
● Fast set up
● Fully isolated microservices
● Possibility to run cluster
locally
● Keeping environment
configuration alongside the
code
Cons:
● Time costs for learning
new technology
Stage 3: Docker + Docker-Compose

20. Title
Stage 3 review
Thanks
Thank you for attention!