How to upgrade to MongoDB 4.0 - Percona Europe 2018

Upgrade to MongoDB 4.0
Antonios Giannopoulos
DBA @ Rackspace/ObjectRocket
linkedin.com/in/antonis/
1

Introduction
www.objectrocket.com
2
Antonios Giannopoulos

Overview
• Upgrade Procedure
• Application Layer
• Middleware
• Database Layer
• Rollback Procedure
• Why 4.0?
3

MongoDB 4.0
4
MongoDB 4.0 released on June 26,2018
Current minor version is 4.0.3
You can obtain it from:
- Mongo Inc Download Center
- Percona Server Download Center
- Repos like yum, apt-get…

MongoDB 4.0
5

Upgrade Replica-Set
6
Upgrade the secondary, one at a time
o Shut down the mongod instance
o Replace the binary with the 4.0 binary
o Restart the member
Connect a mongo shell to the primary
o Issue rs.stepDown()
o Ensure a new Primary is elected
Upgrade the ex-Primary
Enable backwards-incompatible 4.0 features
db.adminCommand( { setFeatureCompatibilityVersion: "4.0" }

Sharded Cluster
7
s1 s2
Stop Balancer
o sh.stopBalancer()
o sh.getBalancerState()
Upgrade config servers
o Use the replica-set steps
Upgrade the shards
Upgrade the mongos
o One at a time
o Replace binary and restart
Enable backwards-incompatible 4.0 features
db.adminCommand( {
setFeatureCompatibilityVersion: "4.0" } )
Restart the mongos

Application
• Upgrade Driver
• Upgrade Language
• Inspect the code base
8

Driver
9
Your driver must be compactible with 4.0

Driver
10

Driver
11

So just upgrade the driver?
12
The underling language version must also be compactible with your driver
Mongo 4.0 Compatible

Upgrade the driver/language
13
Perform safe upgrades /always have rollback in mind
Python supports “virtual environments”
Python “virtualenv” creates an isolated environment for Python projects.
Each project can have its own dependencies, regardless of what dependencies
every other project has.
There are no limits to the number of environments ,they’re just directories
containing scripts

14

15
What about other languages:
There are equivalents of “virtualenv” like
JAVA use classpath
Ruby use Ruby Version Manager (RVM)

16
Do not forget frameworks and their dependencies
o Very popular in PHP
o They have their own compatibility matrix
Should I always use the latest driver?
o Check the change log
o Check the open bugs

Driver version? Dunno
17
Inspect the logs
Manipulate the logs to export driver name & version
Quick hack:
less /var/log/messages |grep "driver: { name" | awk '{print $16, $17, $18, $19}' | uniq

Driver version? Dunno
18
Inspect the system.profile
Collect a representative sample
On a sharded cluster, all shards must examined.

Inspect the codebase
19
Each version Removes & Deprecates operators.
Remove: Operator is no longer available
Deprecate: Operator will be removed to the next version
Take Actions:
o Patched the codebase against the removed operators
o Plan to replace the deprecated operators

20
Removed Operators:
o $isolated operator: If you have an existing partial index that includes
the $isolated operator or a view that includes a $isolated operator, recreate the index
or view without the operator in the definition before upgrading.
Deprecated Operators:
o maxScan
o geoNear command
o copydb and the clone commands

21
Exception handling
o Error descriptions may change between versions
o Error format may also change
o New error codes may introduced
o New warning codes may introduced

Middleware
• Monitoring
• Backups
• Deployments
• Utilities
• OS changes
• TLS/SSL
22

Monitoring
23
The monitoring system must be able to connect to MongoDB
Vast majority relies on db.serverStatus()
A newer version may be necessary to installed prior to upgrade

Deployment & Managment
24
Automatic deployment scripts also connect to MongoDB
For example:
o Deploy a replica-set (rs.add(), rs.status())
o Add a shard (sh.addShard())
A change to error code formats may affect deployment scripts
User-roles may also change
Management tools may also affected

Backup
25
- Filesystems Snapshots
- Copy files
- Hot Backup (Percona Server)
- Mongodump
They should all work but testing is recommended.
In the case of restore, a downgrade may be required.

OS
26
Removes support for:
o SLES 11
o Ubuntu 12.04
o Debian 7
Deprecates:
o Windows 7/2008R2
o Windows 8/2012
o Windows 8.1/2012R2
o Ubuntu 14.04
Compatibility matrix: https://bit.ly/2ys8Crs

TLS
27
Removes support for TLS 1.0
o On systems where TLS 1.1+ is available
Latest PCI compliance standards require the use of TLS 1.1+
If you need to support TLS 1.0 (hopefully temporarily):
o Set to none either net.ssl.disabledProtocols or --sslDisabledProtocols
Test:
openssl s_client -connect <host>:<port> -tls1 (tls1_1 and tls1_2)

Database Layer
• Configuration files
• Prerequisites
• Deprecated items
• Storage engine
• Miscellaneous
• FCV
28

mongo.conf
29
net.transportLayer cant be set to legacy anymore
net.host: Resolves localhost IP address as configured instead of assuming 127.0.0.1
Can’t combine storage.journal.enabled: false with WiredTiger storage engine
Can’t combine storage.indexBuildRetry with replication.replSetName
When ssl.allowInvalidCertificates: true with x.509 authentication, an invalid
certificate is only sufficient to establish a TLS/SSL connection but is insufficient for
authentication.

Prerequisites
30
In order to upgrade featureCompatibilityVersion must be set to 3.6
How to check:
o db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
How to set FCV to 3.6:
o db.adminCommand( { setFeatureCompatibilityVersion: <version> } )
,where <version> ”3.6”
It automatically enables 3.6 new features

Prerequisites
31
3.6 new features
o UUID for collections
o $jsonSchema document validation
o Change Streams
o Chunk aware secondaries
o View definitions, document validators, and partial index filters that use 3.6 query features
o Sessions and retryable writes
o Users and roles with authenticationRestrictions
Sessions:
o Creates a system.session collection
o On sharded clusters is sharded on {_id:1}
o Has a 30 minute TTL index
o Pre-3.6.7 bug, wasn’t creating TTL and sharding
o May trigger a performance overhead (updates/deletes)

Prerequisites
32
In order to upgrade pv must be set to pv1
How to check:
o pv=rs.conf().protocolVersion
How to set pv1:
o Login to every replica-set and execute
cfg = rs.conf();
cfg.protocolVersion=1;
rs.reconfig(cfg);
Enjoy the benefits of pv1

Prerequisites
33
Master-slave replication, no longer supported
You must upgrade to a replica set Prior the upgrade
Conversion involves downtime & affects high availability
o Stop mongo
o Start Master with --replSet <setname>
o Initialize the replica set
rs.initiate( { _id: "<setname>", members: [ { _id: 0, host: "<host:port>" } ] } )
o Add nodes, rs.add("<host:port>" )

Prerequisites
34
Removes MongoDB Challenge-Response (MONGODB-CR) auth mechanism
Deprecated since 3.0 version. Only possible if you upgraded from 2.6
How to check:
db.getSiblingDB('admin').system.users.find({"credentials.MONGODB-CR":{$exists:true}})
How to upgrade to SCRAM:
db.adminCommand({authSchemaUpgrade: 1});
Considerations
o authMechanism on connection string
o Local users on sharded clusters

Storage engine
35
MongoDB 4.0 deprecates MMAPv1
It’s likely the next version to support only WiredTiger
You should consider a switch to WiredTiger
Change to WiredTiger may be challenging:
o In-place Updates
o Range Queries
o Different HW specs
o Different shard keys

Storage engine
36
storage:
dbPath: <data dir>
engine: mmapv1
mmapv1:
<mmap configuration>
storage:
dbPath: <data dir>
engine: wiredTiger
mmapv1:
<mmap configuration>
wiredTiger:
collectionConfig:
blockCompressor: <value>
engineConfig:
cacheSizeGB: <value>
directoryForIndexes: true
journalCompressor: <value>
indexConfig:
prefixCompression: <value>
Not mandatory
Better control

Storage engine
37
Switch to WT (without secondary reads)
WT
1)Convert one secondary
- Alter the configuration
- Stop mongo, wipe data dir,start mongodb
- Initial sync converts the secondary to WT
2)Promote the secondary to become Primary
- rs.freeze(300) the MMAPv1 secondary
- rs.stepdown(300)
WT
3)Burn Period
- Convert the remaining nodes
- Rollback to MMAPv1
Freeze

Storage engine
38
Switch to WT (secondary reads)
WT
1)Convert one secondary
3)Promote the secondary to become Primary
WT
2)Burn Period (Secondary)
4)Burn Period (Primary)
Important: Make sure less secondary can
serve your workload
WT

Storage engine
39
What can I do in advance?
o Benchmark with a real workload
o Prepare to “upgrade” your hardware
o Divide and conquer databases/collections
o Optimize range scans, if possible
o Replace updates with inserts, if possible
o Change shard keys, if possible
Workloads that MMAPv1 may perform better
o Read-Only
o Updates that change a small fraction of the document (counter)

Miscellaneous
40
o Removes the limit on the amount of data that can be rolled back, was 300MiB
o The rollback time limit defaults to 1 day, was 30 minutes
o Configurable via rollbackTimeLimitSecs, wasn’t configurable
o If you don’t have enough oplog, rollback will fail
o Make sure oplog duration > rollbackTimeLimitSecs
o Disable Rollbacks createRollbackDataFiles
o The oplog can grow past its configured size limit

Miscellaneous
41
taskExecutorPoolSize: Number of Task Executor connection pools
o New default is 1, was 0.
o Value of 0 means
- number of cores < 4, the number of pools is 4.
- 4 <= number of cores =< 64, the number of pools is the number of cores.
- number of cores> 64, the number of pools is 64.
AsyncRequestsSenderUseBaton: Default true
Enables performance optimization on Linux for scatter/gather operations
on mongos when using a single Task Executor connection pool.
Under investigation!

FCV
42
o To enable 4.0 new features you must execute:
db.adminCommand({setFeatureCompatibilityVersion: ”4.0"})
o Its advisable to wait for a brief period of time
o During this period none of the 4.0 features will be available
o Rollback to 3.6 would be more difficult after raise the FCV
- We will examine rollback in the next chapter

FCV – What does it do?
43
Enables:
o SCRAM-SHA-256
o New type conversion operators and enhancements (typical for major version)
o Multi-document transactions
o $dateToString option changes (onNull)
o New change stream methods
o Change stream resume token data type change (hex-encoded string)

Confidence level
44
Major Version Replica-set Sharded Cluster Reason
2.2 2.2.3
2.4 2.4.3 2.4.6 Chunk Migration
2.6 2.6.6 2.6.6 Optimizer
3.0 3.0.5 3.0.8 Data loss
3.2 3.2.6 3.2.11 ASIO bug
3.4 3.4.4 3.4.7 Minor bugs
3.6 3.6.3 3.6.8 sessions

Downgrade
• Downgrade FCV
• Rollback a replica set
• Rollback a sharded cluster
45

Downgrade to 3.6
46
Downgrade FCV to 3.6
o db.adminCommand({setFeatureCompatibilityVersion: "3.6"})
Remove incompatible features:
o Drop views, document validators, and partial index filters that use 4.0 query features
o Downgrade the authentication mechanism from SCRAM-SHA-2 to "SCRAM-SHA-1"
db.updateUser( ”username",
{ mechanisms: [ "SCRAM-SHA-1" ], pwd: <newpwd> } )
o Take a backup
o Verify that all replica set members are in sync
o Use latest 3.6 if possible

Downgrade Replica-Set
47
Downgrade the secondary, one at a time
o Issue rs.stepDown()
o Ensure a new Primary is elected
Upgrade the ex-Primary

Sharded Cluster
s1 s2
Stop Balancer
o sh.stopBalancer()
Downgrade config servers
Downgrade the shards
Downgrade the mongos
o One at a time
o Replace binary and restart
Start Balancer
o sh.startBalancer()

Why 4.0?
• Transactions
• Secure authentication
• Change streams
• Non blocking Secondary reads
• Performance
• Miscellaneous improvements
49

Transactions
50
MongoDB 4.0 supports multi-document transactions
o Only available on replica sets
o Requires FCV = “4.0”
o Requires 4.0 compatible drivers
o Requires WiredTiger storage engine
o Transactions buffer lives in cache
o 16MB document size limit due to the oplog

Transactions
51

SCRAM-SHA-256
52
o SHA1 has been deprecated due to its security vulnerabilities
o SHA2 is the successor of SHA1 and SHA-256 its one of its variants
o SHA-256 produces a 256 bits hash (vs 160 on SHA1)
o SHA2 is less vulnerable to collision attacks
o SHA2 is vulnerable to collision attacks length extension attack

Change Streams
53
o Open a change stream cursor for a single database*
o Open a change stream cursor for a deployment**
o Adds the startAtOperationTime option
o Resume token is now a hex-encoded string. Allows comparison and sort.
*excluding admin, local, and config database
** In 3.6 the type is BinData

Secondary Reads
54
Before 4.0
o Writes are applied as batches.
o Block reads during batches to avoid a "wrong" order.
o Periodically the readers have to wait for replication batches to be applied
o Batches needs a lock that requires all reads to complete before it can be taken
o Write heavy workloads increase latency
In 4.0
o Timestamps in the storage engine
o Using transactions to get a consistent view of data at a specific "cluster time".
o Secondary reads takes advantage of snapshots isolation
o Relax the replication lock/allows reads while writes are happening.
o Index format changed to avoid collisions (unique indexes)

WiredTiger
55
o WiredTiger 3.1.1: (July 12, 2018) vs WiredTiger 2.9.2: (December 23, 2016)
o Improved Eviction (WT-3683, WT-3437, WT-4141)
o Improved Checkpoints (WT-4111)
o Reduces the number of internal writes by 50%
o WiredTiger timestamps (uses MongoDB timestamps)

Miscellaneous
56
$near and $nearSphere supports querying on sharded collections
Adds key option for the $geoNear aggregation operator and geoNear command
Mongos can log slow statements:
Command line options: slowms
Configuration file options: operationProfiling.slowOpThresholdMs
Mongos can rate limit slow statements:
Command line options: --slowOpSampleRate
Configuration file options: operationProfiling.slowOpSampleRate
Mongo shell method supports convertShardKeyToHashed

Miscellaneous
57
Aggregation framework supports $convert
Converts a value to specified type:

Questions?
58

Feedback
59
antonios.giannopoulos@rackspace.co.uk
@iamantonios
linkedin.com/in/antonis/
Please share:
Your feedback regarding this presentation
What are you most interest to hear from us in the future

60
We’re Hiring!
Looking to join a dynamic & innovative
team?
https://www.objectrocket.com/careers/
or email careers@objectrocket.com

Thank you!
Address:
401 Congress Ave Suite 1950
Austin, TX 78701
Support:
1-800-961-4454
Sales:
1-888-440-3242
61

How to upgrade to MongoDB 4.0 - Percona Europe 2018

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à How to upgrade to MongoDB 4.0 - Percona Europe 2018

Similaire à How to upgrade to MongoDB 4.0 - Percona Europe 2018 (20)

Plus de Antonios Giannopoulos

Plus de Antonios Giannopoulos (10)

Dernier

Dernier (20)

How to upgrade to MongoDB 4.0 - Percona Europe 2018