Introducing…

db.coll.insert({
_id: 1,
name: "Doris",
ssn: "457-55-5462"
})

doc = db.coll.find_one({
ssn: "457-55-5462"
})

print (doc)

{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}

db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
{
insert: "coll",
documents: [{
name: "Doris",
ssn: BinData(6, "a10x…")
}]
}
You see: MongoDB sees:
Encrypt before sending

{
_id: 1
name: "Doris",
ssn: BinData(6, "a10x…")
}
Driver receives: You see:
{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}
Decrypt after receiving

How does this differ from…?
•… encryption in-transit (TLS)
•… encryption at-rest (encrypted storage engine)

Attacker
query
App
(Client)
Disk
insert write
MongoDB
Auth
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })

Disk
insert write
MongoDB
Attacker
snoop
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

Disk
insert write
MongoDB
Attacker
insert
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

Disk
insert write
MongoDB
Attacker
steal
ESE
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

Disk
insert write
MongoDB
Attacker
login
Client Side Encryption
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
App
(Client)

Disk
insert write
MongoDB
Boundaries of unencrypted data
App
(Client)

Disk
insert write
MongoDB
… with Encrypted Storage Engine
App
(Client)

Disk
insert write
MongoDB
… and TLS
App
(Client)

Disk
insert write
MongoDB
with Client Side Encryption
App
(Client)

Disk
insert write
MongoDB
ssn: BinData(6, "a10x…")
App
(Client)

db.coll.update({}, {
$set: { ssn: "457-55-5462" }
})
{
update: "coll",
updates: [{
q:{},
u: {
$set: { ssn: BinData(6, "a10x…") }
}
}]
}
You see: MongoDB sees:
Update that overwrites value

db.coll.aggregate([{
$project: { name_ssn: {$concat: [ "$name", " - ", "$ssn" ] } }
}]
Aggregate acting on the data

Find with equality query
* For deterministic encryption
db.coll.find({ssn: "457-55-5462" }) {
find: "coll",
filter: { ssn: BinData(6, "a10x…") }
}
You see: MongoDB sees:

Find with equality query
* For deterministic encryption
db.test.find(
{
$and: [
{
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
)
You see:

Find with equality query
* For deterministic encryption
MongoDB sees:
{
find: "coll",
filter: {
$and: [
{
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
}

MongoDB
Attacker
Login

Destroy the key
Provably delete all user data.
GDPR "right-to-be-forgotten"

Doris
Private stuff in storage

PoliceDoris
Private stuff in storage

Vault key
Held only by you
Vault

Encrypted Data
MongoDB
Encryption Key

{ _id: 1, ssn: BinData(0, "A81…"), name: "Kevin" }
{ _id: 2, ssn: BinData(0, "017…"), name: "Eric" }
{ _id: 3, ssn: BinData(0, "5E1…"), name: "Albert" }
…

client = MongoClient(
auto_encryption_opts=opts)

Not sensitive
{

One key for all vaults

One key per vault

{
name: "Doris"
ssn: "457-55-5462",
email: "Doris@gmail.com",
credit_card: "4690-6950-9373-8791",
comments: [ …. ],
avatar: BinData(0, "0fi8…"),
profile: { likes: {…}, dislikes: {…} }
}

Describes JSON
{
bsonType: "object",
properties: {
a: {
bsonType: "int"
maximum: 10
}
b: { bsonType: "string" }
},
required: ["a", "b"]
}
{
a: 5,
b: "hi"
}
{
a: 11,
b: false
}
JSON Schema

{
bsonType: "object",
properties: {
ssn: {
encrypt: { … }
}
},
required: ["ssn"]
}
JSON Schema "encrypt"

encrypt: {
keyId: (…),
algorithm: (…),
bsonType: (…)
}
bsonType indicates the type of underlying data.
algorithm indicates how to encrypt (Random or Deterministic).
keyId indicates the key used to encrypt.

opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> }
…)

Remote Schema Fallback
db.createCollection("coll", { validator: { $jsonSchema: … } } )
Misconfigured
Client insert "457-55-5462"
error, that should be
encrypted
MongoDB

What if
… the server lies about the schema?
Misconfigured
Client insert "457-55-5462"
Evil MongoDB
ok :)

schema_map
Sub-options

Key vault
Key vault key
Held only by you

Stores encrypted keys

opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault"
…)

schema_map
Sub-options
key_vault_namespace

What if
… attacker drops key vault collection?

Keep at home
opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault",
key_vault_client = <client>
…)

schema_map
Sub-options
key_vault_namespace
key_vault_client

(Key Management Service)

Protects keys Stores keys
KMS
Key vault key
Key vault
Key vault
collection

Decryption requires

opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keys",
kms_providers = <creds>
…)

schema_map
Sub-options
key_vault_namespace
key_vault_client
kms_providers

db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Get encrypted key
Decrypt the key with KMSDecrypt the key with KMS
Encrypt 457-55-5462
Send insert
Compare to JSON schema

You need…
•MongoDB 4.2 server
•Beta client (shell, Java, Python, NodeJS, Go, C#)
•Enterprise license for auto encryption
•Community for explicit encryption

*

Authenticated Encryption with Associated Data using the Advance

AEAD_AES_256_CBC_HMAC_SHA_512
Provides confidentiality + integrity

AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic
AEAD_AES_256_CBC_HMAC_SHA_512-Random

coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "f991…") }
…Random

coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
…Deterministic

Can be queried
doc = db.coll.find({
ssn: "457-55-5642"
})
…Deterministic

Only for binary comparable types.
db.coll.find({ a: NumberDecimal("1.2")})
{ a: NumberDecimal("1.2") }
{ a: NumberDecimal("1.20") }
MongoDB returns:
…Deterministic

{ a: BinData(6, "b515…") }
{ a: BinData(6, "801f…") }
Encrypted as:
{ a: NumberDecimal("1.2") }
{ a: NumberDecimal("1.20") }
Value:

db.coll.find({ a: NumberDecimal("1.2") })
{ a: NumberDecimal("1.2") }
MongoDB returns:
"a" encrypted

Encrypt-able values
Deterministic encryption valid for…
•String
•Binary
•ObjectID
•Date
•Regex
•DBPointer
•Javascript
•Symbol
•Int
•Timestamp
•Long
Random encryption valid for…
•(all of deterministic)
•Document
•Array
•JavascriptWithScope
•Double
•Decimal128
•Bool

{ ssn: BinData(6, "AWNkTYTCw89Ss1DPzV3/2pSRDNGNJ9NB" }
New binary subtype
Older drivers and older MongoDB will treat as a black box.

byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Ciphertext

byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
key_id + algorithm describes how to decrypt.
No JSON Schema necessary!
Ciphertext

byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Provides extra server-side validation.
But prohibits single-value types (MinKey, MaxKey, Undefined, Null)
Ciphertext

byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Payload includes encoded IV and padding block, and HMAC.
Ciphertext adds between 66 to 82 bytes of overhead.
Ciphertext

ce = ClientEncryption(opts)
encrypted = ce.encrypt("457-55-5462", opts))
decrypted = ce.decrypt(BinData(6, "a10x…")))
id = ce.createDataKey(opts)

db.test.find({
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ name: "Doris" }
]
})
db.test.find({
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ name: "Doris" }
]
})

Limitations
If we cannot parse…
or it is impossible…
we err for safety.

{ passport_num: "281-301-348", ssn: "457-55-5462" }
{ passport_num: "390-491-482", ssn: "482-38-5899" }
{ passport_num: "104-201-596" }
passport_num and ssn encrypted with different keys

db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: "457-55-5462" } }
How do we encrypt 457-55-5462?

opts = AutoEncryptionOptions(
bypass_auto_encryption = True
…)
client = MongoClient(auto_encryption_opts=opts)
(Decryption still occurs)

ce = ClientEncryption(opts)
db.test.aggregate([
])
{ $project: { identifier: { $ifNull: ["$ssn", "$passport_num" ] } } },
{ $match: { identifier: ce.encrypt("457-55-5462", opts) } }

One key
ce = ClientEncryption(opts)
id = ce.createDataKey(keyopts)
Create with ClientEncryption

One key
Specify with JSONSchema
…
ssn: {
encrypt: {
keyId: [id],
algorithm: (…),
bsonType: (…)
}
}

One key
db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Query key vault by _id
Compare to JSON schema

Labeled keys
ce = ClientEncryption(opts)
id = ce.createDataKey(keyopts, keyAltNames=["Doris"])
Create with ClientEncryption

Labeled keys
Specify JSON Pointer with JSONSchema
…
ssn: {
encrypt: {
keyId: "/name",
algorithm: (…),
bsonType: (…)
}
}

db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Query key vault by label "Doris"
Compare to JSON schema
Labeled keys

db.coll.insert({
name: "Kevin",
ssn: "457-55-5462"
})
Query key vault by label "Kevin"
Compare to JSON schema
Labeled keys

DorisDB
MongoDB Cloud Hosting - By Doris ™

App Server
User
insert
MongoDB
(Key Vault)
fetch key
decrypt key DorisDB
(Storage)
encrypted insert

{
email: (encrypted),
pwd: (encrypted)
}
Email deterministic, pwd random, uses collection key

{
user_id: "…",
title: (encrypted),
body: (encrypted)
}
Encrypted randomly with per-user key

User
MongoDB
(Key Vault)
delete user key
DorisDB
(Storage)
delete all posts
App Server
"Delete my data"
… but was it really deleted?

Users :) Latency :(

EAST
DICTATORLAND
Global Shards

EAST
DICTATORLAND

EAST
DICTATORLAND
{ _id: 1, body: BinData(6, "A81…") }
{ _id: 2, body: BinData(6, "017…") }
{ _id: 3, body: BinData(6, "5E1…") }
…