Problem
In my Node.JS local development environment, using Compass, connected to a CSFLE-enabled MongoDB Enterprise client, I can see all the fields that are supposed to be encrypted by my JSON schema as clear text.
Background
- I am using MongoDB Enterprise for development purposes and have Atlas Cloud for my production deployment.
- I’m planning to use AWS KMS for my production key and a I’ve attempted both a local key and AWS IAM user credentials to implement automatic Client-Side Field-Level Encryption locally.
- I’ve checked that
mongocryptdis in fact running. - During startup, my server connects to multiple instances of MongoDB that correspond to different micro-services I maintain. This server is where I’ve set up my encryption credentials.
- My microservices receive the DEK and create schema maps based on their specific requirements.
- I don’t see any errors in the MongoDB logs access via
mongosh - I’ve dropped my database and started fresh
- I’m using
mongoose
Package Versions
MongoDB
MongoDB Enterprise 7.0.2
Node Packages
"mongodb": "^6.2.0",
"mongodb-client-encryption": "^6.0.0",
"mongoose": "^8.0.0",
How I’m Initializing CSFLE
- At the top level of my server, I’m calling an
initializemethod in myEncryptionclass that takes in options and returns an object that contains common configuration parameters, including the DEK, that my micro-services can use. This method creates an encryption client to create the encryption database, key vault collection and produce the DEK if it does not already exist, then it closes that connection. - I pass the returned config params from the previous step into the microservice init methods, which create the database models (that do not have any server-side encryption schemas defined), and create regular MongoDB client connections to each service using the
autoEncryptionconfig param.
My Code
NOTE: Much stuff has been removed to help clarify the code.
CSFLE Helper Class
export default class Encryption implements IEncryption {
// ... There are several private and public variables not shown here
// private constructor to enforce calling `initialize` method below, which calls this constructor internally
private constructor(opts?: EncryptionConfigConstructorOpts) {
this.tenantId = opts?.tenantId;
this.keyVaultDbName = opts?.keyVaultDbName;
this.keyVaultCollectionName = opts?.keyVaultCollectionName;
this.DEKAlias = opts?.DEKAlias;
// Detect a local development environment
if (process.env?.ENVIRONMENT === LOCAL_DEV_ENV) {
const keyBase64 = process.env?.LOCAL_MASTER_KEY;
const key = Buffer.from(keyBase64, 'base64');
// For testing, I'm manually switching between a local key and remote KMS
// I'll leave out the production-detection code
if (_debug) {
this.provider = KMS_PROVIDER;
this.kmsProviders = {
aws: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
},
};
this.masterKey = {
key: process.env.KMS_MASTER_ARN,
region: opts?.masterRegion,
};
} else {
this.kmsProviders = {
local: {
key,
},
};
}
}
const keyVaultNamespace = `${this.keyVaultDbName}.${this.keyVaultCollectionName}`;
const encryptionOptions: ClientEncryptionOptions = {
keyVaultNamespace,
kmsProviders: this.kmsProviders,
};
this.encryptionOptions = encryptionOptions;
}
public static async initialize(
url: string,
opts?: EncryptionConfigConstructorOpts
): Promise<Encryption> {
// Set internal attributes
const encryption = new Encryption(opts);
// Create key vault collection (this is idempotent, afaik)
const client = new MongoClient(url);
const keyVaultDB = client.db(encryption.keyVaultDbName);
const keyVaultColl = keyVaultDB.collection(encryption.keyVaultCollectionName);
await keyVaultColl.createIndex(
{ keyAltNames: 1 },
{
unique: true,
partialFilterExpression: { keyAltNames: { $exists: true } },
}
);
let dek: UUID | undefined = undefined;
// This checks for an existing DEK, then creates/assigns or just assigns when necessary
try {
// Initialize client encryption
const clientEncryption = new ClientEncryption(client, encryption.encryptionOptions!);
const keyOptions = {
masterKey: encryption.masterKey,
keyAltNames: [encryption.DEKAlias],
};
dek = await clientEncryption.createDataKey(encryption.provider, keyOptions);
} catch (err: any) {
// Duplicate key error is expected if the key already exists, so we fetch the key if that happens
if (String(err?.code) !== '11000') {
throw err;
} else {
// Check if a DEK with the keyAltName in the env var DEK_ALIAS already exists
const existingKey = await client
.db(encryption.keyVaultDbName)
.collection(encryption.keyVaultCollectionName)
.findOne({ keyAltNames: encryption.DEKAlias });
if (existingKey?._id) {
dek = UUID.createFromHexString(existingKey._id.toHexString());
} else {
throw new Error('DEK could not be found or created');
}
}
} finally {
await client.close();
}
encryption.dek = dek;
encryption.isReady = !!encryption.dek;
return encryption;
}
// Defined as an arrow function to preserve the `this` context, since it is called as a callback elsewhere
// This gets called after the `initialize` method from within each micro-service
public getSchemaMap = (
jsonSchema: Record<string, unknown>,
encryptionMetadata?: Record<string, unknown>
): Record<string, unknown> => {
if (!this?.isReady) {
throw new Error('Encryption class cannot get schema map until it is initialized');
}
const schemaMapWithEncryption = {
encryptMetadata: {
keyId: [this.dek],
algorithm: process.env.ALG_DETERMINISTIC,
...encryptionMetadata,
},
...jsonSchema,
};
return schemaMapWithEncryption;
};
}
Startup code
// ... Start up code
const encryption = await Encryption.initialize(process.env.DB_CONN_STRING);
const opts = {
autoEncryption: {
...encryption.encryptionOptions
},
};
await Service1Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);
await Service2Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);
// ... More start up code and API route config
Typical Service’s initialize method
// ... Init code and then assigning the schema-generated model (which does not contain any encryption syntax)
Service1Model.service1DataModel = model<IService1Document>('Service1Doc', Service1Schema, 'Service1Docs');
// Finally, connecting to the DB with a schema map generated for this service, specifically
mongoose.connect(url, {
...opts,
autoEncryption: opts?.autoEncryption
? {
...opts?.autoEncryption,
schemaMap: getSchemaMap(importedSchemaJson),
}
: undefined,
} as ConnectOptions);
My encryption JSON schema
{
"MyCollection1": {
"properties": {
"myDataString": {
"encrypt": {
"bsonType": "string"
}
},
"myDataArray": {
"encrypt": {
"bsonType": "array"
}
},
"myDataObject": {
"bsonType": "object",
"properties": {
"myNestedProperty1": {
"encrypt": {
"bsonType": "string"
}
},
"myNestedProperty2": {
"bsonType": "string"
}
}
}
}
}
}
