MongoDB Node.js Driver 6.11.0 Released

neal · 2024-11-22T16:32:26.932Z

MongoDB Node.js Driver 6.11.0 Released (2024-11-22)

The MongoDB Node.js team is pleased to announce version 6.11.0 of the mongodb package!

Release Notes

Client Side Operations Timeout (CSOT)

We’ve been working hard to try to simplify how setting timeouts works in the driver and are excited to finally put Client Side Operation Timeouts (CSOT) in your hands! We’re looking forward to hearing your feedback on this new feature during its trial period in the driver, so feel free to file Improvements, Questions or Bug reports on our Jira Project or leave comments on this community forum post!

CSOT is the common drivers solution for timing out the execution of an operation at the different stages of an operation’s lifetime. At its simplest, CSOT allows you to specify one option,timeoutMS that determines when the driver will interrupt an operation and return a timeout error.

For example, when executing a potentially long-running query, you would specify timeoutMS as follows:

await collection.find({}, {timeoutMS: 600_000}).toArray(); // Ensures that the find will throw a timeout error if all documents are not retrieved within 10 minutes
// Potential Stack trace if this were to time out:
// Uncaught MongoOperationTimeoutError: Timed out during socket read (600000ms)
//    at Connection.readMany (mongodb/lib/cmap/connection.js:427:31)
//    at async Connection.sendWire (mongodb/lib/cmap/connection.js:246:30)
//    at async Connection.sendCommand (mongodb/lib/cmap/connection.js:281:24)
//    at async Connection.command (mongodb/lib/cmap/connection.js:323:26)
//    at async Server.command (mongodb/lib/sdam/server.js:170:29)
//    at async GetMoreOperation.execute (mongodb/lib/operations/get_more.js:58:16)
//    at async tryOperation (mongodb/lib/operations/execute_operation.js:203:20)
//    at async executeOperation (mongodb/lib/operations/execute_operation.js:73:16)
//    at async FindCursor.getMore (mongodb/lib/cursor/abstract_cursor.js:590:16)

Warning:
This feature is experimental and subject to change at any time. We do not recommend using this feature in production applications until it is stable.

What’s new?

`timeoutMS`

The main new option introduced with CSOT is the timeoutMS option. This option can be applied directly as a client option, as well as at the database, collection, session, transaction and operation layers, following the same inheritance behaviours as other driver options.

When the timeoutMS option is specified, it will always take precedence over the following options:

socketTimeoutMS
waitQueueTimeoutMS
wTimeoutMS
maxTimeMS
maxCommitTimeMS

Note, however that timeoutMS DOES NOT unconditionally override the serverSelectionTimeoutMS option.

When timeoutMS is specified, the duration of time allotted to the server selection and connection checkout portions of command execution is defined by min(serverSelectionTimeoutMS, timeoutMS) if both are >0. A zero value for either timeout value represents an infinite timeout. A finite timeout will always be used unless both timeouts are specified as 0. Note also that the driver has a default value for serverSelectionTimeoutMS of 30000.

After server selection and connection checkout are complete, the time remaining bounds the execution of the remainder of the operation.

Note:
Specifying timeoutMS is not a hard guarantee that an operation will take exactly the duration specified. In the circumstances identified below, the driver’s internal cleanup logic can result in an operation exceeding the duration specified by timeoutMS.

AbstractCursor.toArray() - can take up to 2 * timeoutMS in 'cursorLifetimeMode' and (n+1) * timeoutMS when returning n batches in 'iteration' mode
AbstractCursor.[Symbol.asyncIterator]() - can take up to 2 * timeoutMS in 'cursorLifetimeMode' and (n+1)*timeoutMS when returning n batches in 'iteration' mode
MongoClient.bulkWrite() - can take up to 2 * timeoutMS in error scenarios when the driver must clean up cursors used internally.
CSFLE/QE - can take up to 2 * timeoutMS in rare error scenarios when the driver must clean up cursors used internally when fetching keys from the keyvault or listing collections.

In the AbstractCursor.toArray case and the AbstractCursor.[Symbol.asyncIterator] case, this occurs as these methods close the cursor when they finish returning their documents. As detailed in the following section, this results in a refreshing of the timeout before sending the killCursors command to close the cursor on the server.
The MongoClient.bulkWrite and autoencryption implementations use cursors under the hood and so inherit this issue.

Cursors, `timeoutMS` and `timeoutMode`

Cursors require special handling with the new timout paradigm introduced here. Cursors can be configured to interact with CSOT in two ways.
The first, 'cursorLifetime' mode, uses the timeoutMS to bound the entire lifetime of a cursor and is the default timeout mode for non-tailable cursors (find, aggregate*, listCollections, etc.). This means that the initialization of the cursor and all subsequent getMore calls MUST finish within timeoutMS or a timeout error will be thrown. Note, however that the closing of a cursor, either as part of a toArray() call or manually via the close() method resets the timeout before sending a killCursors operation to the server.

e.g.

// This will ensure that the initialization of the cursor and retrieval of all docments will occur within 1000ms, throwing an error if it exceeds this time limit
const docs = await collection.find({}, {timeoutMS: 1000}).toArray();

The second, 'iteration' mode, uses timeoutMS to bound each next/hasNext/tryNext call, refreshing the timeout after each call completes. This is the default mode for all tailable cursors (tailable find cursors on capped collections, change streams, etc.). e.g.

// Each turn of the async iterator will take up to 1000ms before it throws
for await (const doc of cappedCollection.find({}, {tailable: true, timeoutMS: 1000})) {
    // process document
}

Note that timeoutMode is also configurable on a per-cursor basis.

GridFS and `timeoutMS`

GridFS streams interact with timeoutMS in a similar manner to cursors in 'cursorLifeTime' mode in that timeoutMS bounds the entire lifetime of the stream.
In addition, GridFSBucket.find, GridFSBucket.rename and GridFSBucket.drop all support the timeoutMS option and behave in the same way as other operations.

Sessions, Transactions, `timeoutMS` and `defaultTimeoutMS`

ClientSessions have a new option: defaultTimeoutMS, which specifies the timeoutMS value to use for:

commitTransaction
abortTransaction
withTransaction
endSession

Note:
If defaultTimeoutMS is not specified, then it will inherit the timeoutMS of the parent MongoClient.

When using ClientSession.withTransaction, the timeoutMS can be configured either in the options on the withTransaction call or inherited from the session’s defaultTimeoutMS. This timeoutMS will apply to the entirety of the withTransaction callback provided that the session is correctly passed into each database operation. If the session is not passed into the operation, it will not respect the configured timeout. Also be aware that trying to override the timeoutMS at the operation level for operations making use of the explicit session inside the withTransaction callback will result in an error being thrown.

const session = client.startSession({defaultTimeoutMS: 1000});
const coll = client.db('db').collection('coll');
// ❌ Incorrect; will throw an error
await session.withTransaction(async function(session) {
	await coll.insertOne({x:1}, { session, timeoutMS: 600 });
})

// ❌ Incorrect; will not respect timeoutMS configured on session
await session.withTransaction(async function(session) {
	await coll.insertOne({x:1}, {});
})

ClientEncryption and `timeoutMS`

The ClientEncryption class now supports the timeoutMS option. If timeoutMS is provided when constructing a ClientEncryption instance, it will be used to govern the lifetime of all operations performed on instance, otherwise, it will inherit from the timeoutMS set on the MongoClient provided to the ClientEncryption constructor.
If timeoutMS is set on both the client and provided to ClientEncryption directly, the option provided to ClientEncryption takes precedence.

const encryption = new ClientEncryption(new MongoClient('localhost:27027'), { timeoutMS: 1_000 });
await encryption.createDataKey('local'); // will not take longer than 1_000ms

const encryption = new ClientEncryption(new MongoClient('localhost:27027', { timeoutMS: 1_000 }));
await encryption.createDataKey('local'); // will not take longer than 1_000ms

const encryption = new ClientEncryption(new MongoClient('localhost:27027', { timeoutMS: 5_000 }), { timeoutMS: 1_000 });
await encryption.createDataKey('local'); // will not take longer than 1_000ms

Limitations

At the time of writing, when using the driver’s autoconnect feature alongside CSOT, the time taken for the command doing the autonnection will not be bound by the configured timeoutMS. We made this design choice because the client’s connection logic handles a number of potentially long-running I/O and other setup operations including reading certificate files, DNS lookups, instantiating server monitors, and launching external processes for client encryption.
We recommend manually connecting the MongoClient if intending to make use of CSOT, or otherwise ensuring that the driver is already connected when running commands that make use of timeoutMS.

const client = new MongoClient(uri, { timeoutMS: 1000 });
// ❌ No guarantee to finish in specified time
await client.db('db').collection('coll').insertOne({x:1});

// ✔️ Will have expected behaviour
await client.connect();
await client.db('db').collection('coll').insertOne({x:1});

Explain helpers support `timeoutMS`

Explain helpers support timeoutMS:

await collection.deleteMany({}, { timeoutMS: 1_000, explain: true });
await collection.find().explain(
  { verbosity: 'queryPlanner' },
  { timeoutMS: 1_000 }
)

Note:
Providing a maxTimeMS value with a timeoutMS value will throw errors.

MONGODB-OIDC Authentication now supports Kubernetes Environments.

For k8s environments running in Amazon’s EKS (Elastic Kubernetes Service), Google’s GKE (Google Kubernetes Engine), or Azure’s AKS (Azure Kubernetes Service) simply provide an ENVIRONMENT auth mechanism property in the URI or MongoClient options of “k8s”.

Example:

const client = new MongoClient('mongodb://host:port/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:k8s');

ConnectionClosedEvents always follow PoolClearedEvents

When Connection Monitoring and Pooling events are listened for, ConnectionClosedEvents are now always emitted after PoolClearEvents.

Features

NODE-5682: set maxTimeMS on commands and preempt I/O (#4174) (e4e6a5e)
NODE-5844: add iscryptd to ServerDescription (#4239) (c39d443)
NODE-6069: OIDC k8s machine workflow (#4270) (82c931c)
NODE-6090: Implement CSOT logic for connection checkout and server selection (bd8a9f4)
NODE-6231: Add CSOT behaviour for retryable reads and writes (#4186) (2ffd5eb)
NODE-6274: add CSOT support to bulkWrite (#4250) (c5a9ae5)
NODE-6275: Add CSOT support to GridFS (#4246) (3cb8187)
NODE-6304: add CSOT support for non-tailable cursors (#4195) (131f6ed)
NODE-6305: Add CSOT support to tailable cursors (#4218) (2398fc6)
NODE-6312: add error transformation for server timeouts (#4192) (c2c0cb9)
NODE-6313: add CSOT support to sessions and transactions (#4199) (5f1102f)
NODE-6387: Add CSOT support to change streams (#4256) (4588ff2)
NODE-6389: add support for timeoutMS in StateMachine.execute() (#4243) (c55f965)
NODE-6390: Add timeoutMS support to auto encryption (#4265) (55e08e7)
NODE-6391: Add timeoutMS support to explicit encryption (#4269) (f745b99)
NODE-6392: add timeoutMS support to ClientEncryption helpers part 1 (#4281) (e86f11e)
NODE-6403: add CSOT support to client bulk write (#4261) (365d63b)
NODE-6421: add support for timeoutMS to explain helpers (#4268) (5b2629b)
NODE-6446: deprecate legacy timeout options (#4279) (c28608b)
NODE-6551: update bson to 6.10.0 (#4329) (adb15fe)

Bug Fixes

NODE-6374: MongoOperationTimeoutError inherits MongoRuntimeError (#4237) (9fb896a)
NODE-6412: read stale response from previously timed out connection (#4273) (fd8f3bd)
NODE-6454: use timeoutcontext for state machine execute() cursor options (#4291) (5dd8ee5)
NODE-6469: pool is cleared before connection checkin on error (#4296) (06a2e2c)
NODE-6523: deleteMany in gridfs passes timeoutMS to predicate, not options (#4319) (1965ed5)

Performance Improvements

NODE-6525: remove setPrototype and defineProperty from hot path (#4321) (48ed47e)

Documentation

Reference
API
Changelog

We invite you to try the mongodb library immediately, and report any issues to the NODE project.

system · 2025-02-20T16:32:47.072Z

This topic was automatically closed after 90 days. New replies are no longer allowed.