MongoDB Node.js Driver 4.3.0 Released

dariakp · 2022-01-07T00:47:46.523Z

The MongoDB Node.js team is pleased to announce version 4.3.0 of the mongodb package!

Release Highlights

This release includes SOCKS5 support and a couple of other important features and bug fixes that we hope will improve your experience with the node driver.

The SOCKS5 options can be configured via the proxyHost, proxyPort, proxyPassword and proxyUsername options in the connection string passed to the MongoClient instance. Big thanks to @addaleax for helping with this feature!

The other notable features address performance and TypeScript as detailed below.

Performance

The original release of the 4.x driver relied on a new version of the BSON library that enables UTF-8 validation by default, resulting in noticeable performance degradation over the 3.x driver when processing over string data. This release introduces an option to opt out of this validation by specifying enableUtf8Validation: false at the client, database, collection, or individual operation level.

For example:

// disable UTF-8 validation globally on the MongoDB client
const client = new MongoClient('mongodb://localhost:27017', { enableUtf8Validation: false });

// disable UTF-8 validation for a particular operation
const client = new MongoClient('mongodb://localhost:27017');
const db = client.db('database name');
const collection = db.collection('collection name');

await collection.find({ name: 'John Doe'}, { enableUtf8Validation: false });

TypeScript

Type inference for nested documents

Thanks to an amazing contribution from @avaly we now have support for key auto-completion and type hinting on nested documents! MongoDB permits using dotted keys to reference nested keys or specific array indexes within your documents as a shorthand for getting at keys beneath the top layer. Typescript’s Template Literal types allow us to take the interface defined on a collection and calculate at compile time the nested keys and indexes available.

For example:

interface Human {
  name: string;
  age: number;
}

interface Pet {
  name: string
  bestFriend: Human
}


const pets = client.db().collection<Pet>('pets');
await pets.findOne({ 'bestFriend.age': 'young!' }) // typescript error!

WARNING: There is a known shortcoming to this feature: recursive types can no longer be used in your schema. For example, an interface that references itself or references another schema that references back to the root schema cannot be used on our Collection generic argument. Unlike at runtime where a “recursive” shaped document has an eventual stopping point we don’t have the tools within the language to declare a base case enumerating nested keys. We hope this does not cause friction when upgrading driver versions: please do not hesitate to reach out with any feedback you have about this feature.

Consistent type inference for the _id type

We have also enhanced the type inference for the _id type. Now, when performing operations on a collection, the following holds true based on the type of the schema:

If no _id is specified on the schema, it is inferred to be of type ObjectId and is optional on inserts.
If an _id is specified on the schema as required, then the _id type is inferred to be of the specified type and is required on inserts.
If an _id is specified on the schema as optional, it is inferred to be of the specified type and is optional on inserts: this format is intended to be used with the pkFactory option in order to ensure a consistent _id is assigned to every new document.

Features

NODE-3589: support dot-notation attributes in Filter (#2972) (76fff97)
NODE-3633: add Socks5 support (#3041) (451627a)
NODE-3784: Add enableUtf8Validation option (#3074) (4f56409)

Bug Fixes

gridfs: make GridFSBucketWriteStream.prototype.end() return this for compat with @types/node@17.0.6 (#3088) (7bb9e37)
NODE-2899: sort and correct circular imports (#3072) (48cc729)
NODE-3675: SRV option bug correctly defaults authSource to $external (#3079) (30f2a2d)
NODE-3803: Fix _id typing on collection create operations (#3077) (f1979db)

Documentation

Reference: https://docs.mongodb.com/drivers/node
API: mongodb
Changelog: node-mongodb-native/HISTORY.md at main · mongodb/node-mongodb-native · GitHub

We invite you to try the mongodb library immediately, and report any issues to the NODE project.

Theodor_Diaconu · 2022-01-07T07:41:22.478Z

It was extremely convenient to have a model which referenced other models or referenced itself, as these were later decorated by an ORM.

Regarding TS, can this workaround not be applied and benefit of best of both worlds? Version 3.4-dev breaks recursive types · Issue #30188 · microsoft/TypeScript · GitHub

dariakp · 2022-01-07T20:24:00.092Z

Hi there, thank you for reaching out with your feedback - we are going to look into this further and see if we can get some sort of a patch out that allows recursive types to still compile. You can follow our progress here: https://jira.mongodb.org/browse/NODE-3852

Theodor_Diaconu · 2022-01-11T07:23:51.900Z

I think I found a working example where we can specify the depth of this. https://www.examplefiles.net/cs/25063 this explains the concept and will take you to this playground link

In theory we can have tree-like structures as long as we set a limit. Now… the problem? You guessed it, it generates a shitload of combinations and how do we define this limit?

I’ll keep this in background and once I compute a solution will leave comm here.

Bailey_Pearson · 2022-01-12T20:51:47.469Z

Hey @Theodor_Diaconu! We’ve been working on a potential fix to our breaking recursive types in 4.3.0. The changes are included in this PR. Feel free to take a look if you’d like! There are two other small TS fixes included in that PR as well but feel free to ignore those changes.

Our solution isn’t perfect - it doesn’t provide true type safety on recursive types. Instead, it just allows recursive types to be used with the node driver (with some limitations). Three limitations to our solution are:

Recursive types that contain a union on the recursive parameter
Recursive types where the recursion isn’t a direct recursion (the recursive type needs to be a direct child property on the parent type)
type checking only occurs up to the first recursive call. after that, the compiler allows anything but doesn’t provide type safety

This seems to be a good balance between allowing some of the behavior we broke in 4.3 back while keeping our types maintainable for the future though.

Let us know if you have any thoughts or feedback!

Theodor_Diaconu · 2022-01-15T13:05:51.108Z

I believe you found the sweet spot for this, I’ve looked through the code and indeed it looks like the perfect compromise. Thank you. Looking forward to 4.3.1 release to see if we have any breaking changes in our code-bases.

Theodor_Diaconu · 2022-02-09T11:15:18.131Z

@Bailey_Pearson sorry to bother you again with this, but this is the errors I get:

    await comments.updateOne(
      { _id: c1.insertedId },
      {
        $set: {
          title: "Lifecycle Updated",
        },
      }
    );

Type of property 'author' circularly references itself in mapped type '{ [Key in keyof Post]: Post[Key] extends Post ? [Key] : Post extends Post[Key] ? [Key] : Post[Key] extends readonly (infer ArrayType)[] ? Post extends ArrayType ? [...] : ArrayType extends Post ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'comments' circularly references itself in mapped type '{ [Key in keyof Post]: Post[Key] extends Post ? [Key] : Post extends Post[Key] ? [Key] : Post[Key] extends readonly (infer ArrayType)[] ? Post extends ArrayType ? [...] : ArrayType extends Post ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'comments' circularly references itself in mapped type '{ [Key in keyof User]: User[Key] extends User ? [Key] : User extends User[Key] ? [Key] : User[Key] extends readonly (infer ArrayType)[] ? User extends ArrayType ? [...] : ArrayType extends User ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'post' circularly references itself in mapped type '{ [Key in keyof Comment]: Comment[Key] extends Comment ? [Key] : Comment extends Comment[Key] ? [Key] : Comment[Key] extends readonly (infer ArrayType)[] ? Comment extends ArrayType ? [...] : ArrayType extends Comment ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'posts' circularly references itself in mapped type '{ [Key in keyof User]: User[Key] extends User ? [Key] : User extends User[Key] ? [Key] : User[Key] extends readonly (infer ArrayType)[] ? User extends ArrayType ? [...] : ArrayType extends User ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'posts' circularly references itself in mapped type '{ _id: ["_id"]; title: ["title"]; posts: ["posts", number, ...any[]]; }'.ts(2615)
Type of property 'tags' circularly references itself in mapped type '{ [Key in keyof Post]: Post[Key] extends Post ? [Key] : Post extends Post[Key] ? [Key] : Post[Key] extends readonly (infer ArrayType)[] ? Post extends ArrayType ? [...] : ArrayType extends Post ? [...] : [...] : [...]; }'.ts(2615)
Type of property 'user' circularly references itself in mapped type '{ [Key in keyof Comment]: Comment[Key] extends Comment ? [Key] : Comment extends Comment[Key] ? [Key] : Comment[Key] extends readonly (infer ArrayType)[] ? Comment extends ArrayType ? [...] : ArrayType extends Comment ? [...] : [...] : [...]; }'.ts(2615)

export class Post {
  constructor(data: Partial<Post> = {}) {
    Object.assign(this, data);
  }

  _id?: ObjectId;
  title: string;

  comments: Comment[] = [];

  authorId: ObjectId | any;
  author: User;

  number?: string | number;
  tags: Tag[] = [];
  tagsIds: ObjectId[] = [];
}

export class Comment {
  _id?: ObjectId;
  title: string;
  date: Date;

  // virtual
  titleAndDate: string;

  // virtual
  get titleWithUserId() {
    return this.title + " " + this.userId;
  }

  userId: ObjectId;
  user: User;

  postId: ObjectId;
  post: Post;
}


export class User {
  constructor(data: Partial<User> = {}) {
    Object.assign(this, data);
  }

  _id?: ObjectId;
  name: string;
  title?: string;

  comments: Comment[] = [];

  posts: Post[] = [];
}

Bailey_Pearson · 2022-02-14T20:19:14.005Z

Hey @Theodor_Diaconu! Thanks for reaching out and the thorough example.

Unfortunately this is a limitation with the current implementation of recursive types in the driver. We currently only support recursive types where the recursion is direct - meaning that a type has a property of the same type. Your example is an example of mutual recursion, which we don’t support. This occurs because the Post class contains a field of type User but the User class also has a field of type Post[].

As always, a workaround is to cast your Filter as any.

We’re planning on improving the Filter type more in the future but we want to take the time to get it correct, rather than piece-mealing changes onto it release-by-release so the fix might not be ready in the near future.

Theodor_Diaconu · 2022-06-02T10:32:10.041Z

@Bailey_Pearson not sure if helpful for you but the elegant solution I found was to exclude the keys of Types which contain ‘_id’ from the filters list, solving all my problems easily and still keeping the nice “ORM”-like structure

system · 2022-09-14T01:06:34.697Z