Query MongoDB - Node.js SDK

Atlas Device SDKs are deprecated. Refer to the deprecation page for details.

You can query data stored in MongoDB Atlas directly from your client application code by using the Realm Node.js SDK's MongoDB client with the Query API. Atlas App Services provides data access rules on collections to securely retrieve results based on the logged-in user or the content of each document.

Note

Example Dataset

The examples on this page use a MongoDB collection that describes inventory in a chain of plant stores. For more information on the collection schema and document contents, see Example Data.

Use Cases

There are a variety of reasons you might want to query a MongoDB data source. Working with data in your client via Atlas Device Sync is not always practical or possible. You might want to query MongoDB when:

The data set is large or the client device has constraints against loading the entire data set
You are creating or updating custom user data
You are retrieving documents that are not modeled in Realm
Your app needs to access collections that don't have strict schemas
A non-Realm service generates collections that you want to access

While not exhaustive, these are some common use cases for querying MongoDB directly.

Prerequisites

Before you can query MongoDB from your Node.js application, you must set up MongoDB Data Access in your App Services App. To learn how to set up your backend App to let the Realm SDK query Atlas, refer to Set Up MongoDB Data Access in the App Services documentation.

Example

Example Data

The examples on this page use the following MongoDB collection that describes various plants for sale in a chain of plant stores:

{ _id: ObjectId("5f87976b7b800b285345a8c4"), name: "venus flytrap", sunlight: "full", color: "white", type: "perennial", _partition: "Store 42" },
{ _id: ObjectId("5f87976b7b800b285345a8c5"), name: "sweet basil", sunlight: "partial", color: "green", type: "annual", _partition: "Store 42" },
{ _id: ObjectId("5f87976b7b800b285345a8c6"), name: "thai basil", sunlight: "partial", color: "green", type: "perennial", _partition: "Store 42" },
{ _id: ObjectId("5f87976b7b800b285345a8c7"), name: "helianthus", sunlight: "full", color: "yellow", type: "annual", _partition: "Store 42" },
{ _id: ObjectId("5f87976b7b800b285345a8c8"), name: "petunia", sunlight: "full", color: "purple", type: "annual", _partition: "Store 47" }

Documents in the plants collection use the following schema:

{
  "title": "Plant",
  "bsonType": "object",
  "required": ["_id", "_partition", "name"],
  "properties": {
    "_id": { "bsonType": "objectId" },
    "_partition": { "bsonType": "string" },
    "name": { "bsonType": "string" },
    "sunlight": { "bsonType": "string" },
    "color": { "bsonType": "string" },
    "type": { "bsonType": "string" }
  }
}

type Plant = {
  _id: BSON.ObjectId;
  _partition: string;
  name: string;
  sunlight?: string;
  color?: string;
  type?: string;
};

Connect to a Linked Cluster

To access a linked cluster from your client application, pass the cluster name to User.mongoClient(). This returns a MongoDB service interface that you can use to access databases and collections in the cluster.

const mongodb = app.currentUser.mongoClient("mongodb-atlas");
const plants = mongodb.db("example").collection("plants");

const mongodb = app.currentUser.mongoClient("mongodb-atlas");
const plants = mongodb.db("example").collection<Plant>("plants");

Read Operations

Find a Single Document

To find a single document, pass a query that matches the document to collection.findOne(). If you do not pass a query, findOne() matches the first document it finds in the collection.

The following snippet finds the document that describes "venus flytrap" plants in the collection of documents that describe plants for sale in a group of stores:

const venusFlytrap = await plants.findOne({ name: "venus flytrap" });
console.log("venusFlytrap", venusFlytrap);

{
  _id: ObjectId("5f87976b7b800b285345a8c4"),
  name: "venus flytrap",
  sunlight: "full",
  color: "white",
  type: "perennial",
  _partition: "Store 42",
}

Find Multiple Documents

To find multiple documents, pass a query that matches the documents to collection.find(). If you do not pass a query, find() matches all documents in the collection.

The following snippet finds all documents that describe perennial plants in the collection of documents that describe plants for sale in a group of stores:

const perennials = await plants.find({ type: "perennial" });
console.log("perennials", perennials);

[
  { _id: ObjectId("5f87976b7b800b285345a8c4"), name: 'venus flytrap', sunlight: 'full', color: 'white', type: 'perennial', _partition: 'Store 42' },
  { _id: ObjectId("5f87976b7b800b285345a8c6"), name: 'thai basil', sunlight: 'partial', color: 'green', type: 'perennial', _partition: 'Store 42' },
  { _id: ObjectId("5f879f83fc9013565c23360e"), name: 'lily of the valley', sunlight: 'full', color: 'white', type: 'perennial', _partition: 'Store 47' },
  { _id: ObjectId("5f87a0defc9013565c233611"), name: 'rhubarb', sunlight: 'full', color: 'red', type: 'perennial', _partition: 'Store 47' },
  { _id: ObjectId("5f87a0dffc9013565c233612"), name: 'wisteria lilac', sunlight: 'partial', color: 'purple', type: 'perennial', _partition: 'Store 42' },
  { _id: ObjectId("5f87a0dffc9013565c233613"), name: 'daffodil', sunlight: 'full', color: 'yellow', type: 'perennial', _partition: 'Store 42' }
]

Count Documents

To count documents, pass a query that matches the documents to collection.count(). If you do not pass a query, count() counts all documents in the collection.

The following snippet counts the number of documents in a collection of documents that describe plants for sale in a group of stores:

const numPlants = await plants.count();
console.log(`There are ${numPlants} plants in the collection`);

"There are 9 plants in the collection"

Write Operations

Insert a Single Document

To insert a single document, pass it to collection.insertOne().

The following snippet inserts a single document describing a "lily of the valley" plant into a collection of documents that describe plants for sale in a group of stores:

const result = await plants.insertOne({
  name: "lily of the valley",
  sunlight: "full",
  color: "white",
  type: "perennial",
  _partition: "Store 47",
});
console.log(result);

{
  insertedId: "5f879f83fc9013565c23360e",
}

Insert Multiple Documents

To insert multiple documents at the same time, pass them as an array to collection.insertMany().

The following snippet inserts three documents describing plants into a collection of documents that describe plants for sale in a group of stores:

const result = await plants.insertMany([
  {
    name: "rhubarb",
    sunlight: "full",
    color: "red",
    type: "perennial",
    _partition: "Store 47",
  },
  {
    name: "wisteria lilac",
    sunlight: "partial",
    color: "purple",
    type: "perennial",
    _partition: "Store 42",
  },
  {
    name: "daffodil",
    sunlight: "full",
    color: "yellow",
    type: "perennial",
    _partition: "Store 42",
  },
]);
console.log(result);

{
  insertedIds: [
    "5f87a0defc9013565c233611",
    "5f87a0dffc9013565c233612",
    "5f87a0dffc9013565c233613",
  ],
}

Update a Single Document

To update a single document, pass a query that matches the document and an update document to collection.updateOne().

The following snippet updates a single document in a collection of documents that describe plants for sale in a group of stores. This operation queries for a document where the name field contains the value "petunia" and changes the value of the first matched document's sunlight field to "partial":

const result = await plants.updateOne(
  { name: "petunia" },
  { $set: { sunlight: "partial" } }
);
console.log(result);

{ matchedCount: 1, modifiedCount: 1 }

Update Multiple Documents

To update multiple documents simultaneously, pass a query that matches the documents and an update description to collection.updateMany().

The following snippet updates multiple documents in a collection of documents that describe plants for sale in a group of stores. This operation queries for documents where the _partition field contains the value "Store 47" and changes the value of the _partition field of each matching document to "Store 51":

const result = await plants.updateMany(
  { _partition: "Store 47" },
  { $set: { _partition: "Store 51" } }
);
console.log(result);

{ matchedCount: 3, modifiedCount: 3 }

Upsert Documents

To upsert a document, set the upsert option to true in your update operation. If the operation's query does not match any document in the collection, an upsert automatically inserts a single new document into the collection that matches the provided query document with the update applied to it.

The following snippet updates a document in a collection of documents that describe plants for sale in a group of stores with an upsert operation. The query doesn't match any existing documents, so MongoDB automatically creates a new one.

const result = await plants.updateOne(
  {
    sunlight: "full",
    type: "perennial",
    color: "green",
    _partition: "Store 47",
  },
  { $set: { name: "sweet basil" } },
  { upsert: true }
);
console.log(result);

{
  matchedCount: 0,
  modifiedCount: 0,
  upsertedId: ObjectId("5f1f63055512f2cb67f460a3"),
}

Delete a Single Document

To delete a single document from a collection, pass a query that matches the document to collection.deleteOne(). If you do not pass a query or if the query matches multiple documents, then the operation deletes the first document it finds.

The following snippet deletes one document in a collection of documents that describe plants for sale in a group of stores. This operation queries for a document where the color field has a value of "green" and deletes the first document that matches the query:

const result = await plants.deleteOne({ color: "green" });
console.log(result);

{ deletedCount: 1 }

Delete Multiple Documents

To delete multiple document from a collection, pass a query that matches the documents to collection.deleteMany(). If you do not pass a query, deleteMany() deletes all documents in the collection.

The following snippet deletes all documents for plants that are in "Store 51" in a collection of documents that describe plants for sale in a group of stores:

const result = await plants.deleteMany({
  _partition: "Store 51",
});
console.log(result);

{ deletedCount: 3 }

Real-time Change Notifications

You can call collection.watch() to subscribe to real-time change notifications that MongoDB emits whenever a document in the collection is added, modified, or deleted. Each notification specifies a document that changed, how it changed, and the full document after the operation that caused the event.

Note

collection.watch() returns an async generator that allows you to asynchronously pull change events for operations as they occur.

Important

Serverless Limitations

You cannot watch for changes if the data source is an Atlas serverless instance. MongoDB serverless currently does not support change streams, which are used on watched collections to listen for changes.

Watch for All Changes in a Collection

To watch for all changes in a collection, call collection.watch() with no arguments:

for await (const change of plants.watch()) {
  switch (change.operationType) {
    case "insert": {
      const { documentKey, fullDocument } = change;
      console.log(`new document: ${documentKey}`, fullDocument);
      break;
    }
    case "update": {
      const { documentKey, fullDocument } = change;
      console.log(`updated document: ${documentKey}`, fullDocument);
      break;
    }
    case "replace": {
      const { documentKey, fullDocument } = change;
      console.log(`replaced document: ${documentKey}`, fullDocument);
      break;
    }
    case "delete": {
      const { documentKey } = change;
      console.log(`deleted document: ${documentKey}`);
      break;
    }
  }
}

Watch for Specific Changes in a Collection

To watch for specific changes in a collection, pass a query that matches change event fields to collection.watch():

for await (const change of plants.watch({
  filter: {
    operationType: "insert",
    "fullDocument.type": "perennial",
  },
})) {
  // The change event will always represent a newly inserted perennial
  const { documentKey, fullDocument } = change;
  console.log(`new document: ${documentKey}`, fullDocument);
}

Aggregation Operations

Aggregation operations run all documents in a collection through a series of stages called an aggregation pipeline. Aggregation allows you to filter and transform documents, collect summary data about groups of related documents, and other complex data operations.

Run an Aggregation Pipeline

To execute an aggregation pipeline, pass an array of aggregation stages to collection.aggregate(). Aggregation operations return the result set of the last stage in the pipeline.

The following snippet groups all documents in the plants collection by their type value and aggregates a count of the number of each type:

const result = await plants.aggregate([
  {
    $group: {
      _id: "$type",
      total: { $sum: 1 },
    },
  },
  { $sort: { _id: 1 } },
]);
console.log(result);

[
  { _id: "annual", total: 1 },
  { _id: "perennial", total: 5 },
]

Filter Documents

You can use the $match stage to filter documents according to standard MongoDB query syntax.

{
  "$match": {
    "<Field Name>": <Query Expression>,
    ...
  }
}

Example

The following $match stage filters documents to include only those where the type field has a value equal to "perennial":

const perennials = await plants.aggregate([
  { $match: { type: { $eq: "perennial" } } },
]);
console.log(perennials);

[
  { "_id": ObjectId("5f87976b7b800b285345a8c4"), "_partition": "Store 42", "color": "white", "name": "venus flytrap", "sunlight": "full", "type": "perennial" },
  { "_id": ObjectId("5f87976b7b800b285345a8c6"), "_partition": "Store 42", "color": "green", "name": "thai basil", "sunlight": "partial", "type": "perennial" },
  { "_id": ObjectId("5f87a0dffc9013565c233612"), "_partition": "Store 42", "color": "purple", "name": "wisteria lilac", "sunlight": "partial", "type": "perennial" },
  { "_id": ObjectId("5f87a0dffc9013565c233613"), "_partition": "Store 42", "color": "yellow", "name": "daffodil", "sunlight": "full", "type": "perennial" },
  { "_id": ObjectId("5f1f63055512f2cb67f460a3"), "_partition": "Store 47", "color": "green", "name": "sweet basil", "sunlight": "full", "type": "perennial" }
]

Group Documents

You can use the $group stage to aggregate summary data for one or more documents. MongoDB groups documents based on the expression defined in the _id field of the $group stage. You can reference a specific document field by prefixing the field name with a $.

{
  "$group": {
    "_id": <Group By Expression>,
    "<Field Name>": <Aggregation Expression>,
    ...
  }
}

Example

The following $group stage arranges documents by the value of their type field and calculates the number of plant documents that each unique type value appears in.

const result = await plants.aggregate([
  {
    $group: {
      _id: "$type",
      numItems: { $sum: 1 },
    },
  },
  { $sort: { _id: 1 } },
]);
console.log(result);

[
  { _id: "annual", numItems: 1 },
  { _id: "perennial", numItems: 5 },
]

Paginate Documents

To paginate results, you can use range aggregation queries with the $match, $sort, and $limit operators. To learn more about paginating documents, refer to Using Range Queries in the MongoDB Server documentation.

Example

The following example paginates through a collection of documents in ascending order.

// Paginates through list of plants
// in ascending order by plant name (A -> Z)
async function paginateCollectionAscending(
  collection,
  nPerPage,
  startValue
) {
  const pipeline = [{ $sort: { name: 1 } }, { $limit: nPerPage }];
  // If not starting from the beginning of the collection,
  // only match documents greater than the previous greatest value.
  if (startValue !== undefined) {
    pipeline.unshift({
      $match: {
        name: { $gt: startValue },
      },
    });
  }
  const results = await collection.aggregate(pipeline);
  return results;
}
// Number of results to show on each page
const resultsPerPage = 3;
const pageOneResults = await paginateCollectionAscending(
  plants,
  resultsPerPage
);
const pageTwoStartValue = pageOneResults[pageOneResults.length - 1].name;
const pageTwoResults = await paginateCollectionAscending(
  plants,
  resultsPerPage,
  pageTwoStartValue
);
// ... can keep paginating for as many plants as there are in the collection

Project Document Fields

You can use the $project stage to include or omit specific fields from documents or to calculate new fields using aggregation operators. Projections work in two ways:

Explicitly include fields with a value of 1. This has the side-effect of implicitly excluding all unspecified fields.
Implicitly exclude fields with a value of 0. This has the side-effect of implicitly including all unspecified fields.

These two methods of projection are mutually exclusive: if you explicitly include fields, you cannot explicitly exclude fields, and vice versa.

Note

The _id field is a special case: it is always included in every query unless explicitly specified otherwise. For this reason, you can exclude the _id field with a 0 value while simultaneously including other fields, like _partition, with a 1. Only the special case of exclusion of the _id field allows both exclusion and inclusion in one $project stage.

{
  "$project": {
    "<Field Name>": <0 | 1 | Expression>,
    ...
  }
}

Example

The following $project stage omits the _id field, includes the name field, and creates a new field named storeNumber. The storeNumber is generated using two aggregation operators:

$split separates the _partition value into two string segments surrounding the space character. For example, the value "Store 42" split in this way returns an array with two elements: "Store" and "42".
$arrayElemAt selects a specific element from an array based on the second argument. In this case, the value 1 selects the second element from the array generated by the $split operator since arrays index from 0. For example, the value ["Store", "42"] passed to this operation would return a value of "42".

const result = await plants.aggregate([
  {
    $project: {
      _id: 0,
      name: 1,
      storeNumber: {
        $arrayElemAt: [{ $split: ["$_partition", " "] }, 1],
      },
    },
  },
]);
console.log(result);

[
  { "name": "venus flytrap", "storeNumber": "42" },
  { "name": "thai basil", "storeNumber": "42" },
  { "name": "helianthus", "storeNumber": "42" },
  { "name": "wisteria lilac", "storeNumber": "42" },
  { "name": "daffodil", "storeNumber": "42" },
  { "name": "sweet basil", "storeNumber": "47" }
]

Add Fields to Documents

You can use the $addFields stage to add new fields with calculated values using aggregation operators.

{ $addFields: { <newField>: <expression>, ... } }

Note

$addFields is similar to $project but does not allow you to include or omit fields.

Example

The following $addFields stage creates a new field named storeNumber where the value is the output of two aggregate operators that transform the value of the _partition field.

const result = await plants.aggregate([
  {
    $addFields: {
      storeNumber: {
        $arrayElemAt: [{ $split: ["$_partition", " "] }, 1],
      },
    },
  },
]);
console.log(result);

[
  { "_id": ObjectId("5f87976b7b800b285345a8c4"), "_partition": "Store 42", "color": "white", "name": "venus flytrap", "storeNumber": "42", "sunlight": "full", "type": "perennial" },
  { "_id": ObjectId("5f87976b7b800b285345a8c6"), "_partition": "Store 42", "color": "green", "name": "thai basil", "storeNumber": "42", "sunlight": "partial", "type": "perennial" },
  { "_id": ObjectId("5f87976b7b800b285345a8c7"), "_partition": "Store 42", "color": "yellow", "name": "helianthus", "storeNumber": "42", "sunlight": "full", "type": "annual" },
  { "_id": ObjectId("5f87a0dffc9013565c233612"), "_partition": "Store 42", "color": "purple", "name": "wisteria lilac", "storeNumber": "42", "sunlight": "partial", "type": "perennial" },
  { "_id": ObjectId("5f87a0dffc9013565c233613"), "_partition": "Store 42", "color": "yellow", "name": "daffodil", "storeNumber": "42", "sunlight": "full", "type": "perennial" },
  { "_id": ObjectId("5f1f63055512f2cb67f460a3"), "_partition": "Store 47", "color": "green", "name": "sweet basil", "storeNumber": "47", "sunlight": "full", "type": "perennial" }
]

Unwind Array Values

You can use the $unwind stage to transform a single document containing an array into multiple documents containing individual values from that array. When you unwind an array field, MongoDB copies each document once for each element of the array field but replaces the array value with the array element in each copy.

{
  $unwind: {
    path: <Array Field Path>,
    includeArrayIndex: <string>,
    preserveNullAndEmptyArrays: <boolean>
  }
}

Example

The following example uses the $unwind stage for each object's type and color combination. The aggregation pipeline has the following steps:

Use $group stage with $addToSet to create new documents for each type with a new field colors that contains an array of all the the colors for that flower type that occur in the collection.
Use $unwind stage to create separate documents for each combination of type and color.
Use $sort stage to sort the results in alphabetical order.

const result = await plants.aggregate([
  { $group: { _id: "$type", colors: { $addToSet: "$color" } } },
  { $unwind: { path: "$colors" } },
  { $sort: { _id: 1, colors: 1 } },
]);
console.log(result);

[
  { "_id": "annual", "colors": "yellow" },
  { "_id": "perennial", "colors": "green" },
  { "_id": "perennial", "colors": "purple" },
  { "_id": "perennial", "colors": "white" },
  { "_id": "perennial", "colors": "yellow" },
]

Back

Call a Function

Manage Users