Run Aggregation Pipelines

On this page

Understand the Aggregation Syntax

Example

You can run aggregation pipelines on your collections using the MongoDB Shell. Aggregation pipelines transform your documents into aggregated results based on selected pipeline stages.

Common uses for aggregation include:

Grouping data by a given expression.
Calculating results based on multiple fields and storing those results in a new field.
Filtering data to return a subset that matches a given criteria.
Sorting data.

When you run an aggregation, MongoDB Shell outputs the results directly to the terminal.

Understand the Aggregation Syntax

The MongoDB aggregation pipeline consists of stages. Each stage transforms the documents as they pass through the pipeline. Pipeline stages do not need to produce one output document for every input document; e.g., some stages may generate new documents or filter out documents.

To create an aggregation pipeline, use the following syntax in the MongoDB Shell:

1 db.<collection>.aggregate([
2   {
3     <$stage1>
4   },
5   {
6     <$stage2>
7   }
8   ...
9 ])

Example

The examples on this page reference the Atlas sample dataset. You can create a free Atlas cluster and populate that cluster with sample data to follow along with these examples. To learn more, see Get Started with Atlas.

The example below uses the movies collection in the Atlas sample_mflix sample dataset.

Example Document

Each document in the movies collection describes a movie:

1 {
2   _id: 573a1397f29313caabce8347,
3   fullplot: 'In a cyberpunk vision of the future, man has developed the technology to create replicants, human clones used to serve in the colonies outside Earth but with fixed lifespans. In Los Angeles, 2019, Deckard is a Blade Runner, a cop who specializes in terminating replicants. Originally in retirement, he is forced to re-enter the force when four replicants escape from an off-world colony to Earth.',
4   imdb: { rating: 8.2, votes: 419589, id: 83658 },
5   year: 1982,
6   plot: 'A blade runner must pursue and try to terminate four replicants who stole a ship in space and have returned to Earth to find their creator.',
7   genres: [ 'Sci-Fi', 'Thriller' ],
8   rated: 'R',
9   metacritic: 88,
10   title: 'Blade Runner',
11   lastupdated: '2015-09-04 00:05:51.990000000',
12   languages: [ 'English', 'German', 'Cantonese', 'Japanese', 'Hungarian' ],
13   writers: [
14     'Hampton Fancher (screenplay)',
15     'David Webb Peoples (screenplay)',
16     'Philip K. Dick (novel)'
17   ],
18   type: 'movie',
19   tomatoes: {
20     viewer: { rating: 4, numReviews: 331213, meter: 91 },
21     dvd: 1997-08-27T00:00:00.000Z,
22     critic: { rating: 8.5, numReviews: 102, meter: 90 },
23     lastUpdated: 2015-09-12T17:48:21.000Z,
24     consensus: "Misunderstood when it first hit theaters, the influence of Ridley Scott's mysterious, neo-noir Blade Runner has deepened with time. A visually remarkable, achingly human sci-fi masterpiece.",
25     rotten: 10,
26     production: 'Warner Bros. Pictures',
27     fresh: 92
28   },
29   poster: 'https://m.media-amazon.com/images/M/MV5BNzQzMzJhZTEtOWM4NS00MTdhLTg0YjgtMjM4MDRkZjUwZDBlXkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY1000_SX677_AL_.jpg',
30   num_mflix_comments: 1,
31   released: 1982-06-25T00:00:00.000Z,
32   awards: {
33     wins: 13,
34     nominations: 15,
35     text: 'Nominated for 2 Oscars. Another 11 wins & 15 nominations.'
36   },
37   countries: [ 'USA', 'Hong Kong', 'UK' ],
38   cast: [
39     'Harrison Ford',
40     'Rutger Hauer',
41     'Sean Young',
42     'Edward James Olmos'
43   ],
44   directors: [ 'Ridley Scott' ],
45   runtime: 117
46 }

The documents aggregated in this tutorial reside in the sample_mflix.movies collection. Use the following command to switch to the database that contains this collection:

use sample_mflix

Example Aggregation Pipeline

Consider the following pipeline:

1 db.movies.aggregate([
2 
3   // First Stage
4 
5   { $project: { _id: 0, genres: 1, imdb: 1, title: 1 } },
6 
7   // Second Stage
8 
9   { $unwind: "$genres" },
10 
11   // Third Stage
12 
13   { $group:
14     { _id: "$genres",
15       averageGenreRating: { $avg: "$imdb.rating" }
16     }
17   },
18 
19    // Fourth Stage
20 
21    { $sort: { averageGenreRating: -1 } }
22 ] )

This pipeline performs an aggregation in four stages:

First Stage

The $project stage passes documents that contain the following fields to the next pipeline stage:

genres
imdb
title

Documents from the collection that don't include all of these fields are not passed to the next pipeline stage.

Note

The $project stage specifies _id: 0 to suppress the _id field from the documents it passes to the next stage.

For more information, see the MongoDB Manual.

The $project stage transforms the example document and passes the following output to the next pipeline stage:

1 {
2   imdb: { rating: 8.2, votes: 419589, id: 83658 },
3   genres: [ 'Sci-Fi', 'Thriller' ],
4   title: 'Blade Runner'
5 }

Second Stage

The $unwind stage passes a document for each element in the genres array to the next pipeline stage.

The $unwind stage generates the following two documents from the original example document, then passes them to the next pipeline stage:

1 {
2   imdb: { rating: 8.2, votes: 419589, id: 83658 },
3   genres: 'Sci-Fi',
4   title: 'Blade Runner'
5 }

1 {
2   imdb: { rating: 8.2, votes: 419589, id: 83658 },
3   genres: 'Thriller',
4   title: 'Blade Runner'
5 }

Third Stage

The $group stage:

Retrieves the distinct genre values from the documents it receives from the previous pipeline stage,
Creates a document for each distinct genre where the _id is the genre name,
Adds a field, averageGenreRating, to each new document that contains the average imdb.rating of all documents that match the genre, and
Passes the new documents to the next pipeline stage.

This stage sends documents that look similar to the following to the next pipeline stage:

1 { _id: 'Sport', averageGenreRating: 6.781233933161954 },
2 { _id: 'History', averageGenreRating: 7.202306920762287 },
3 { _id: 'Biography', averageGenreRating: 7.097142857142857 },
4 { _id: 'Adventure', averageGenreRating: 6.527788649706458 },
5 { _id: 'Family', averageGenreRating: 6.36096256684492 },
6 { _id: 'Crime', averageGenreRating: 6.730478683620045 },
7 { _id: 'Western', averageGenreRating: 6.879197080291971 },
8 { _id: 'Fantasy', averageGenreRating: 6.42495652173913 },
9 { _id: 'Talk-Show', averageGenreRating: 7 },
10 { _id: 'Documentary', averageGenreRating: 7.365266635205286 },
11 { _id: 'War', averageGenreRating: 7.183944374209861 },
12 { _id: 'Short', averageGenreRating: 7.355813953488372 },
13 { _id: 'Horror', averageGenreRating: 5.84110718492344 },
14 { _id: 'Film-Noir', averageGenreRating: 7.503809523809523 },
15 { _id: 'News', averageGenreRating: 7.254901960784314 },
16 { _id: 'Thriller', averageGenreRating: 6.322121555303888 },
17 { _id: 'Action', averageGenreRating: 6.3774842271293375 },
18 { _id: 'Music', averageGenreRating: 6.923452380952381 },
19 { _id: 'Animation', averageGenreRating: 6.917993795243019 },
20 { _id: 'Drama', averageGenreRating: 6.830528688822631 }

Fourth Stage

The $sort stage sorts the documents it receives from the previous stage in descending order based on the value of the averageGenreRating field.

When you run the example pipeline, the MongoDB Shell prints documents similar to the following to the terminal:

1 [
2   { _id: 'Film-Noir', averageGenreRating: 7.503809523809523 },
3   { _id: 'Documentary', averageGenreRating: 7.365266635205286 },
4   { _id: 'Short', averageGenreRating: 7.355813953488372 },
5   { _id: 'News', averageGenreRating: 7.254901960784314 },
6   { _id: 'History', averageGenreRating: 7.202306920762287 },
7   { _id: 'War', averageGenreRating: 7.183944374209861 },
8   { _id: 'Biography', averageGenreRating: 7.097142857142857 },
9   { _id: 'Talk-Show', averageGenreRating: 7 },
10   { _id: 'Music', averageGenreRating: 6.923452380952381 },
11   { _id: 'Animation', averageGenreRating: 6.917993795243019 },
12   { _id: 'Western', averageGenreRating: 6.879197080291971 },
13   { _id: 'Drama', averageGenreRating: 6.830528688822631 },
14   { _id: 'Sport', averageGenreRating: 6.781233933161954 },
15   { _id: 'Crime', averageGenreRating: 6.730478683620045 },
16   { _id: 'Musical', averageGenreRating: 6.696913580246913 },
17   { _id: 'Romance', averageGenreRating: 6.695711554220159 },
18   { _id: 'Mystery', averageGenreRating: 6.563317384370015 },
19   { _id: 'Adventure', averageGenreRating: 6.527788649706458 },
20   { _id: 'Comedy', averageGenreRating: 6.479626461362988 },
21   { _id: 'Fantasy', averageGenreRating: 6.42495652173913 }
22 ]