Group stage in aggregation is being skipped

Stephen_Lynx · 2024-08-27T17:21:37.831Z

So, this is my query, being performed using the legacy driver for node.

  files.aggregate([ {
    $match : {
      filename : {
        $in : name
      }
    }
  }, {
    $group : {
      _id : '$onDisk',
      ids : {
        $push : '$_id'
      }
    }

And my issue is, if the amount of documents matched exceeds a little over 500 , the group stage is completely skipped and I get an array of matched documents that contain just the _id. I do not get an error about maximum size for bson exceeded, the aggregation simply returns me what it has before the group stage. I can’t tell if this issue is on the driver or mongo itself. The mongod version is 6 and legacy driver is 5.0.0, with the mongodb driver being 5.9.2.

I don’t know if it’s relevant, but ‘onDisk’ can be null, and it was when it failed, I can’t tell if it failed with large amounts an non-null values for onDisk.

Stephen_Lynx · 2024-08-27T21:07:19.935Z

I don’t know if this is relevant, but this aggregation is performed immediately after another aggregation on the same collection.

 files.aggregate([ {
    $match : {
      'metadata.type' : 'captcha',
      'metadata.expiration' : {
        $lte : new Date()
      }
    }
  }, {
    $group : {
      _id : 0,
      files : {
        $push : '$filename'
      }
    }

Artur_Trapp · 2024-08-28T11:05:02.140Z

We are having the same issue as of yesterday (2024-08-27).

Mongo Compass runs the query fine, but even when trying to export the results, the group stage gets skipped, which is causing a bunch of our tools to break.

Stephen_Lynx · 2024-08-28T16:57:35.890Z

That points for an issue on the database itself. What is your mongod version?

Jack_Woehr · 2024-08-28T17:57:06.695Z

Stephen_Lynx:

I don’t know if it’s relevant, but ‘onDisk’ can be null, and it was when it failed, I can’t tell if it failed with large amounts an non-null values for onDisk.

mongodb.com

$group (aggregation) - MongoDB Manual v7.0

The _id expression specifies the group key. If you specify an _id value of null, or any other constant value, the $group stage returns a single document that aggregates values across all of the input documents. See the Group by Null example.

Federico_Ferrioli · 2024-08-28T18:27:13.118Z

We are experiencing the same issue since yesterday morning (2004-08-27).
The problem seems to appear whenever we use a $in or $nin condition followed by a $group stage. If the number of elements in $in / $nin array is below 1000, the problem doesn’t occurr…

Jack_Woehr · 2024-08-28T19:24:39.384Z

@Federico_Ferrioli @Artur_Trapp @Stephen_Lynx

File an issue in MongoDB Jira Issue Tracker!

Federico_Ferrioli · 2024-08-29T11:52:53.755Z

I’ll share with you the message I received from support team:

" I understand your concern here and I would like to let you know that our internal team identified aggregation issues introduced in version v6.0.17 and scheduled a downgrade for all the clusters to v6.0.16.

There are numbers of aggregation stages affected by this bug in different pipeline usages, including some like [ {$match: { … }}, {$group: { …}}, … ]. Other stages are also affected which includes: $documents, $lookup, $replaceRoot, $let with $expr.

We sincerely apologize for any inconvenience the recent minor version upgrade may have caused."

So the problem seems to be solved since the update has been reverted to previous version.

Stephen_Lynx · 2024-08-29T14:19:22.804Z

It was solved if you are using their SAAS. For everyone else it boils down to “yeah, we have a botched update out there lmao”.

Stephen_Lynx · 2024-09-19T19:17:04.794Z

It’s been a month and 6.0.17 is still out there. Just updated a vm I had around and it went to the broken version.