$lookup is not working after $search

Hello All:

I am running an aggregation with a $Search (atlas search) as follows:

[
  {
    $search: {
      index: "indexDeos",
      // Replace with the name of your search index
      text: {
        query: "trumpet",
        // Use the text from the Leo
        path: "text",
        // Path to the text field in the Deos collection
        fuzzy: {}, // Enables fuzzy matching
      },
    },
  },
]

Upon running this I get some results. However if I run a $lookup on the resultant data as follows, I get null user data.

[
  {
    $search: {
      index: "indexDeos",
      // Replace with the name of your search index
      text: {
        query: "trumpet",
        // Use the text from the Leo
        path: "text",
        // Path to the text field in the Deos collection
        fuzzy: {}, // Enables fuzzy matching
      },
    },
  },
  {
    $lookup: {
      from: "users",
      localField: "userId",
      foreignField: "_id",
      as: "user",
    },
  },
]

Here is the resultant data, note that the user is null.

_id
677e2193830fb96c3c90e62d
createdAt
2025-01-08T06:56:19.619+00:00

status
Array (1)
text
"Trumpat"
userId
677e0ff7674b72e983cb87e4

user
Array (empty)```

In the above case I am searching through a collection and creating a resultant set of data which I am joing with my user collection on userId.

However if i run the $lookup without applying it to the resultant data from the $search as follows, it works and gives me the user data.

[
{
$lookup: {
from: “users”,
localField: “userId”,
foreignField: “_id”,
as: “user”,
},
},
]```

Here is the resultant data:

createdAt
2025-01-07T10:42:31.061+00:00

status
Array (1)
text
"This is the first deo in the new system"
userId
64f56b4134b25512c4d86e2d

user
Array (1)

0
Object
_id
677d0517bc7c5e17bfc25eac

My question is why is the $lookup (I am using it to join to my users table) not working with the data returned by $search, when otherwise the $lookup is working. Is the data returned by $search in a previous aggregation pipeline stage, special in some way?

If I were to put the $lookup before the $search or inside it as recommended here
https://www.mongodb.com/docs/atlas/atlas-search/tutorial/lookup-with-search/
then it would first join my entire collection and then do a $search on the entire joined collection, which would be operationally inefficient.

Why is it not possible to simply do the $lookup after the $search in the aggregation pipeline? Afterall that is what the pipeline is for right? To be able to perform further actions on data returned by a previous stage.

Please help Mongo team. Both me and Chatgpt are out of ideas on this one!!!

If you want to ask me why I am using $lookup to join to collections and not nesting them please look at this
https://www.mongodb.com/community/forums/t/how-to-get-access-to-only-the-object-in-sub-document-where-there-was-a-hit-match-in-atlas-search/286867?u=arjun_kochhar_leodeo

With $search we cannot access the keys of sub documents. Therefore I need to create separate collections and join them. But even that is not working with $search. As described, I am able to do a $Search and then the $lookup on the search results is not working.

Hello:

I got this to work. There was a bug in my code. The $lookup works after the $search. There is no issue with that. You CAN do a $lookup after a $Search in an aggregation pipeline.