How to read the fully replicated data using the atlas search aggregation pipeline?

Kalinga_Raj · 2024-10-23T11:33:18.632Z

I want to read the fully replicated data using the atlas search aggregation pipeline. But, Atlas search is not allowing me to read with concern “majority”.

Command aggregate does not support { readConcern: { level: "majority", provenance: "clientSupplied" } } :: caused by :: Aggregation stage $search cannot run with a readConcern other than 'local'. Current readConcern: majority'.

So, How can we ensure the consistency in the search result?

I’ve read the documentation and understood that the $search stage is performed by the mongot service and it won’t work with non-local read concern. I don’t know why and how can MongoDB guarantee the consistency in search results.

My Cluster Architecture Details:

Cluster tier : Dedicated M10
Node Type : electable
cluster : 3 node replicaSet
Mongodb version : 7.0.14
pymongo : 4.10.1
writeConcern : majority
readConcern : majority
read preference : secondary

I have not enabled workload-isolation; So,mongot and mongod services are running in each nodes of the replicaSet cluster.

amyjian · 2024-10-23T15:05:24.567Z

Hi @Kalinga_Raj , Atlas Search is eventually consistent and we do not support read concerns. This is typically sufficient for our customers.

Can you share more about your use case and why stronger consistency is required?

Jennysson_Junior · 2024-10-23T20:44:35.539Z

Hi @Kalinga_Raj

Great short anwser @amyjian!

As mention, the $search stage is performed by a different process, which indexes and processes search queries. These indexes are built asynchronously from the data on the mongod nodes. Because of this, $search queries bypass the typical replica set mechanisms.

The update data on search index is doing by read a change stream of the collection, as it mention on docs.

You can see the process on the following diagram, available [here].(https://www.mongodb.com/docs/atlas/atlas-search/about/deployment-options/)

fts-architecture703×442 15 KB