Atlas search pagination navigation issue

angelo_capone · 2024-11-13T13:19:06.626Z

Hi,

i am running an atlas search aggregation pipeline using count and pagination,
i’m facing an issue while navigating through paged results: after several pages i get no results even though the number of elements is lower than total.
The following query generate results and the count value in the result set is 625,
the pagination is implemented with the addition of “searchAfter” clause with the value of the last paginationToken of the previous page (commented out in the code below) .
After 6 pages (each of 8 documents) i get no more results.
If i raise the limit value to 626 i get 625 results, then the counted value is correct but something strange happens with the pagination token and the searchAfter operator.

[
  {
    $search: {
      index: "templates_search",
      compound: {
        filter: [
          {
            in: {
              path: "codiceScheda",
              value: ["P1_16","P1_17","P1_18","P1_19","P1_20","P1_21","P1_23","P1_24","P2_16","P2_17","P2_18",
                "P2_19","P2_20","P2_21","P2_23","P2_24","P3_4","P3_5","P4_1","P4_2","P4_3","P4_4","P4_5",
                "P4_6","ISDA1","ISDA2"
              ]
            }
          },
          {
            range: {
              path: "dataPubblicazione",
              gte: ISODate(
                "2024-04-19T00:00:00Z"
              ),
              lte: new ISODate(
                "2024-04-19T23:59:59Z"
              )
            }
          }
        ],
        must: []
      },
      sort: {
        score: {
          $meta: "searchScore"
        },
        idAvviso: 1
      }, 
      "scoreDetails" : true, 
      "count" : { "type" : "lowerBound"}, 
      //"searchAfter" : "CPakUhoCEAAaAmABIg5aDGZE14JjfLenKYye8Q=="
    }
  },
  {
    $limit: 8
  },
  {
    $addFields: {
      scoreDetails: {
        $meta: "searchScoreDetails"
      },
      count: "$$SEARCH_META.count.lowerBound",
      paginationToken: {
        $meta: "searchSequenceToken"
      },
      score: {
        $meta: "searchScore"
      }
    }
  },
  {
    $lookup: {
      from: "avvisi",
      localField: "idAvviso",
      foreignField: "idAvviso",
      pipeline: [
        {
          $project: {
            scheda: 0
          }
        }
      ],
      as: "avviso"
    }
  }
]

The behaviuor described is observerd in mongoDB Atlas 6 and 7, both with Compass and with java application with latest mongodb driver.

Any help is appreciated, thanks in advance.

amyjian · 2024-11-13T17:12:38.178Z

Hi @angelo_capone , can you provide a sample document and the index definition that you are using?

angelo_capone · 2024-11-14T19:55:05.824Z

Hi all,
after further investigation,
we found that after removing the readPreference=secondaryPreferred used in the connection string,
pagination works as expected.
It appears that paginationToken is not a cluster-level value but rather a node-local value, could it be possible?