Vector search is not working as expected

Abhishek_Shivale · 2024-09-25T19:57:39.826Z

My indexing :-

{
  "fields": [
    {
      "numDimensions": 1536,
      "path": "comapnies_embedding",
      "similarity": "cosine",
      "type": "vector"
    },
    {
      "path": "description",
      "type": "filter"
    },
    {
      "path": "line_of_business",
      "type": "filter"
    }
  ]
}```

actually i trying to implement vector search but results are not as expected but empty documents coming on top of the result with 9+ vector score, I don't know what's happing but then started looking for filters option there is not an single example for  filters.
i am using openai text-embedding-ada-002 for embedding i really dont understand why empty document coming on top,
i actually what i want after search by vector search how can i do search with query 
following things is my aggrigation

const queryembedding = await getEmbedding(query);

  const aggregation = [
    {
      $vectorSearch: {
        exact: true,
        index: 'vector_index',
        path: 'comapnies_embedding',
        queryVector: queryembedding,
        //  numCandidates: 100,
        limit: 100

      },
    },
    {
      $project: {
        'companyName': 1,
        'employee_count': 1,
        industry: 1,
        country: 1,
        scope: 1,
        description: 1,
        _id: 1,
        comapnyLogo: 1,
        'score': {
          '$meta': 'vectorSearchScore'
        }
      }
    },
    {
      $sort: {
        score: -1 
      }
    }
  ];

  const toUpdate = await Company.aggregate(aggregation).exec()

Henry_Weller · 2024-09-25T20:17:06.825Z

Hi @Abhishek_Shivale! thank you for the question.

Sometimes empty document sets can be returned when the query path specifies an index name that is different from what is defined in the index definition. Would you mind confirming that the index definition has the name vector_index? A second check would be to confirm that the index has documents indexed, which you should be able to inspect from the Atlas UI.

Thanks,
Henry

Abhishek_Shivale · 2024-09-27T10:14:47.423Z

Henry_Weller:

Sometimes empty document sets can be returned when the query path specifies an index

thanks, appreciate your response!
yes my index name is :- vector_index
yes i am bale to see documents indexed in atlas ui,
additionally this is query i am doing in mongo dB compass for aggregation

{
  queryVector: [here is my vector but i am not including this here],
  path: 'comapnies_embedding',
  numCandidates: 100,
  index: 'vector_index',
  limit: 100,
}

if either index name or path is wrong i would get an error but, i got some empty document which is not related to query

Henry_Weller · 2024-10-07T18:58:42.961Z

Abhishek_Shivale:

if either index name or path is wrong i would get an error but, i got some empty document which is not related to query

Actually, if you misspell an index or path we wouldn’t error out, we would provide empty results, so this might be expected behavior related to that root cause.