SearchDefinitionBuilder.Equals - Unexpected Restrictions on DataTypes

Joe_Harvey · 2025-03-24T11:40:09.627Z

I am attempting to use the C# drivers to recreate a $search stage which I have written manually as a BSON document in Mongo Compass, where it is working exactly has expected. However, I am getting unexpected behaviour/type restrictions when attempting to us SearchDefinitionBuilder.Equals as the drivers only allow for data types of boolean, numeric, ObjectId and date which is unexpected as the Atlas Search documentation for equals clearly states that this operator can match against strings as long as they are indexed with the Atlas Search token type as mine are (see my search index below).

When I attempt to use a TField of type string with the Equals method I get the error:

The type ‘string’ must be a non-nullable value type in order to use it as parameter ‘TField’ in the generic type or method 'SearchDefinitionBuilder.Equals(Expression<Func<LegalPerson, TField>>, TField, SearchScoreDefinition)'CS0453

Similarly, if I type to use a TField of type enum (in my case Entity.SubType.Company which is backed by and stored as the data type Int32 and a value of 2 I get the error:

The type ‘Vaiie.Onboard.Api.Models.Entity.SubType’ cannot be used as type parameter ‘TField’ in the generic type or method ‘SearchDefinitionBuilder.Equals(Expression<Func<LegalPerson, TField>>, TField, SearchScoreDefinition)’. There is no boxing conversion from ‘Vaiie.Onboard.Api.Models.Entity.SubType’ to ‘System.IComparable<Vaiie.Onboard.Api.Models.Entity.SubType>’.CS0315

For example:

Screenshot 2025-03-21 at 11.52.00696×643 54.3 KB

As you will see from my search index and $search stage samples below, the use of exists on these fields with these data types works exactly as hoped when I form this $search BSON document manually in Compass and my results include all expected documents however, I seem unable to build up this same $search stage via the C# drivers due to these type restrictions.

Am I doing something wrong, or is this a bug/oversight with the drivers?

Sample Document:
``{
“_t”: [
“Profile”,
“LegalPerson”
],
“status”: 2,
“subType”: 2,
“contactDetails”: {
“email”: “info@acme.co.uk”,
“countryCode”: 44,
“subscriberNumber”: “7797812345”
},
“countryOfRegistration”: “GB”,
“registrationNumber”: “123456789”,
“name”: “Acme Corporation UK”
}`

Search Index:

  "mappings": {
    "dynamic": false,
    "fields": {
      "_t": {
        "type": "token",
        "normalizer": "none"
      },
      "subType": {
        "type": "number",
        "representation": "int64",
        "indexIntegers": true
      },
      "status": {
        "type": "number",
        "representation": "int64",
        "indexIntegers": true
      },
      "contactDetails": {
        "type": "document",
        "fields": {
          "email": {
            "type": "string",
            "analyzer": "basicEmailAddressAnalyzer"
          }
        }
      },
      "name": {
        "type": "string",
        "analyzer": "lucene.standard"
      },
      "countryOfRegistration": {
        "type": "token",
        "normalizer": "lowercase"
      },
      "registrationNumber": {
        "type": "string",
        "analyzer": "lucene.keyword"
      }
    }
  },
  "analyzers": [
    {
      "name": "basicEmailAddressAnalyzer",
      "tokenizer": {
        "type": "uaxUrlEmail"
      }
    }
  ]
}`

`$search`  Stage:

`/**
 * index: The name of the Search index.
 * text: Analyzed search, with required fields of query and path, the analyzed field(s) to search.
 * compound: Combines ops.
 * span: Find in text field regions.
 * exists: Test for presence of a field.
 * near: Find near number or date.
 * range: Find in numeric or date range.
 */
{
  index: "profileLegalPerson",
  compound: {
    filter: [
      // LegalPerson based Dossiers
      {
        equals: {
          path: "_t",
          value: "LegalPerson"
        }
      },
      // Company based LegalPersons
      {
        equals: {
          path: "subType",
          value: 2
        }
      },
      // In Draft or Live state
      {
        in: {
          path: "status",
          value: [1, 2]
        }
      }
    ],
    should: [
      {
        text: {
          path: "contactDetails.email",
          query: "info@acme.co.uk",
          score: {
            boost: {
              value: 4
            }
          }
        }
      },
      {
        text: {
          path: "name",
          query: "acme",
          score: {
            boost: {
              value: 2
            }
          }
        }
      },
      {
        text: {
          path: "registrationNumber",
          query: "123456789",
          score: {
            boost: {
              value: 10
            }
          }
        }
      },
      {
        equals: {
          path: "countryOfRegistration",
          value: "GB"
        }
      }
    ],
    minimumShouldMatch: 1
  }
}`

papafe · 2025-03-24T15:19:34.552Z

Hi @Joe_Harvey, welcome to the forum

It should be possible to do what you’re asking from version 3.2.0 of the driver.

Joe_Harvey · 2025-03-24T16:04:17.922Z

Update: I was using version 2.27.0 of the C# driver and have since found out that this seeming oversight was fixed as part of version 3.1.0 where these SearchDefinitionBuilder.Equals() method(s) were updated to remove the where TField : struct requirement and are documented as accepting string types.

Upgrading to > 3.1.0 resolved some of these issues for me however, I am still getting unexpected issues when trying to make use of enum based fields/values when building search definitions, for example, both of these attempts to add a filter results in an exception being thrown with the message:

Specified cast is not valid

A:

              .Filter( // Company based LegalPersons
                  Builders<LegalPerson>.Search.Equals<Entity.SubType>(
                      lp => lp.SubType,
                      Entity.SubType.Company
                  )
              )

B:

                .Filter( // In Draft or Live state
                    Builders<LegalPerson>.Search.In<ProfileStatus>(
                        lp => lp.Status,
                        new [] { ProfileStatus.Draft, ProfileStatus.Live }
                    )
                )

So instead I have had to resort to specifying the POCO property/fiueld name manually as a string and manually casting the enum options to their underlying Int32 value e.g.:

                .Filter( // Company based LegalPersons
                    Builders<LegalPerson>.Search.Equals<int>(
                        "subType",
                        (int)Entity.SubType.Company
                    )
                )
                .Filter( // In Draft or Live state
                    Builders<LegalPerson>.Search.In<int>(
                        "status",
                        new [] { (int)ProfileStatus.Draft, (int)ProfileStatus.Live }
                    )
                )

Which is I guess workable but quite annoying, any guidance on how I can achieve closer to the hope A/B above would be gratefully received?

papafe · 2025-03-24T16:54:39.897Z

Glad to hear that the update solved some of the issues
Regarding your problem with the enums, I’ll take a look and I’ll come back to you.

papafe · 2025-03-25T17:04:34.057Z

Hey @Joe_Harvey,

I confirmed that what you’re trying to do will work in one of the next versions of the driver, it should be either the next one or the following. It could be you’ll need to slightly modify the filter, but it’s not decided yet. This is the PR that should help you: https://github.com/mongodb/mongo-csharp-driver/pull/1583

In the meanwhile I can suggest a “halfway” solution

              .Filter( // Company based LegalPersons
                  Builders<LegalPerson>.Search.Equals(
                      lp => (int)lp.SubType,
                      (int)Entity.SubType.Company
                  )
              )

Practically you just need to add the casting also to your property, so you won’t need to specify the string manually.

Joe_Harvey · 2025-03-26T09:29:45.768Z

Thank you @papafe, I have implemented that suggestion and have added myself as a subscriber to notifications on the GitHub PR you linked to so I will keep an eye on this and see what the outcome is.