Document Filtering

On this page

Syntax

Parameter Fields
Behavior
Limitations
Verification
Examples
Start mongosync with a Document Filter
Learn More

Important

Cluster-to-Cluster Sync Beta Program

This feature is only available in mongosync beta. To learn more, see Cluster-to-Cluster Sync Beta Program.

Starting in mongosync beta 1.8, you can selectively migrate documents based on specific conditions. To further limit which documents migrate to the destination cluster, you can combine document filtering and namespace filtering.

To use document filtering, both the source cluster and destination cluster must use a supported MongoDB version.

Syntax

The start API endpoint accepts an optional documentFilter parameter with the following syntax:

"documentFilter" : {
   "field" : <field-name>,
   "matchValues" : [<field-values>]
}

Parameter Fields

Field	Type	Necessity	Description
`field`	String	Required	Field name
`matchValues`	Array	Required	Field value(s) that a document must have in order to migrate.

Behavior

Limitations

field names can't contain dots (.) or dollar signs ($).
matchValues array items can be the following BSON types:
- All number types
- Binary
- Booleans
- Datetime
- ObjectID
- Strings
Documents must not move in or out of the filter during migration.
If the document filter contains a string and at least one migrated collection with non-default collation, mongosync fails immediately.
If you call the progress API endpoint and use a document filter, the estimatedTotalBytes response field returns null.
The destination cluster must not contain pre-existing data that matches the filter.
You can't specify a document filter and set the reversible flag to true.

Verification

Document Filtering is not compatible with the Embedded Verifier. The /start command returns an error if you enable both the verifier and Document Filtering.

For more information, see Verify Data Transfer.

Note

Invalid Document Filter Handling

If you provide an invalid document filter in the start request, mongosync does not crash. Instead, the start request fails. You can re-issue the start request with a valid document filter without needing to restart mongosync.

Examples

Start `mongosync` with a Document Filter

The following example starts a sync job between cluster0 and cluster1. The source cluster is cluster0 and the destination cluster is cluster1.

cluster0 contains the travel database, which includes the restaurants, attractions, and lodging collections. Documents in the restaurants, attractions, and lodging collections all contain a city field.

The documentFilter document in this example filters for documents where the city field is either Los Angeles, New York, or San Francisco.

"documentFilter" : {
          field : "city",
          matchValues: [ "Los Angeles", "New York", "San Francisco" ]
      }

Learn More

Back

Many-To-One Migrations

Pre-Existing Data Handling