Configure a Time Series Collection

On this page

About this Task

Before you Begin
Steps
Example
Learn More

You can configure mapping rules to migrate a table that contains a datetime column to a time series collection. Time series collections efficiently store time series data. In time series collections, writes are organized so that data from the same source is stored alongside other data points from a similar point in time.

About this Task

You can use field customizations to convert string columns to datetime fields in Relational Migrator.
CDC jobs that have time series configurations do not create the time series collections until the CDC job completes.
Compared to normal collections, storing time series data in time series collections improves query efficiency and reduces the disk usage for time series data and secondary indexes.
Use cases for time series collections include IoT, forecasting, and time based analysis.

Before you Begin

For full details on each time series field component, see Time Series Field Reference.
If the time series data in your sync job does not require any complex transformations you may benefit from migrating your time series data as a separate sync job. This can make it easier to preload a large time series collection and migrate the rest of your data after the initial job completes.
If you're migrating large amounts of data, you might want to use table filters to divide your sync jobs into batches. For example, you might import one year of historical data at a time. This allows you to break down a large job into more manageable segments.
If a long-running sync job fails unexpectedly (for example, due to a database outage) and is not able to automatically recover, you may be able to use table filters to create a new sync job that picks up from where the previous one left off. This approach lets you avoid having to repeat the entire sync job.

Steps

Select a collection with a datetime field

Navigate to the Mapping screen.
On the Schema model pane under the MongoDB header, click on a collection that has a datetime field.

Configure the time series collection

On the Mappings pane, click the Configure button.
Select the Time Series Collection radio button.
Select a Time Field.
Time Field is the name of the field which contains the datetime field in each time series document. Documents in a time series collection must have a valid BSON date data type. For details on converting a field to a datetime type in Relational Migrator, see field customizations.
Note
If you have multiple datetime columns must select a single field per timeseries collection.
(Optional) Select a Meta Field
Meta Field is the name of the field that contains metadata in each time series document. The metadata in the specified field is used to label a unique series of documents. The metadata should rarely change.
Select a Granularity.
Specify a Granularity value that most closely matches the time between consecutive incoming timestamps. Setting the correct granularity improves performance by optimizing how MongoDB stores data in the collection. Possible values are seconds, minutes, and hours.
(Optional) Select a Expire After Seconds.
Expire After Seconds enables the automatic deletion of documents in a time series collection by specifying the number of seconds after which documents expire. MongoDB deletes expired documents automatically.
Click Save And Close.
Note
When a collection is configured with a time series field, the time series icon displays next to the collection name on the Schema model pane.

Example

The following example converts the rental_date column into a time series field in MongoDB:

rental_id	rental_date	inventory_id	customer_id	return_date	staff_id	last_updated
1	2005-05-24 22:53:30	367	130	2005-05-26 22:04:30	1	2006-02-15 20:30:53

{
rental_date: {
   $date: "2005-05-24T22:53:30.000Z",
},
last_updated: "2006-02-16T01:30:53Z",
customerId: 130,
rental_date: {
   $date: "2005-05-26T22:04:30.000Z",
},
_id: {
   $oid: "661597470a883992d56d60d6",
},
inventory_id: 367,
staff_id: 1,
rental_id: 1,
}

Learn More

Back

Rename Collection

Fields