Migrate Data into a Time Series Collection with an Aggregation Pipeline

Create a metadata field.

If your collection doesn't include a field you can use to identify each series, transform your data to define one. In this example, the metaData field becomes the metaField of the time series collection that you create.

The pipline below performs the following operations:

• Uses $addFields to add a metaData field to the weather_data collection.

• Uses $project to include or exclude the remaining fields in the document.

db.weather_data.aggregate([
{
   $addFields: {
      metaData: {
         "st": "$st",
         "position": "$position",
         "elevation": "$elevation",
         "callLetters": "$callLetters",
         "qualityControlProcess": "$qualityControlProcess",
         "type": "$type"
      }
   },
},
{
   $project: {
      _id: 1,
      ts: 1,
      metaData: 1,
      dataSource: 1,
      airTemperature: 1,
      dewPoint: 1,
      pressure: 1,
      wind: 1,
      visibility: 1,
      skyCondition: 1,
      sections: 1,
      precipitationEstimatedObservation: 1
   }
}
])

Create your time series collection and insert your data.

Add an $out aggregation stage to your pipeline to create a time series collection and insert your data into it. The pipeline below performs the following operations:

• Uses $out with the timeseries option to create a weathernew time series collection in the mydatabase database.

• Defines the metaData field as the metaField of the weathernew collection.

• Defines the ts field as the timeField of the weathernew collection.

  Note

  The timeField of a time series collection must be a date type.

{
   $out: {
      db: "mydatabase",
      coll: "weathernew",
      timeseries: {
         timeField: "ts",
         metaField: "metaData",
         granularity: "seconds"
      }
   }
}

For the aggregation stage syntax, see $out. For a full explanation of the time series options, see the Time Series Field Reference.

Review your data.

After you run this aggregation pipeline, you can use findOne() to view a document in your weathernew time series collection:

db.weathernew.findOne()

The operation returns the following document:

   {
      _id: ObjectId("5553a998e4b02cf7151190b8"),
      ts: ISODate("1984-03-05T13:00:00Z"),
      metaData: {
         st: "x+47600-047900",
         position: {
            type: "Point",
            coordinates: [ -47.9, 47.6 ]
         },
         elevation: 9999,
         callLetters: "VCSZ",
         qualityControlProcess: "V020",
         type: "FM-13"
      },
      dataSource: "4",
      airTemperature: { value: -3.1, quality: "1" },
      dewPoint: { value: 999.9, quality: "9" },
      pressure: { value: 1015.3, quality: "1" },
      wind: {
         direction: { angle: 999, quality: "9" },
         type: "9",
         speed: { rate: 999.9, quality: "9" }
      },
      visibility: {
         distance: { value: 999999, quality: "9" },
         variability: { value: "N", quality: "9" }
      },
      skyCondition: {
         ceilingHeight: { value: 99999, quality: "9", determination: "9" },
         cavok: "N"
      },
      sections: [ "AG1" ],
      precipitationEstimatedObservation: { discrepancy: "2", estimatedWaterDepth: 999 }
   }