Deploy a Federated Database Instance

1. Select the dataset for your federated database instance from the Data Sources section.

   Click Add Data Sources to select your data store.

2. Specify your data store.

   Choose HTTP(S) to configure a federated database instance for data in publicly accessible HTTP and HTTPS URLs.

   Corresponds to stores.[n].provider JSON configuration setting.

3. Enter a name for your HTTP data store into the HTTP(S) Store Name field.

   Note

   The data store's name must be unique within your federated database instance.

   Corresponds to stores.[n].name JSON configuration setting.

4. Enter the publicly accessible URL of the file where data is stored.

   Atlas Data Federation supports JSON, BSON, CSV, TSV, Avro (gzipped or uncompressed), Parquet, and ORC file types.

   Tip

   Click Use Sample URL to add a sample HTTP data store.

   For each additional HTTP data store that you want to add, click Add Another URL, then enter the HTTP data store URLs.

   Corresponds to stores.[n].urls JSON configuration setting.

5. Click Next to configure virtual databases and collections.

6. Create the virtual databases, collections, and views and map the databases, collections, and views to your data store.

   1. (Optional) Click the for the:

      • Database to edit the database name. Defaults to VirtualDatabase[n].

        Corresponds to databases.[n].name JSON configuration setting.

      • Collection to edit the collection name. Defaults to VirtualCollection[n].

        Corresponds to databases.[n].collections.[n].name JSON configuration setting.

      • View to edit the view name.

      You can click:

      • Add Database to add databases and collections.

      • associated with the database to add collections to the database.

      • associated with the collection to add views on the collection. To create a view, you must specify:

        • The name of the view.

        • The pipeline to apply to the view.

          The view definition pipeline cannot include the $out or the $merge stage. If the view definition includes nested pipeline stages such as $lookup or $facet, this restriction applies to those nested pipelines as well.

        To learn more about views, see:

        • Views

        • db.createView

      • associated with the database, collection, or view to remove it.

   2. Select HTTP from the dropdown in the Data Sources section.

   3. Drag and drop the data store to map with the collection.

      Corresponds to databases.[n].collections.[n].dataSources JSON configuration setting.

Your configuration for an HTTP data store should look similar to the following:

1
{
2
  "stores" : [
3
    {
4
      "name" : "<string>",
5
      "provider": "<string>",
6
      "defaultFormat" : "<string>",
7
      "allowInsecure": <boolean>,
8
       "urls": ["<string>"]
9
    }
10
  ],
11
  "databases" : [
12
    {
13
      "name" : "<string>",
14
      "collections" : [
15
        {
16
          "name" : "<string>",
17
          "dataSources" : [
18
            {
19
              "storeName" : "<string>",
20
              "allowInsecure" : <boolean>,
21
              "urls" : ["<string>"],
22
              "defaultFormat" : "<string>",
23
              "provenanceFieldName": "<string>"
24
            }
25
          ]
26
        }
27
      ],
28
      "views" : [ 
29
        {
30
          "name" : "<string>", 
31
          "source" : "<string>", 
32
          "pipeline" : "<string>" 
33
        }
34
      ] 
35
    }
36
  ]
37
}

For more information on the configuration settings, see HTTP URL.

1. Define your HTTP data store.

   Edit the JSON configuration settings shown in the UI for stores. Your stores cofiguration setting should resemble the following:

   "stores" : [
     {
       "name" : "<string>",
       "provider" : "<string>",
       "allowInsecure": <boolean>,
       "urls" : ["<string>"],
       "defaultFormat" : "<string>"
     }
   ]

   To learn more about these configuration settings, see stores.

2. Define your federated database instance virtual databases, collections, and views.

   Edit the JSON configuration settings shown in the UI for databases. Your databases cofiguration setting should resemble the following:

   "databases" : [
     {
       "name" : "<string>",
       "collections" : [
         {
           "name" : "<string>",
           "dataSources" : [
             {
               "storeName" : "<string>",
               "allowInsecure" : <boolean>,
               "urls" : ["<string>"],
               "defaultFormat" : "<string>",
               "provenanceFieldName": "<string>"
             }
           ]
         }
       ]
     }
   ]

   To learn more about these configuration settings, see databases.