Docs Menu
Docs Home
/
Development
/
Data Federation
Docs Home
/
Development
/
Data Federation
Docs Home
/
Development
/
Data Federation

Configure Data Stores for a Federated Database Instance

This page contains general commands that you can use to set, update, and retrieve federated database instance storage configuration. For a specific Data Store configuration, such as AWS S3 Encryption, see its specific Data Store documentation.

Define a Data Store for a Federated Database Instance

To configure a data store for a federated database instance, you need to create a storage configuration JSON file that defines your data stores and maps them to collections you can query. MongoDB provides federated database instance support for the following data stores:

  • AWS S3 Bucket

  • Azure Blob Storage

  • Google Cloud Storage Bucket

  • Atlas Cluster

  • HTTP URL

  • Online Archives

Important

Information in your storage configuration is visible internally at MongoDB and stored as operational data to monitor and improve the performance of Atlas Data Federation. We recommend that you don't use PII in your configurations.

Data Access Types

When you create a federated database instance, you grant Atlas one of the following types of data access to the AWS buckets in your AWS account.

  • read only access: It cannot make any changes to the data or add new data.

  • read and write access: It can update existing files, delete files, or add new data to the storage.

To access your Atlas clusters, Atlas uses your existing Role Based Access Controls. You can view and edit the acces of your user following the steps in Manage Users and Roles on Self-Managed Deployments.

Privilege Actions

Privilege actions are the operations that you can perform on your federated database instance. You can grant the privilege action through one of the following:

  • When you create or modify custom roles from the Atlas User Interface

  • In the actions.action request body parameter when you create or update a custom role from the Atlas API

The following table shows the available federated database instance privilege actions:

Command
Description
sqlGetSchema

Retrieve the schema stored for a collection or view, see View Schema Using mongosh for details.

sqlSetSchema

Set or delete the schema for a collection or view, see View Schema Using mongosh for details.

viewAllHistory

Retrieve details about the queries that were run in the past using $queryHistory.

outToAzure

Write data from any one of the supported federated database instance stores or multiple supported federated database instance stores to your Azure Blob Storage container using $out.

outToS3

Write data from any one of the supported federated database instance stores or multiple supported federated database instance stores to your S3 bucket using $out.

outToGCP

Write data from any one of the supported federated database instance stores or multiple supported federated database instance stores to your Google Cloud Storage bucket using $out.

storageGetConfig

Retrieve your federated database instance storage configuration using the storageGetConfig command.

storageSetConfig

Set or update your federated database instance storage configuration using the storageSetConfig command.

Retrieve Configuration File

To retrieve your Federated Database Instance Configuration File, connect to your instance and use the following command:

use admin
db.runCommand( { "storageGetConfig" : 1 } )

The command returns the current federated database instance configuration.

Validate Configuration File

To validate your federated database instance configuration, connect to your instance and use the following command:

use admin
db.runCommand( { "storageValidateConfig" : <config> } )

Replace <config> with the configuration for the federated database instance.

If the configuration is valid, the command returns the following answer:

{ "ok" : 1 }

If the configuration is invalid, the command returns the list of errors in the errs field:

{
       "ok" : 1,
       "errs" : [
               "<error>",
               "<error>",
               ...
       ]
}

Update your Federated Database Instance Configuration

To update your federated database instance configuration, connect to your instance and use the following command:

use admin
db.runCommand( { "storageSetConfig" : <config> } )

Replace <config> with the configuration for the federated database instance.

Tip

MongoDB recommends you to validate your Federated Database Instance configuration before applying any changes.

To update your Federated Database Instance configuration Data stores, data sources, databases, collections, and views, follow these steps:

1

Log into MongoDB Atlas

2

Select Data Federation option from the Services on the left

3

Click the Configuration button besides the name of your federated database instance

4

Select your prefered configuration interface method on the top-right corner

  • For a visual interface, click Visual.

  • For editing directly in the JSON file, click JSON Editor.

5

Make your changes to the federated database instance storage configuration.

Manage Data Sources

Add an Additional Data Source

Corresponds to stores JSON configuration setting.

  1. Click Add Data Store Sources.

  2. Select the data source provider you want to configure: - For AWS S3, continue to step 5 of Deploy a Federated Database Instance Data Store. - For Google Cloud, continue to step 7 of Deploy a Federated Database Instance Data Store. - For Azure, continue to step 7 of Deploy a Federated Database Instance Data Store. - For Atlas Cluster, continue to step 7 of Deploy a Federated Database Instance Data Store. - For HTTP data store, continue to step 7 of Deploy a Federated Database Instance Data Store. - For Atlas Online Archive, continue to step 7 of Deploy a Federated Database Instance Data Store.

Edit Your Data Source Name

Corresponds to stores.[n].name JSON configuration setting.

  1. Select your data store source from the list on the left.

  2. Click the Edit Store Name icon.

  3. Change the current name in the Data Store Name field.

  4. Click Done.

Delete a Data Source

  1. Select your Data Store source from the list on the left

  2. Click the Delete this Store icon

  3. Click the Delete button.

Manage Databases

Create a New Database

Corresponds to databases JSON configuration setting.

  1. Click the Add Database button on the right-top corner

  2. Fill in the Database Name field

  3. Fill in the Collection Name field

  4. Click the Create button

Edit a Database name

Corresponds to databases.[n].name JSON configuration setting.

  1. Select your Database from the list.

  2. Click the Edit Database Name icon.

  3. Update the Database Name.

  4. Click the Done button.

Delete a Database

  1. Select your Database from the list.

  2. Click Delete this Database icon.

  3. Click the Delete button.

Manage Collections

Create a new Collection in a Database

Corresponds to databases.[n].collections JSON configuration setting.

  1. Select the Database from where you want to create a Collection

  2. Click the Add Collection icon

  3. Fill in the Collection Name field

  4. Click the Create button

Edit a Collection name

Corresponds to databases.[n].collections.[n].name JSON configuration setting.

  1. Select your Collection from the list

  2. Click the Edit Collection Name icon

  3. Change the current name in the Collection Name field for your new name

  4. Click the Create button

Delete a Collection

  1. Select your Collection from the list

  2. Click the Delete this Collection icon

  3. Click the Delete button.

Manage Views

Create a new View in a Collection

Corresponds to databases.[n].views JSON configuration setting.

  1. Select the Collection from where you want to create a View

  2. Click the Add View icon

  3. Fill in the View Name field

  4. Fill in the :manual:`Pipeline </core/aggregation-pipeline/>` field

  5. Click the Create button

Edit a View name or Pipeline

Corresponds to databases.[n].views.[n].name and databases.[n].views.[n].pipeline JSON configuration settings respectively.

  1. Select your View from your Collection

  2. Click the Edit View Name icon

  3. Change the current name in the View Name field for your new name

  4. Change the current pipeline in the Pipeline field for your new pipeline

  5. Click the Save button

Delete a View

  1. Select your View from the list

  2. Click the Delete this View icon

  3. Click the Delete button.

Edit the JSON configuration file objects to add, create, or remove stores, databases, or views

To learn more about the configuration settings of your specific Data Store provider, see their Configuration File page:

To learn how to set and manage storage configuration using MongoDB Shell (mongosh), see the page Manage a Federated Database Instance.

Generate Wildcard Collections

Important

The Wildcard collections function is available only for data sources from AWS S3 and Atlas Clusters.

When using Data Sources from AWS S3 or Atlas Clusters, you can dynamically set collection names that map to data. To generate a collection name, specify the wildcard symbol * as the value for the collection name setting in your federated database instance storage configuration.

Tip

You can use the storageSetConfig command to configure the settings for generating wildcard (*) collections.

Back

Advanced User Configuration

Next

Deploy a Data Store

On this page