Add an Azure Blob Storage Private Link Connection

Set up Azure Service Principal Access.

Follow the procedure described in Set Up and Manage Azure Service Principal Access.

Grant your Service Principal the following permissions:

• Storage Blob Data Contributor for your storage account

• Storage Blob Data Contributor for your blob storage containers

• Reader permissions for the Azure Subscription containing both the storage account and blob containers.

Note the Service Principal ID you used for later in this procedure.

Enable Atlas IP Access to Azure.

The Atlas Administration API provides an endpoint to Return All Control Plane IP Addresses.

Note the values of the IP addresses in the outbound.aws.us-east-1 field of the response.

Configure your Azure Storage Account to permit access to as described in the network security page of the Azure Blob Storage documentation, providing the IP addresses, stripped of CIDR notation, that you noted earlier.

Create your Azure private endpoint.

Perform the procedure outlined on the Create a private endpoint page of the Azure documentation. Note the service endpoint ID for later in this procedure.

In Atlas, go to the Users page.

1. If it's not already displayed, select your desired organization from the Organizations menu in the navigation bar.

2. Click All Projects in the sidebar under the Identity & Access section, and select your desired project.

3. Click Project Identity & Access in the sidebar under the Security section.

The Users page displays.

Configure a Service Account.

Create a service account.

Grant this service account the following roles within your Atlas Stream Processing project:

• Project Owner

• Project Stream Processing Owner

Create the Private Link endpoint.

The Atlas Administration API provides an endpoint to Create One Private Link.

For an Azure Blob Storage Private Link connection, set the following key-value pairs:

Use Return All Private Link Connections to check the state of your endpoint. When it reaches a DONE state, note the value of the _id field.

Connect your stream processor to the Azure Private Link endpoint.

The Atlas Administration API provides an endpoint to Create One Stream Connection.

For an Azure Blob Storage Private Link connection, set the following key-value pairs:

Set up Azure Service Principal Access.

Follow the procedure described in Set Up and Manage Azure Service Principal Access.

Grant your Service Principal the following permissions:

• Storage Blob Data Contributor for your storage account

• Storage Blob Data Contributor for your blob storage containers

• Reader permissions for the Azure Subscription containing both the storage account and blob containers.

Note the Service Principal ID you used for later in this procedure.

Enable Atlas IP Access to Azure

Retrieve your Atlas control plane IP addresses.

Note the values of the IP addresses in the outbound.aws.us-east-1 field of the response.

Configure your Azure Storage Account to permit access to as described in the network security page of the Azure Blob Storage documentation, providing the IP addresses, stripped of CIDR notation, that you noted earlier.

Create your Azure private endpoint.

Perform the procedure outlined on the Create a private endpoint page of the Azure documentation. Note the service endpoint ID for later in this procedure.

In Atlas, go to the Users page.

1. If it's not already displayed, select your desired organization from the Organizations menu in the navigation bar.

2. Click All Projects in the sidebar under the Identity & Access section, and select your desired project.

3. Click Project Identity & Access in the sidebar under the Security section.

The Users page displays.

Configure a Service Account

Create a service account.

Grant this service account the following roles within your Atlas Stream Processing project:

• Project Owner

• Project Stream Processing Owner

Create the Private Link endpoint.

To a Private Link endpoint for a stream processor connection using the Atlas CLI, run the following command:

atlas streams privateLinks create [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams privateLinks create.

In your configuration file, set the following key-value pairs:

Use the atlas streams privateLinks list command to check the state of your endpoint. When it reaches a DONE state, note the value of the _id field.

Connect your stream processor to the Azure Private Link endpoint.

To create one connection for the stream processing workspace you specify using the Atlas CLI, run the following command:

atlas streams connections create [connectionName] [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections create.

In your configuration file, set the following key-value pairs:

Set up Azure Service Principal Access.

Follow the procedure described in Set Up and Manage Azure Service Principal Access.

Grant your Service Principal the following permissions:

• Storage Blob Data Contributor for your storage account

• Storage Blob Data Contributor for your blob storage containers

• Reader permissions for the Azure Subscription containing both the storage account and blob containers.

Note the Service Principal ID you used for later in this procedure.

Enable Atlas IP Access to Azure.

The Atlas Administration API provides an endpoint to Return All Control Plane IP Addresses.

Note the values of the IP addresses in the outbound.aws.us-east-1 field of the response.

Configure your Azure Storage Account to permit access to as described in the network security page of the Azure Blob Storage documentation, providing the IP addresses, stripped of CIDR notation, that you noted earlier.

Create your Azure private endpoint.

Perform the procedure outlined on the Create a private endpoint page of the Azure documentation. Note the service endpoint ID for later in this procedure.

In Atlas, go to the Users page.

1. If it's not already displayed, select your desired organization from the Organizations menu in the navigation bar.

2. Click All Projects in the sidebar under the Identity & Access section, and select your desired project.

3. Click Project Identity & Access in the sidebar under the Security section.

The Users page displays.

Navigate to the Atlas Stream Processing private endpoint interface.

1. In the sidebar, click Private Endpoint.

2. Click the Atlas Stream Processing tab.

   If you have not created an Atlas Stream Processing private endpoint previously, click Create endpoint. If you have, click Add ASP Endpoint.

Create the Private Link endpoint.

1. Set Cloud Provider to Azure.

2. Set Vendor to Blob Storage.

3. Click Next, enter service details

4. Provide the Azure service endpoint ID you noted earlier.

5. Provide the Endpoint region in which you created the endpoint. Atlas Stream Processing supports Azure Private Link connections only in the same region as their respective endpoints.

6. Provide the Host name of the Azure Storage Account to which you want to connect.

7. Click Next, generate endpoint ID

In Atlas, go to the Stream Processing page for your project.

1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

3. In the sidebar, click Stream Processing under the Streaming Data heading.

The Stream Processing page displays.

Go to the Connection Registry.

1. Locate the overview panel of the stream processing workspace you want to modify and click Manage.

2. In the sidebar, click Connection Registry.

Connect your stream processor to the Azure Private Link endpoint.

1. Click + Add connection.

2. Select an Azure Blob Storage connection.

3. Provide a Connection Name. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing aggregations.

4. Click the Private Link button.

5. Toggle Enable Private Link Networking on.

6. From the dropdown menu, select the private endpoint you created earlier.

7. From the Azure Service Principal dropdown, select the service principal defined earlier in this procedure.

8. Provide the Storage Account Name for the Azure Storage Account to which you want to connect.

9. Click + Add Connection.