AWS Bedrock Knowledge Bases + MongoDB Atlas Failure

Brandon_Axtmann · 2024-06-05T10:47:44.162Z

Hello,
I’m trying to setup AWS Bedrock Knowledge Bases with MongoDB Atlas, however I’m encountering an error.

I’ve been trying to follow this guide:

Launch a Fully Managed RAG Workflow With MongoDB Atlas and Amazon Bedrock |...

Atlas Vector Search and Amazon Bedrock enable the vector database capabilities of Atlas to bring up-to-date context to Foundational Model outputs.

The MongoDB cluster I have is an M10 cluster, in AWS ca-central-1. For simplicity, I temporarily added 0.0.0.0/0 to the network rules, since I’m just testing for now. I’ve validated that the username/password is correct.

What I’m seeing is that I can create (hit the create button in bedrock) for the KB, AWS will say it’s creating, but then after about 10 minutes, the stats changes from In Progress to failed, with no errors on the AWS side. I’ve checked everywhere I can think, but I see no hints. I’ve tried this multiple times over the past few days.

I’ve tried both creating the KB via the console and the CLI. I was able to create a KB via the CLI with AWS Aurora Postgres, (but I did get console errors doing that, so I had to use that CLI… but that’s not relevant here).

I’ve also checked in cloudtrail and I can see events showing that AWS is creating the KB, but I do not see any events after that, indicating completion or an error.

Here is what my CLI command looks like to create the KB for mongo:

aws bedrock-agent create-knowledge-base \
    --name "security-survey-test-2-mongo" \
    --knowledge-base-configuration '{
        "type": "VECTOR",
        "vectorKnowledgeBaseConfiguration": {
            "embeddingModelArn": "arn:aws:bedrock:us-west-2::foundation-model/amazon.titan-embed-text-v2:0"
        }
    }' \
    --role-arn "arn:aws:iam::123456789:role/service-role/AmazonBedrockExecutionRoleForKnowledgeBase_pzmvo" \
    --storage-configuration '{
        "mongoDbAtlasConfiguration": {
            "collectionName": "ai-security-surveys-bedrock",
            "credentialsSecretArn": "arn:aws:secretsmanager:us-west-2:123456789:secret:mongodb/it-ops-prod/bedrock-w1I6Jg",
            "databaseName": "bedrock",
            "endpoint": "it-ops-prod-mongo.ABCDE.mongodb.net",
            "fieldMapping": {
                "metadataField": "bedrock_metadata",
                "textField": "bedrock_text_chunk",
                "vectorField": "bedrock_embedding"
            },
            "vectorIndexName": "bedrock_index"
        },
        "type": "MONGO_DB_ATLAS"
    }' \

In AWS secrets manager, I have two values, username and password. The Role has full-admin permissions, in addition to the bedrock perms (this is in a sandboxed account for testing).

The dataset for now, is very simple. And with that CLI command, it’s not even creating the dataset (which is possible, I validated that with I did the Aurora test), so it’s most likely not a problem with the data set or the s3 bucket.

This is what the search index looks like in Atlas:

image1688×697 31.8 KB

And this is what the KB looks like in AWS bedrock console

screencapture-us-west-2-console-aws-amazon-bedrock-home-2024-06-04-18_22_11 (1)1920×1701 290 KB

I’m just not sure what the problem is at this point, has anyone else encountered this? Please let me know if there are any questions!

Thanks,

Brandon

Prakul_Agarwal · 2024-06-05T21:23:05.224Z

Hi @Brandon_Axtmann - MongoDB Atlas vector store in Knowledge Bases for Amazon Bedrock is currently available in the US East (N. Virginia) and US West (Oregon) Regions. Can you please try in one of those regions?
Let me check on why the proper error is not showing up.

Brandon_Axtmann · 2024-06-06T11:31:37.475Z

Hi Prakul,

In AWS, the KB is in us-west-2 (Oregon) region already. Does my MongoDB database need to be in one of those regions as well? It’s currently in AWS MTL (on the Atlas side of things)

Thanks