Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector

Kushal_Somkuwar · 2021-08-12T13:35:14.889Z

I am getting the following error

Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}.

Client view of cluster state is {type=REPLICA_SET, servers=[
  {address=mongo01:27017, type=UNKNOWN, state=CONNECTING*},
  {address=mongo02:27017, type=REPLICA_SET_SECONDARY, roundTripTime=2.5 ms, state=CONNECTED},
  {address=mongo03:27017, type=REPLICA_SET_SECONDARY, roundTripTime=1.1 ms, state=CONNECTED}
];

using spring data mongo

its a multitenant environment
change/create the mongoclient connection per request. Database/user/password change as tenant request.

Issue is intermittent and keeps happening randomly.

Ross_Lawley · 2021-08-16T11:50:46.424Z

Hi @Kushal_Somkuwar,

Looking at the logging message - the driver is having trouble connecting to the primary node. The client view of the cluster state shows the state as seen from the Java driver (which spring data uses underneath).

Connecting appears to take longer than the 30s timeout to connect to the mongo01 node. So things to check are the server logs - has there been a change in the replicaset topology - eg new primary and secondaries? Also have there been any networking issues between the application and the mongodb nodes?

Ross

Kushal_Somkuwar · 2021-08-18T11:11:50.851Z

Hi @Ross_Lawley ,

so application connection with mongodb works fine initially but if the data json is big than it become slow for application and start showing the above error.

we have checked the server logs there so no such of connection and I primary node is running fine there is no issue with cluster as well we suspect the same but why would it connect initially if there is a network issue.

Kushal

Ross_Lawley · 2021-08-18T13:27:43.277Z

Hi @Kushal_Somkuwar,

If the driver initially connects then later cannot connect, then that points to some networking issue. It may be intermittent and not happen all the time. But from the drivers perspective it is timing out trying to select a server. The log message just shows you the current view of the servers after selection has failed, which is it is in the process of connecting to the mongo1 node.

Ross

Kushal_Somkuwar · 2021-08-19T10:56:04.306Z

Hi @Ross_Lawley

I also think the same but we are not getting any exception on mongodb server logs and application just get that error. Is there a way to identify what is exactly is the network issue ?

Sandhya_Kripalani · 2022-04-13T07:38:56.821Z

@Kushal_Somkuwar We are facing a similar issue and we are connecting to mongo from Google Cloud.
Did you find out what was the issue? If so, how did you resolve it?

Sandhya_Kripalani · 2022-04-13T07:45:57.848Z

@Ross_Lawley Kindly help. Below is the stack trace
“No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=xxxx, type=UNKNOWN, state=CONNECTING, exception={java.lang.OutOfMemoryError: Java heap space}}, ServerDescription{address=xxx, type=UNKNOWN, state=CONNECTING, exception={java.lang.OutOfMemoryError: Java heap space}}, ServerDescription{address=xxxx, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=1401391508, setName=xxx’, canonicalAddress=xxx, hosts=[xxx], passives=[], arbiters=[], primary=xxxx’, tagSet=TagSet{[Tag{name=‘nodeType’, value=‘ELECTABLE’}, Tag{name=‘provider’, value=‘GCP’}, Tag{name=‘region’, value=‘EASTERN_US’}, Tag{name=‘workloadType’, value=‘OPERATIONAL’}]}, electionId=null, setVersion=6, lastWriteDate=Wed Apr 13 03:33:15 UTC 2022, lastUpdateTimeNanos=6819537012507}]}. Waiting for 30000 ms before timing out”

Ross_Lawley · 2022-04-13T08:01:51.382Z

Hi @Sandhya_Kripalani,

I noticed this java.lang.OutOfMemoryError: Java heap space in the error message.

I’ve never seen that reported before in such a way (as part of the No server chosen exception) but its a sign that your JVM needs more resources.

Ross

Kushal_Somkuwar1 · 2022-04-16T07:17:13.562Z

@Sandhya_Kripalani you will need to check mongoclient and the number of connections it is making with MongoDB it should be on the higher side. This issue comes when your application keep creating mongoclient object and not closing the existing mongoclient connections. Hope that will help

Hassan_EL-MZABI · 2022-05-11T11:06:59.309Z

Hello @Ross_Lawley ,

I’m having the same issue here, but I use reactive mongodb in a spring boot application.

This is my logs messages:

org.springframework.data.mongodb.core.ReactiveMongoTemplate - Streaming aggregation: [{ "$match" : { "type" : { "$in" : ["INACTIVE_SITE", "DEVICE_NOT_BILLED", "NOT_REPLYING_POLLING", "MISSING_KEY_TECH_INFO", "MISSING_SITE", "ACTIVE_CIRCUITS_INACTIVE_RESOURCES", "INCONSISTENT_STATUS_VALUES", "TEST_SUPERVISION_RANGE"]}}}, { "$project" : { "extractionDate" : 1, "_id" : 0}}, { "$group" : { "_id" : null, "result" : { "$max" : "$extractionDate"}}}, { "$project" : { "_id" : 0}}] in collection kpi
org.mongodb.driver.cluster - No server chosen by com.mongodb.reactivestreams.client.internal.ClientSessionHelper$$Lambda$1183/0x000000080129dd60@292878db from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
luster - No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
com.obs.dqsc.interceptor.GlobalExceptionHandler - Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, {address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, {address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]
org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver - Resolved [org.springframework.dao.DataAccessResourceFailureException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, {address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=10.235.79.213:7915, type=UNKNOWN, state=CONNECTING}, {address=10.235.79.89:7915, type=UNKNOWN, state=CONNECTING}]]
org.springframework.web.servlet.DispatcherServlet - Completed 500 INTERNAL_SERVER_ERROR

And this is my mongodb config:

    @Override
    public MongoClient reactiveMongoClient() {
        ConnectionString connectionString = new ConnectionString(
                "mongodb://" + username + ":" + password +
                        "@" + host1 + ":" + port
                        + "," + host2 + ":" + port +
                        "/?authSource=" + authSource +
                        "&replicaSet=" + replica
        );

        MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
                .applyToSslSettings(builder ->
                        builder.enabled(true).invalidHostNameAllowed(true)
                )
                .applyToConnectionPoolSettings(
                        builder -> builder.minSize(10)
                                .maxSize(100)
                                .maxWaitTime(8,TimeUnit.MINUTES)
                )
                .applyToSocketSettings(builder -> builder.applySettings(SocketSettings.builder().connectTimeout(5,TimeUnit.MINUTES).readTimeout(5,TimeUnit.MINUTES).build()))
                .applyConnectionString(connectionString)
                .build();

        return MongoClients.create(mongoClientSettings);
    }

I tried to use different networks to avoid this problem, bit it always gives me the same error.
Could you help me please?

Prateek_Sumermal_Bagrecha · 2022-09-25T00:36:11.802Z

Hi Fellow developers,

I have recently started seeing this issues again as soon as I switched to ReactMongoTemplate.
In my case I am doing multiple find operations as part of flux streams and the connection does not go through after first few subscriptions. It also many times bring the whole cluster down.

Please tell me if someone managed to fix this issue.

sunny_tyagi · 2022-10-13T19:03:44.117Z

Facing similar issue, any solution?

sunny_tyagi · 2022-10-13T19:04:38.370Z

Did you find the root cause as similar issue i am facing.

kevinadi · 2022-10-14T02:38:24.887Z

Hi @sunny_tyagi welcome to the community!

By “similar issue”, I assume you see an error message that looks like this:

Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}.

Is this correct?

If yes, then this is a typical timeout error that is shared across all official drivers, where it tries to connect to the primary member of a replica set, and gives up when it cannot find one after 30 seconds (default). See MongoClient settings serverSelectionTimeout.

Typically this is caused by network issues where the driver cannot reach the servers. It could be caused by network partition, security settings that prevent the client to reach the server (e.g. IP whitelisting issues), DNS issues, blocked port, among many others.

If your app used to be able to connect without issues and now it cannot, then perhaps there is something different now. Things to check may include whether you’re connecting from the same network as before, whether there are DNS changes to the server, whether security settings was changed in the server, or any other network reachability issues that wasn’t there before.

If you have checked everything and all seems to be in order, I would suggest you to consult your network administrator to troubleshoot your connectivity issues.

Just to be complete, if anyone else is encountering this error in the future, it’s best to create a new topic describing the exact error encountered, along with more specific information such as MongoDB version, driver version, topology description, and the complete error message, since everyone’s situation is different and the same error may be the result of two very different causes

Best regards
Kevin

Prateek_Sumermal_Bagrecha · 2023-02-08T13:21:58.621Z

It was fixed my increasing the socket settings to the approximate time taken for cluster to come up again or reelect a leader. Also increase maxSize to like 10, spring could not handle 100 connections adequately. Now process is a bit slow but still more reliable.

Madhan_Kumar · 2023-03-06T08:37:13.794Z

could you please add the code snippet ? i am also facing the same issue

Muhammed_Alghwell · 2023-10-26T14:10:20.550Z

Hi, my configuration for the services that is using the mongodb service, using application.properties, to contact it with the uri mongodb://mongodbservice:27015/dbname. I found in the logs it is calling localhost:27017. Any clue?

jon_rutter · 2024-06-18T14:57:55.953Z

Apologies if this doesn’t help anyone else, I had this error when using Atlas and I just had to the IP address for the server to the Network Access tab.

yuta_takeda · 2025-02-12T15:50:32.949Z

I had the same issue with an AWS EC2 instance where outbound IPs were limited by the security group. Allowing the IP address of following domain resolved the issue.
-00-00.lacss.mongodb.net
-00-01.lacss.mongodb.net"
-00-02.lacss.mongodb.net"

_N_A149 · 2025-03-24T12:56:21.969Z

Hello Team, We do face similar issue, we are working in a docker containers where as the Mongo Client driver connects to Mongo DB, and getting the below stack trace… We able to resolve this issue when restart the application or the containers. This issue seen randomly on any of the containers or some times in all the containers. for any use case restarting the application / containers resolving the issue. We use org.mongodb.Mongo-java_driver 3.12.9 & org.mongodb.driver-async 3.12.9 drivers… below is the trace we see…

Please suggest on the same, what can be the root cause:

Container-s118 2025-03-07 06:57:49,205 [pool-92-thread-1] ERROR audit.log - Storage failed - terminating flow message: {DB-Timed out after 500 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[{address=2606:ae00:XXXX:XXX::14:27022, type=REPLICA_SET_SECONDARY, roundTripTime=23.8 ms, state=CONNECTED}, {address=2606:ae00:XXXX:XXX::14:27022, type=REPLICA_SET_SECONDARY, roundTripTime=14.8 ms, state=CONNECTED}, {address=2606:ae00:XXXX:XXX::13:27022, type=REPLICA_SET_SECONDARY, roundTripTime=21.3 ms, state=CONNECTED}, {address=2606:ae00:XXXX:XXX::13:27022, type=UNKNOWN, state=CONNECTING}]}