MongoDB Cursor Error Spike After Spring Boot Upgrade

_N_A138 · November 11, 2024, 4:13am

Hello, I’m experiencing an issue while using MongoDB with Spring Boot and would appreciate some advice.

Issue Description

After upgrading Spring Boot from version 2.5.8 to 2.7.18, I’ve been seeing a significant increase in MongoDB cursor errors. The specific error message is as follows:

reactor.core.Exceptions$ErrorCallbackNotImplemented: com.mongodb.MongoException: next() called after the cursor was closed.
Caused by: com.mongodb.MongoException: next() called after the cursor was closed.
	at com.mongodb.internal.operation.AsyncQueryBatchCursor.next(AsyncQueryBatchCursor.java:166)
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.MonoCreate] :
	reactor.core.publisher.Mono.create
	com.mongodb.reactivestreams.client.internal.BatchCursor.next(BatchCursor.java:35)
Error has been observed at the following site(s):
	*_______Mono.create ⇢ at com.mongodb.reactivestreams.client.internal.BatchCursor.next(BatchCursor.java:35)
	|_        Mono.from ⇢ at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.recurseCursor(BatchCursorFlux.java:83)
	|_  Mono.doOnCancel ⇢ at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.recurseCursor(BatchCursorFlux.java:84)
	|_   Mono.doOnError ⇢ at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.recurseCursor(BatchCursorFlux.java:85)
	|_ Mono.doOnSuccess ⇢ at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.recurseCursor(BatchCursorFlux.java:92)
Original Stack Trace:
		at com.mongodb.internal.operation.AsyncQueryBatchCursor.next(AsyncQueryBatchCursor.java:166)
		at com.mongodb.reactivestreams.client.internal.BatchCursor.lambda$next$0(BatchCursor.java:35)
		at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:58)
		at reactor.core.publisher.Mono.subscribe(Mono.java:4490)
		at reactor.core.publisher.Mono.subscribeWith(Mono.java:4605)
		at reactor.core.publisher.Mono.subscribe(Mono.java:4318)
		at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.recurseCursor(BatchCursorFlux.java:104)
		at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.lambda$subscribe$0(BatchCursorFlux.java:56)
		at reactor.core.publisher.LambdaMonoSubscriber.onNext(LambdaMonoSubscriber.java:171)
		at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
		at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
		at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
		at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
		at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
		at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180)
		at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:172)
		at com.mongodb.reactivestreams.client.internal.MongoOperationPublisher.lambda$sinkToCallback$30(MongoOperationPublisher.java:549)
		at com.mongodb.reactivestreams.client.internal.OperationExecutorImpl.lambda$execute$2(OperationExecutorImpl.java:92)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.async.function.AsyncCallbackSupplier.lambda$whenComplete$1(AsyncCallbackSupplier.java:95)
		at com.mongodb.internal.async.function.RetryingAsyncCallbackSupplier$RetryingCallback.onResult(RetryingAsyncCallbackSupplier.java:114)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.async.function.AsyncCallbackSupplier.lambda$whenComplete$1(AsyncCallbackSupplier.java:95)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.async.function.AsyncCallbackSupplier.lambda$whenComplete$1(AsyncCallbackSupplier.java:95)
		at com.mongodb.internal.operation.FindOperation$2.onResult(FindOperation.java:792)
		at com.mongodb.internal.operation.CommandOperationHelper.lambda$transformingReadCallback$10(CommandOperationHelper.java:331)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor$2.onResult(DefaultServer.java:276)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.connection.CommandProtocolImpl$1.onResult(CommandProtocolImpl.java:84)
		at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection$1.onResult(DefaultConnectionPool.java:684)
		at com.mongodb.internal.connection.UsageTrackingInternalConnection$2.onResult(UsageTrackingInternalConnection.java:159)
		at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
		at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:521)
		at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:498)
		at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:821)
		at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:785)
		at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
		at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
		at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
		at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
		at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
		at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:160)
		at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:573)
		at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276)
		at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297)
		at com.mongodb.internal.connection.AsynchronousSocketChannelStream$AsynchronousSocketChannelAdapter.read(AsynchronousSocketChannelStream.java:144)
		at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:118)
		at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:107)
		at com.mongodb.internal.connection.InternalStreamConnection.readAsync(InternalStreamConnection.java:642)
		at com.mongodb.internal.connection.InternalStreamConnection.access$600(InternalStreamConnection.java:86)
		at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:775)
		at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:760)
		at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
		at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
		at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
		at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
		at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
		at java.base/sun.nio.ch.Invoker$2.run(Invoker.java:221)
		at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
		at java.base/java.lang.Thread.run(Thread.java:833)

This error used to occur occasionally in the past, but since the upgrade, its frequency has doubled or even tripled. While the system itself is still functioning correctly, the constant occurrence of this error is problematic and cluttering the logs.

Additionally, I previously updated only reactor-core to version 3.5.11, and I experienced a similar increase in the frequency of these cursor errors at that time as well.

Environment Details

MongoDB Server Version: 4.2
Spring Boot Version: 2.7.18 (previously 2.5.8)
MongoDB Driver Version: Using the version compatible with the upgraded Spring Boot (driver was updated alongside the upgrade)
Technologies Used: Spring WebFlux (Reactor), MongoDB Reactive Streams Driver
Dependencies Changed:
- Reactor Core: 3.4.13 → 3.4.34
- Spring Data MongoDB: 3.2.7 → 3.4.18
- MongoDB Driver Core: 4.2.3 → 4.6.1

Actions Taken So Far

MongoDB Driver Upgrade: Updated the MongoDB driver to match the new Spring Boot version.
Using Reactor Hooks: Set up Hooks.onErrorDropped to log errors, but it hasn’t helped in pinpointing the root cause of the increased error frequency.

Questions

Could the increase in cursor closures be due to changes in Spring Boot 2.7.18 affecting MongoDB cursor management?
Are there any known compatibility issues or changes in behavior between Spring Boot 2.7.18 and MongoDB that could lead to this?
What other configurations or approaches would you recommend trying to address this issue?

Although the application itself seems to work fine, the sheer volume of these errors in the logs is concerning. Any suggestions or insights from those who have faced similar issues would be greatly appreciated.

Thank you!