1.26.2 libmongoc driver crashing

Surendra_Paddala · 2024-05-31T02:20:35.923Z

Hi All, Recently I deployed application using libmongoc driver version 1.26.2. I see there is crash.

especially while doing the find. Here is the GDB trace. Any leads appreciated.

as per the GDB, its failing at assertion in the below file and line
mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mcd-rpc.c
uint32_t
mcd_rpc_op_msg_get_flag_bits (const mcd_rpc_message *rpc)
{
ASSERT_MCD_RPC_ACCESSOR_PRECONDITIONS;
BSON_ASSERT (rpc->msg_header.op_code == MONGOC_OP_CODE_MSG);
return rpc->op_msg.flag_bits;
}

Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:50
50 …/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007f6a6f534537 in __GI_abort () at abort.c:79
#2 0x00007f6a6d3d263f in mcd_rpc_op_msg_get_flag_bits (rpc=rpc@entry=0x560b9cef1cd0) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mcd-rpc.c:2156
#3 0x00007f6a6d3e5d9a in _mongoc_cluster_run_opmsg_recv (cluster=cluster@entry=0x560b9cfcd498, cmd=cmd@entry=0x7fffd71be160, rpc=rpc@entry=0x560b9cef1cd0, reply=reply@entry=0x560b9cfd7770,
error=error@entry=0x560b9cfdb260) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cluster.c:3716
#4 0x00007f6a6d3e6006 in mongoc_cluster_run_opmsg (cluster=cluster@entry=0x560b9cfcd498, cmd=cmd@entry=0x7fffd71be160, reply=reply@entry=0x560b9cfd7770, error=error@entry=0x560b9cfdb260)
at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cluster.c:3795
#5 0x00007f6a6d3e6643 in mongoc_cluster_run_command_monitored (cluster=0x560b9cfcd498, cmd=cmd@entry=0x7fffd71be160, reply=reply@entry=0x560b9cfd7770, error=error@entry=0x560b9cfdb260)
at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cluster.c:611
#6 0x00007f6a6d3f62ec in _mongoc_cursor_run_command (cursor=cursor@entry=0x560b9cfdb180, command=command@entry=0x7fffd71be470, opts=opts@entry=0x560b9cfdb1a0, reply=reply@entry=0x560b9cfd7770,
retry_prohibited=retry_prohibited@entry=false) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cursor.c:1090
#7 0x00007f6a6d3f8c22 in _mongoc_cursor_response_refresh (cursor=cursor@entry=0x560b9cfdb180, command=command@entry=0x7fffd71be470, opts=opts@entry=0x560b9cfdb1a0, response=response@entry=0x560b9cfd7770)
at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cursor.c:1747
#8 0x00007f6a6d3f9e49 in _prime (cursor=0x560b9cfdb180) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cursor-find-cmd.c:36
#9 0x00007f6a6d3f696d in _call_transition (cursor=0x560b9cfdb180) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cursor.c:1256
#10 mongoc_cursor_next (cursor=0x560b9cfdb180, bson=0x7fffd71be740) at /home/surendra.paddala/mongo-c-driver-1.26.2/src/libmongoc/src/mongoc/mongoc-cursor.c:1332
#11 0x00007f6a6d4fc198 in db_mongodb_store_result (h=0x7f6a6d58a480, r=0x7fffd71be9e0) at mongodb_dbase.c:859
#12 0x00007f6a6d4ff77f in db_mongodb_query (h=0x7f6a6d58a480, k=0x7fffd71be9d0, op=0x0, v=0x7fffd71be990, c=0x7fffd71be9f0, n=2, nc=1, o=0x0, _r=0x7fffd71be9e0) at mongodb_dbase.c:1073
#13 0x00007f6a6ab60f30 in get_session_data (session_id=0x0, framed_ip_address=0x7f666cf92078) at pcc_data_load.c:3521
#14 0x00007f6a6aa54960 in get_gx_session_from_db (session_id=0x0, framed_ip_address=0x7f666cf92078) at gx_controller_hash.c:154
#15 0x00007f6a6aa66abb in is_gx_exist (framed_ip_address=0x7f666cf92078, session_id=0x7fffd71bee30) at gx_controller_hash.c:1133
#16 0x00007f6a6aa26614 in process_rx_queue_message (msg=0x7f666cf91f50) at rx_controller_queue.c:101
#17 0x00007f6a6aa2ad71 in rx_queue_process (id=133) at rx_controller_queue.c:402
#18 0x00007f6a6aa02978 in child_init (rank=0) at mod.c:200
#19 0x0000560b9ac3d052 in init_mod_child (m=0x7f6a6d577c50, rank=0) at core/sr_module.c:780
#20 0x0000560b9ac3ccd0 in init_mod_child (m=0x7f6a6d577ee0, rank=0) at core/sr_module.c:776
#21 0x0000560b9ac3ccd0 in init_mod_child (m=0x7f6a6d578300, rank=0) at core/sr_module.c:776
#22 0x0000560b9ac3d817 in init_child (rank=0) at core/sr_module.c:825
#23 0x0000560b9aa655ed in main_loop () at main.c:1753
#24 0x0000560b9aa6d79a in main (argc=9, argv=0x7fffd71bf848) at main.c:2800

Rishabh_Bisht · 2024-05-31T09:09:07.095Z

Hi @Surendra_Paddala

Thanks for reporting. Is the crash consistently reproducible? Can you also share the MongoDB Server version and Operating System?

Surendra_Paddala · 2024-06-04T15:59:31.893Z

we are using mongo DB server version 6.X

Surendra_Paddala · 2024-06-05T03:04:35.599Z

Crash is able to reproduce and operating system us debian 11. mongo db version 6,x

Rishabh_Bisht · 2024-06-05T08:12:03.040Z

Thanks for sharing. Would you be able to share the minimal steps/code to reproduce this issue so that one of our engineers can investigate this?

I have logged CDRIVER-5584 for the investigation.

Surendra_Paddala · 2024-06-13T09:10:05.080Z

Sure, firstly if the mongo DB server is dedicated, we really see no issues. When application get connected to cluster where replication enabled between two sites. Shard key based. On this server we experience the issue. Data is very simple. we have two collections. Store it and find using a key.
write operations are no issue. Only with read(find query)