Issue with int64 stored in the field _id

Nikita_Kozelko · 2024-04-26T16:45:15.755Z

Hi,

There is a problem with using the int64 value in the field _id.

Looks like MongoDB ignores the last 2 digits of the int64 number. So it affects searching, adding new documents, etc.

For example, create a document:

{
  "_id": 1233393892850663424
}

After that please try to create the next document:
``
{
“_id”: 1233393892850663426
}
`

It doesn't allow to add a new document because new id isn't unique. But it's not the same as previous.
As far as I see in the documentation, _id can contain any valid Bson data type. So there it shouldn't be an issue to work with int64. 

Used version of MongoDB: 6.0.

Could you please help with understand the problem of MongoDb?
Thanks.

chris · 2024-04-27T15:45:40.852Z

Hi @Nikita_Kozelko

This will be the language or interpreter losing precision of the number before it is inserted to mongodb.

For example with mongosh (a javascript REPL environment) the default number type is a double, once this number exceeds the MAX_SAFE_INTEGER value precision is lost and duplicate numbers are a result of that. Instead native(or bson library provided) types need to be used.

// javascript default number
new Array(10).fill().map((x,i)=>{return {_id: Number.MAX_SAFE_INTEGER + i}})
[
  { _id: 9007199254740991 },
  { _id: 9007199254740992 },
  { _id: 9007199254740992 },
  { _id: 9007199254740994 },
  { _id: 9007199254740996 },
  { _id: 9007199254740996 },
  { _id: 9007199254740996 },
  { _id: 9007199254740998 },
  { _id: 9007199254741000 },
  { _id: 9007199254741000 }
]

//javascript Long
new Array(10).fill().map((x,i)=>{return {_id: Long(Number.MAX_SAFE_INTEGER.toString()).add(i)}})
[
  { _id: Long('9007199254740991') },
  { _id: Long('9007199254740992') },
  { _id: Long('9007199254740993') },
  { _id: Long('9007199254740994') },
  { _id: Long('9007199254740995') },
  { _id: Long('9007199254740996') },
  { _id: Long('9007199254740997') },
  { _id: Long('9007199254740998') },
  { _id: Long('9007199254740999') },
  { _id: Long('9007199254741000') }
]

This example show a sequence inserted by python starting at the max signed int32 value and the result of reading this in both python and mongosh(javascript).

#!/usr/bin/env python3
import os
from pprint import pprint
import pymongo

maxint32=0xFFFFFFFF >> 1

client = pymongo.MongoClient(os.getenv('MONGODB_URI')) 

client.test.foo.drop()

for i in range(10):
  client.test.foo.insert_one({ '_id': maxint32 + i })

pprint([ _ for _ in client.test.foo.find() ])

> python3 test
[{'_id': 2147483647},
 {'_id': 2147483648},
 {'_id': 2147483649},
 {'_id': 2147483650},
 {'_id': 2147483651},
 {'_id': 2147483652},
 {'_id': 2147483653},
 {'_id': 2147483654},
 {'_id': 2147483655},
 {'_id': 2147483656}]

Notice the change from ‘Number’ to Long.

mongosh ${MONGODB_URI} --eval 'db.foo.find()'
[
  { _id: 2147483647 },
  { _id: Long('2147483648') },
  { _id: Long('2147483649') },
  { _id: Long('2147483650') },
  { _id: Long('2147483651') },
  { _id: Long('2147483652') },
  { _id: Long('2147483653') },
  { _id: Long('2147483654') },
  { _id: Long('2147483655') },
  { _id: Long('2147483656') }
]

Nikita_Kozelko · 2024-04-27T17:50:28.426Z

Hi,
Looks like there is a problem in the compass. The right query for the compass is:

{_id: Long("1233393892850663424")}

This issue was reported to the MongoDB’s jira: https://jira.mongodb.org/browse/COMPASS-7891.
Thanks.