I am using PyMongo. Do I have to close a MongoClient after use?

Marv · 2023-02-16T02:37:52.476Z

I am using PyMongo, and I have some questions about connection and cursor.

Do I have to close a cursor? I want to know because I really want to do something like this.

 for s in collection.find():
       print(s)

Do I have to close a MongoClient manually? Will it affect the connection polling feature of PyMongo?

from pymongo import MongoClient

url = 'mongodb://localhost:27017'
client = MongoClient(url)               # create MongoClient

db = client['test']                     # client.dbname
collection = db['students']             # db.collname

# QUERY
cursor = collection.find()
for s in cursor:
    print(s)

cursor .close()      # Is this required?
client.close()         # Is this required?

Kobe_W · 2023-02-16T05:32:53.259Z

https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html

the document doesn’t say if that is required or not. But generally if a “close” is provided, it’s better to call it when the object is no longer needed. (to save compute resources).

For “client” i suppose it can be kept without closing there until the application terminates (process exits), but for a “cursor” i would suggest to close it when the its use is done.

Shane · 2023-02-16T18:54:07.580Z

Thank you for your question.

@Kobe_W is correct, best practice is to call close() when the client is no longer needed or use the client in a with-statement:

with MongoClient(url) as client:
    # use client here

It’s also a good idea to call close() on the cursor too, the best way would be to use a with-statement like this:

with collection.find() as cursor:
    for s in cursor:
        print(s)

When PyMongo’s MongoClient and cursor objects are garbage collected without being closed pymongo will clean up application-side resources automatically (connections, threads, etc…) however some server-side resources could be left open (cursors, sessions). Calling close() explicitly (or using a with-statement) will clean up these server-side resources that would otherwise be left around until they timeout.

I just opened this ticket to improve our documentation around these ideas: https://jira.mongodb.org/browse/PYTHON-3606

Ole_Jorgen_Bronner · 2023-04-18T17:47:45.313Z

So I understand correctly that exhausting the cursor does not close it? I guess failing to close the cursors could cause problems with “too many open files” in the server? It could explain some issues I’m having.

(guess that makes sense since it’s probably possible to call explain etc. after exhaustion)

Shane · 2023-04-18T20:27:30.553Z

Ole_Jorgen_Bronner:

So I understand correctly that exhausting the cursor does not close it?

Exhausting a cursor (ie fully iterating all the results) does close the cursor automatically. Using the cursor in a with-statement is helpful when the cursor is intentionally or accidentally not fully iterated. For example if an exception is raised in the middle of processing the results the cursor with-statement will cleanup the cursor in a more timely manner:

with collection.find() as cursor:
    for s in cursor:
        print(s)
        raise ValueError('oops')

Swastik_K · 2024-01-17T05:47:17.018Z

@Shane I too have a similar doubt and didn’t find the conclusion from this discussion… so we should not close client everytime but we should close cursor everytime. Is this the conclusion

Shane · 2024-01-17T19:07:04.749Z

Correct, a client should be kept open as long as it is being used and should be closed when the app no longer needs it (or before the application shuts down). This ensures resources like server sessions are cleaned up promptly.

For cursors, this kind of code works:

for doc in collection.find():
    print(doc)

But it is more robust to close the cursor resource explicitly (eg via a with-statement):

with collection.find() as cursor:
    for doc in cursor:
        print(doc)

Rajesh_Golani · 2024-10-15T09:58:19.171Z

Hey recently I ran in to limit of too many open files while using mongo connection.
In pymongo my code looks like this,
self.collection is poiting to a collection.
obj = self.db.find_one(match)
I understand that in find_all we get returned with the cursor, but now in this case of find_one, how does one close the cursor? is it already closed by the pymongo code?

Shane · 2024-10-15T18:42:37.320Z

find_one never leaves a cursor open. Internally it runs the find command with a limit of 1 which causes the server to return the document without opening a cursor.

Shane · 2024-10-15T18:44:21.056Z

Are you hitting “too many open files” in the python application or in the MongoDB server? What’s your ulimit -n output (in the same terminal where you hit the error)?

Rajesh_Golani · 2024-10-17T20:09:29.772Z

@Shane thanks for responding. Actually I did run in to ulimit in the python applicaton and not on the mongodb server. Both are sitting on the same LAN. setting a higher ulimit -n, on terminal where I ws running python application resolved the issue. But this is not a great way to fix thigns and I am wondering if my code base is leaving lot of cursors open.

The code is way to complex for me to give full description. But it is mix of find_one and find.
here is another snippet (this occurs multiple times in the code), Will the below code leave the cursor open or does list method close the cursor. Is there any way for me to find out if cursor is closed.

match = {"price" : {"$gte" : 1 }}
objs = self.collection.find(match)
objs = list(objs)

Shane · 2024-10-17T22:03:30.982Z

Fully iterating the cursor with list() will automatically close the cursor.

Also an open cursor does not consume a file descriptor.

Rajesh_Golani · 2024-10-17T22:52:18.351Z

@Shane
Thanks shane really appreciated