Error performing an async for on an AgnosticCursor with _get_more method

Robert_DiFalco · 2023-05-05T16:12:24.735Z

Hey folks, I’m using motor for the first time, version 2.5.1. I have a twisted app that is using the asyncio reactor. My tests uses the pytest_twisted plugin with --reactor asyncio. I’m able to reproduce the error with the following code.

async def print_results():
    client = AsyncIOMotorClient(test_db_uri)
    db = client.get_default_database()
    col = db.get_collection("user")
    async for doc in col.find({}):
        print(doc)


@defer.inlineCallbacks
def test_motor():
    createSomeUsersWithMongoEngine()
    yield defer.ensureDeferred(print_results())

The error I get is:

  File "/usr/local/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks
    result = context.run(gen.send, result)
  File "test/test_reporting.py", line 318, in print_results
    async for doc in col.find({}):
  File "/usr/local/lib/python3.10/site-packages/motor/core.py", line 1158, in next
    if self.alive and (self._buffer_size() or await self._get_more()):
RuntimeError: await wasn't used with future

It’s almost as if _get_more is not returning a future. Any ideas? This is using Python 3.10.11, twisted 22.10.0, and PyMongo 3.12.0.

Thanks in advance for any pointers.

Steve_Silvester · 2023-05-08T14:51:03.653Z

Hi @Robert_DiFalco, I believe the error you are seeing is the same as python - RuntimeError: await wasn't used with future when using twisted, pytest_twisted plugin, and asyncio reactor - Stack Overflow, and is related to pytest_twisted itself. The following pure-asyncio code runs with pytest without error:

import asyncio
from motor.motor_asyncio import AsyncIOMotorClient


test_db_uri = 'mongodb://localhost/test'

async def print_results():
    client = AsyncIOMotorClient(test_db_uri)
    db = client.get_default_database()
    col = db.get_collection("test")
    async for doc in col.find({}):
        print(doc)


def test_this():
    asyncio.run(print_results())

Robert_DiFalco · 2023-05-11T06:20:10.576Z

Yes turns out you are right! Nothing to do with Motor at all. I have to turn the Motor future into a Deferred and then it works fine.

async def print_results():
    client = AsyncIOMotorClient(test_db_uri)
    db = client.get_default_database()
    col = db.get_collection("user")
    async for doc in col.find({}):
        print(doc)


@defer.inlineCallbacks
def test_motor():
    createSomeUsersWithMongoEngine()
    yield Deferred.fromFuture(print_results())

Well, one thing I learned in this process is that Motor is actually not an asyncio driver for MongoDB. It’s actually just a wrapper on sync sockets (a wrapper of PyMongo actually) that simply runs the IO bound methods in a ThreadExecutorPool. Kind of disappointing. I was using txmongo that while old and dusty at least is a real async socket driver implementation for Mongo. I wonder if Mongo will ever have a modern trully async driver. I guess I can hope!

system · 2023-05-16T06:20:19.291Z

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.