I am using a python 3.10 and pymongo.
We are using ddd.
We are handling saga transactions using unit of work with inbox and outbox transactional box.
We are using mongodb to save the aggregates of the system For each aggregate we have its own collection and we have collection for inbox and outbox.
We use mongodb session to handle transactions using db context and unit of work.
For each aggregate and for the inbox and outbox we have the same db_context.
Because we want to solve the “duel write” problem in outbox and update the state of the message in inbox according to if the transaction on the aggregates has been successful or not.
(Atomacity and consistency in ACID concepts)
If anything fails we want to abort the transaction.
For handling the events in the inbox, we are using a field called message_is_processed which can be true or false.
We are filtering all the messages whos message_is_processed is false and handle each one.
For the mongodb session we use casual consistency and read and write concern - “majority”.
Our problem is that we use horizontal scale on our application that contain the inbox and outbox.
Because each kubernetes pod uses the same mongodb, multiple pods can read specific message from the inbox before a pod finish the transaction and mark the message as processed.
Which leads to multiple logic flows which result in multiple different outbox messages.
So the ratio of inbox and outbox messages can be 1 to many,
and we want to make it 1 to 1.
I really need you help to solve this issue.