Multi-Document ACID Transactions in MongoDB with Go
Rate this quickstart
The past few months have been an adventure when it comes to getting started with MongoDB using the Go programming language (Golang). We've explored everything from create, retrieve, update, and delete (CRUD) operations, to data modeling, and to change streams. To bring this series to a solid finish, we're going to take a look at a popular requirement that a lot of organizations need, and that requirement is transactions.
So why would you want transactions?
There are some situations where you might need atomicity of reads and writes to multiple documents within a single collection or multiple collections. This isn't always a necessity, but in some cases, it might be.
Take the following for example.
Let's say you want to create documents in one collection that depend on documents in another collection existing. Or let's say you have schema validation rules in place on your collection. In the scenario that you're trying to create documents and the related document doesn't exist or your schema validation rules fail, you don't want the operation to proceed. Instead, you'd probably want to roll back to before it happened.
There are other reasons that you might use transactions, but you can use your imagination for those.
In this tutorial, we're going to look at what it takes to use transactions with Golang and MongoDB. Our example will rely more on schema validation rules passing, but it isn't a limitation.
Since we've continued the same theme throughout the series, I think it'd be a good idea to have a refresher on the data model that we'll be using for this example.
In the past few tutorials, we've explored working with potential podcast data in various collections. For example, our Go data model looks something like this:
1 type Episode struct { 2 ID primitive.ObjectID `bson:"_id,omitempty"` 3 Podcast primitive.ObjectID `bson:"podcast,omitempty"` 4 Title string `bson:"title,omitempty"` 5 Description string `bson:"description,omitempty"` 6 Duration int32 `bson:"duration,omitempty"` 7 }
The fields in the data structure are mapped to MongoDB document fields through the BSON annotations. You can learn more about using these annotations in the previous tutorial I wrote on the subject.
While we had other collections, we're going to focus strictly on the
episodes collection for this example.Rather than coming up with complicated code for this example to demonstrate operations that fail or should be rolled back, we're going to go with schema validation to force fail some operations. Let's assume that no episode should be less than two minutes in duration, otherwise it is not valid. Rather than implementing this, we can use features baked into MongoDB.
Take the following schema validation logic:
1 { 2 "$jsonSchema": { 3 "additionalProperties": true, 4 "properties": { 5 "duration": { 6 "bsonType": "int", 7 "minimum": 2 8 } 9 } 10 } 11 }
The above logic would be applied using the MongoDB CLI or with Compass, but we're essentially saying that our schema for the
episodes collection can contain any fields in a document, but the duration field must be an integer and it must be at least two. Could our schema validation be more complex? Absolutely, but we're all about simplicity in this example. If you want to learn more about schema validation, check out this awesome tutorial on the subject.Now that we know the schema and what will cause a failure, we can start implementing some transaction code that will commit or roll back changes.
Before we dive into starting a session for our operations and committing transactions, let's establish a base point in our project. Let's assume that your project has the following boilerplate MongoDB with Go code:
1 package main 2 3 import ( 4 "context" 5 "fmt" 6 "os" 7 8 "go.mongodb.org/mongo-driver/bson/primitive" 9 "go.mongodb.org/mongo-driver/mongo" 10 "go.mongodb.org/mongo-driver/mongo/options" 11 ) 12 13 // Episode represents the schema for the "Episodes" collection 14 type Episode struct { 15 ID primitive.ObjectID `bson:"_id,omitempty"` 16 Podcast primitive.ObjectID `bson:"podcast,omitempty"` 17 Title string `bson:"title,omitempty"` 18 Description string `bson:"description,omitempty"` 19 Duration int32 `bson:"duration,omitempty"` 20 } 21 22 func main() { 23 client, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(os.Getenv("ATLAS_URI"))) 24 if err != nil { 25 panic(err) 26 } 27 defer client.Disconnect(context.TODO()) 28 29 database := client.Database("quickstart") 30 episodesCollection := database.Collection("episodes") 31 32 database.RunCommand(context.TODO(), bson.D{{"create", "episodes"}}) 33 }
The collection must exist prior to working with transactions. When using the
RunCommand, if the collection already exists, an error will be returned. For this example, the error is not important to us since we just want the collection to exist, even if that means creating it.Now let's assume that you've correctly included the MongoDB Go driver as seen in a previous tutorial titled, How to Get Connected to Your MongoDB Cluster with Go.
The goal here will be to try to insert a document that complies with our schema validation as well as a document that doesn't so that we have a commit that doesn't happen.
1 // ... 2 3 func main() { 4 // ... 5 6 wc := writeconcern.New(writeconcern.WMajority()) 7 rc := readconcern.Snapshot() 8 txnOpts := options.Transaction().SetWriteConcern(wc).SetReadConcern(rc) 9 10 session, err := client.StartSession() 11 if err != nil { 12 panic(err) 13 } 14 defer session.EndSession(context.Background()) 15 16 err = mongo.WithSession(context.Background(), session, func(sessionContext mongo.SessionContext) error { 17 if err = session.StartTransaction(txnOpts); err != nil { 18 return err 19 } 20 result, err := episodesCollection.InsertOne( 21 sessionContext, 22 Episode{ 23 Title: "A Transaction Episode for the Ages", 24 Duration: 15, 25 }, 26 ) 27 if err != nil { 28 return err 29 } 30 fmt.Println(result.InsertedID) 31 result, err = episodesCollection.InsertOne( 32 sessionContext, 33 Episode{ 34 Title: "Transactions for All", 35 Duration: 1, 36 }, 37 ) 38 if err != nil { 39 return err 40 } 41 if err = session.CommitTransaction(sessionContext); err != nil { 42 return err 43 } 44 fmt.Println(result.InsertedID) 45 return nil 46 }) 47 if err != nil { 48 if abortErr := session.AbortTransaction(context.Background()); abortErr != nil { 49 panic(abortErr) 50 } 51 panic(err) 52 } 53 }
In the above code, we start by defining the read and write concerns that will give us the desired level of isolation in our transaction. To learn more about the available read and write concerns, check out the documentation.
After defining the transaction options, we start a session which will encapsulate everything we want to do with atomicity. After, we start a transaction that we'll use to commit everything in the session.
A
Session represents a MongoDB logical session and can be used to enable casual consistency for a group of operations or to execute operations in an ACID transaction. More information on how they work in Go can be found in the documentation.Inside the session, we are doing two
InsertOne operations. The first would succeed because it doesn't violate any of our schema validation rules. It will even print out an object id when it's done. However, the second operation will fail because it is less than two minutes. The CommitTransaction won't ever succeed because of the error that the second operation created. When the WithSession function returns the error that we created, the transaction is aborted using the AbortTransaction function. For this reason, neither of the InsertOne operations will show up in the database.Starting and committing transactions from within a logical session isn't the only way to work with ACID transactions using Golang and MongoDB. Instead, we can use what might be thought of as a more convenient transactions API.
Take the following adjustments to our code:
1 // ... 2 3 func main() { 4 // ... 5 6 wc := writeconcern.New(writeconcern.WMajority()) 7 rc := readconcern.Snapshot() 8 txnOpts := options.Transaction().SetWriteConcern(wc).SetReadConcern(rc) 9 10 session, err := client.StartSession() 11 if err != nil { 12 panic(err) 13 } 14 defer session.EndSession(context.Background()) 15 16 callback := func(sessionContext mongo.SessionContext) (interface{}, error) { 17 result, err := episodesCollection.InsertOne( 18 sessionContext, 19 Episode{ 20 Title: "A Transaction Episode for the Ages", 21 Duration: 15, 22 }, 23 ) 24 if err != nil { 25 return nil, err 26 } 27 result, err = episodesCollection.InsertOne( 28 sessionContext, 29 Episode{ 30 Title: "Transactions for All", 31 Duration: 2, 32 }, 33 ) 34 if err != nil { 35 return nil, err 36 } 37 return result, err 38 } 39 40 _, err = session.WithTransaction(context.Background(), callback, txnOpts) 41 if err != nil { 42 panic(err) 43 } 44 }
Instead of using
WithSession, we are now using WithTransaction, which handles starting a transaction, executing some application code, and then committing or aborting the transaction based on the success of that application code. Not only that, but retries can happen for specific errors if certain operations fail.You just saw how to use transactions with the MongoDB Go driver. While in this example we used schema validation to determine if a commit operation succeeds or fails, you could easily apply your own application logic within the scope of the session.
If you want to catch up on other tutorials in the getting started with Golang series, you can find some below:
Since transactions brings this tutorial series to a close, make sure you keep a lookout for more tutorials that focus on more niche and interesting topics that apply everything that was taught while getting started.