Building invincible applications with Temporal and MongoDB
Rate this tutorial
The past 15 years have transformed how we approach software. Cloud providers enable us to provision infrastructure across the globe—on demand—and scale it as our needs change. Microservices give us greater flexibility for implementing, updating, and scaling each component. Modern applications rely on interconnected services, often spanning different networks, regions, or even organizations. Modern applications are distributed systems.
These applications are more scalable than their monolithic counterparts but have more potential points of failure. Consider the basic e-commerce application depicted above, which relies on three services, each with its own database. There’s an inventory service that provides information about available products, a payment service that charges customers for purchases, and a notification service that sends e-mail updating customers on the status of their orders.
The application depends on the availability of all three services, but they may become unresponsive due to heavy load. They could also suffer a network outage, undergo system maintenance, or lose their database connection.
Sometimes, applications experience network unreliability on their own side of the connection. Worse yet, it might crash due to a software bug or hardware failure. If that occurs in the middle of order processing, the items might be removed from inventory but the customer is never charged. If the application is restarted, the customer might receive duplicate confirmation e-mails or be charged twice for a single purchase. Perhaps you’ve experienced this as a customer.
In this tutorial, you’re going to learn how to prevent these types of problems by using an open source platform called Temporal that enables applications to withstand service outages and application crashes. Although you can’t prevent those problems from occurring, Temporal handles them automatically, so you can code as if they never happen.
Temporal delivers Durable Execution, an abstraction that guarantees your application will continue running despite adverse conditions. It abstracts away the complexity of building scalable distributed systems, which in turn reduces the amount of code you need to develop, debug, and maintain.
The Temporal platform consists of two parts, both of which are open source. First is a library, known as a Temporal SDK, which provides the APIs and implementations needed to write Temporal applications in a specific programming language. Temporal currently offers SDKs for Java, Go, TypeScript, Python, PHP, and .NET. Second is server software, which tracks the progress and state of your applications. If your application crashes, this history data makes it possible to reconstruct the pre-crash state in a new process, and then resume execution as if it had never crashed at all. Your application can even overcome hardware failure, because this recovery process can take place on a different machine.
Temporal traces its origins to an earlier system that its creators built while working as software engineers at Uber, which was itself based on large-scale distributed systems they previously built at Amazon and Microsoft. Today, thousands of companies use Temporal in production, with a diverse range of use cases that includes AI model training, order management, infrastructure provisioning, reservation booking, and financial transaction processing.
Imagine that Maria’s favorite band is playing a concert in her city. Unfortunately, tickets went on sale when Maria was delivering an important presentation to the CEO. Luckily, her friend David texts her to explain that he has a spare front-row seat. Maria pulls out her phone and uses an app to transfer $100 for the ticket from her account at Bank A to David’s account at Bank B.
From the app user’s perspective, moving money between two banks is a single operation. In reality, it involves two steps: withdrawing the money from Maria’s account and then depositing it into David’s account. Both steps must happen and they both must happen exactly once.
This underscores the need for the application to be resilient. As with the e-commerce example described above, it depends on the availability of both banks, yet either could experience an outage for any number of reasons. An application crash could have serious consequences.
For example, suppose the power goes out between the withdrawal and deposit steps. In this case, the account balances will be off by $100 since the money was removed from Maria’s account but never deposited into David’s. Since terminating an application clears its state, meaning that the program has no memory of what it did before, restarting the application won’t fix it. If the money transfer runs to completion after being restarted, it will withdraw another $100 from Maria’s account before depositing $100 into David’s account. The original $100 is still missing!
During this tutorial, you’ll experiment with a money transfer application implemented with Temporal. You’ll see for yourself how it’s able to overcome a service outage and even a crash of the application itself. You’ll also see how to support a human-in-the-loop scenario, in which the transfer awaits manual approval by a manager before proceeding.
In addition to the Temporal application code, we provide code to simulate the banking services (although for the sake of simplicity, this is done using a single service and database). We provide a GUI that shows the current account balances and controls whether the service is available for a given account, allowing you to create an outage on demand.
- Java 11 or higher
- Get started with MongoDB Atlas for free! Use the MongoDB M0 cluster and follow along.
- Temporal CLI, a command-line tool for using Temporal
The main building block of a Temporal application is called a Workflow, which is a function or method that has the benefit of Durable Execution. Once started, a Workflow will continue its execution until it either succeeds or you choose to end it.
Unlike Workflow engines that favor a low-code or no-code approach, which inevitably disappoint and constrain the developer, a Temporal Workflow is defined by writing code using any programming language supported by one of the Temporal SDKs. This Workflows-as-code approach means that you can write, test, compile, and package a Temporal application using the same tools you already use.
Temporal places few restrictions on how you structure or implement your code. There is one notable exception: The Workflow must be deterministic. This means that, for a given input, it must do the same thing every time. For example, if you create a Workflow that returns the sum of two numbers, then passing it
5 and 7 as input must always return the value 12. Operations involving random numbers are an obvious source of non-deterministic behavior, but interacting with external systems is also non-deterministic because the result depends on the availability and behavior of that external system at a given moment in time. You can still perform non-deterministic operations in a Temporal application; you just need to put that code in an Activity.Activities encapsulate code that is prone to failure. If an Activity fails, it is automatically retried according to a customizable policy. These retries enable a Temporal application to withstand service outages and other transient or intermittent failures. When the failure is resolved—regardless of how long it takes— the next retry will succeed and the application will continue on as if the failure never occurred. Temporal also provides built-in support for heartbeats and timeouts, enabling rapid failure detection even in long-running operations.
As explained earlier, Temporal has a client-side library (the Temporal SDK) and server software. The Temporal Service is started by running that server software, regardless of where it happens to be running. It has two primary roles. The first is to maintain Task Queues used to schedule work to be done and the second is to maintain a history of events for work already completed.
Workers are responsible for executing your Workflow and Activity code. It uses a Temporal Client to communicate with the Temporal Service, polling a Task Queue to find out what it should do next. Each Task contains details about a Workflow or Activity to execute, along with the data to provide as input.
After a Worker runs the code corresponding to the Task, it reports the outcome back to the Temporal Service, which records the details in the Event History. If this Task was for an Activity that failed or timed out, the Temporal Service will schedule another Task for the retry. Otherwise, it will continue by adding new Tasks until the Workflow Execution is complete.
This is a command-line tool for interacting with Temporal. It also provides a lightweight version of the Temporal Service that’s handy for local development, which you will use for this tutorial.
This diagram illustrates how all of the above relate to one another. The dashed vertical line indicates the separation between the Worker and Temporal Service. In this tutorial, you’ll run both on the same machine. In a production system, the Workers and Temporal Service are typically deployed to different systems, just as a production web application will have its web servers and database servers deployed to different systems. A production system will have multiple workers, often hundreds or thousands of them, since this increases the application’s availability and throughput.
While Temporal orchestrates the Workflow and Activities, MongoDB will serve as the persistent storage layer for the application, maintaining our consistent and reliable data management. MongoDB stores all account information, including account IDs, names, and current balances. By keeping this data in MongoDB, the application ensures that Workflows always have access to accurate account information, even during unexpected outages.
MongoDB complements Temporal by providing:
- Reliable data persistence: MongoDB stores all account information, ensuring Temporal Workflows always have access to up-to-date data, even after restarts or failures.
- Scalability: MongoDB’s ability to scale out across multiple servers allows the application to handle growing amounts of account data and transaction activity without impacting performance.
- Resilience to outages: MongoDB’s replication keeps data available during outages, allowing workflows to continue without interruption.
Temporal comes with a web-based interface for viewing and interacting with Workflow Executions. It gives you visibility into what’s running right now and what has previously run. The Web UI provides a convenient way to view the Event History for each Workflow Execution, allowing you to see the input, current status, and output of your Workflow, as well as those same details about each Activity executed as part of that Workflow. During this tutorial, you’ll use this tool to confirm and resolve an outage with the banking service.
Run the following command to fetch a copy of the code used in this tutorial:
1 git clone https://github.com/mongodb-developer/mongodb-java-money-transfer
This repository you cloned has two subdirectories. Each is a separate project with its own codebase and Maven build configuration file. The
bank-services subdirectory contains the code that manages the bank accounts, offers an HTTP API for performing operations on those accounts, and provides a GUI for viewing and controlling the status of each account. The money-transfer-application directory contains the Temporal application that transfers money between bank accounts by using an HTTP client to invoke withdrawal and deposit operations through the bank services API. This tutorial focuses on the money transfer application code.In the Temporal Java SDK, a Workflow is defined using an interface, which is identified by a
@WorkflowInterface annotation. The developer then uses the @WorkflowMethod annotation to specify which of its methods to run when the Workflow is executed. In this example, the Workflow interface specifies a Workflow method called transfer that accepts an object containing details (such as the sender’s account, recipient’s account, and amount) of the transfer.For brevity, the code that follows omits import statements, log statements, code comments, and other elements that may distract from the explanation. You can ignore the
approve method for now, since this is covered later in the tutorial.1 package org.mongodb.workflows; 2 3 4 public interface MoneyTransferWorkflow { 5 6 7 String transfer(TransferDetails input); 8 9 10 void approve(String managerName); 11 }
The interface also has a corresponding implementation class, which overrides those methods, providing the code to run during Workflow Execution.
Since that implementation references Activities that perform the withdrawal and deposit operations, it’s helpful to first see how those are defined. Similar to the Workflow, they are defined using an interface that uses annotations to specify which methods to call:
1 package org.mongodb.activities; 2 3 4 public interface AccountActivities { 5 6 7 String withdraw(String account, int amount, String referenceId); 8 9 10 String deposit(String account, int amount, String referenceId); 11 }
In this case, both the
withdraw and deposit Activity methods take three arguments: the account identifier, the amount of money to debit or credit, and a reference number that uniquely identifies this request. The banking service uses this reference number as an idempotency key, enabling it to detect and ignore duplicate requests. Each method returns a String, which is the transaction ID provided by the banking service when the operation succeeds.As with the Workflow interface, the Activity interface has a corresponding implementation class, which provides the code to run when the Activity is executed. The Activity implementation uses an HTTP client to issue a request to the banking service, parses the response, and returns the transaction ID. The only Temporal-specific code in either method is a utility method,
Activity.wrap, which converts a checked exception (such as IOException) into an unchecked exception. This simplifies the code, since Temporal will catch any exception that occurs and handle it by retrying the Activity in a subsequent attempt.Now that you understand how the Workflow and Activities are defined, you have the context needed to understand the Workflow implementation class. The excerpt shown below omits the code related to manager approvals for larger transfers, which will be covered later in this tutorial.
1 public class MoneyTransferWorkflowImpl implements MoneyTransferWorkflow { private final ActivityOptions options = ActivityOptions.newBuilder() 2 .setStartToCloseTimeout(Duration.ofSeconds(10)) 3 .setRetryOptions(RetryOptions.newBuilder() 4 .setInitialInterval(Duration.ofSeconds(1)) 5 .setMaximumInterval(Duration.ofSeconds(60)) 6 .setBackoffCoefficient(2.0) 7 .setDoNotRetry( 8 InsufficientFundsException.class.getName()) 9 .build()) 10 .build(); 11 12 private final AccountActivities activitiesStub = Workflow.newActivityStub(AccountActivities.class, options); 13 14 15 public String transfer(TransferDetails input) { 16 String withdrawKey = String.format("withdrawal-for-%s", input.getReferenceId()); 17 String withdrawResult = activitiesStub.withdraw(input.getSender(), input.getAmount(), withdrawKey); 18 19 String depositKey = String.format("deposit-for-%s", input.getReferenceId()); 20 String depositResult = activitiesStub.deposit(input.getRecipient(), input.getAmount(), depositKey); 21 22 String confirmation = String.format("withdrawal=%s, deposit=%s", withdrawResult, depositResult); 23 24 return confirmation; 25 } 26 }
The first statement in the code above defines the options that control Activity execution. There are two parts to this. The first defines a Start-to-Close Timeout, which specifies the maximum duration (10 seconds, in this case) that the Activity is allowed to run. If it exceeds this without returning a result or error, it will be timed out and retried. The second part is the Retry Policy for the Activities. If omitted, the Workflow will use the default policy, but this example provides a custom policy for easier experimentation. This policy states that the Activity should be retried one second after failure, doubling the delay between subsequent attempts, with a maximum delay of 60 seconds. It also declares that there will be no retry if the Activity throws an
InsufficientFundsException.The next statement calls the
Workflow.newActivityStub method, passing in the Activity interface and the options described above. This returns a stub with the methods defined in the Activity interface. You’ll call those methods to execute the Activities from the Workflow. The Worker observes those calls, sending messages to the Temporal Service that result in new Tasks being added to its Task Queue, which in turn results in the Event History being updated and a Worker executing those Activities.Finally, the
transfer method fulfills the money transfer request by calling the withdraw and deposit Activities, concatenating their transaction IDs into a confirmation message returned by the Workflow. The call to the withdraw and deposit Activities includes an idempotency key created from the reference ID. Those keys are passed along in the call to the banking service, ensuring that neither the withdrawal nor deposit are duplicated.As explained earlier, the Worker is responsible for executing the Workflow and Activity code. It does this by polling a Task Queue maintained by the Temporal Service, which provides details about what code to run. It then executes the code and reports the status back to the Temporal Service, which updates the Event History. It uses a Temporal Client object, provided by the Temporal SDK, to communicate with the Temporal Service. The Worker implementation is provided by the Temporal SDK, but the
org.mongodb.workers.ApplicationWorker class in this application is what configures and starts it:1 package org.mongodb.workers; 2 3 public class ApplicationWorker { 4 5 public static final String TASK_QUEUE_NAME = "money-transfer-task-queue"; 6 7 public static void main(String[] args) { 8 WorkflowServiceStubs serviceStub = WorkflowServiceStubs.newLocalServiceStubs(); 9 WorkflowClient client = WorkflowClient.newInstance(serviceStub); 10 WorkerFactory factory = WorkerFactory.newInstance(client); 11 Worker worker = factory.newWorker(TASK_QUEUE_NAME); 12 worker.registerWorkflowImplementationTypes(MoneyTransferWorkflowImpl.class); 13 14 AccountActivities activities = new AccountActivitiesImpl(); 15 worker.registerActivitiesImplementations(activities); 16 17 factory.start(); 18 } 19 }
The first statement in the
main method below returns a WorkflowServiceStubs instance, which represents a connection to a local Temporal Service. The next statement creates a Temporal Client that will communicate with that service. The two statements that follow create the Worker instance, which will poll the Task Queue identified by the constant defined as a String at the top of this class. As you’ll see in a moment, the Task Queue name is also referenced in another class, and defining it as a constant that’s used by both ensures that the value is the same in both places. The next three lines register the Workflow and Activity types that this Worker will support. The final line starts the Worker, causing it to begin polling the Task Queue, picking up Tasks corresponding to the Workflow and Activity types it supports, and running the code as specified in the Task.Workflow Execution begins when the Temporal Service receives a request to run a specific Workflow Type with specific input data. That request comes from a Temporal Client. The Starter class is a small program that creates a Temporal Client (just as the Worker did) and then uses it to issue a Workflow Execution request to the Temporal Service, passing in the input data for the Workflow.
1 package org.mongodb; 2 3 public class Starter { 4 private static final Logger logger = LoggerFactory.getLogger(Starter.class); 5 6 public static void main(String[] args) { 7 // The sender, recipient, and transfer amount are parsed from 8 // command-line arguments. The parsing code here is omitted for brevity. 9 10 String referenceId = UUID.randomUUID().toString(); 11 12 TransferDetails details = new TransferDetails(sender, recipient, transferAmount, referenceId); 13 14 String workflowId = String.format("transfer-%d-%s-to-%s", transferAmount, sender, recipient).toLowerCase(); 15 16 WorkflowServiceStubs serviceStub = WorkflowServiceStubs.newLocalServiceStubs(); 17 WorkflowClient client = WorkflowClient.newInstance(serviceStub); 18 WorkflowOptions options = WorkflowOptions.newBuilder() 19 .setTaskQueue(ApplicationWorker.TASK_QUEUE_NAME) 20 .setWorkflowId(workflowId) 21 .build(); 22 23 MoneyTransferWorkflow workflow = client.newWorkflowStub(MoneyTransferWorkflow.class, options); 24 String confirmation = workflow.transfer(details); 25 26 logger.info("Money Transfer complete. Confirmation: {}", confirmation); 27 28 System.exit(0); 29 } 30 }
When you reviewed the Workflow Interface code above, you saw that the Workflow method takes a
TransferDetails object as input. This contains information about the sender’s account, the recipient’s account, the amount being transferred, and a reference ID. The reference ID is an identifier used by banks for transaction tracking and processing, and the Starter class generates this value using a UUID. As explained earlier, we use this to generate idempotency keys for each Activity, which the banking service uses to detect duplicate requests.While this reference ID is used only by the application, the Workflow ID is an identifier used by Temporal itself. Temporal guarantees that no more than one Workflow Execution with a given ID will be running at a given time. It is specified when submitting a Workflow Execution request and typically has a value that is meaningful within the underlying application. For example, an order processing application might use the order number as the Workflow ID, which would guard against an accidental click in a web application submitting the same order twice. In this case, the application constructs a Workflow ID based on the sender, recipient, and amount. That is, it will ignore a new request to transfer the same amount of money between the same people as another transfer already in progress.
The
workflow.transfer call, which references the method defined in the Workflow interface, instructs the Worker to submit that request to the Temporal Service. This is a synchronous call, and the code will block here until the Workflow Execution concludes. If that execution was successful, the call returns the output from the Workflow method, which is the confirmation message that joins the transaction IDs for the withdrawal and deposit operations. While this example uses a synchronous call to the Workflow, the Temporal APIs also provides an asynchronous counterpart that you can use if you don’t want to block while the Workflow Execution completes.Now that you understand the code, it’s time to run it and see it in action.
The banking service uses MongoDB to store information about the bank accounts, including their current balances. It locates the MongoDB cluster through an environment variable (
MONGO_CONNECTION_STRING), which you must set before launching the service. The banking service will create a database named bankingdemo in that cluster.Follow the instructions below that correspond to your environment, and adjust the connection string to match that for your MongoDB cluster.
1 export MONGO_CONNECTION_STRING="mongodb+srv://<username>:<password>@cluster.abcde.mongodb.net/"
1 set MONGO_CONNECTION_STRING="mongodb+srv://<username>:<password>@cluster.abcde.mongodb.net/"
1 env:MONGO_CONNECTION_STRING="mongodb+srv://<username>:<password>@cluster.abcde.mongodb.net/"
Once you have set the environment variable and verified that your MongoDB cluster is running, change into the
bank-services directory in your terminal:1 cd bank-services
Finally, run the following Maven command to compile the code and start the banking services and GUI:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.banking.Main"
The
README.md file in that project describes how you can manually interact with the banking services through an HTTP client, but you won’t need to do that in this tutorial.When you run this for the first time, you’ll need to add at least two bank accounts so you can transfer money between them.
Click the Add Account button to do this. The examples that follow will use the names Maria and David for those accounts. You are free to use any name you like, but you’ll need to use the names you selected when you run the money transfer Workflow.
The screenshot below shows the GUI after adding these two accounts. Each has an initial balance of $1000 (the currency symbol displayed may vary based on your location) and is available for API calls.
The Temporal CLI is a command-line tool for interacting with a Temporal Service. For example, you can use it to start, signal, or cancel Workflow Executions, check which are currently running, and view their Event Histories or results. The Temporal CLI also includes a lightweight version of the Temporal Service. It has no external dependencies and starts up in seconds, making it a great choice for local development.
Since the upcoming steps require that the Temporal Service is running, start it now by opening another terminal and running this command:
1 temporal server start-dev --db-filename temporal-service.db
This starts the Temporal Service, with the Web UI listening on its default port (8233). If that port is not available, append
--ui-port to this command, followed by the port number of your choice.The
--db-filename option specifies the location of the path to a file where the Temporal Service will persist Event Histories and other data. If the file does not exist, it will be created. If it does exist, as will be the case if you restart the Temporal Service, it will use it to load data from the previous session. If you omit this option, the Temporal Service stores the data in memory, so it will not persist across restarts.The Worker communicates with the Temporal Service, polling a Task Queue that indicates what to run in order for Workflow Execution to progress. When the Worker accepts those tasks, it runs the appropriate Workflow and Activity methods, and then reports the result back to the Temporal Service.
You will run all remaining commands in this tutorial from the
money-transfer-application directory, so change to that directory now.1 cd money-transfer-application
Execute the Maven command below to compile the application code and run the
main method in the class that configures and starts the Worker for this project:1 mvn compile exec:java -Dexec.mainClass="org.mongodb.workers.ApplicationWorker"
The Temporal Service is now running and the Worker you started is polling the Task Queue for the money transfer. The next step is to submit a request to the Temporal Service to initiate the money transfer Workflow. The Temporal Service will then add the first Task to the queue and will later add more Tasks as the Workflow Execution progresses.
Run the command below to execute the
org.mongodb.Starter class, which submits the execution request and then displays the result when execution is complete. This example transfers $100 from Maria to David. If you set up accounts with different names, modify the command to use those instead.1 mvn compile exec:java -Dexec.mainClass="org.mongodb.Starter" -Dexec.args="Maria David 100"
You should see a few lines of output. The final one should include the result returned by the Workflow, which is a confirmation message that contains the transaction IDs from the withdrawal and deposit operations. Those are randomly generated by the banking service, so they will be different from the example shown here:
1 [org.mongodb.Starter.main()] INFO org.mongodb.Starter - Money Transfer complete. Confirmation: withdrawal=W4842620791, deposit=D7932687105
You should also see that the balances shown in the GUI have changed. Before you ran the Workflow, both balances had the same amount. After performing the transfer, Maria’s account has been reduced by the amount of the transfer, while David’s had a corresponding increase.
You can also use the Temporal Web UI to see the results—and many of other details—for this Workflow Execution. Launch your browser and navigate to http://localhost:8233/. If you used the
--ui-port option to specify a different port when starting the Temporal Service, replace 8233 in that URL with the port you specified. You should see a page similar to the one in this screenshot:
The main page lists current and recent Workflow Executions, displaying the status, Workflow ID, start, and end times for each. Since it was your first, it’s the only one shown right now. Click the link in the Workflow ID column to view the detail page. You should see something similar to the screenshot below, which has been annotated to label the four main sections.
Below is an excerpt showing the two sections near the top, with numbered callouts to draw your attention to specific parts. This shows 1) when the Workflow Execution started and ended, 2) the type of Workflow it was, 3) the Task Queue it used, 4) the data it was provided as input, and 5) the result it returned as output.
The Timeline section that follows this section illustrates the time and relative sequence for each Activity. The Event History table, located below that, shows the details for each event that took place during the Workflow Execution. These details are not only helpful for debugging, but they also enable the Worker to completely reconstruct the application state following a crash.
You have now successfully run the money transfer Workflow. Now it’s time for you to see what happens when things don’t go as planned.
You will now do another transfer from Maria to David, but this time, you’ll simulate a service outage with Maria’s bank.
Click the Stop button for Maria’s account in the GUI. This disables access to her account through the banking service, simulating an outage at her bank. Since the GUI also uses this service, it shows UNKNOWN for the balance, since it cannot access her account.
The Temporal Service and your Worker should still be running, so you need only re-run the Workflow to perform another transfer. Do that now by running the executing the following command:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.Starter" -Dexec.args="Maria David 100"
Unlike earlier, when this quickly showed the result returned by the Workflow, the output you’ll see in the terminal now suggests that the Workflow started but has not yet finished. The Temporal Web UI is a convenient tool for debugging problems like this.
Navigate to the detail page for this Workflow Execution in the Web UI. The status shown near the upper-left corner indicates that this is still running. The Timeline section shows a horizontal line made up of vertical red lines, which indicates that the corresponding Activity (Withdraw) is failing. The Pending Activities tab (highlighted with an orange box in this screenshot) has the number 1 next to it, which indicates that one Activity has failed and is awaiting another retry attempt.
Click the Pending Activities tab now, which will show a screen similar to the one below. This identifies the type of Activity that is failing, how many retries have already been attempted, how many attempts remain, and how long it will be until the next attempt. The Last Failure section contains the details of the failure, which includes the stack trace. In this case, the message (duplicated here in large text) indicates that the failure is caused by an
IllegalStateException thrown by the application because Maria’s bank is unavailable.
Click the History tab to the left of the Pending Activities tab to return to the previous page for this Workflow Execution.
Click the Start button for Maria’s account in the GUI. With the outage now resolved, the Workflow will run to completion upon the next retry attempt, as if there was never an outage at all. You’ll know this has happened when the Workflow Execution status changes to Completed in the Web UI. You should also observe that the balances shown in the GUI changed to reflect the completed transfer.
Earlier, you learned about the problems that can occur if a crash occurs between the withdrawal and deposit. Temporal provides Durable Execution, which allows it to continue execution as if the crash never happened. Now, it’s time for you to experience that for yourself by killing the Worker after the withdrawal but before the deposit.
Because the Workflow Execution happens so quickly, you’ll need to introduce a delay between the two Activities that will give you time to kill the Worker.
Edit the
src/main/java/org/mongodb/workflows/MoneyTransferWorkflowImpl.java file in the money-transfer-application project. Between the withdraw and deposit Activities in the transfer method in that class, you’ll find the following statement commented out:1 //Workflow.sleep(Duration.ofSeconds(30));
This uses a method provided by the Temporal Workflow API to start a Durable Timer, which pauses Workflow Execution for the specified duration of 30 seconds. That should provide enough time for you to kill the Worker.
Uncomment this statement. Any time you modify the Workflow or Activity code, you must restart all of the Workers so that they begin using the updated version. Therefore, you must locate the terminal window where you started the Worker, press Ctrl-C to kill that process, and then re-run the command you used to start it:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.workers.ApplicationWorker"
You have just introduced the delay that will give you time to crash the application before it completes. Now it’s time to run the Workflow again:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.Starter" -Dexec.args="Maria David 100"
Once it starts, switch back to the terminal session where you started the Worker. As soon as it reports the withdrawal as complete, press Ctrl-C to kill the Worker.
You may now use the Web UI to find this Workflow Execution. The Timeline and Event History Table should both confirm that the
withdraw Activity has completed but the deposit Activity has not.Now that you crashed the application, it’s time to see it resume from where it left off. Re-run the command you used to start your Worker:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.workers.ApplicationWorker"
Return to the Web UI and locate the detail page for this Workflow Execution. You should observe that the Workflow runs to completion, although this may take additional time due to the delay caused by the
Workflow.sleep statement and the timeout that the Temporal Service uses to detect Worker failure.Once it has completed, notice the timeline view, which illustrates when the withdrawal, sleep, and deposit took place. You’ll also see that each of them happened only once. If you want further evidence that the withdrawal wasn’t repeated, despite it having run before the Worker crash, check the terminal window where you started the banking services. The log messages shown there will confirm that there was only a single withdrawal.
After you killed the Worker, Workflow Execution progress ceased because there were no Workers running. A production deployment of a Temporal application will have multiple Workers. One reason for this is scalability, since each additional Worker expands the capacity to run more Workflow Executions concurrently. Another reason is availability, since other Workers will automatically take over for one that crashed.
If you’d like to see this for yourself, start a second Worker and then repeat the steps in this section, killing the Worker that ran your
withdraw Activity. You should find that the remaining Worker is able to complete the Workflow Execution without any intervention on your part.A Workflow can have both automated and manual steps. Temporal supports both and you can use both in a single Workflow. This part of the tutorial will show you how.
The business requirements for the money transfer application state that transfers for relatively small amounts of money should complete automatically, as you’ve already seen. Transfers that exceed $500 are placed on hold until they can be reviewed and approved by a manager.
There are two parts to this approval process. The first is in the Workflow, which checks whether the amount exceeds the threshold. If it does, it sets the variable
hasManagerApproval to false. Just after the statement that does this, the code calls Workflow.await to block until that variable has a true value. Temporal’s design allows Workflow Executions to efficiently and reliably await a condition or Timer, even for durations lasting several months or years. Here’s an excerpt showing the relevant code (some comments have been modified or omitted here for brevity):1 // Large transfers must be explicitly approved by a manager 2 if (input.getAmount() > 500) { 3 logger.warn("This transfer is on hold awaiting manager approval"); 4 hasManagerApproval = false; 5 } 6 7 // Workflow Execution will not progress until hasManagerApproval is true 8 Workflow.await(() -> hasManagerApproval); 9 10 // Transfer approved. Will now withdraw money from the sender's account 11 logger.info("Starting withdraw operation");
How does the
hasManagerApproval variable get set to true? That’s the second part, and it’s done in the approve method, which was annotated with @SignalMethod in the Workflow interface. A Signal is a named message that can supply data to a running Workflow.Signals are sent to the Temporal Service, which passes them onto the Worker, which in turn invokes the corresponding method in the Workflow Definition. In this example, the Signal name is
approve and its data is the name of the manager approving the transfer. This method updates the hasManagerApproval variable, which enables the transfer to continue because it’s no longer blocked on the Workflow.await call. Here is the code for the approve method:1 2 public void approve(String managerName) { 3 logger.info("This transfer has now been approved by {}", managerName); 4 hasManagerApproval = true; 5 }
There are multiple ways to send a Signal. You can use the Temporal CLI or Web UI, which are generic approaches that don’t require writing any code. You can also use APIs provided by the Temporal SDK, which allows you to send a Signal from your application, as the banking GUI in this tutorial does. Clicking its Approve a transaction button prompts you for the Workflow ID and approver’s name. When you click the OK button, it calls these three lines of code:
1 WorkflowServiceStubs service = WorkflowServiceStubs.newLocalServiceStubs(); 2 WorkflowClient client = WorkflowClient.newInstance(service); 3 client.newUntypedWorkflowStub(workflowId).signal("approve", managerName);
The first two lines of code create a Temporal Client and are identical to those you saw in the code used to start the Workflow. The last line of code uses this Client and the Workflow ID to locate the Workflow Execution, sending it a Signal named
approve with the manager’s name as the input data.You may notice that this code uses
newUntypedWorkflowStub. While it’s possible to use a typed stub, as the code used to start the Workflow did, using an untyped stub allows the GUI to send the Signal generically. In other words, this approach avoids a dependency between the GUI and the money transfer application code.Now that you understand how the approval process is implemented, it’s time for you to see it in action by initiating and approving a transfer.
The Temporal Service and your Worker should still be running from earlier, so you need only re-run the Workflow to perform another transfer. Specify an amount that requires approval by running the command below:
1 mvn compile exec:java -Dexec.mainClass="org.mongodb.Starter" -Dexec.args="Maria David 600"
Since this transfer is on hold pending approval, you will not see a confirmation message until it has completed.
Switch to the terminal where you started the Worker. The last line of output should be a message that says, “This transfer is on hold awaiting manager approval.”
Navigate to the detail page for this Workflow Execution in the Web UI. The Timeline and Event History should look similar to the following screenshot:
Notice that neither the
withdraw nor the deposit Activity appear here. This is because the Workflow code does not call those until the transfer is approved.As you proceed with the next step, make sure that the Web UI remains visible on your screen (if possible), so that you can see the Workflow Execution progress.
In the Banking GUI, click the Approve a Transaction button. This displays a dialog box. Copy the Workflow ID from the Web UI (it appears at the top of the detail page) and paste it into the Workflow ID field. Enter a name in the Manager Name field.
Finally, click the OK button to send the Signal and approve the transfer. You should soon see in the Web UI that the Workflow Execution runs to completion.
The Event History also reflects the approval. The pink icon in the Timeline view represents the
approve Signal being sent. The Workflow Execution Signaled Event in the history table provides additional detail, including the timestamp and name of the approver.
Congratulations on completing the tutorial! You've seen how a Temporal application retries an operation that fails due to a service outage, automatically recovering as if the outage never occurred at all. You've seen how Durable Execution enables the application to automatically recover from unexpected termination, resuming from where it left off instead of starting over at the beginning. Finally, you have seen how Temporal applications support human-in-the-loop Workflows, enabling you to build applications which involve tasks that require manual processing steps.
The code offers many more opportunities to explore. For example, you might experiment with changes to the Retry Policy in the Workflow. You might also implement the Saga pattern to undo an in-progress transfer if the recipient’s account is unavailable. The money transfer Workflow also provides some corresponding tests, including one which tests the human-in-the-loop scenario.
Another great way to continue learning is to sign up for one of Temporal's free hands-on training courses. You should also check out the Temporal community page, where you'll find information about upcoming events, links to documentation and code samples, and an invitation to join our Slack workspace. If you want to learn more about MongoDB, check out our Community Forums, or head over to our developer center, where we have tutorials such as how to use Kubernetes for Spring Boot microservices.
Top Comments in Forums
There are no comments on this article yet.
Rate this tutorial