Spring Data Unlocked: Advanced Queries With MongoDB
Ricardo Mello7 min read • Published Nov 08, 2024 • Updated Nov 08, 2024
FULL APPLICATION
Rate this tutorial
I bet you’re here because you’ve already read the first article in this series, Getting Started With Java and MongoDB. But even if you haven’t, don’t worry—you’re in the right place.
In the first part of this series, we explored key concepts related to Spring and learned how to create a project and configure it to seamlessly integrate with MongoDB. This is the second part of the series Spring Data Unlocked, where we will continue working on our project and explore Spring Data's capabilities by creating advanced queries and seeing how seamlessly it can be done.
- Get started with MongoDB Atlas for free! If you don’t already have an account, MongoDB offers a free-forever Atlas cluster.
- IDE of your choice
If you don’t remember what we discussed earlier, don’t worry. Read the previous article on how to get started with Java and MongoDB or refresh your memory with our data model that we are working on:
1 { 2 "id": "672182814338f60133ee26e1", 3 "transactionType": "Debit", 4 "amount": 888.0, 5 "currency": "USD", 6 "status": "In Progress", 7 "description": "Transfer to Ricardo", 8 "createdAt": "2024-10-09T14:00:00", 9 "accountDetails": { 10 "originator": { 11 "accountNumber": "2376543213", 12 "name": "Maria", 13 "bank": "Bank G" 14 }, 15 "beneficiary": { 16 "accountNumber": "2234987651", 17 "name": "Ricardo Mello", 18 "bank": "Bank V" 19 } 20 } 21 }
Our transaction model contains this structure, and we will proceed
with it.
Still focusing on productivity and speed, we will continue using our
MongoRepository
interface and create a query to retrieve our transactions by type. Open the TransactionRepository
class and add the following method:1 List<Transaction> findByTransactionType(String type);
As you can see, we are finding all transactions by the type. Spring, through naming conventions (derived queries), allows you to create queries based on the method names.
Tip: We can monitor the log process by enabling a DEBUG setting in the
application.properties
file:1 logging.level.org.springframework.data.mongodb=DEBUG
We will be able to see the query in the console:
1 2024-10-15T18:30:33.855-03:00 DEBUG 28992 2 [SpringShop] [nio-8080-exec-6] o.s.data.mongodb.core.MongoTemplate: 3 find using query: { "transactionType" : "Transfer"} fields: Document{{}} sort: { "transactionType" : "Transfer"} for class: class com.mongodb.Transaction in collection: transactions
Now, let’s say we want all transactions where the amount is greater than 3000:
1 List<Transaction> findByAmountGreaterThan(double amount);
It is possible to delete records in the same way. Take a look:
1 void deleteByTransactionType(String type);
If everything is working correctly, your code should be similar to this:
1 package com.mongodb; 2 import org.springframework.data.mongodb.repository.MongoRepository; 3 import org.springframework.stereotype.Repository; 4 import java.util.List; 5 6 7 public interface TransactionRepository extends MongoRepository<Transaction, String>{ 8 List<Transaction> findByTransactionType(String type); 9 List<Transaction> findByAmountGreaterThan(double amount); 10 void deleteByTransactionType(String type); 11 }
Continuing our development, we can use the Query class that Spring provides instead of working with derived queries. This alternative is also interesting. Below is an example of how to fetch transactions by status, displaying (Projection) only the fields
createdAt
, accountDetail
, and amount
, and sorting by createdAt
in descending order:1 2 3 4 5 6 List<Transaction> findByStatus(String status);
We can also perform update operations on our database using this annotation. In the first part, we apply the filter, and then we proceed to update the status.
1 2 3 void updateStatus(String id, String status);
Another important annotation in Spring Data is @Aggregation. As the name suggests, it allows us to create aggregation stages. Let’s imagine the following scenario:
We want to return the total
amount
grouped by transaction type. To do this, we'll create the following method:1 2 3 4 5 6 List<Transaction> getTotalAmountByTransactionType(String transactionType);
- In the first stage,
$match
, we filter the transactions by the specified type. - In the second stage,
$group
, we group the transactions bytransactionType
and sum the values from theamount
field. - Finally, in the
$project
stage, we display the totalamount
.
A new request from our product team has come in, and they want us to save all transactions with an error status in a new collection. We could set up a job to run once a day and trigger a method to handle this for us. For that, we can use MongoDB’s
$out
stage:1 2 3 4 5 6 void exportErrorTransactions();
After this method is executed, the aggregation will process, and any documents with an error status will be inserted into a new collection within the same database called
error_transactions
.Let's break it down again:
$match
to filter transactions with an error status$project
to specify which fields we want to include in the error collection$out
to define the collection where we will export the error transactions
Note: While we won’t go into the specifics of
$out
, I highly recommend reviewing its behavior before using it in a production
environment.As you can see, this annotation is very powerful and can provide features for increasingly complex queries. Let’s assume we now have a new collection called
customers
with some additional fields:1 { 2 "name": "Ricardo Mello", 3 "email": "ricardo.mello@mongodb.com", 4 "accountNumber": "2234987651", 5 "phone": "5517992384610", 6 "address": { 7 "street": "Street 001", 8 "city": "New York" 9 } 10 }
We can use
$lookup
to perform a join between the transaction
and customer
collections using the accountNumber
field and bring in extra values like phone
and address
.1 2 3 4 5 6 7 8 9 10 11 12 13 List<CustomerOriginatorDetail> findTransactionsWithCustomerDetails();
In the code above, we’re using the
$lookup
operator to join the new customer
collection through the accountNumber
field, and returning a new list of CustomerOriginatorDetail
.1 public record CustomerOriginatorDetail( 2 double amount, 3 String status, 4 Transaction.AccountDetails accountDetails, 5 List<OriginatorCustomerDetails> originatorCustomerDetails 6 ) { 7 private record OriginatorCustomerDetails( 8 String name, 9 String accountNumber, 10 String phone, 11 Address address) { 12 13 private record Address(String city, String street) {} 14 } 15 }
Note: Remember, we're not focusing on data modeling here, just showing
that it’s possible to work with $lookup in this context.
Another approach with the
@Aggregation
annotation is to create an interface that represents an aggregation, which simplifies its usage. Our SearchAggregate
annotation enables the creation of reusable search functionality using MongoDB's aggregation framework, facilitating efficient and structured querying of text data in your database.1 import org.springframework.data.mongodb.repository.Aggregation; 2 3 import java.lang.annotation.ElementType; 4 import java.lang.annotation.Retention; 5 import java.lang.annotation.RetentionPolicy; 6 import java.lang.annotation.Target; 7 8 9 10 11 12 13 SearchAggregate {14 }
Now, to utilize our annotation in a method, we can pass the arguments for the query and path in the
TransactionRepository
class:1 2 List<Transaction> search(String query, String path);
An important aspect to note is that if we look through the interfaces extended by
MongoRepository
, we’ll find PagingAndSortingRepository
, which includes the method:Page<T> findAll(Pageable pageable);
This method is crucial for working with pagination. Let’s go back to our
TransactionService
and implement the following code:1 public Page<Transaction> findPageableTransactions( 2 Pageable pageable 3 ) { 4 return transactionRepository.findAll(pageable); 5 }
Now, in the TransactionController, we can make the paginated call as follows:
1 2 public PagedModel<Transaction> findAll( int page, 3 int sizePerPage, 4 String sortField, 5 Sort.Direction sortDirection) { 6 Pageable pageable = PageRequest.of(page, sizePerPage, Sort.by(sortDirection, sortField)); 7 return new PagedModel<>(transactionService.findPageableTransactions(pageable)); 8 }
And finally, we can call our method using the following
curl
command.1 curl --location 'http://localhost:8080/transactions?page=0&sizePerPage=10&sortField=description&sortDirection=ASC'
A very interesting alternative to achieve flexibility and control over MongoDB operation is MongoTemplate. This template supports operations like update, insert, and select and also provides a rich interaction with MongoDB, allowing us to map our domain objects directly to the document model in the database. Let's start by creating a new
MongoConfig
class:1 package com.mongodb; 2 3 import com.mongodb.client.MongoClient; 4 import com.mongodb.client.MongoClients; 5 import org.springframework.context.annotation.Bean; 6 import org.springframework.context.annotation.Configuration; 7 import org.springframework.data.mongodb.core.MongoOperations; 8 import org.springframework.data.mongodb.core.MongoTemplate; 9 10 11 public class MongoConfig { 12 13 14 public MongoClient mongoClient() { 15 MongoClientSettings settings = MongoClientSettings.builder() 16 .applyConnectionString(new ConnectionString("<your connection string>")) 17 .build(); 18 return MongoClients.create(settings); 19 } 20 21 22 MongoOperations mongoTemplate(MongoClient mongoClient) { 23 return new MongoTemplate(mongoClient, "springshop"); 24 } 25 }
We can observe the @Bean annotation from Spring, which allows us to work with the injected MongoTemplate in our services. Continuing with our development, we will work with the Customer class to manipulate it. To do so, create the
Customer
record:1 package com.mongodb; 2 3 public record Customer( 4 String name, 5 String email, 6 String accountNumber, 7 String phone, 8 Address address 9 ) { 10 public record Address( 11 String street, 12 String city 13 ) {} 14 }
Let's start with the basic insertion model and then evolve to other operations. To do this, we will create our
CustomerService
with the following method:1 import org.springframework.data.mongodb.core.MongoOperations; 2 import org.springframework.stereotype.Service; 3 4 5 public class CustomerService { 6 7 private final MongoOperations mongoOperations; 8 9 CustomerService(MongoOperations mongoOperations) { 10 this.mongoOperations = mongoOperations; 11 } 12 13 public Customer newCustomer() { 14 var customer = new Customer( 15 "Ricardo", 16 "ricardohsmello@gmail.com", 17 "123", 18 "1234", 19 new Customer.Address("a", "Sp") 20 ); 21 22 return mongoOperations.insert(customer); 23 } 24 25 }
Our service will work with mongoOperations, which will handle the insertion of our document.
Note that I am creating (new Customer) to illustrate the insertion. A
good practice would be to receive this Customer as an argument in our
function.
One way to handle bulk insertions is by using bulkWrite. The MongoTemplate provides us with the bulkOps method, where we can provide a list, and it will be executed in batches:
1 public int bulkCustomerSample(List<Customer> customerList) { 2 BulkWriteResult result mongoOperations.bulkOps(BulkOperations.BulkMode.ORDERED, Customer.class) 3 .insert(customerList) 4 .execute(); 5 6 return result.getInsertedCount(); 7 }
Moving on to queries, we have the implementation of queries in the MongoTemplate class. In the code below, we are searching for a customer by email and returning only one:
1 public Customer findCustomerByEmail(String email) { 2 return mongoOperations.query(Customer.class) 3 .matching(query(where("email").is(email))) 4 .one() 5 .orElseThrow(() -> new RuntimeException("Customer not found with email: " + email)); 6 }
The query method (imported statically) expects a Criteria (where), also imported statically, where we can apply various filters such as gt(), lt(), and(), and or(). For reference, see the Spring documentation.
Let’s imagine we need a query that returns the total number of customers by city. To do this, we need to group by city and count the number of customers. To achieve this goal, first, let's create a new record to handle it:
1 public record CustomersByCity( 2 String id, 3 int total 4 ){}
And then, we will create a totalCustomerByCity method:
1 public List<CustomersByCity> totalCustomerByCity() { 2 3 TypedAggregation<Customer> aggregation = newAggregation(Customer.class, 4 group("address.city") 5 .count().as("total"), 6 Aggregation.sort(Sort.Direction.ASC, "_id"), 7 project(Fields.fields("total", "_id"))); 8 9 AggregationResults<CustomersByCity> result = mongoOperations.aggregate(aggregation, CustomersByCity.class); 10 return result.getMappedResults(); 11 }
The static method newAggregation offers us a good approach to manipulate our aggregation stages. We are grouping by city and using the count() function, which internally performs a sum on the total number of customers, sorting by _id in ascending order and displaying both the total and id fields.
In this second part of our series, Spring Data Unlocked, we explored how to create complex queries with MongoRepository and MongoTemplate. This tutorial covered concepts such as pagination, custom annotations, bulk inserts, and powerful aggregation queries.
If you have any questions, feel free to leave them in the comments.
Top Comments in Forums
There are no comments on this article yet.