Dissecting Open Banking with MongoDB: Technical Challenges and Solutions

Thank you to Ainhoa Múgica for her contributions to this post.

Unleashing a disruptive wave in the banking industry, open banking (or open finance), as the term indicates, has compelled financial institutions (banks, insurers, fintechs, corporates, and even government bodies) to embrace a new era of transparency, collaboration, and innovation. This paradigm shift requires banks to openly share customer data with third-party providers (TPPs), driving enhanced customer experiences and fostering the development of innovative fintech solutions by combining ‘best-of-breed’ products and services. As of 2020, 24.7 million individuals worldwide used open banking services, a number that is forecast to reach 132.2 million by 2024. This rising trend fuels competition, spurs innovation, and fosters partnerships between traditional banks and agile fintech companies. In this transformative landscape, MongoDB, a leading developer data platform, is vital in supporting open banking by providing a secure, scalable, and flexible infrastructure for managing and protecting shared customer data. By harnessing the power of MongoDB's technology, financial institutions can lower costs, improve customer experiences, and mitigate the potential risks associated with the widespread sharing of customer data through strict regulatory compliance.

The essence of open banking/finance is about leveraging common data exchange protocols to share financial data and services with 3rd parties. In this blog, we will dive into the technical challenges and solutions of open banking from a data and data services perspective and explore how MongoDB empowers financial institutions to overcome these obstacles and unlock the full potential of this open ecosystem.

Dynamic environments and standards

As open banking standards evolve, financial institutions must remain adaptable to meet changing regulations and industry demands. Traditional relational databases often struggle to keep pace with the dynamic requirements of open banking due to their rigid schemas that are difficult to change and manage over time. In countries without standardized open banking frameworks, banks and third-party providers face the challenge of developing multiple versions of APIs to integrate with different institutions, creating complexity and hindering interoperability. Fortunately, open banking standards or guidelines (eg. Europe, Singapore, Indonesia, Hong Kong, Australia, etc.) have generally required or recommended that the open APIs be RESTful and support JSON data format, which creates a basis for common data exchange.

MongoDB addresses these challenges by offering a flexible developer data platform that natively supports JSON data format, simplifies data modeling, and enables flexible schema changes for developers. With MongoDB’s wide range of idiomatic drivers and libraries, developers can easily access MongoDB using their own programming language. The Stable API feature ensures compatibility during database upgrades, preventing code breaks and providing a seamless transition. Additionally, MongoDB provides productivity-boosting features like full-text search, vector search, data visualization, and data federation to empower developers with a rich suite of capabilities unified on a single data platform. This enables financial institutions and third-party providers to roll out new features more rapidly and capture opportunities in the changing open banking landscape more effectively and timely.

An example of a client who leverages MongoDB’s native JSON data management and flexibility is Natwest. Natwest is a major retail and commercial bank in the United Kingdom-based in London, England. The bank has moved from zero to 900 million API calls per month within years as open banking uptake grows and is expected to grow ten times in the coming years. At a MongoDB event on 15 Nov 2022, Jonathan Haggarty, Natwest’s Head of “Bank of APIs” Technology – an API ecosystem that brings the retail bank’s services to partners – shared in his presentation titled Driving Customer Value using API Data that Natwest’s growing API ecosystem lets it “push a bunch of JSON data into MongoDB [which makes it] “easy to go from simple to quite complex information" and also makes it easier to obfuscate user details through data masking for customer privacy. Natwest is enabled to surface customer data insights for partners via its API ecosystem, for example “where customers are on the e-commerce spectrum,” the “best time [for retailers] to push discounts” as well insights on “most valuable customers” – with data being used for problem-solving; analytics and insight; and reporting.

Performance

In the dynamic landscape of open banking, meeting the unpredictable demands for performance, scalability, and availability is crucial. The efficiency of applications and the overall customer experience heavily rely on the responsiveness of APIs. However, building an open banking platform becomes intricate when accommodating third-party providers with undisclosed business and technical requirements. Without careful management, this can lead to unforeseen performance issues and increased costs. Open banking demands the APIs to perform well under all kinds of workload volumes. OBIE recommends an average TTLB (time to last byte) of 750 ms per endpoint response for all payment invitations (except file payments) and account information APIs. Compliance with regulatory service level agreements (SLAs) in certain jurisdictions further adds to the complexity. Legacy architectures and databases often struggle to meet this demanding criteria, necessitating extensive changes to ensure scalability and optimal performance.

That's where MongoDB comes into play. MongoDB is purpose-built to deliver exceptional performance with its WiredTiger storage engine and compression capabilities. Additionally, MongoDB Atlas improves performance by following its intelligent index and schema suggestions, automatic data tiering, and workload isolation for analytics. One prime illustration of its capabilities is demonstrated by Temenos, a renowned financial services application provider, achieving remarkable transaction volume processing performance and efficiency by leveraging MongoDB Atlas. They recently ran a benchmark with MongoDB Atlas and Microsoft Azure and successfully processed an astounding 200 million embedded finance loans and 100 million retail accounts at a record-breaking 150,000 transactions per second. This showcases the power and scalability of MongoDB with unparalleled performance to empower financial institutions to effectively tackle the challenges posed by open banking. MongoDB ensures outstanding performance, scalability, and availability to meet the ever-evolving demands of the industry.

Scalability

Building a platform to serve TPPs, who may not disclose their business usages and technical/performance requirements, can introduce unpredictable performance and cost issues if not managed carefully. For instance, a bank in Singapore faced an issue where their Open APIs experienced peak loads and crashes every Wednesday. After investigation, they discovered that one of the TPPs ran a promotional campaign every Wednesday, resulting in a surge of API calls that overwhelmed the bank's infrastructure. A scalable solution that can perform under unpredictable workloads is critical, besides meeting the performance requirements of a certain known volume of transactions.

MongoDB's flexible architecture and scalability features address these concerns effectively. With its distributed document-based data model, MongoDB allows for seamless scaling both vertically and horizontally. By leveraging sharding, data can be distributed across multiple nodes, ensuring efficient resource utilization and enabling the system to handle high transaction volumes without compromising performance. MongoDB's auto-sharding capability enables dynamic scaling as the workload grows, providing financial institutions with the flexibility to adapt to changing demands and ensuring a smooth and scalable open banking infrastructure.

Availability

In the realm of open banking, availability becomes a critical challenge. With increased reliance on banking services by third-party providers (TPPs), ensuring consistent availability becomes more complex. Previously, banks could bring down maintenance services during off-peak hours. However, with TPPs offering 24x7 experiences, any downtime is unacceptable. This places greater pressure on banks to maintain constant availability for Open API services, even during planned maintenance windows or unforeseen events.

MongoDB Atlas, the fully managed global cloud database service, addresses these availability challenges effectively. With its multi-node cluster and multi-cloud DBaaS capabilities, MongoDB Atlas ensures high availability and fault tolerance. It offers the flexibility to run on multiple leading cloud providers, allowing banks to minimize concentration risk and achieve higher availability through a distributed cluster across different cloud platforms. The robust replication and failover mechanisms provided by MongoDB Atlas guarantee uninterrupted service and enable financial institutions to provide reliable and always available open banking APIs to their customers and TPPs.

Security and privacy

Data security and consent management are paramount concerns for banks participating in open banking. The exposure of authentication and authorization mechanisms to third-party providers raises security concerns and introduces technical complexities regarding data protection. Banks require fine-grained access control and encryption mechanisms to safeguard shared data, including managing data-sharing consent at a granular level. Furthermore, banks must navigate the landscape of data privacy laws like the General Data Protection Regulation (GDPR), which imposes strict requirements distinct from traditional banking regulations.

MongoDB offers a range of solutions to address these security and privacy challenges effectively. Queryable Encryption provides a mechanism for managing encrypted data within MongoDB, ensuring sensitive information remains secure even when shared with third-party providers. MongoDB's comprehensive encryption features cover data-at-rest and data-in-transit, protecting data throughout its lifecycle. MongoDB's flexible schema allows financial institutions to capture diverse data requirements for managing data sharing consent and unify user consent from different countries into a single data store, simplifying compliance with complex data privacy laws. Additionally, MongoDB's geo-sharding capabilities enable compliance with data residency laws by ensuring relevant data and consent information remain in the closest cloud data center while providing optimal response times for accessing data.

To enhance data privacy further, MongoDB offers field-level encryption techniques, enabling symmetric encryption at the field level to protect sensitive data (e.g., personally identifiable information) even when shared with TPPs. The random encryption of fields adds an additional layer of security and enables query operations on the encrypted data. MongoDB's Queryable Encryption technique further strengthens security and defends against cryptanalysis, ensuring that customer data remains protected and confidential within the open banking ecosystem.

Activity monitoring

With numerous APIs offered by banks in the open banking ecosystem, activity monitoring and troubleshooting become critical aspects of maintaining a robust and secure infrastructure. MongoDB simplifies activity monitoring through its monitoring tools and auditing capabilities. Administrators and users can track system activity at a granular level, monitoring database system and application events.

MongoDB Atlas has Administration APIs, which one can use to programmatically manage the Atlas service. For example, one can use the Atlas Administration API to create database deployments, add users to those deployments, monitor those deployments, and more. These APIs can help with the automation of CI/CD pipelines and monitoring the activities on the data platform enabling developers and administrators to be freed of this mundane effort and focus on generating more business value. Performance monitoring tools, including the performance advisor, help gauge and optimize system performance, ensuring that APIs deliver exceptional user experiences.

MongoDB Atlas Charts, an integrated feature of MongoDB Atlas, offers analytics and visualization capabilities. Financial institutions can create business intelligence dashboards using MongoDB Atlas Charts. This eliminates the need for expensive licensing associated with traditional business intelligence tools, making it cost-effective as more TPPs utilize the APIs. With MongoDB Atlas Charts, financial institutions can offer comprehensive business telemetry data to TPPs, such as the number of insurance quotations, policy transactions, API call volumes, and performance metrics. These insights empower financial institutions to make data-driven decisions, improve operational efficiency, and optimize the customer experience in the open banking ecosystem.

Real-Timeliness

Open banking introduces new challenges for financial institutions as they strive to serve and scale amidst unpredictable workloads from TPPs. While static content poses fewer difficulties, APIs requiring real-time updates or continuous streaming, such as dynamic account balances or ESG-adjusted credit scores, demand capabilities for near-real-time data delivery.

To enable applications to immediately react to real-time changes or changes as they occur, organizations can leverage MongoDB Change Streams that are based on its aggregation framework to react to data changes in a single collection, a database, or even an entire deployment. This capability further enhances MongoDB’s real-time data and event processing and analytics capabilities.

MongoDB offers multiple mechanisms to support data streaming, including a Kafka connector for event-driven architecture and a Spark connector for streaming with Spark. These solutions empower financial institutions to meet the real-time data needs of their open banking partners effectively, enabling seamless integration and real-time data delivery for enhanced customer experiences.

MongoDB has also released Atlas Stream Processing, the MongoDB-native way to process streaming data, transforming the way developers build modern applications. Developers can now use the document model and the MongoDB aggregation framework to process rich and complex streams of data from platforms such as Apache Kafka. This unlocks powerful opportunities to continuously process streaming data (e.g., aggregate, filter, route, alert, and emit) and materialize results into Atlas or back into the streaming platform for consumption by other systems such as the Open Banking system for providing customers and transaction data updated in real-time.

Conclusion

MongoDB's technical capabilities position it as a key enabler for financial institutions embarking on their open banking journey. From managing dynamic environments and accommodating unpredictable workloads to ensuring scalability, availability, security, and privacy, MongoDB provides a comprehensive set of tools and features to address the challenges of open banking effectively. With MongoDB as the underlying infrastructure, financial institutions can navigate the ever-evolving open banking landscape with confidence, delivering innovative solutions, and driving the future of banking. Embracing MongoDB empowers financial institutions to unlock the full potential of open banking and provide exceptional customer experiences in this era of collaboration and digital transformation.