Mainframe Data Modernization with MongoDB Powered by Wipro's "ModerniZ" Tool
September 16, 2021 | Updated: October 21, 2022
#Partners#Modernization
This post will highlight a practical and incremental approach to modernizing mainframe data and workloads into cloud-hosted microservices using MongoDB’s modern, general purpose database platform.
Enterprises modernize their mainframes for a number of reasons—to increase agility, lower the risk of aging legacy skills, and reduce total cost of ownership (TCO). But the greatest underlying benefit and reason for modernization lies in a company’s ability to access and make sense of their own data more quickly. Gleaning valuable business insights through the use of real-time data and AI/ML models is at the heart of today’s most successful and innovative companies.
Consider the following business processes and their reliance on real-time insights:
- Real-time fraud detection, KYC, and score calculation
- Supporting new API requirements – PSD2, Open Banking, Fintech APIs etc
- Payment pattern analysis
- Moving away from hundreds of canned reports to template-based self-service configurable reporting
- Real-time management reporting
With the continued emergence of mobile and web apps, enterprises are looking to render content even faster, as well as scale up and down on demand. However, mainframes often serve as the true system of record (SoR) and maintain the golden copy of core data. In a typical enterprise, inquiry transactions on the mainframe contribute to over 80% of the overall transaction volume—in some cases up to 95%. The goal for organizations is to increase the throughput of these inquiry transactions with improved response time.
However, during real-time transaction processing, middleware must orchestrate multiple services, transform service response, and aggregate core mainframe and non-core applications. This architecture prevents the legacy services from seamlessly scaling on-demand with improved response time. At the same time, CIOs have significant concerns around the risks of big bang mainframe exit strategies, especially in the financial services, insurance and retail sectors where these complex applications serve the core business capabilities.
Wipro and MongoDB’s joint offering, “ModerniZ,” can help alleviate these risks significantly by introducing a practical, incremental approach to mainframe modernization.
An incremental solution: Offloading inquiry services data with ModerniZ
To keep up with changing trends in an agile manner, and to bridge the gap between legacy monoliths and digital systems of engagement, a tactical modernization approach is required. While complete mainframe modernization is a strategic initiative, offloading inquiry services data and making it available off the mainframe is a popular approach adopted by several enterprises.
Below are a few business requirements driving mainframe data offload:
- Seamlessly scale volume handling by up to 5X
- Improve response time by up to 2X
- Reduce mainframe TCO by up to 25%
- Direct API enablement for B2B and Partners (change the perception of being antiquated)
- Improve time to market on new enhancements by up to 3X
- Provision separate security mechanism for inquiry-only services
- Access single view data store for intra-day reporting, analytics and events handling
These business requirements, as well as the challenges in current mainframe environments, warrant an offloaded data environment with aggregated and pre-enriched data from different sources in a format which can be directly consumed by channels and front-end systems.
This is where our joint offering for mainframe modernization with MongoDB and Wipro’s ModerniZ tool comes into play. ModerniZ is Wipro’s IP platform specially focused on modernizing the UI, services and data layer of System Z (Mainframe). ModerniZ has multiple in-house tools and utilities to accelerate every phase of the legacy modernization journey.
Let’s dig deeper into the solution elements.
"CQRS with ModerniZ" for transactional and operational agility
Command Query Responsibility Segregation (CQRS) is an architectural principle that can prescribe an operation as a ‘command,’ which performs an action, or a ‘query,’ which returns data to the requestor—but not both. CQRS achieves this by separating the read data model from the write data model. Separation of these two operations in a business process helps optimize performance, reduce the cost associated with inquiry transactions, and create a new model which can grow vertically and horizontally.
MongoDB, with its document model and extensive set of capabilities, is best suited to house this offloaded ‘read data model.’ MongoDB’s JSON/BSON document model helps in pre-enriching the data and storing it in ‘inquiry ready’ format which simplifies the overhead for the front-end consumers.
Enterprise use cases for CQRS:
- Customer demographic information – e.g. monetary and nonmonetary transaction inquiries in banking and financial services
- Payer view – Healthcare
- Single view across policy administration engines and pricing engines
- Consolidated participant view in benefit administration platforms
- Single view of manufacturing production control systems across plants or countries
The below process indicates the step-by-step approach in enabling CQRS by offloading the data and transactional volumes into a modernized landscape, and continuously syncing the data (based on the business criticality) across platforms.
Figure 1. Mainframe data modernization process
The visual below indicates the conceptual target architecture where the service delivery platform (API/Middleware layer) identifies and routes the transaction to the respective systems. Any update which happens in the legacy system will be cascaded to the target MongoDB based on the business criticality of the fields.
Figure 2. Post data modernization process view
Mainframe services will continue to get exposed as domain APIs via zCEE for the Command Services (Update Transactions) and the newly built microservices will serve the inquiry transactions by fetching data from MongoDB. Any data updates in the mainframe will be pushed to MongoDB.
The below table indicates how different fields can be synced between the mainframe and MongoDB, as well as their corresponding sync intervals. Java programs/Spark consumes the JSON document from Kafka Cluster, merges into MongoDB and creates new documents.
| Sync Type | Field Classsification | Sync Strategy |
|---|---|---|
| Type-1 | Near Real Time Sync for critical fields | Using Kafka Queues / CICS Event Triggers / DB2 / 3rd Pardy CDC Replicators / Queue Replicators |
| Type-2 | Scheduled Batch Polling sync for less critical fields | Using Mini Intra-day Batch / Replicators / ELT |
| Type-3 | EoD Batch sync for non-critical fields | Batch CDC Sync Sync / Update / ELT |
How MongoDB and Wipro's ModerniZ helps
Wipro’s ModerniZ platform provides multiple tools to accelerate the modernization journey across phases, from impact analysis to design to build to deployment. For data modernization, ModerniZ has 5 tool sets like PAN (Portfolio Analyzer), SQL Converter, automated data migration, and so on—all which can be leveraged to yield a minimum committed productivity gain of 20%.
Figure 3. Mainframe to cloud transformation using ModerniZ
Why MongoDB for mainframe modernization?
MongoDB is built for modern application developers and for the cloud era. As a general purpose, document-based, distributed database, it facilitates high productivity and can handle huge volumes of data. The document database stores data in JSON-like documents and is built on a scale-out architecture that is optimal for any kind of developer who builds scalable applications through agile methodologies. Ultimately, MongoDB fosters business agility, scalability and innovation.
Some key benefits include:
- Deploys across cloud in nearly all regions
- Provides a document model that is flexible and maps to how developers think and code
- Costs a fraction of the price compared to other offloading solutions built using relational or other NoSQL databases and even a larger, sharded MongoDB environment brings magnitudes of savings compared to the traditional mainframe MIPS-based licensing model
- Allows complex queries to be run against data via an extremely powerful aggregation framework. On top of that, MongoDB provides a BI connector for dedicated reporting/business intelligence tools as well as specific connectors for Hadoop and Spark to run sophisticated analytical workloads
- Offers enterprise-grade management through its EA product and includes security features which cover all areas of security – authentication, authorization, encryption and auditing. Competitors often only offer a subset of those capabilities
- Provides a unified interface to work with any data generated by modern applications
- Includes in-place, real-time analytics with workload isolation and native data visualization
- Maintains distributed multi-document transactions that are fully ACID compliant
MongoDB has successfully implemented mainframe offloading solutions before and customers have even publicly talked about the success (e.g. a top financial service enterprise in the EU)
Roadmap and contact details
Work in progress on offloading mainframe read workloads to GCP native services and MongoDB.
Contact partner-presales@mongodb.com for more information.