This post is for Day 20 of Merpay Advent Calendar 2022, brought to you by pedro from the Merpay Credit Design Backend team.
In the Merpay system, invoice microservice manages customers’ invoices of deferred payment service so called Merpay Smart Payments(メルペイのあと払い) and small-sum loan service so called Merpay Smart Money(メルペイスマートマネー). It also provides various methods to repay those invoices.
But in the very beginning of the Merpay system, there was no invoice microservice. This post describes why we decided to create the invoice microservice and how we are growing it from just a concept to an independent running microservice.
The System Without Invoice Microservice
Before Merpay released the Merpay Smart Money, there was only one service issuing invoices to customers, which was Merpay Smart Payments. It was not necessary to have a dedicated invoice microservice to manage the invoices.
Smartpay microservice supported the entire process of using Merpay Smart Payments, from deferred payment, issuing monthly invoices to repayment of those invoices.
As shown in the diagram, the following invoicing related functionalities were all provided by smartpay microservice.
- calculate the service fee based on the usage, and issue monthly invoices.
- repay the invoices at the convenience store or using Merpay balance.
- automatically charge from the bank account to repay the invoices.
- schedule the day for the automatic charging.
Because smartpay microservice contains all Merpay Smart Payments related data, providing these functionalities in the same microservice is much simpler than having a separate invoice microservice.
Invoice Microservice Inside Smartpay
In 2021, Merpay decided to launch a small-sum loan service called Merpay Smart Money. While analyzing the requirements of smart money, we found out there were so many common functions with Merpay Smart Payments, especially for those related with invoicing.
There were several system design plans discussed at that time, with pros and cons for each. Finally we decided to have a dedicated invoice microservice to manage invoices of both Merpay Smart Payments and Merpay Smart Money, because it would bring us several benefits.
- the common functions can be reused as much as possible.
- charge only once from the bank account and repay both invoices if the customer schedules the same repayment date.
- easy to manage repayment priority of different invoices, especially when there is limited repayment source.
Ideally, we have to create two new microservices, one for the Merpay Smart Money domain and the other for the invoice domain. Also a big change is needed in smartpay microservice, to split the invoice related functions to invoice microservice.
While considering the limited development resources and the release schedule, we have to give up the plan of creating the ideal system in the very beginning of the Merpay Smart Money launch, instead we decided to implement the system step by step.
As for the first step, only minimum development was needed for the Merpay Smart Money launch, including the new smart money microservice and extending smartpay microservice to provide invoice functionalities by a new gRPC server. Implementing a new invoice gRPC server instead of mixing the functionalities within the existing ones for Merpay Smart Payments, it also left us the opportunity to migrate invoice microservice outside of smartpay in the future.
Although in this step we just created a gRPC server inside the existing microservice, and the invoice microservice was more like a concept, it is still important to clarify the responsibilities of each microservice at that point. Invoice microservice supports both invoices of Merpay Smart Payments and Merpay Smart Money, it should only deal with abstracted invoice models. As the clients, smart money and smartpay microservices should handle each domain’s specialized logic and data, for example: calculating service fee or interest to be invoiced each month.
To achieve this, remodeling some invoice related tables inside the smartpay microservice was necessary. Take the invoices table as an example, we needed to distinguish between the fields only for the Merpay Smart Payments domain and those for an abstract invoice, and add new fields if necessary.
The Migration Project
Although we were able to launch the Merpay Smart Money with minimum development, it was not the final goal as there were still several technical debts to be repaid.
Problems
The problems can be mainly categorized into two types, one with the codebase and the other with the database.
Although gRPC servers as the interface are separated, the source code for invoice domain and for Merpay Smart Payments domain are still in the same smartpay codebase. Apparently this brings us the following issues.
- increased complexity of codebase.
- risk of unintentional effect on invoice microservice with Merpay Smart Payments function update.
- risk of unintentional effect on Merpay Smart Payments with invoice function update.
As for the database, entities of some tables represent both the Merpay Smart Payments domain and the abstract invoice domain. This increases the difficulty to understand the data model as well as the complexity and maintainability of data analysis queries.
Migration Plan
When talking about the migration of invoice microservice from smartpay to an independent microservice, three systems are involved.
- At first, an independent invoice microservice needs to be created to provide the same invoice gRPC server and run the same invoice related batches.
- Current client of invoice microservice, the smart money microservice needs to switch the request destination from invoice inside smartpay to the new invoice microservice.
- The smartpay microservice needs to use invoice microservice to achieve invoice related functionalities.
We want to avoid introducing all these changes to all the systems at the same time, so we use the current invoice gRPC server inside smartpay as a proxy, and process the migration on each customer’s basis. Based on the customer’s migration status, the invoice gRPC server inside smartpay decides to proxy the request to the new invoice microservice or process it within smartpay microservice. Thus we can control the migration progress easily, and nothing needs to be changed on the client side before finishing all customers’ migration.
After finishing the migration for all the customers, the client side could switch the destination from invoice microservice inside smartpay to the new invoice microservice. And the invoice microservice inside smartpay could also be deprecated.
The migration project is still ongoing, and there are several interesting topics in the migration process, for example: how to keep the consistency of data, TDD practice, etc. Hopefully we can share more with you in the future.
In The End
It‘s never an easy thing to design and implement a perfect system architecture. The perfect architecture today may not suit tomorrow’s requirements. Rather than always aiming to build a perfect architecture, making the decisions based on current situation, recognizing the technical debts in those decisions and repaying those debts timely with a plan might be a more practicable way to run a microservices system.
Tomorrow’s article will be by @fivestar. Look forward to it!
