The 5th day’s post of Merpay Tech Openness Month 2020 is brought to you by Robert from the Merpay Cash I/O Team.
I talked a little bit about what our team does in my previous post and the one before, but at that time we were still part of the Bank Team.
Today I want to talk about why we changed our team name to the subjectively cooler Cash I/O and how we went about migrating the user payout system from our legacy PHP codebase to a Go based microservice architecture.
Bank Team? Cash I/O Team?
What’s the difference you ask?
Since even before the launch of Merpay our team was known as the Bank Team and we were primarily in charge of anything bank related at Merpay.
That meant connecting your bank accounts to Merpay, allowing you to recharge your Merpay balance in real-time as well as providing internal APIs to other microservices that can use them to create an even better product, like give our users the ability to use Merpay Smart Payments
Apart from purely bank related functionality we also integrated with 7 Bank ATM so our users can top up their Merpay balance by going to any 7 Bank ATM
Bank recharges and ATM topups make up the "Cash In" part.
While it was still being discussed at that time, the next big project we wanted to do was to migrate user payouts from the legacy PHP codebase that has served Mercari since the beginning to a more modern Go based microservice architecture.
Let’s leave the discussion about which one is better for another time, but what’s important is that with this we would be able to increase the reliability and improve our team’s understanding of the whole "Cash Out" part as well.
So Cash In and Cash Out together make Cash I/O.
User payouts
Going back to the main topic: User payouts. What is this you ask?
The Japanese term that we use is 振込申請 and it deals with "getting money out of Mercari".
Mercari is not only the best place to buy great items, it’s also the best place to sell them!
When you sell something, your sales proceeds in Mercari will increase and you can use them as part of your Merpay balance to buy anything you want immediately using many different payment methods like
But what if you don’t want to buy anything and just like the feeling of cold hard cash?
Well, to get your money out of Mercari/Merpay, you need to apply for that and we call the process "User payout".
Migration
As you can imagine, the migration for something as core to the concept of selling took some time.
One reason why it took a long time is that it was, and still is, being done in several phases.
The first big phase was completed last year when we launched Merpay and the ownership of anything payment related in Mercari was transferred to Merpay. So, with Merpay having a completely microservice based architecture, we had to at least support the concept of sales proceeds and, consequently, user payouts.
Here you can see the original flow:
And here the flow with Merpay in the mix:
That was a bit of history, but then what did we do this time?
As you can see from the diagram we still call the Mercari Monolith to execute the actual payouts.
In the latest phase, we wanted to migrate this part into a microservice so it would look something like this:
There are two things here that you might notice:
1.we still call the Mercari Monolith
2.what’s up with Mercari Microservices?
To answer 1., we still need to call the Mercari Monolith for some data that hasn’t been migrated to a microservice.
To answer 2., Merpay is using microservices so to unify the architecture engineers working primarily on Mercari are also migrating several functions from the Monolith into microservices.
While this made it seem more complex, that’s not exactly the case. It is complex, but, in order to migrate the whole process to microservices, we had to clearly define all APIs and separate responsibilities.
This was one of the parts where we were able to see the biggest benefits.
Previously, we had calls to the DB being done not only inside our main backend system, but also inside the administrative tool for customer support, which is located in a separate repository. This led to the tricky situation where a change in the main backend repository needed to be done in consideration of this administrative tool. Thus we had to be extremely careful in the order of deployments so as to not affect our customer support.
With the migration done, both systems can communicate with our microservice only via API calls and the DB is just an abstraction they don’t have to consider.
We were also able to fix a few bugs that could lead to inconsistencies in rare cases and we also improved the audit trail of what’s happening inside the system.
This made the whole process more robust and if any specific part fails, we can reliably rollback as well as know exactly where the issue occured.
Future
As I said before, this migration is being done in several phases and it’s not over yet.
We want to completely remove our dependency on the Mercari Monolith so in the future all communication will happen between microservices like:
Of course all diagrams presented here have been greatly simplified and there’s a lot of secret sauce in the code so if you want to work with us on improving the reliability and make Mercari/Merpay even better then do join us!
Special thanks
Lastly, I want to say a big thank you to everyone that worked on this project, and there were a lot of people involved. Without you it definitely wouldn’t have been possible! Special thanks also go to our mysterious photographer 😉
Finishing with the two moments that made me most happy, and afraid, because I know that many people depend on this for their livelihood.
and then the final
