Introduction
Robust Foundation for Speed (RFfS) is a project that was newly launched at Mercari. It is an effort to solve complex technical problems within the current shared business infrastructure so that Mercari can continue to pursue rapid, strong growth into the future.
This effort involves analyzing and improving the existing system, and doing so rapidly and on a large scale. Mercari sees this refactoring as a project for the medium-term, given the highly challenging, company-wide nature of the project. In this article, we asked @mtsuka, the RFfS lead, about the project, the motivations behind it, and its goals.
What Is This "RFfS" that Mercari is Focusing On?
— Before we ask about the details concerning RFfS, why are you working to strengthen the shared business infrastructure now?
Up until now, to establish our services as a business, Mercari has emphasized product market fit. In other words, we have focused on creating features that customers need and providing value. Even now, many engineers approach their work each day with a desire to create new things and have an impact on the world.
Amidst all of this, there are some areas that have been more or less overlooked as a result of the balancing of priorities. One such area is maintenance of business infrastructure, which is the function that supports and provides the foundation for Mercari’s business operations. If we overly neglect this area, this will lead to what is called technical debt. We just weren’t able to find a good opportunity to address this. But this time, with the encouragement of our CEO and CTO, we decided to invest in strengthening our business infrastructure. The result was the launch of the Robust Foundation for Speed, or RFfS. RFfS aims to bring significant value to business continuity and expansion by focusing technical resources on improving business infrastructure.
Even the so-called tech giants are investing most of their engineering resources into platforms and underlying technologies. For this reason, we believe that RFfS is a very important initiative.
— What are your specific goals?
Mercari has been working on microservices migration for two years (since 2019). The basic idea here is the same as with that initiative. It’s just more comprehensive and meta in nature. We believe that creating a system that is easy to develop and maintain is key to continuing to deliver functionality and value. Monolithic systems are difficult to keep deployed within a large organization, as small changes often affect many parts of the system. Shifting the architecture to a microservices architecture makes it easier to deploy and change, but RFfS aims to make this even more reliable and speed it up.
The Challenges RFfS Will Address
— Please tell us about the areas that RFfS will focus on.
We figured that tackling the most difficult issues would have the greatest impact. With this in mind, we analyzed the current situation and found that there were areas that had been left untouched because they were complex and had a large impact on users, despite those areas having a large business impact. There were also areas that we didn’t know anything about due to the absence of the engineers and staff that had previously been in charge.
For these reasons, we defined the following three areas as our key areas.
— Let’s talk about the challenges in each of these three areas. First, what are the challenges when it comes to C2C Transactions?
C2C Transactions refers to the system that handles all of Mercari’s business transactions. In other words, it is the business logic of Mercari itself. The features here are still tightly coupled. Since this system is related to transactions, it is also the part that is called by every feature. Our goal is to divide this area into different domains suitably. We need to consider not only the business logic, but also the user experience and future business expansion so that we can divide the domain in a skillful manner and convert it into microservices, or a modular monolith. The biggest challenge is figuring out how to divide this functionality in the most optimal manner. I think this is a difficult task.
— What are the challenges associated with the ID platform?
In general, I think that in many cases, the account-related parts of today’s services can be handled using a social platform or a type of service called IDaaS. When Mercari needed an authentication infrastructure, there was nothing that could meet those kinds of requirements, so we had to create our own. As Mercari has continued to grow, so has the scale and complexity of its systems. That kind of authentication infrastructure needs to meet the specifications defined by NIST 800-63, which is difficult to understand and implement correctly. Plus, not many engineers have expertise in these ID domains, so it is difficult to recruit personnel.
As for the authentication authorization mechanism, because Mercari handles personal information, making mistakes during implementation can lead to business risks. In addition, in terms of our Group companies (Merpay, Mercoin, and Souzoh), the ID platform is also very important for collaboration. Failing to implement the authentication platform correctly could hinder the expansion of our businesses. Because speed is important here at Mercari, this is a difficult issue.
— Finally, what are the challenges you face with regard to CS tools?
As Mercari’s services continued to grow, customer support tools (CS tools) were created on an ad hoc basis. Basically, small CS tools were created for each feature, and it has become increasingly difficult for humans to manage them. That’s why we would like to turn these CS tools into a platform.
Since individual CS tools are tightly coupled to each feature, analyzing them is a major challenge. It may seem like a good idea to divide them by business domain or company, but along with the architecture, they would need to be redesigned and integrated. Buying and selling alone is not the only value provided by Mercari. The safety provided by these CS tools is an important part of the customer experience as well. CS tools also handle personal information, which is a difficult issue from the perspective of project management.
The Present and Future States of RFfS
— What are you careful about when proceeding with a project?
First of all, it’s important not to deny the past. I myself have only been with the company for about two years. So I don’t know what arguments were made in the past, and many solutions were implemented according to the conclusions that seemed best at the time. That’s not a problem. Because the situation has changed now, so have the solutions that are considered optimal. For this reason, we have adopted an approach in which we listen to those involved first before proceeding.
— What is your roadmap for the future?
First, by the end of the year, we will explore what we still don’t know and discover what’s there. After that, we would like to start domain analysis and API partitioning design next year. We will probably be able to start implementing the actual migration toward the end of mid-2022. The implementation will take about a year from that to complete.
— How do you feel about the project thus far?
To be honest, it is a difficult project. However, it seems we’ll be able to get a handle on what we don’t know once we finish our deep dive . What remains to be seen is how many unknown areas will be left after that. It will be a lot of hard work and require a lot of time, but I think we’ll be able to accomplish our goal.
The Kinds of Engineers Needed for RFfS
— What qualities and mindset should an engineer have to effectively contribute to RFfS?
In this project, engineers are exposed to technology that is, at times, several years old. We need engineers who are not put off by dealing with older tech. In the end, technology is just a tool, so it does not matter how old or new it is. One’s objectives are important, naturally, but what’s more important is whether you can make reliable efforts to handle complex issues.
We are looking for so-called problem solvers who can focus on figuring out how to solve complex problems and provide value. I think this kind of experience is very useful for an engineering career.
This is the kind of mindset with which the team is approaching the project.
— Can you tell us about the technical aspects?
Mercari was originally developed using the LEMP stack (Linux, nginx, MySQL, PHP). Today, it uses Go and is shifting more toward microservices, but there are still on-premise environments. In fact, the core of Mercari is a system written in monolithic PHP.
Since there are still a lot of PHP systems in use, I would love to work with someone who is knowledgeable about LEMP stack development. There are plenty of opportunities for such individuals to play an active role.
— So committing to RFfS will not only help people grow as engineers, but also contribute to the industry itself.
Mercari has adopted a microservices architecture to scale its business operations. We have achieved wonderful results, but there are still areas that remain unexplored. The RFfS project is the solution to this problem.
Given current conditions, it seems that the microservices architecture is not a silver bullet. Through RFfS, I feel that we can get a glimpse of the world of the future. I hope that we lead the way in innovations beyond microservices, which will in turn contribute to the IT industry.
It’s going to take a long time and a lot of work, but this is the vision that is driving us forward!
— Thank you very much!
Hiring information
Mercari is currently hiring. If you are at all interested in joining us, please take a look at our career page and the special page for Robust foundation for Speed.
