Introduction
Hello, I’m otter from the Search Backend team at Mercari JP.
In this article, I’d like to talk about the improvements we made when we revamped the interface of the Search API, which is in charge of a BFF-like role.
Issues that the API had
As many of you may know, Mercari has adopted a microservice architecture for its entire service development.
The Search API was also converted from the monolithic API to a microservice, and in order to ensure backward compatibility with clients such as iOS and Android, the API interface and specifications were migrated as is. As a result, it was difficult to flexibly respond to new measures on the service side, and on the system side, error handling and pagination, which should be standardized for each microservice, did not follow the internal guidelines of Mercari. In addition, the API protocol was not gRPC, which is recommended for all Mercari services, but HTTP, which meant that the system performance and development experience could be improved.
What has been improved in Search API v2
I would like to explain how exactly we improved these issues.
Multi-templates for search results
One of the main reasons why we needed the Search API v2 was to display products from Mercari Shops in the Mercari search results.
This is where we came up with the idea of creating multiple templates for search results. In order to display non-Mercari products in the search results in the future, we will add types to the search result items and change the content displayed for each type.
In addition, the default data source for the API request parameters used to be only Mercari items, but we have changed the interface to specify the data source for multi-sourcing.
In the search result shown in the above figure, "Mercari, Shops, Service A, Service B" is specified as the dataset, and the figure shows that the template for each product type is applied.
Componentization of the search UI
The client-side had information on where and under what conditions each element other than the items to be displayed in the search results would be displayed. However, there were issues such as not being able to flexibly respond to changes and not being able to delete old elements from old clients, so we decided to make each element of the UI a component and return the component position, such as how many lines to display, from the API.
Also, for Facet Filter (refining search criteria), the client application used to call the Facet Filter API which has a complex API interface, but we have changed it to return the Facet Filter responses from the Search API as BFF.
gRPC and Protocol Buffers
The Search API also provided an interface definition using Protocol Buffers as well as a gRPC server for the endpoints provided to other microservices, but as mentioned above, for clients, only the HTTP protocol was supported, following the interface of the monolithic API era.
The API guidelines created by the Mercari architect team also strongly recommend gRPC as the API protocol.
gRPC is strongly touted for its performance benefits, but I think the biggest benefit for APIs that are provided to clients like this one is the definition of interfaces using Protocol Buffers.
At Mercari JP, the definition of Protocol Buffers for each microservice API is centrally managed, and CI automatically generates the source code for the gRPC server and client for every language we support.
When we want to communicate the API interface to iOS and Android engineers or get feedback, we can show them the Protocol Buffers pull request to clarify. And the best part is that type-safety is guaranteed between client and server.
Error Handling and Pagination
For clients, error handling and pagination specifications are some of the items they would most like to see standardized.
These specifications are also defined in the API guidelines of Mercari, but basically, they are in line with the error handling and pagination of the Google API guidelines.
However, even if it is one of the most important things to standardize, the specifications for error response and pagination can not be changed easily since we need to ensure backward compatibility.
Therefore, when creating API v2, it was essential to unify the specifications in this area.
Conclusion
In this article, I introduced the issues and improvements that led us to publish a new version of the Search API interface.
In creating the API v2, we spent a lot of time in the design phase because it would become the foundation for the future, but we did not have that much trouble in the actual implementation phase of the API v2. I think the main reason for this was that the Search API had already been carved out as a microservice.
Migrating and replacing services like this every few years will improve the current development experience and make future replacements easier for members like me who take over later. I think one of the strengths of Mercari is that we are able to move forward with such large-scale modifications of system requirements as a project.
The Search Backend team is looking for members.
If you are interested in it, please see the application guidelines.
