Hi! I’m takady, a backend engineer at Cross Border (XB) Engineering. In this post, I’ll share how we designed and built a flexible order management system from the ground up.
Background
While the existing Mercari Japan’s marketplace has a mature order management system, it’s not easily expandable to global marketplace requirements because of the following challenges;
- One state transition pattern: The existing system’s order lifecycle is coupled to a specific business flow. Adding/Removing steps impacts the large areas of the system.
- Dependencies in DB level: Maintain the data consistency between core resources by sharing DB transactions.
Rather than retrofitting the existing system, we chose to build a new order management system.
Design Decisions That Enable Scale
As the business grows rapidly, business requirements will vary significantly. Making the order management system easier to expand for future use-cases is the key to long-term success.
Flexible Lifecycle of Order Items
State transitions of each item can be defined differently depending on the product type. This flexibility allows the product to easily handle different business scenarios.
Note: These flows are examples to illustrate what our flexible lifecycle system enables. Not all flows are currently implemented, but the architecture makes it straightforward to add new ones as business evolves.
How Lifecycles Work: Data Structure
Each order item references a lifecycle that defines its available state transitions:
This approach means:
- Adding new transaction types only requires defining a new lifecycle without significant code changes
- Multiple items in the same order can follow different lifecycles independently
- State validation is enforced at the lifecycle level, preventing invalid transitions
Orchestrating Distributed Transactions
One of the biggest challenges in any order management system is coordinating actions across multiple services while maintaining data consistency.
Saga Pattern with Orchestration
We employ the Saga pattern with an orchestration approach to manage distributed transactions in comparison with the following options:
-
Saga vs TCC: We primarily use the Saga pattern, which rolls back completed operations through compensating transactions when a step fails. While TCC (Try-Confirm-Cancel) provides stronger consistency guarantees, it requires extra effort as all participant modules need to implement separate Try, Confirm, and Cancel APIs. We only consider TCC on a case-by-case basis for operations that are difficult to roll back.
-
Orchestration vs Choreography: We chose orchestration over choreography because it provides a centralized coordinator that manages all interactions between services. This gives us a holistic view of the system, making it simpler to implement, maintain, and troubleshoot compared to a distributed choreography approach.
Our in-house orchestration tool, “Magician” (developed by Merpay), handles this complexity elegantly. Magician is an orchestration engine for distributed transactions that provides essential functionalities for our use cases, including workflow management (Saga), retry activity, and async workers. A significant advantage is that it’s already proven in production—several microservices at Merpay and Mercoin use it for mission-critical payment orchestration. This means we can reference real-world implementations and leverage shared knowledge across teams.
Example orchestration flow for crossborder order placement:
- Validate order
- Consume coupon from the promotion module
- Process payment through Merpay
- Place order in the order module
- Request proxy partner to purchase items from seller
If any step fails, Magician automatically triggers compensating transactions to roll back previous steps, ensuring eventual consistency across services without requiring distributed locks.
These technical foundations—flexible lifecycles and robust orchestration—work together to create a system that can evolve with business needs while maintaining reliability and consistency.
Why It Matters: Business Impact
These technical decisions directly translate to business value:
Faster Time-to-Market for New Features
- New transaction types can be launched without modifying existing order flows
- Teams can develop and test new features in isolation, reducing coordination overhead
- Example: Adding a new luxury goods verification step doesn’t require regression testing all existing product types
Reduced Downtime and Improved Reliability
- Orchestration with automatic rollback prevents partial failures from leaving the system in an inconsistent state
- If payment processing fails, coupons are automatically restored—no manual intervention needed
- Centralized coordination makes it easier to monitor and troubleshoot issues before they impact customers
Building Checkout and Payment with Merpay’s Solution
Beyond orchestration for order management, we also needed to build checkout and payment capabilities. Rather than building from scratch, we integrated with Merpay’s established checkout and payment foundation —allowing us to focus our engineering efforts on marketplace-specific features.
What Merpay’s Solution Provides
Merpay’s checkout solution provides the foundation for our checkout flow:
- Flexible checkout UI: The element concept allows us to customize the checkout experience
- Multiple payment methods to be supported: Currently credit cards are supported, more payment methods to be supported in the future
- Provider abstraction: Seamless integration with Merpay payment service while encapsulating the actual payment providers behind it
This integration enabled us to:
- Accelerate time-to-market: Launch checkout capabilities quickly while focusing on marketplace-specific features
- Leverage proven reliability: Benefit from Merpay’s battle-tested payment infrastructure used across Mercari’s marketplaces
- Maintain orchestration consistency: Payment processing integrates seamlessly as one step in our Saga workflow
For more technical details about Merpay’s payment solution, you can refer to the blog post by Foghost (Japanese only).
Current Status and Future
We’ve just implemented a basic crossborder transaction flow in this order management system. Going forward, we plan to add more features and roll out to additional regions. We aim to quickly support these expansions by leveraging this foundation.
As the business grows, we can expect to face new challenges. To address these challenges and strengthen the foundation, we will balance infrastructure enhancements with feature development.
Conclusion
In this post, I’ve shared how we designed and built a flexible order management system. The design principles for our order management system—flexible lifecycles and robust orchestration—will be validated through future expansions as we onboard more transaction types and business scenarios, including both crossborder and local transactions.
I’m excited to be part of this project and look forward to sharing more lessons learned as the system evolves.
