Every month we receive hundreds of thousands of inquiries from our customers spanning hundreds of categories. To help our customers resolve their problems, we use a routing algorithm to route each inquiry to a suitable customer support agent among hundreds of agents. What does such a routing algorithm look like?
This post is for Day 24 of Mercari Advent Calendar 2022, brought to you by Prashant Anand from the Mercari Contact Center team.
You want to get your certificate of residence (住民票) [1]. What do you do?
You go to your ward office. You know that this certificate is issued by the Family register (戸籍住民サービス課) counter, so you go to that counter and get a ticket number. Then you wait until your number is up, give the application to the person behind the counter, and then the certificate is issued.
If you think about this process, once you arrive at the ward office, there are two main steps to get your certificate:
- find the counter that you need to go to in the ward office
- after creating a ticket, wait until your number is up
We handle inquiries sent by our customers at Mercari in a similar two-step process. First, we need to find out what the inquiry is about and then wait until a customer support (CS) agent that can handle the inquiry is available. In this article, we’ll learn about the design of a system that can achieve the former, i.e., a system that helps us find the category of the inquiry. We’ll NOT be looking at how we manage the queue of inquiries after we have identified their categories.
How to Find a Suitable Agent for an Inquiry
When Yuito [2], a 35-year-old school teacher living in Katori, could not log into his Mercari account and reset the password, he sent us an inquiry to recover his account. After Akari [3], a lawyer in Osaka, bought a t-shirt on Mercari, which was shipped via Raku-Raku Mercari Shipping, but has yet to receive the shipment even after five days, she contacted us to resolve the problem. Similarly, we receive inquiries about payments, counterfeit items, registration, and more.
To handle these inquiries, we could assign them randomly to any of the available agents but that would require all agents to know how to handle all kinds of inquiries before they can start working.
So, our CS team has created about 150 different categories of inquiry. Each of these categories is called a Skill. Categorizing inquiries helps agents develop specializations. One agent can become proficient in handling a particular type of inquiry.
In practice, each of our CS agents has a few skills. If an agent has four different skills, it means they have the expertise to handle four categories of inquiries. If we could identify the skill required to handle an inquiry, we could send it to one of the agents with that skill. In the rest of the article, we’ll see how to identify the skill required to handle an inquiry.
The Goal of the Routing Algorithm
If you want to get in touch with our customer support, we have a chat interface. This chat interface allows you to talk back and forth until your problem is solved. We call each message sent by a customer Contact and each response sent by our CS agents Reply, and the group of Contacts and Replies between a customer and our CS agents to solve a problem is called a Case.
So, whenever a customer sends us a message about a problem for the first time, we create a new case. Now the first message is quite important. In the first message, customers explain the problem they are facing. So, we use it to determine the skill required to respond to the inquiry.
Fig 1: Interaction between a customer and CS agents in a chat interface
Our routing algorithm aims to assign one of the skills to a case after receiving the first contact. In the next section, we’ll learn how to do exactly that.
Ticket Routing Algorithm
Ticket routing, i.e., assigning skill to a case, happens after we receive the first contact and before our CS agents send the first reply. Our routing algorithm works in two steps. In the first step, we assign a skill to each case. In the second step, depending on certain criteria, the skill of the case might or might not be updated. Let’s take a look at these steps one by one.
Step 1: Contact Type Routing
Fig 2: Screens to contact customer support through the Mercari app
As shown in the figure above, when customers want to send us an inquiry, they have to choose what the inquiry is about. This category chosen by the customer is what we call Contact Type. For each of these contact types, we have a corresponding skill stored in a table. Think of this table as a mapping from contact type to skill.
Fig 3: Contact type to skill mapping table
So when a new case is created, we look at the contact type of the case, find the corresponding skill from the lookup table, and assign that skill to the case. Since the lookup table contains a corresponding skill for all possible contact types, all the cases get assigned a skill in this step.
Step 2: Case Assort Rule Routing
The skill assigned to the case in step 1 based on the contact type is not always correct. Even for a single contact type, we need different skills depending on the situation. So far, we also have yet to consider the inquiry text. In step 2, Case Assort Rules help us reassign skills to the cases taking the inquiry text into account so that the assigned skills are more accurate.
A case assort rule has four attributes:
- Contact type: One of the contact types
- Include words: A list of words
- Exclude words: A list of words
- Skill: One of the skills
We say a case assort rule matches with an inquiry if all of the following three criteria are satisfied:
- the rule and the inquiry have the same contact type
- all of the include words are present in the inquiry text
- none of the exclude words are present in the inquiry text
If all three criteria are satisfied, then we can assign the skill of the case assort rule to the inquiry. However, if any of the three criteria is not satisfied, the skill of the inquiry can’t be updated by the rule.
Let’s understand this through an example. Let’s consider the following case assort rule:
- Contact type: Shipping
- Include words: [らくらく]
- Exclude words: [振込み, アカウント]
- Skill: Logistics_Rakuraku_Mercari_Shipping
Let’s check if the following inquiry matches the case assort rule we just created:
- Contact type: Shipping
- Text: お取引相手の住所に不備があり、転居先不明と取引画面で出ています。正しい住所を入れるようにコメントしたのですが、返事が一向に帰ってきません。私はらくらくメルカリ便で送ったのですが、このような場合送った荷物はどうなるのでしょうか。
- Skill: Logistics_Gen
As you can see, both the rule and the inquiry have the same contact type, Shipping. There is only one word in the include words list in the rule, らくらく, and it is present in the inquiry text. There are two words in the exclude words list in the rule, 振込み, and アカウント, and none of them are present in the inquiry text. Since all three criteria are satisfied, we say the rule matches the inquiry. So, we can update the inquiry skill from Logistics_Gen to Logistics_Rakuraku_Mercari_Shipping. This example is shown in the graphic below as well.
Fig 4: A sample inquiry that matches with a sample case assort rule
Let’s take a look at another example where the inquiry and the case assort rule created above do not match. Let’s consider the following inquiry:
- Contact type: Shipping
- Text: 発送方法をメルカリ便にしようと思って操作しようとしたのですが、誤って商品を発送する前に発送ボタンを押してしまいました。できれば発送前の状態に戻していただきたです。
- Skill: Logistics_Gen
As you can see, both the rule and the inquiry have the same contact type, Shipping. However, the include words present in the rule, らくらく, is not present in the inquiry text. So, we say that the rule didn’t match the inquiry, and this rule can’t update the skill of the inquiry. We’ll leave the skill of the inquiry, Logistics_Gen, as it is. This example is shown in the graphic below as well.
Fig 5: A sample inquiry that doesn’t match with a sample case assort rule
We have a set of about 250 active case assort rules. When a new case is created and assigned a skill based on contact type in step 1, we come to the case assort rules in step 2. We start with the first case assort rule and check whether it matches the inquiry. We go down the list of rules until we find a match. As soon as we find a match, we assign the skill suggested by that rule to the case and stop going further down the list of rules. If we went through all the rules and none matched with the inquiry, the skill of the case is not updated.
So the overall process of inquiry routing has the following four steps:
- Create a new case when a customer sends the first contact
- Assign a skill to case based on the contact type selected by customer
- Update the skill of the case if the inquiry matches any of the case assort rules
- Assign the case to one of the CS agents who has the skill present in the case
Fig 6: 4 step customer inquiry routing algorithm
Conclusion
We looked at the design of a routing algorithm for customer inquiries that takes the inquiry text and inquiry category selected by the customer into the account for routing. The routing happens in two steps. In the first step, a skill is assigned to the case based on the inquiry category (contact type) selected by the customer. In the second step, the skill of the case is updated only if the case satisfies all 3 criteria for any of the case assort rules. After this step, the inquiry is routed to an agent with the skill present in the case.
Now looking at this system does leave us with many questions. How do we evaluate the performance of this system? Since both contact types and skills represent the categories of inquiries, why do we need both? Why can’t we just use the contact types for routing? How do we create case assort rules? Since we start matching an inquiry from the first rule and then go down the list of rules, is the order of the rules important? If yes, how can we find the optimal order? Can machine learning help us in improving the performance of this system? If yes, how can it be integrated into this system?
If you want answers to these questions, stay tuned to our engineering blog. What are your thoughts about these questions? Feel free to reach out, and I would love to discuss more.
The final article in the Mercari Advent Calendar 2022 series will be published tomorrow by @yasu_shiwaku. Look forward to it!
Footnotes
[1] This situation is specific to residents of Japan. A certificate of residence is a document that you can get from your local ward office. Read more details about it here.
[2] The story, all names, characters, and incidents portrayed in this example are fictitious. No identification with actual persons (living or deceased), places, buildings, and products is intended or should be inferred.
[3] The story, all names, characters, and incidents portrayed in this example are fictitious. No identification with actual persons (living or deceased), places, buildings, and products is intended or should be inferred.
