Automation/karakuri for increasing engineering productivity

This is the 15th post for Mercari Bold Challenge Month.

Hi, I’m @celia, an iOS Engineer working at Merpay. In this article, I want to share about how we use automation to increase the team’s productivity. We use tools like fastlane and appium for automated tests, but that’s a topic for another blog post.
Today I want to introduce how our automation infrastructure allows engineers to implement features that help the team to be more productive, and automates repetitive manual tasks. Mercari’s engineering culture encourages us to take ownership, automate routine tasks and create tools to keep the team’s daily operation running smoothly (If you want to learn more, see this, and this blog post). Hope by the end of this article, you will gain a broader understanding of automation, and maybe play with your own toolkit a bit more.

Firstly, I will give you a quick introduction on the tools and platforms we use and how it’s all connected. Next I will talk about how I created karakuri to improve the team’s productivity. The first one is a slackbot feature that helps the team to pick up code reviews faster. The second one is a command that automates the integration between merpay and mercari iOS codebase, that saves us from doing repetitive manual work.

Introducing mobile team infrastructure

Our infrastructure is made up of four main components:

• Automation: Tasks triggered by github webhook, scheduled (cron) tasks
• Bot: Tasks triggered by slackbot command, such as running tests, deployment, release, and integration
• CI: Bitrise, CircleCI
• Data: Unit test coverage, test running time, CI success rates

Automation, Bot

We have a project mercari-mobile-infrastucture that holds code for automation and bot. While there are many ways to build a service that integrates github API and slackbot, we use AWS SAM. As a mobile engineer who doesn’t have any prior experience with AWS Lambda, I appreciate SAM CLI’s friendly interface, the fact that it allows me to debug my code locally while developing, and that when used with Github actions it does the heavy lifting for us when deploying.

CI

We use CircleCI to run scheduled tasks, and to trigger workflow when changes are made to a Github repository (To learn more about how we use CircleCI, check out this post from the Android team). We also use Bitrise, to run jobs for integration, deploy, release, etc. Ideally we’d love to have to only maintain one CI platform, but when the project was started, CircleCI didn’t have an available REST API to trigger specific workflows. Recently they released API for that, we are considering migrating to it after it becomes more stable. At the time of writing, we still use Bitrise for all of the workflows that are triggered by using API.

Data

We gather reports about test coverage, test running time and CI success rates, and use tools like kibana or looker to help our team to make sense of the data.

How does it work?

If you are not familiar with how serverless application, webhook/API, and CI platform interacts with each other, let me explain a bit further. Feel free to skip this part if you already know what these are.

The serverless application is a group of functions running on AWS, it’s subscribed to webhook events and contains code implementation that has the ability to trigger events by sending requests to the API. In our case, mercari-mobile-infrastucture’s main features include:

1. Github integration. When something new happened on Github, the webhook sends an HTTP POST payload to it’s configured URL, so our functions know an issue is created, someone just approved a pull request, etc. Backwardly, the functions can call Github API to do things just like a human can do on the website, such as creating an issue, commenting, or approve someone’s pull request etc.

2. Slackbot integration. We use slack-apps’s interactive messaging and incoming webhooks to allow members to initiate tasks that’s actually ran by our functions. We use this for managing releases, distributing beta app, trigger tests on CI, manage permission for Apple Developer and AppStoreConnect and more. When the bot is mentioned in a message, slack forwards it to mercari-mobile-infrastucture, then the message content gets parsed using yargs so it knows what the user is trying to do. Then it performs the task by sending requests to Github API or triggering a build on bitrise by sending a request to its API.

3. Scheduled tasks running on AWS. The scheduled tasks for our iOS codebase are running on circleCI, but there are a few other tasks that involve a schedule (for example, release) or integration with other services that is implemented in mercari-mobile-infrastucture.

Karakuri

Any of the above feature can be extended to a blog post of its own, but this one is about Bold Challenge, so I won’t go into all the details. Instead, I will talk about some new features I added, and the problem it solves.

Code Review Reminder

There are tools like pull panda that can help with team’s collaboration on Github, but after trying it I found it works better for smaller teams. At the time, it was common when someone finished a pull request, he or she would post a slack message and ask for review. I thought automating that process could be enough.

We use labels to organize our issues and pull requests on Github. So I set up a webhook event for type PullRequest, and wrote a function that will post a slack message when the Can Review label is added [*1]:

(this message shows up in slack channel when someone adds the label Can Review to a pull request.)

When the review is finished and someone approves the pull request, the webhook will send event for type PullRequestReview, and if the review state is approved, it will edit the slack message so everyone knows it’s already reviewed:

(this message shows up in slack channel when someone approves a pull request.)

I also added a scheduled job that runs every morning on weekdays to show a list of open pull requests:

It fetches a list of pull requests with state open, and filter the results using labels. It’s very similar to pull panda’s Waiting for review list, but we don’t count hours stale and send out mentions. We designed it this way to protect our voluntary-based code review culture and reduce unnecessary distractions.

Implementation details and code sample are not shared here, but we use octokit for Github API and @slack/client (there are better alternatives these days) for Slack API. The official API documentation and wrapper packages make it easy to get started on your own.

Integration Helper Bot Command

We use carthage to manage dependencies at Mercari. As a matter of fact, Merpay’s iOS code is in its own repository, and it’s integrated into the Mercari app using carthage too. This means Merpay iOS members constantly do carthage updates for Merpay in the Mercari iOS codebase. Keeping separate repositories reduce the complexity of the team and our projects, but it adds a lot of overhead to our QA and release cycle. So we came out the idea to create a bot command to update carthage dependencies.

Typically, to update dependencies using carthage, you update the version or git position in the Cartfile and run carthage update. The same logic applies when it’s automated. Here’s how it’s actually running:

The bot’s command update_library will create a pull request toward branch feature/merpay to update carthage dependency for merpay-ios-sdk to tag v1.13.2.

After clicking yes in the interactive message, it will update links to the bitrise build:

And after the build finishes, it will update the slack message to show a link to the pull request:[*2]

On bitrise, the build is triggered to run a workflow that runs fastlane action update_library, which is where we implement the code to update carthage. The key step is to use a ruby regular expression to match the line for the library, overwrite it, and run the command to update carthage:

# Code sample in update_library.rb
# Overwrite in Cartfile
begin
project_directory = params[:project_directory]
UI.message("Current root directory is #{project_directory}")
Dir.chdir(project_directory.to_s) do
UI.message("Moved to directory: #{project_directory}")
File.readlines('Cartfile').each do |line|
next unless line =~ /#{library}/
start_idx = line =~ /#{library}/
updated_line = is_tag ? "== #{position}n" : ""#{position}"n"
offset = library.length + 2
File.write("Cartfile", File.open("Cartfile", &:read).gsub(line.from(start_idx + offset), updated_line))
UI.message("Updated Cartfile sucsessfully!")
break
end
end
rescue => ex
UI.user_error!("Overwrite in Cartfile failed, please check if Cartfile includes target library.n Error detail: #{ex}")
end
# run carthage update for library
begin
Dir.chdir(project_directory.to_s) do
carthage(
command: "update",
dependencies: ["#{library}"],
use_ssh: true,
use_submodules: true,
no_build: true,
)
end
rescue => ex
UI.user_error!("Carthage update failed. Please try update carthage locally on this branch and see what went wrong.n Error detail: #{ex}")
end

Summary

By having a flexible infrastructure for automation, many of our engineer’s daily tasks can be made simpler. Mercari’s productivity team owns the mercari-mobile-infrastucture project, and they are in charge of maintaining tools and codebase for CI/CD and automation for Mercari’s mobile team. I’m very grateful that they helped me with the features I wanted to make and accepted my work. Working on automation certainly brings a new perspective to me as an iOS engineer.

Next, @ogataka50 will write about microservices化に伴うデータ移行について. Happy reading!

Footnotes

[*1] The slackbot’s icon and name, link to Github is masked because it’s internal use only.

[*2] The link to Github has been masked due to security concerns.