E2E Tests Every Developer Can Write — Test Platform Built with Plain go test

Introduction

Hi! My name is @ryotarai, and I’m responsible for SRE & Enabling for Crossborder (XB) Engineering.

As part of the series, Behind the Scenes of Developing Mercari’s First Global App, “Mercari Global App,” this post takes a deep dive into end-to-end (E2E) testing for the project’s backend APIs. Specifically, I’ll share how we built an E2E testing foundation that any developer can maintain, and I’ll cover the design philosophy and its implementation.

Why We Needed to Improve E2E Tests

Challenges with conventional E2E testing

E2E tests for backend APIs play a crucial role in verifying that the entire system functions correctly. Despite this, many projects run into the following problems.

1. Complex setup: Preparing the test environment takes time, which does not allow developers to run tests readily.
2. Hard to run test in parallel: Test must compete for resources, leading to long runtimes.
3. Reliance on individuals: The QA team is the principal organization in charge of maintaining tests, which makes it hard for developers to work with tests themselves.
4. High learning cost: Testers have to learn how to use specialized frameworks or DSLs.

At the outset, our project faced these issues too. Especially when only the QA team maintained the E2E tests, we ran into a number of problems like the following:

• API changes pushed E2E test updates down the priority list.
• Slow test additions led to lower coverage.
• Developers did not understand test implementations, complicating debugging.

Our Goal: E2E tests that allow everyone to contribute

Our goal was a structure that allowed every developer writing API code to maintain the E2E tests, instead of just the QA team.

To make that possible, the setup needed to meet the following requirements:

• Be able to write tests using technologies developers already use daily
• Ensure a low learning cost so testers can get to work immediately
• Be able to use IDE features like code completion and refactoring
• Be able to run the same way locally and in CI

Framework Design Philosophy

The philosophy: “Write it with plain go test”

Mercari Global App backend APIs are implemented in Go. Ultimately, we chose to write E2E tests as ordinary Go code using go test. There were a few reasons for this:

1. Zero learning cost: Developers already know how to write code in go test.
2. Type safety: Developers can directly use Connect’s generated clients and get compile-time checks.
3. IDE support: Completion, refactoring, go-to-definition, and more are all available.
4. Easy debugging: Team members can debug it like any regular Go program.
5. Leverage existing code: Test helpers, mocks, etc. can be reused.

This move changed E2E tests from something “apart” from our work into a part of our everyday development workflow.

At the core of our E2E framework is the design principle: “You can write it with plain go test.”

Let’s look at a real test example:

func TestUpdateNickname(t *testing.T) {
    t.Parallel()

    tests := []struct {
        name     string
        userID   int64
        nickname string
        wantCode connect.Code
    }{
        {
            name:     "Success",
            userID:   createTestUser(t).ID,
            nickname: "NewNickname",
            wantCode: connect.CodeOK,
        },
        {
            name:     "Blank nickname returns error",
            userID:   readonlyUser().ID,
            nickname: "",
            wantCode: connect.CodeInvalidArgument,
        },
        {
            name:     "Non-logged in user returns error",
            userID:   0,
            nickname: "TestNickname",
            wantCode: connect.CodeUnauthenticated,
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            t.Parallel()

            testenv.Run(t, func(params env.RunParams) {
                client := accountv1connect.NewBFFAccountServiceClient(
                    http.DefaultClient,
                    params.Server.URL,
                )

                req := connect.NewRequest(&accountv1.UpdateNicknameRequest{
                    Nickname: tt.nickname,
                })

                if tt.userID != 0 {
                    // Set authentication header
                    setAuthHeader(t.Context(), req.Header(), tt.userID)
                }

                _, err := client.UpdateNickname(t.Context(), req)
                if connect.CodeOf(err) != tt.wantCode {
                    t.Errorf("error code = %v, want %v",
                        connect.CodeOf(err), tt.wantCode)
                }
            })
        })
    }
}

This code uses Go’s standard table-driven test pattern:

• Use t.Parallel() to specify enable parallel execution (same as regular go test)
• Define test cases in a slice of structs
• Use t.Run() for subtests; each subtest also runs in parallel
• Inside testenv.Run(), obtain the test server URL
• Use Connect’s auto-generated client as-is
• Use the same assertions as the regular go test

There’s almost no complexity related specifically to E2E, you simply write the test like you would a regular unit test.

Additionally, because you can write in plain Go code, you can also effectively leverage AI coding tools like Claude Code. With AI assistance, you can add test cases and flush out edge cases more efficiently. Even for team members outside backend engineering (like QA) who aren’t yet accustomed to Go, AI helps them author their test code.

We also leaned heavily on AI when migrating existing E2E tests implemented in Jest to this framework. We managed to make the migration efficient by referencing the existing tests, having AI generate Go test code, and then having developers review and tweak it.

Overall architecture

One option for running E2E tests is to point them at an app deployed in a shared development environment accessible to everyone. However, there is an issue with this approach, namely that it makes it hard to test in-progress backend changes immediately.

We prioritized having an environment where we could run E2E tests while changing application code, and add or modify tests on the fly. To achieve that, we adopted a design where we dynamically started a server for each test. This allowed developers to validate their changes with E2E tests immediately and even do test-driven development.

Main responsibilities of the framework:

1. Automatic startup and management of test servers: Start servers on demand and manage them in a pool.
2. Automatic database preparation: Start AlloyDB Omni, create logical databases, and run migrations.
3. Parallel-execution support: Manage resources so multiple tests can run concurrently.
4. Automatic cleanup: On test completion, automatically clean up data and return resources to the pool.

From a developer’s perspective, all this complexity is fully hidden. Just call testenv.Run() and your test environment is ready.

Implementation Details

Next, let’s take a look at the implementation of these ideas to see how the framework achieves parallel execution and resource management.

Parallel execution via resource pools

To enable parallel E2E execution, we manage servers with a pool.

Crucially, when the function passed to testenv.Run() returns, the server is automatically returned to the pool. Developers don’t need to manually release resources. They simply write tests as usual and the framework handles cleanup and pooling.

This setup provides the following:

• No resource contention during parallel runs
• Minimized server startup cost (reuse from the pool)
• Prevention of data contamination between tests (initialize with TRUNCATE)
• Transparent resource management (developers don’t need to think about it)

Database management

For the database, we start with only one AlloyDB Omni container. Inside the container, the framework automatically creates a logical database for each test and runs migrations.

This design provides the following:

• Reduced startup cost (only one DB container has to start)
• Data isolation even under parallel execution (each logical DB is independent)
• Automated migration (developers don’t have to think about this

Logical databases are also managed with a pool. After a test, we truncate to clean the data and then reuse the database.

Collecting code coverage

The framework supports go build -cover, introduced in Go 1.20+.

Ordinary test coverage (go test -cover) only measures execution within test code, but E2E needs to measure the server process itself. This is what go build -cover enables.

Our framework implementation covers the following:

1. Automatic creation of an independent coverage directory per server
   • Create a temp directory on each server startup
   • Automatically set the GOCOVERDIR environment variable
2. Accurate coverage collection even with parallel execution
   • Each server writes to its own directory, so there are no conflicts
3. Automatic merge when test ends
   • Consolidate all server coverage data with go tool covdata merge
   • Produce a single, consolidated coverage dataset

Developers only need to set specific environment variables to automatically collect and merge coverage across multiple servers:

# Build the server binary with coverage
go build -cover -o server ./server

# Run tests while collecting coverage
GLOBAL_GOCOVERDIR=/tmp/coverage go test ./e2etest/...

# Generate a coverage report
go tool covdata percent -i /tmp/coverage

This setup enables accurate code coverage for E2E tests, helping to quantify API quality.

Running on Kubernetes

We run E2E tests locally during development and on Kubernetes in CI. Here are some interesting tricks for running on Kubernetes.

Fast deployment with go test -c

Here are some of the tasks you might typically do to run tests on Kubernetes:

1. Build a container image
2. Push the image to a registry
3. Pull the image in a Kubernetes Pod
4. Start the container

However, each of these steps takes time to complete. Since speed matters for E2E, we took a different approach:

# Build the test binary
go test -c \
    -o package/e2etest \
    ./path/to/e2etest

# Build the server binary
go build \
    -o package/server \
    ./path/to/server

# Archive with tar and transfer via kubectl exec
tar -czf - -C ./package . | \
    kubectl exec -c main -i -n ${POD_NAMESPACE} ${POD_NAME} -- \
    tar xzf - -C /tmp/e2e

# Run directly inside the Pod
kubectl exec -c main -it -n ${POD_NAMESPACE} ${POD_NAME} -- \
    /path/to/entrypoint.sh

By using go test -c, you can compile tests into an executable binary. That translates into three things::

• No need to build a container image
• No pushing/pulling from a registry
• Direct file transfer via kubectl exec

Using this method, we cut the lead time to start running tests significantly; from build to test start takes about a minute and a half.

We run on Kubernetes to secure enough resources for parallel execution. As you increase the degree of parallelism, you need that many servers to avoid pitting tests against each other, so resource needs grow linearly—hence the cluster.

Conclusion

In this post, we introduced our E2E testing approach for the Mercari Global App backend APIs. With this approach, E2E tests are no longer “apart” from our work and are instead a part of our everyday development flow. Now, when developers change APIs, they can add or modify E2E tests freely.

Of course, there’s still room to improve:

• Further reduce test execution time
  • We’re working on running only the tests relevant to the changes using AI
• Simplify test data setup
• Improve test result reporting

Still, by prioritizing the developer experience, we believe we’ve built a sustainable E2E testing foundation.

We hope sharing our work will be helpful to projects grappling with similar challenges.