What We Talk About When We Talk About Quality Assurance.

* This article is a translation of the Japanese article written on September 16, 2021.

Hello! This is @myajiri, a QA engineer at Merpay. This article is for day 12 of Merpay Tech Openness Month 2021.

After joining Merpay, PMs and engineers undergo a process of onboarding (training to understand our products and work). I’m responsible for the Quality Assurance (QA) Policy section of this process. In this article, I’d like to cover what is discussed during this process.

(Isn’t it interesting that all PMs and engineers learn about QA when first hired?)

Why did we create a QA Policy?

To instill the creed of quality assurance throughout the entire product organization!

As Merpay’s business expands, our projects and organizations continue to change at a rapid pace. Even if we establish rules and regulations, they tend to become outdated before they are even applied. However, even as projects and organizations change, our fundamental creed (policy) does not change very much.

I believe that our top priority is to instill a policy of committing to end-to-end quality in products and projects.

I’ll cover this in more detail later, but if we want to bring the best products to the market, it’s not enough to just leave it to the QA engineers to work on testing.

Merpay’s quality assurance: "All for Quality"

At Merpay, the goal is to have all roles working together in an “All for One” way throughout the product/project life cycle, instead of each role acting alone. We call this state "All for Quality" within the QA Team.

What is "All for Quality?"

In our QA Policy, "All for Quality" is defined as:

An approach of improving the total reliability through all processes (everyone), without being dependent on a certain process (such as testing)

This concept has its roots in Lusser’s law. This law concerns system reliability and is based on concepts proposed by Robert Lusser, a German aircraft engineer during World vvvvvWar II. Imagine a product composed of 10 parts, where the individuals responsible for each part create their parts at 90% quality. 90% might seem like a pretty high number. However, the finished product assembled from these 10 parts will have a reliability of only 35%.

In other words, aiming to always achieve the best quality for each part is crucial in improving the overall reliability of a product. The concept of "All for Quality" means that each person involved in each part does their best.

Quality assurance also applies to processes

I also believe that quality assurance applies not only to products themselves, but to each process involved in creating a product. This includes the product planning stage, project planning, the specification definition phase, operations leading up to release, and in some cases testing itself.

Let’s release it with “good enough” testing

Testing is crucial for releasing great products. But why do we test? The goal is to provide value to users as quickly as possible. Although testing is important, there’s no need for it to serve as a "gatekeeper" preventing a product from being released. Products released for Merpay, however, provide many crucial features that handle important assets of our users. That doesn’t mean it’s something to be scared of, but do I believe that it’s important to gain plenty of experience in work with clear objectives and foundations in order to determine the right balance between taking and hedging risks. What to focus on varies depending on the project or feature. However, I’d like to share the “good enough testing" aspect of the PRISMA method as a mindset to apply as criteria for making decisions.

What are tests?

Within a given process will be multiple steps named something like "XX test." What is a test? It is recommended to combine “checking" and "exploring" when testing, and for these to operate as separate processes.

• Check: Confirm that something operates according to requirements or specifications
• Explore: Continue to learn and explore with regard to the product

At Merpay, both QA engineers and developers write test code and sometimes automate testing. These tests are suited for efficiently checking that a program operates as written and operates according to requirements. Meanwhile, exploratory testing is a method for obtaining ideas while actually using the program (for example, "will it break if I do this?"), in order to confirm reliability in an exploratory manner. When you decide to start testing, it’s important for engineering and QA to already be working together, in order to maximize component quality.

"Good enough testing”

1. There are no defects that could cause a serious incident
2. Sufficient value can be provided in the current state
3. The value of the product is greater than any latent risks
4. The benefit of releasing now is greater than the benefit of delaying the release

Conclusion

In this article, I provided a brief introduction to the QA Policy content that Merpay engineers learn during the onboarding process when they first join the company. I plan to enhance this content by further breaking it down into test process guidelines and indicators.