Introduction
Hello everyone! I’m @Udit, an Engineering Manager (QA) at Mercari.
At Mercari, we emphasize every team member’s role in writing tests, automating tests, and reporting bugs. While it is important for QA experts to be able to optimize testing, automate processes, and provide solid QA processes within the team to ensure high-quality testing experiences, it is also very important to shift away from the notion that QA being solely responsible for all testing. Instead, we emphasize collaboration within QA and other teams. We encourage QA members to become integral team players, aligning their efforts with the team’s objectives and iterating on processes together.
In this article, we will describe how we achieve the goal of delivering quality with speed through a collaborative and team-oriented approach to quality assurance, in line with the QA team’s mission.
Role and Responsibilities
QA plays a vital role in assisting teams to independently accomplish all the quality-related tasks specified in this guideline. Their primary focus lies in enhancing the testing process, offering valuable test tools, sharing their extensive testing knowledge, and monitoring essential testing metrics. By providing this support, QA empowers teams to effectively manage their own quality assurance responsibilities.
QA responsibilities are:
- Actively participate in team meetings, ceremonies, and feature kickoffs.
- Collaborate with the team on test planning activities.
- Review and ensure the implementation and execution of planned tests.
- Maintain a healthy Test Pyramid structure.
- Develop both manual and automated tests as required.
- Execute tests and provide assistance during test execution.
- Share comprehensive knowledge about our product.
- Facilitate onboarding of teams and members on testing techniques.
- Categorize and execute automated tests for Release Judgment.
- Continuously iterate and improve the QA processes within the teams.
- Support and assist teams without QA in implementing QA processes.
- Share quality metrics with teams for retrospective analysis.
Overview of the Testing Process
The diagram provided illustrates the various stages involved in testing activities, starting from the initial feature definition to its final release. Let me explain how each stage works.
Definition
Scrum Team members (PMs, Developers, QAs) review and agree on requirements/specifications based on the perspectives such as follows;
- Ensuring comprehensive feature definition.
- Eliminating ambiguity from requirements.
- Making requirements testable.
- Maintaining up-to-date supporting documentation.
To provide further context, these activities are achieved through various practices such as reviewing feature test scenarios and actively participating in kickoff meetings with PMs, etc.
QA Kickoff
The QA member and the software engineer, along with other stakeholders, collaborate on a story/bug, ensuring that sufficient testing is being planned for the ticket. These plannings include;
- Planning tests for each acceptance criteria thoroughly.
- Planning tests for the happy path of the feature.
- Planning tests for edge cases to cover all potential scenarios.
- Planning tests for the interaction of the feature with other features.
- Evaluating non-functional requirements such as performance and accessibility.
- Planning post-release testing activities.
To ensure traceability, the mentioned tests should be referenced and linked to the development ticket.
In the test plan, it is crucial to identify and distinguish between manual and automated tests, specifying the appropriate level for each. This helps in achieving a balanced and healthy test pyramid structure. Additionally, nailing down common and edge cases is another valuable aspect to consider in the test plan.
Instead of including an end-to-end (E2E) test for the edge case, such as verifying that an item priced at ¥0 cannot be listed, consider the following alternatives:
- Write a unit test to evaluate the business logic.
- Create an API test for the listing endpoint.
- Develop a minimal UI test (with backend fake/mocks) to validate the error message displayed on the UI.
Test Development
When working on a feature, it is important to include planned automated tests in the same ticket alongside the coding process. During the code review, ensure that the tests are implemented at the appropriate level in the pyramid structure, rather than higher up.
For teams, the Definition of Done includes creating, executing, and passing all the planned tests, including unit, integration, end-to-end (e2e), manual, and others. It is also crucial to triage all bugs discovered during testing, especially P0 (Urgent) and P1 (Important) bugs, as they can have a significant impact on the functionality of the feature as a whole. Reviewing adherence to the Definition of Done falls within the domain of QA. Moreover, any new code should not break the automated tests of other features, and it is the responsibility of the development team to maintain them.
Some test development can be deferred until the feature is complete, especially if certain tests are not necessary during the early stages, such as UI tests for experiments that might be dropped. However, it is important to create a separate task for handling the development of these tests in the future and ensure that it is associated with the appropriate Epic or Story.
In addition to automated tests, it is valuable to conduct manual testing using techniques like exploratory testing, scenario testing, and dogfooding.
Exploratory testing allows testers to uncover bugs and issues in an unscripted manner, while scenario testing helps validate specific use cases. By engaging in exploratory-like tests, testers can explore the software to discover potential issues and evaluate its behavior in real-world scenarios.
Dogfooding, also known as "eating your own dog food," is the practice of using your own products or services within your own organization. By implementing this approach, we prioritize becoming the primary users and testers of the products we create. This enables us to gain valuable insights from a user’s perspective, identify potential issues, and gather feedback that helps us continuously improve the quality of our offerings. Through dogfooding, we validate our products in real-world scenarios, enhancing their usability and aligning our development team with the needs of users.
Demo
After completing the ticket and ensuring that the feature has reached a mature stage, it is beneficial to convene a review session with the requirements writer, QA member, and software engineer. This review aims to evaluate whether the feature effectively fulfills its intended purpose.
Additionally, it is worth considering recording a feature review to showcase the usage and functionality of the feature. This recording serves as a valuable resource for sharing information with other teams or members within the organization.
Ship
Once a feature reaches the branch cut and becomes part of the upcoming release build, it becomes the responsibility of the entire team to ensure its continued functionality. This accountability extends beyond just QA members.
To properly track and identify the versions in which the features were shipped, it is essential to complete the correct "Fix version" field in all tickets.
During the release process, it is crucial to organize testing across the entire team to verify that the submitted features function correctly in the released build. The team should execute and maintain tests specifically related to these features.
If any tests have been added to the automated test suite as a must-pass test before every release, it is important to ensure that they are passing and are not unreliable or outdated.
Furthermore, teams are expected to take ownership of their features and screens even after they have been shipped. This includes conducting regular regression tests and actively addressing feedback received through internal and customer feedback channels. Rotating team members helps ensure that the entire team shares ownership and responsibility for the features.
Benefits and Challenges
The approach of empowering teams with QA expertise has numerous benefits. Firstly, it reduces the count of hotfixes and blockers, resulting in a smoother development and release process. With QA members actively involved in the entire testing lifecycle, requirements are better defined and ambiguity is eliminated. This leads to more comprehensive feature definition and test planning. Additionally, the focus on collaboration and iterative processes improves communication, alignment, and overall efficiency.
While the approach of empowering teams with QA expertise brings significant benefits, it also presents challenges. The transition towards a fully collaborative model requires ongoing effort and coordination. Bringing all team members on board and ensuring consistent adherence to QA processes may require time and training. Achieving a balance between manual and automated testing, as well as identifying common and edge cases, poses additional challenges. As with any evolving approach, there is always room for improvement as we strive for perfection.
Conclusion
The structure of QA teams at Mercari has proved to be very effective for us. Our approach of promoting collaboration between QA experts with other teams has been a vital part of this success.
The primary focus has been on fostering clarity and unity between QA and teams, highlighting the importance of collaboration rather than QA being solely responsible for all testing. The article encourages QA members to become integral team players, aligning their efforts with the team’s objectives and iterating over processes together.
By adopting this collaborative and team-oriented approach, Mercari aims to deliver quality with speed and empowering teams to achieve better quality outcomes. Through active participation, comprehensive test planning, and efficient test development and execution, teams can strive for continuous improvement in software quality.
Overall, by implementing this approach to QA, Mercari enables teams to work together seamlessly, leading to enhanced communication, improved efficiency, and ultimately delivering high-quality software to users.
Join us in revolutionizing the way we approach quality and become a valued contributor at Mercari!
