How is Security Testing Different from Typical Software Testing?

This post is for Day 10 of Mercari Advent Calendar 2021, brought to you by @gloria from the Mercari Product Security team.

————

Hello, this is @gloria from the Product Security Team. Previously, I wrote about DevSecOps as a part of last year’s Advent Calendar event.

Recently, I noticed that the topic of transitioning from QA to Security Tester seems to be often talked about on Twitter and LinkedIn. To many people, this transition might seem strange, or even intimidating, because intuitively the two roles appear to be in two vastly different branches in the world of tech. Having experienced a similar career change in the past, I wanted to share my viewpoint on this topic and offer some tips for people who want to step into the field of security testing.

Why is Security Testing a Possible Career Path for QA?

As DevOps and reliance on automated testing increases, the future of QA has also become heavily debated. I believe that while QA as a role will not disappear completely, the number of positions will certainly decrease due to a shift to using test automation and having developers play a more active role in testing. For those who choose to leave the field of QA, traditionally there are two possible career paths to choose from.

QA who are not particularly interested in coding or directly handling technology tend to move into roles that either manage people (Engineering Manager, Project Manager), or products (Business Analyst, Product Owner, etc.). Since soft skills such as communication and people skills, tactfulness, attention to detail, and the ability to think from multiple perspectives are vital to the work of QA, the transition into managing people and products is a very natural and fulfilling one that allows them to make full use of their soft skills.

On the other hand, QA who are technologically-inclined tend to move into developer roles, or highly-specialized technical roles such as test automation. Security testing also falls under this category.

The possible career paths of QA (from https://qa-academy.lv/en/qa-engineer-job-4-possible-career-paths/)

At first glance, security testing might sound like a difficult role for QA to move into but surprisingly, many skills and knowledge used by QA are also applicable in security testing. In the next section, we will look at some differences and similarities between the two roles.

Security Testing vs Typical Software Testing

Types of Test Cases

In my career transition, what I found is that security testing is not actually that different from typical software testing. As I mentioned in a previous blog article, there is a natural overlap between QA and security testing. The two types of testing are conducted from different viewpoints— while QA tests from the perspective of a normal or slightly curious user, security tests from the perspective of a malicious user. Therefore, some types of testing like input validation, error cases, and business logic are done by both QA and security.

The starred items (*) are the natural overlap between QA and Security testing (from https://engineering.mercari.com/en/blog/entry/20200930-testdeck/)

To give a more concrete example, let’s look at how each would test a standard login form. QA might conduct the test on the left to verify what would happen if the user accidentally mistypes their email address while logging in. On the other hand, a security tester might conduct the test on the right to verify that the form is not vulnerable to SQL Injection, a common security issue that can have devastating effects when found on login forms. In both cases, they are testing input validation and error handling on the login screen, but their intentions are different. Since QA tests normal behavior and security testers test malicious behavior, sometimes the words “use case” and “misuse/abuse case” are used respectively to refer to their test cases.

A typical test conducted by QA (left) vs security tester (right)

Hard Skills (Tools and Technology)

While there are some tools that can be used in both QA and security testing such as Charles Proxy, Postman, and JMeter, the biggest difference between the two roles is in the hard skills required.

Security testing requires deeper technical knowledge because often, an understanding of the technology behind the feature is necessary to come up with suitable test cases. For example, in order to come up with the misuse case in the previous login form example, the security tester needs to know two things: 1) the software they are testing uses an SQL database to store user accounts, and 2) software using SQL databases are potentially vulnerable to SQL Injection. In other words, the security tester needs to know the specific technology stack used by the software, and common vulnerabilities for that specific technology stack, before being able to come up with possible cases to test.

QA, on the other hand, tends to test from the user’s perspective so it is not necessary for them to understand the specifics of the software’s inner workings such as the type of database or programming languages used, software and network architecture, etc. Since security testing is performed using the greybox or whitebox approach (either some or all of the system’s inner works must be known in order to effectively test) while QA more often tests from the blackbox perspective, security testing requires more extensive technical background.

Soft Skills

Although the typical stereotype of someone working in security is a lone-wolf hacker hidden away in a dark room, strong soft skills are absolutely necessary to become a successful security tester. Similar to QA, a security tester’s job is to find security bugs in the software and raise them to the development team, so skills such as good communication, presentation skills, tactfulness, and humility can take you far. In addition to people skills, attention to detail, problem-solving skills, logical thinking, and creativity are also needed in order to catch security bugs, and these are skills that are common to typical software testers. Therefore, overall I think the soft skills needed for success in both roles are extremely similar.

How Does One Learn Security Testing?

Learning security testing involves the mastering of two things: tools and testing technique, and the hacker mindset.

In today’s day and age, it could not be easier to learn new tools. There is a wide variety of learning resources available online such as Youtube tutorials, blogs, training courses, etc. Some websites also offer hands-on practice such as PortSwigger’s Web Security Academy. It should be noted that if you do intend to do hands-on practice on real websites or applications, please make sure to obtain permission from the creators first because conducting security tests without consent is illegal.

Personally, I believe that of the two, learning the hacker mindset is the more difficult one because it involves changing the way that we think. While typical software testing requires thinking of creative ways in which a user might accidentally break the software, security testing involves purposefully thinking of ways to gain something through exploiting the software. This can be money, access to things that you normally shouldn’t be able to access (extra features, users’ personal information or passwords, bypassing restrictions, etc.), or even just satisfaction from being able to deface a website or take down the company’s entire service. These are all common motives of hackers and in order to defend against them, it is necessary to think from their perspective.

In order to nurture the hacker mindset, a good question to ask yourself while testing is “What can go wrong?”. This act is called threat modeling and it is a great way to train your mind to see ways in which software may be exploited. In other words, instead of asking “How might a user break this?”, try asking yourself “How can I break this in a way that lets me gain something?”. By shifting your testing motive to focus on what you can gain, your thinking and perception become closer to that of an average hacker, allowing you to find issues before they would have found them. It can be difficult coming up with ideas of how to exploit things at first but as you continue to train this mindset, over time, ideas should start to flow to you more naturally.

Conclusion

Security testing has long been considered as one of the potential career paths for QA. There is a natural overlap in the types of test cases performed, tools that are used, and soft skills that are necessary to be successful in the role. Although security testing certainly does require more extensive hard skills such as system knowledge and technical expertise, I believe that anyone who puts their mind to it can master these skills in no time and go onto become a wonderful security tester.

————

The Mercari Product Security team is looking for talented people to work with us! If you are interested, please check out our job posting for further details.

Tomorrow’s article will be by @gary. Look forward to it!