It’s no secret that software testing is crucial for the delivery of a quality digital product. There are many recommended techniques and guides for best practices in the STLC, and they all have the same end goal – a well-built product that meets the user’s needs.
Negative testing is one of the two major strategies in software testing. Let’s start with the Wikipedia definition:
Negative testing is a method of testing an application or system that ensures that the plot of the application is according to the requirements and can handle unwanted input and user behavior. Invalid data is inserted to compare the output against the given input.
In other words, the purpose of negative testing is to explore what happens when a user provides negative input data and performs unexpected actions. The idea is to ensure the application is working properly and according to expectations in those conditions.
The following guide will provide key information about negative scenarios in software testing as well as some tips and tricks to improve your tests. There are also examples of basic test cases that should give you a better understanding of this technique or maybe serve as a basis for developing your own testing ideas.
Positive vs. negative scenarios in software testing
In software testing, we differentiate between positive and negative testing, i.e., positive and negative test scenarios. These are completely opposite but complementary – each of them presents one side of a coin. The following table gives an overview of the differences in the top three most relevant categories: App/system Behavior, User Behavior, and Data.
| Category | Positive testing | Negative testing |
|---|---|---|
| App/system behavior | Assumes that the app/system will behave as expected under ordinary circumstances and standard conditions (will meet the requirements) | Considers the possibility that the app/system will be “challenged” (e.g., slow Internet connection) |
| User behavior | Assumes that all the users are coming with good intentions; they will use the app for the purpose it was created and will not do any harm to it | Considers the possibility that the app/system will be met with unwanted user activity: – inputting invalid data due to non-user-friendly flows and/or misunderstanding certain aspects of the app/system – third-party attack (e.g. DDoS attack) |
| Data | It assumes that the user will use and provide only valid data | Considers the possibility that the entered data is wrong (wrong data format and type, wrong number and type of added characters, etc.) |
Benefits of using negative scenarios in software testing
Other than improving the overall quality of software, negative testing comes with an array of benefits for both the in-house team and the client.
- Improved QA skills and knowledge (within the team and company-wide). A good software tester covers more ground (scenarios) during testing, and negative testing allows that. If a single person improves their skills in this area, it makes the whole team more efficient. A more efficient team means better products, better delivery times, and improved services overall.
- Clean test data. Testing with invalid data can help differentiate between valid and invalid data and identify which is which. This means that the invalid data can be isolated, updated accordingly, and possibly deleted from the project’s database in large part.
- Improved overall performance. Negative testing improves the application or system being tested from the usability perspective, for example by checking how stable it is. It’s important to think about this aspect before the product reaches the market.
- Improved security. Negative scenarios in software testing deal with negative inputs, either unintentional or malicious ones. Negative testing can help identify how to improve the product’s security and make it more reliable.
- Revealing new software testing directions on a project. Negative testing can add basic principles and rules for making apps/systems more secure. This can lead to applying a whole new branch of software testing, Security Software Testing. Penetration testing can be performed additionally as part of the SST.
- Client satisfaction. The market always demands high-quality software and negative testing has an important part in achieving good client satisfaction. Negative test scenarios can be in a separate folder, which allows the clients to generate separate reports. These give them a clear picture of their app and help in future decision-making. However, the decision when and how to apply negative testing is almost always up to the client and their resources (usually time and money).
When and how to use negative scenarios in software testing
As with any other type of testing, negative testing can traditionally be performed both on the frontend and the backend. Here are some useful techniques and methods:
- Exploratory Testing. A well-known technique that helps software testers get to know more about the app/system they are testing. When they employ this technique for negative testing, taking into consideration they know the product’s requirements and functionalities, their imagination and creativity are the only limits.
- Fuzz Testing. Software testers input random data in the app/system fields and submit that data to see the product’s behavior. These results can be useful for finding patterns in issues that will cause errors and crashes.
- Boundary Values Testing. This is more of an analysis method than a practical technique. The results of the analysis should identify the lowest and the highest number of characters that every input field in the tested app/system allows. Using these results, a software tester can efficiently check and validate the boundaries outside the allowed numbers. This method is very well explained in our QA Handbook, and you can use it to refresh your knowledge.
Additionally, negative testing can be a useful part of some testing heuristics, for example, RCRCRC, FEW HICCUPPS, etc.
In certain situations, these techniques and methods can be useful for finding errors and possible crashes.
- Input field testing. Scenarios like inputting an invalid number of characters, inputting special characters, inputting emojis, etc. are most common in negative testing. One of the most popular examples of input field testing is the famous SQL Injection. Additionally, inputting characters that are not allowed (e.g. letters from the Cyrillic alphabet) is a good example of negative testing on input fields.
- Correspondence between frontend and backend. The behavior of the frontend has to match the behavior of the backend. For example, if an input field on the frontend accepts a maximum input of 30 characters and the user enters 31 or more characters, the frontend should not allow it, and it should display an error. On the other hand, if the backend accepts more than 31 characters and these are submitted to the database, a failure can happen.
- Web session testing. A web session should have some time-out rules. Negative software testing verifies what will happen in case of a slower Internet connection.
Tracking results
Probably the most reasonable, if not the only place for tracking the results of negative testing is the test management tool you are using on your project. This is a good way to track results because these types of tools have user-friendly interfaces for generating reports.
The most efficient way to do this is to use the options available in the test management tool. For example, if you’re using TestRail to write test cases for negative test scenarios, make sure to select Negative Testing as the testing type.
If your test management tool doesn’t support this option but has another option for adding tags and labels to the test (like Xray), use it to properly tag and label the test cases created for the negative test scenarios.
Tracking results from negative test scenarios can benefit the QA team and the client. Using the aforementioned structure, both sides can find the negative test scenarios and navigate through them easier.
Either way, make sure to align with your QA teammates and the client about the structure, how to create test cases, and how to execute them.
Tips & Tricks for dealing with negative test scenarios in software testing
Even though negative testing is a well-known technique in software testing, it is useless if we don’t follow certain principles during the planning and execution of negative test scenarios. The following can help software testers with planning, creating, and executing test scenarios for negative testing.
- Create a separate folder within the project for negative test scenarios. Although sometimes negative test scenarios can be a part of the acceptance criteria from a Story ticket, creating test cases in a separate folder within the same project/directory is good practice because our access to them is easier and quicker.
- Think upfront about negative scenarios. It’s not a big problem if you apply negative testing sometimes later in the project. However, if you think about the negative scenarios upfront, you will save valuable time, energy, and money, and this can give you more confidence before launch.
- Use the Flow-by-Flow structure. In the designated folder for negative test scenarios, create a separate folder for each flow (functionality) in the app/system you will cover with the scenario. This way, it’s easier to organize, understand, and follow the structure for anyone who will use negative test scenarios.
- Always use references. Ticket number, testing type, and tags and labels that clearly state that this is a negative test case can make it easier for us to plan future testing executions.
- Future testing activities. Similar to positive test cases, negative test cases can also be automated by our test automation engineers. Take this into consideration when creating scenarios. It means we should mark the automation candidates from the negative scenarios properly to make the test automation engineers’ job easier.
- Future user activities. Take into consideration the ‘extreme’ user activities like trying to submit an empty form or an empty state of a page (should include some illustrations and informative text, etc.)
- Discuss and agree on the terms of negative testing with the team and the client. Negative testing requires additional time. Therefore, as a software tester, you must speak with your team lead and/or client to agree on when and how to perform negative testing. Feel free to include suggestions on how to improve the testing strategy on the project you’re working on.
- DO NOT reinvent the wheel. If there is no test management tool on the project, but there are some rules and policies for tracking software testing results, don’t use other ways of tracking only negative testing results. Instead, try to improve the existing tracking system.
EXTRA NOTE: When creating negative scenarios, don’t make too many of them (e.g., don’t create a negative scenario where a First Name textbox doesn’t accept every single emoji) – you don’t want to be ‘stuck’ in Exhaustive testing.
Test case examples
Test cases are a solid starting point for getting started with testing. Documenting negative test scenarios is just as important as documenting other test scenarios (positive, regression, smoke tests, etc.). In this section, we’ll provide some specific examples for negative test scenarios that cover both frontend and backend.
Frontend negative test scenarios
- Create a user with an email address that already exists in the system/db
- Log in with the wrong password
- Verify if the First Name and Last Name fields accept special characters, numbers, and emoticons
- Verify if the Phone Number field accepts letters
- Verify if the user can complete an order with an expired credit card
Backend negative test scenarios
- Log in without providing any user token
- Log in with an expired token
- Login with user#1 using the token from user#2
- Complete an order with a user who has not saved any payment methods
Think negative
Negative software testing can be a great asset when it comes to delivering quality software. At Infinum, it is a well-known and well-established technique for software testing. It can be performed both on the frontend and the backend parts of an application, and you can do it manually or by executing previously created automated test scripts.
Negative testing can be hindered by a lack of resources (time and finances), but in the long run, it will save money because the app/system being tested becomes better, more secure, and more reliable for the end users. A well-planned and structured software testing strategy should always include negative scenarios to make sure all areas are covered.
