Test documentation
Last modified on Tue 04 Oct 2022

Quality needs to be constantly improved, but it is just as necessary to make sure that quality never deteriorates. - Shigeru Mizuno

Test case

Test case is a group of input values, execution preconditions, expected execution postconditions, and results. We use test cases for creating test executions (smoke test or regression test) and we use those test executions for going through the application (usually before release) and checking if everything is working as expected.

Writing & structuring test cases:

When writing test cases there is a minimum set of info each test case should have:

Title

The title of a test case should be self-explanatory and easy to read as much as it can. Sure, some test cases are complex as they are, but a person that is doing a test run should be able to know what's it all about.

The title should clearly state what you are testing. Possible ways to format your title are the following:

Example: Verify that after clicking the Login button user can log in

Example: Verify that the profile image is shown when the user taps on his avatar

Steps

We look at the test steps as a set of instructions on how to perform a test run on a specific test case. That being said, test steps must be clear and accurate as much as they can.

Example:

1. Log in as a guest user to any country
2. Open the registration screen
3. After registering a new user, open the email with OTP
4. Copy/Remember OTP
5. Paste it/write it to the verification code field

Expected

At the end of every test case, there should be an expected result after the test has been completed. There we describe what needs to happen when a test run is finished on that specific test case.

Example:

Expected result: Account is created and the user is redirected to the app (profile screen is shown).

Additionally, test case may also contain:

Test case examples

Testrail_test_case.jpg

Xray_test_case.jpg

Maintaining and writing test cases

NOTE: Try to have a reasonable number of test cases for your project (keep in mind that you need to maintain and execute all test cases (or most of them) from your repository when doing your regression test).

Test plan

A test plan is a complete planning document that contains the scope, approach, resources, schedule, devices, etc. of testing activities. It helps us determine the effort needed to validate the quality of the application under test - we usually create a test plan for regression and smoke testing which is done before the app goes to production.

The test plan should be created for every release of your application. Also, if you are the only QA on your project, it's still good to have a test plan. By creating a test plan, you will have all the information about testing in one place. Also, other members of your project can understand the details of testing.

Making a test plan document has many benefits:

The test plan can be made on Confluence (if you use JIRA for your project) or you can use this template. Make sure to first duplicate the sheet before making any changes!

Important things that the test plan should have:

JIRA_test_plan.jpg

Maintaining the test plan:

Maintaining the test plan means that it can be edited and adjusted for every purpose/objective of the project, depending on the scope of the release and the manageable time period.

For every release on the project, it is important to create a new test plan with all the necessary information on how the test run will be done. This also means that if something is changed before the release (scope, devices, etc.), the test plan must be updated as soon as possible so it is up to date.

The current practice that we use at Infinum is that we create and prepare a template that is later used for every release of the application itself. As mentioned before - depending on the scope of the release and the time period that is given to the tester.

Test report

A test report is an organized summary of testing objectives, activities, and results. It is created and used to help stakeholders (product manager, analysts, testing team, and developers) understand product quality and decide whether a product, feature, or defect resolution is on track for release.

A very basic test report for a small application or organization should include, at least, the following:

Every time when a test run is finished and the whole test plan is fulfilled, a tester should generate a test report that is later shared with the whole development team and the stakeholders (clients).

This report can be exported as a PDF file and then sent via e-mail, Slack, or shared via a TestRail link.

How to generate a test report from TestRail

There are several different test reports you can generate. Here's how you can generate the two most used ones.

Run summary

  1. Navigate to a test run
  2. Click on Run > Summary
  3. Click on Add report
  4. Wait for it to generate
  5. Download it

Detailed test report

  1. Navigate to a test run
  2. Click on the Print icon
  3. Select "Details" from the top dropdown
  4. Click on Print and save it as a PDF

Testrail_test_report.jpg

How to generate a test report from Xray

  1. Navigate to a Test Runs List
  2. In the filters dropdown, specify the test plan from which you want to export the executions
  3. In the "Contains" field, add the name of the execution that you will be exporting (sometimes Xray won't return any results so you can try with a couple of key words from your execution's name)
  4. Click "Generate report"
  5. On the right side, specify which columns you want to export (this selection should be enough: Key, Summary, Components, Test execution, Test plan, TestRun Status, Test execution fix version, Open, Closed)
  6. Export
  7. Convert exported file to PDF (open with Numbers and export as PDF)

Xray_test_report.jpg

Test strategy

Test strategy is all the above and extra. Basically, a test strategy is all the documents that explain how the QA process will look like on the specific project. We can also call it a "way of working" for a QA member.

Test strategy documents can contain some information about: