GST Billing Software

header image


Erachana Line

What is Testing Documentation?

Testing documentation involves the documentation of facts that should be developed before or during the testing of Software.

Documentation for software testing helps in estimating the testing effort required, test coverage, requirement tracking/tracing, etc. This section describes some of the commonly used documented facts related to software testing such as:

  • Test Plan
  • Test Scenario
  • Test Case
  • Traceability Matrix
Concepts - Test Plan, Test Scenario, Test Case, Traceability Matrix

Test Plan

A test plan outlines the strategy that will be used to test an application, the resources that will be used, the test environment in which testing will be performed, and the limitations of the testing and the schedule of testing activities.

A test plan includes the following:

  • Introduction to the Test Plan document
  • Assumptions while testing the application
  • List of test cases included in testing the application
  • List of features to be tested
  • What sort of approach to use while testing the software
  • List of deliverables that need to be tested
  • The resources allocated for testing the application
  • Any risks involved during the testing process
  • A schedule of tasks and milestones to be achieved

Test Scenario

It is a one-line statement that notifies what area in the application will be tested. Test scenarios are used to ensure that all process flows are tested from end to end. A particular area of an application can have as little as one test scenario to a few hundred scenarios depending on the magnitude and complexity of the application.

The terms 'test scenario' and 'test cases' are used interchangeably, however a test scenario has several steps, whereas a test case has a single step. Viewed from this perspective, test scenarios are test cases, but they include several test cases and the sequence that they should be executed. Apart from this, each test is dependent on the output from the previous test.

Test Case

Test cases involve a set of steps, conditions, and inputs that can be used while performing testing tasks. The main intent of this activity is to ensure whether a software passes or fails in terms of its functionality and other aspects. There are many types of test cases such as functional, negative, error, logical test cases, physical test cases, UI test cases, etc.

Furthermore, test cases are written to keep track of the testing coverage of a software. Generally, there are no formal templates that can be used during test case writing. However, the following components are always available and included in every test case:

  • Test case ID
  • Product module
  • Product version
  • Revision history
  • Purpose
  • Assumptions
  • Pre-conditions
  • Steps
  • Expected outcome
  • Actual outcome
  • Post-conditions

Many test cases can be derived from a single test scenario. In addition, sometimes multiple test cases are written for a single software which are collectively known as test suites.

Traceability Matrix

Traceability Matrix (also known as Requirement Traceability Matrix - RTM) is a table that is used to trace the requirements during the Software Development Life Cycle. It can be used for forward tracing (i.e. from Requirements to Design or Coding) or backward (i.e. from Coding to Requirements). There are many user-defined templates for RTM.

Each requirement in the RTM document is linked with its associated test case so that testing can be done as per the mentioned requirements. Furthermore, Bug ID is also included and linked with its associated requirements and test case. The main goals for this matrix are:

  • Make sure the software is developed as per the mentioned requirements.
  • Helps in finding the root cause of any bug.
  • Helps in tracing the developed documents during different phases of SDLC.

Test Case Template

A good test case template maintains test artifact consistency for the test team and makes it easy for all stakeholders to understand the test cases.

  1. Test case ID : Each test case should be represented by a unique ID. To indicate test types follow some convention like "TC_UI_1" indicating "User Interface Test Case#1."
  2. Test Priority: It is useful while executing the test.

i. Low

ii. Medium

iii. High

  1. Name of the Module: Determine the name of the main module or sub-module being tested

4. Test Designed by: Tester's Name

5. Date of test designed: Date when test was designed

6. Test Executed by: Who executed the test- tester

7. Date of the Test Execution: Date when test needs to be executed

8. Name or Test Title: Title of the test case

9. Description/Summary of Test: Determine the summary or test purpose in brief

10. Pre-condition: Any requirement that needs to be done before execution of this test case. To execute this test case list all pre-conditions

11. Dependencies: Determine any dependencies on test requirements or other test cases

12. Test Steps: Mention all the test steps in detail and write in the order in which it requires to be executed. While writing test steps ensure that you provide as much detail as you can

13. Test Data: Use of test data as an input for the test case. Deliver different data sets with precise values to be used as an input

14. Expected Results: Mention the expected result including error or message that should appear on screen

15. Post-Condition: What would be the state of the system after running the test case?

16. Actual Result: After test execution, actual test result should be filled

17. Status (Fail/Pass): Mark this field as failed, if actual result is not as per the estimated result

18. Notes: If there are some special condition which is left in above field

Optionally you can have the following fields depending on the project requirements

  • Link / Defect ID: Include the link for Defect or determine the defect number if test status is fail
  • Keywords / Test Type: To determine tests based on test types this field can be used. Eg: Usability, functional, business rules, etc.
  • Requirements: Requirements for which this test case is being written
  • References / Attachments: It is useful for complicated test scenarios, give the actual path of the document or diagram
  • Automation ( Yes/No): To track automation status when test cases are automated
  • Custom Fields: Fields particular your project being tested due to client/project requirements.

A sample Test case template is attached.

What are Test Summary Reports

Test Summary Report is a document which contains

  • A summary of test activities and final test results
  • An assessment of how well the Testing is performed

Based on the test report, the stakeholders can

  • Evaluate the quality of the tested product
  • Make a decision on the software release. For example, if the test report informs that there're many defects remaining in the product, the stakeholder can delay the release until all the defects are fixed.

What does a Test Summary Report contain?

Project Information

All information of the project such as the project name, product name, and version should be described in the test report.

Test Objective

As mentioned in Test Planning tutorial, Test Report should include the objective of each round of testing, such as Unit Test, Performance Test, System Test …Etc.

Test Summary

This section includes the summary of testing activity in general. Information detailed here includes

• The number of test cases executed

• The numbers of test cases pass

• The numbers of test cases fail

• Pass percentage

• Fail percentage

  • Comments

This information should be displayed visually by usingcolor indicator, graph, and highlighted table.

Defect/Bug Report

One of the most important information in Test Report is defect. The report should contain following information

• Total number of bugs

• Status of bugs (open, closed, responding)

• Number of bugs open, resolved, closed

• Breakdown by severity and priority

A Good Test Report contains

  • Detail: You should provide a detailed description of the testing activity, show which testing you have performed. Do not put the abstract information into the report, because the reader will not understand what you said.
  • Clear: All information in the test report should be short and clearly understandable.
  • Standard: The Test Report should follow the standard template. It is easy for stakeholder to review and ensure the consistency between test reports in many projects.
  • Specific: Do not write an essay about the project activity. Describe and summarize the test result specification and focus on the main point.