The first indicator that can be used in almost any test improvement process, and is highly recommended by most test experts, is the Defect Detection Percentage (DDP). If you‟ve found most (if not all) of the important defects during testing, and users/customers found few during real-life usage, your testing is good.
Defect Detection Percentage is defined as the number defects found by testing divided by the total known defects. The DDP can be calculated per test stage (e.g. integration, alpha testing, beta testing) or for all test stages together. DDP is a calculable metric after a project is finished and some time (e.g., three or six months) has passed in which residual defects may be found.
The Advanced Level Test Management Syllabus focuses on the following generic test management activities:
Test management must be able to define a set of test metrics for test monitoring, test control and test completion as part of the test planning activities. Each metric needs to be defined, measured, monitored and reported.
During test planning, appropriate test metrics are defined that match the test objectives from the project test strategy.
The metrics used during test monitoring and test control may be different from those used at test completion. During test monitoring and test control, the metrics are about the progress of the test activities. In test completion, the test objectives should be achieved. One or more metrics could be combined to measure test exit criteria assigned to given test objectives.
The following table provides examples of metrics (there are many more) used in test management activities:
| Metric (planned/monitored for defined milestones) | Test Monitoring and test control | Test Completion |
| Requirements coverage | X | X |
| Product risk coverage | X | X |
| Code coverage | X | |
| Actual vs. planned estimation (in hours) for testing activities | X | |
| Percentage of executed test cases per status (e.g. failed, blocked) vs. planned test cases | X | X |
| Accumulated number of resolved defects vs. the accumulated number of defects | X | |
| Actual automated test cases vs. planned automated test cases | X | |
| Actual vs. planned cost of testing | X |
Table 2: Examples of Metrics Used in Test Management Activities
The metric used in a specific test activity is shown in the table. Metrics with an X in test monitoring and test control are primarily used to measure progress and are reported in test progress reports (CTFL). Metrics with an X in test completion are primarily used to measure the achievement of test objectives and are reported in the test completion report (CTFL). Metrics with an X in both columns can be used for either.
There are also metrics for monitoring test effectiveness (e.g., defect detection percentage (DDP)).
DDP is covered in the ISTQB® Expert Level syllabus, Improving the Test Process module, specifically in the Implementing Test Process Improvement section.