Scenario 1 – Test Project Evaluation and Reporting
Assume you have just been promoted to Senior Test Manager, responsible for
testing your company’s most strategic project, a Personal Video Recorder (PVR)
capable of recording and playing back 3D TV.
The development model for this project is the V-model.
The project is due to launch in six months, and different levels of testing are already
being performed in parallel.
There are six testing teams assigned to test different parts of this project. Four teams
are dedicated to testing different components of the PVR and performing integration
testing between components. The managers of these teams report to you as part of
a matrix management organization. For the other two teams, one team is
responsible for system testing and the other team is responsible for user acceptance
testing. The managers of these two teams report to you directly.
Senior management holds a weekly strategy meeting and you are expected to
prepare a 15-minute update on the test status of this project. You have been given
six weeks to prepare for the first update.
During your first day on the job, you meet with the six Test Managers. You
immediately notice that there is no standardized reporting from the test teams. Each
team is using a different set of metrics to determine quality and exit criteria for their
part of the project and for analyzing efficiency and effectiveness of their test process.
Next, you meet with the managers of the four component development groups, the
Project Manager and the Vice President of Research & Development, in order to
understand what information they need from the test teams in order to make the
correct decisions about the project.
By the end of the first week you realize that you need to define a set of useful
metrics to be used throughout the test project, create test reports to be used by the
test teams, and create a high-level test report and dashboard for reporting to project
management and senior management. 

Scenario 1 “Test Project Evaluation and Reporting” provides the context for this question.
The development teams are considering moving to an Agile development
process.
Consider the following:
i.Percentage of builds that failed
ii.The weekly rate at which actual versus planned automated regression test cases are being written
iii.Average number of iterations run until automated test cases are obsolete
iv.Trend of number of requirements changes
v.Trend of length of time between defect detection and closure
vi.Trend of number of open defects
vii.Number of defects found during development
Which one of the following statements best summarizes the test reporting and
control metrics that will be required if the teams move to an Agile process?

Select ONE option.

Scenario 1 – Test Project Evaluation and Reporting
Assume you have just been promoted to Senior Test Manager, responsible for
testing your company’s most strategic project, a Personal Video Recorder (PVR)
capable of recording and playing back 3D TV.
The development model for this project is the V-model.
The project is due to launch in six months, and different levels of testing are already
being performed in parallel.
There are six testing teams assigned to test different parts of this project. Four teams
are dedicated to testing different components of the PVR and performing integration
testing between components. The managers of these teams report to you as part of
a matrix management organization. For the other two teams, one team is
responsible for system testing and the other team is responsible for user acceptance
testing. The managers of these two teams report to you directly.
Senior management holds a weekly strategy meeting and you are expected to
prepare a 15-minute update on the test status of this project. You have been given
six weeks to prepare for the first update.
During your first day on the job, you meet with the six Test Managers. You
immediately notice that there is no standardized reporting from the test teams. Each
team is using a different set of metrics to determine quality and exit criteria for their
part of the project and for analyzing efficiency and effectiveness of their test process.
Next, you meet with the managers of the four component development groups, the
Project Manager and the Vice President of Research & Development, in order to
understand what information they need from the test teams in order to make the
correct decisions about the project.
By the end of the first week you realize that you need to define a set of useful
metrics to be used throughout the test project, create test reports to be used by the
test teams, and create a high-level test report and dashboard for reporting to project
management and senior management. 

Scenario 1 “Test Project Evaluation and Reporting” above.
1. Given the information in this scenario, describe a test status
dashboard that would provide relevant information to the test team
managers and would provide a test status across the complete
testing project.
2. Given the information in this scenario, design a project dashboard
that would provide relevant information to the following
stakeholders:
a. Project manager
b. Development manager
c. Vice President of Research & Development (VP R&D)
Your answer should demonstrate an understanding of the different
information needs, level of detail, and presentation media for each of the
stakeholders.